There are few options for sharing webroots among multiple webnodes. This is especially true for Drupal, which has a files directory that could be constantly diverging on each webnode. Usually people decide to go with NFSv3. This is what we do on drupal.org and most of my clients do at Tag1. Sadly, this is not really a decision on merit, but on a lack of other options. NFSv3 is pretty much like communicating with two cans and a string.
A major problem with running Apache/PHP/Drupal on NFS in particular is lstat. Drupal uses a PHP function called include_once and uses it often. This function includes other PHP files, but does it only once. (Truly brilliance in naming) It differentiates between files with their full path and it finds their full path with lstat. Doesn't seem like much, but this leads to a lot of lstats and these calls are not at all cached using NFSv3. Using strace to count them on one Apache run leads to results like this:
47.71 0.847578 3 267280 3192 lstat
19.86 0.352825 10 35063 4101 open
8.28 0.147049 2 66478 16474 access
6.83 0.121374 4 34190 1707 stat
5.21 0.092609 3 33295 260 read
The important columns are the first (percentage of time in syscall) and the 4th (number of calls).
Enter PHP 5.1's realpath cache. Again, PHP developers' naming genius shines through. This feature caches the real path of PHP files and it is used to cache the checks that include_once performs. Sadly, the default php.ini sets this to 16K and makes it close to useless. It is a fairly new feature, so isn't that well known yet, but it shows very real results. Changing this setting to 800K on drupal.org's webnodes gave us this strace result:
58.80 5.179350 31 166219 12408 open
12.51 1.101776 5 223492 4308 read
5.49 0.483280 3 158441 20494 stat
4.90 0.431528 3 168837 36562 access
3.72 0.327620 2 154757 write
2.76 0.242768 1 233763 20582 lstat
2.39 0.210757 46 4595 writev
That is a massive difference. In my testing on another site, this setting increased the pages served by 3-4 per second.
Long Live The Real Path Cache