In my
current campaign, we've been doing it this way:
1. Tower shields provide up to complete cover to the person behind them, or 75% cover to an attacking person (+7 AC).
2. The shield guards carry the shield, and stay roughly in line with their buddies. I ruled that this counts for 75% cover against the front (the slight vulnerability is due to imperfections in the shield wall). Shield guards who choose to stay in the line get no dodge or DEX bonus; they can break ranks and gain their usual bonuses.
Note: I allow shield guards to fight within a 3' wide space, since they are not actually fighting, just carrying a shield. Someone without proficiency in the tower shield, however, will jostle those around him, and thus needs the 5' wide space.
3. Spear soldiers stab through the gaps in the shield. The spearmen get 75% cover (+7 AC); the enemy gets 50% cover (+4 AC). Longspear soldiers stand one more rank back and stab through the gaps as well. The enemy gets an additional +2 AC against the longspearmen, but retaliation is almost impossible.
4. A few semi-random modifiers:
Uphill battles: An enemy that is uphill of the shield wall reduces the shield wall's cover to 50%. Shield guards can carry their shield at a high angle, but this is exhausting (checking for being winded, exhausted, collapse each minute).
Tricky terrain: Reduced visibility is a serious hazard, here. On unstable ground, I require Reflex saves not to fall down.
5. The shield guards can take a couple of feats to help them out:
Tight Formation: If you and both adjacent shield guards possess this feat, you receive
total cover from your tower shield while in formation. Your shield can still be attacked and damaged, but you are completely hidden behind it. You may only move at 5 feet per round, as you can not see where you are going. A solid wall on one side counts as a shieldmate with this feat, but walls on both sides negates this feat. People behind you (such as spear soldiers) still only receive 75% cover.
Spear Friend: You are skilled at making openings in the shield wall at just the right moments. The enemy only receives 25% cover (+2 AC) against the spearmen.
Shield High: You have developed the strength and discipline necessary to carry your shield at a high angle, negating the penalty of fighting uphill, without swift exhaustion. Check for tiring every 10 minutes instead of every minute.