If you're using the "rules" for tower shields in the FAQ, yes, you pick a side of your square, but then the following applies:
If a magical attack has you as a specific target (that is it does not merely affect the area that contains you but is aimed right at you), the shield has no effect. All rays fall into this category, as does any spell that has a Target entry in its header and any spell that has an Effect entry and affects creatures (provided, of course, that the attacker can and does choose the shield user as a target).
Spiritual weapon has an Effect entry and affects creatures, so the tower shield would have no effect.
If you're not using the "rules" in the FAQ, then you don't have to pick a side of your square, and the tower shield is ineffective only against targeted spells (which
spiritual weapon is not), so it would provide total cover against
spiritual weapon.