It is just one badly designed rule. The designers must have been drunk and this rule will not survive the next revision.
C'mon people... if I want to play Magic, I will get out my cards.
Maybe...just maybe... the designers jumped to the insane conclusion that this game was meant to be run with a *DM* that can take into account certain things that would otherwise not make sense.
(Somewhere) In the DMG it talks about doing damage to objects, and that the *DM* (there it is again) is to determine when certain weapons/attacks would be less effective dealing damage to certain objects. That Greatsword sure does more damage than a battle axe... guess which will chop down a tree better? Hmmm the warmace and longspear do thesame damage as the battleaxe, they should be good for hitting a tree...
Plus, the rules are aimed towards 'objects'; not really laying seige to a castle. The RAW *include* room for DM interpretation and action... this is one such place.