Aldarc
Legend
There is also the dimension of level. There may be one wizard for 1000 mundanes, but that wizard is only level 1. A nobleman may offer a wizard patronage, granting them housing and a place of study in return for providing counsel and defense. More powerful wizards may require more for their services. So the average magistrate's villa or castle may only have a level 2-3 wizard. Another possibility is that the nobility themselves may be the wizards.Doublespeak, Department of, procedure, standard, Mark I. But why is castle X there in the first place, and who and what is there with an interest in besieging it?
If you really want to talk about what "makes sense", then -- to the extent that the elements have real-world analogs -- you cannot well dismiss real-world experience.
If you really want to talk about what makes sense in the context of a Dungeons & Dragons game, then you cannot well dismiss actual play experience. If there is advantage to be had, then count on players taking it! D&D is "survival of the fittest" on overdrive.
At least it used to be so. If the "magic economy" in 4e (plate +1, leather +1, same price!; no incentive to make ritual scrolls for sale; etc.) is "D&D rules", then certainly so are the Original and Advanced games' assumptions.
If you just want to shoot the breeze about whatever measures and counter measures people posting here can theoretically contrive, then you may as well simply drop those conceits -- contentiously dismissive claims that distract from your desire.
By pressing an argument that castles do not "make sense in a world of dragons and spells" -- and arbitrarily, against all evidence, insisting that the world must (in what becomes a circular argument) conform in every respect to the needs of your case -- what do you expect?
What I think one reasonably ought to expect as a consequence is counter-arguments.
Towns and castles: the presence of magic users, whether they are rare or not, shifts combat and defense towards a much more modern direction. Wizards and sorcerers become medium-range artillery pieces. And while walls may not be built to keep wizards out, they could be used to keep riff-raff peasants, most dangerous beasts out of towns, and to create choke-points for attacking mundane infantry.
As to magical protection, Eberron has the dragonmark of warding, whose guild houses focus on magical protection and defense.