But those are not the most common things you'd have to defend your town from. You're talking about invading armies. But, most communities don't have to worry about that all that often. You don't, generally, have large invading armies coming around every year. Or even every ten years, really. Most communities don't need these anti-flying, anti-high magic defenses for the same reason that the vast majority of homes in Canada or America don't have security services. Your odds of a home invasion are realistically so low that it doesn't make sense to spend that much money on security.
However, roving packs of owlbears or various common monsters and humanoids would be a pretty regular occurrence. Sure, your moat doesn't keep out a dragon, but, it does slow down a troll or an ogre, which, really, is why you have a moat in the first place.
That you cannot protect from everything 100% of the time isn't why you build walls. You build walls for the stuff that happens most of the time.
I would disagree that flying monsters would not be a regular occurrence. I think wyverns, hippogriffs, chimera, harpies, griffons, young dragons, and other flying creatures that require large animals for sustenance would be just as likely to invade populated areas to go after and grab cows, horses, sheep and then fly away as any ground-based invader would. And that's really my point... walls are an expected, obvious and really
only defensive measure from our real-world point of view because we as humans never had to deal with flying invaders. But in a fantasy world... they would have evolved knowing and dealing with
both types of invaders, and thus at the very least there would be a logical
tandem of defensive measures that towns and cities would have come up with.
I am more than willing to go along with the idea that walls could have evolved for towns and cities over the centuries / millennia as a possible defense... so long as there be equal lip service to a simultaneous advancement of aerial defense. I mean at the barest minimum, ballista would be much more prevalent atop these city walls than usually get portrayed... but I also think that with Force magic being so prolific in use at even the lowest levels of magic--
Mage Armor, Shield, and "always hitting"
Magic Missile-- that that magical technology could, would and should have had much more of a build-out over the years for larger-scale defense.
I mean heck... wouldn't one think that over the tens of thousands of years of high elven magic use that at least one of them might have said "Hey, you know... we have this magic that creates a defensive force armor barrier around a person... maybe we ought to see if we could expand that magic out to cover our horses? Or our carts? And then maybe after a while our homes?" And then eventually around entire villages, towns and cities. But nope... they got as far as inventing
Mage Armor and then just said "I think we're good! There's nothing more to be done!"