Morkus from Orkus
I think too much of this discussion devolved into how two spells in particular work. I think the more general thrust would be, given the spells and items in the game, what would a game world realistically look like? Does a setting like Forgotten Realms really work from a magical perspective? For example, take a wall-city or castle. Neither are effective at stopping aerial or teleporting opponents. Spells like Passwall render them moot as well. Plant Growth and Fabricate are just the tip of the iceberg.
It's unlikely that you will have an aerial army to contend with, so walls will still be greatly effective against an army that has a few fliers. As for passwall, you can make walls 25 feet thick, or just have a plan to let in 25-30% of the army and then dispel the passage, splitting the army and allowing it to be more easily destroyed. Using teleport, that's a fool's errand in 5e. Between bounded accuracy and the fail chance, you aren't going to get in than a handful of people, and that doesn't invalidate the need for walls or castles. Sure, it could be useful for getting in a spy or a saboteur, but those don't invalidate the need for castles or walls, either.