A tree does the same thing. Either you can deal with foes at range or you can't. If your foe can't a tree will save your butt about as often as the fly spell does. The only issue I have with flight isn't flight but the insane ranges on some spells. As long as you are well within basic range weapon categories when on offense I don't see it as an issue. Heck if you have some need that they always be in weapon range give the fly spell a flight ceiling or something. Claim its based on telekinesis and it pushes off the ground and after a certain distance from the ground there isn't enough force to push higher.
Ok, I just don't see muddy ground as a big issue to for my adventures I guess, or a mountain. Terrain is more of a back story element to me not a large challenge to overcome. Besides you can easily open up airborne terrain hazards. Wind is much stronger in the air, there isn't any wind breaks to slow it down and you have nothing to brace your self against. Also any weather based problems other than fog usually are still in the air unless someone is flying really high. Heck even in 3e by the time you could fly in anyway that would disrupt these elements you were high enough level that I just don't see these things as appropriate challenges anymore. And the ones that were appropriate challenges generally flight did nothing to solve.
Eh spell casters solve problems with magic, skill dudes with skills etc. My only complaint has been classes like the fighter that don't have any way to solve dilemmas, and skills seem more limited to me than they should. Escape artist should be able to pull off a lot of the features of the freedom of movement spell IMO, for example.(not an epic person maybe it does at epic levels, I think it should be available sooner)