It was this way back to the time that casters could do above 3rd level spells. High level casters were always constructed with the idea that they would be able to alter the world with magic making them more powerful than martials. It is a fantasy genre trope whether it be Merlin, The Dragon Reborn, the various sorcerers in Conan, Elric, Alanon, Gandalf, and the list is nearly endless. Magic is mysterious, powerful, and virtually unlimited.
Except that, in those olden days, high-level Fighters were given their own set of impressive, and more importantly powerful, benefits. A big one is that Fighters had great saves; they could often shrug off the effects that a high-level caster could throw at them. Fighters also had an almost open-ended number of attacks, based on level, against what we would, today, call "minions" (I think back then they were "level 0" or something like that? I don't remember the term).
As for the power and limits of magic--that's in the designer's hands. Certainly if you want to cite things like The Dragon Reborn or Gandalf, you're comparing things that don't look at all the same to me--neither one of them could *possibly* be represented in D&D, though ironically for totally opposite reasons. Channeling is NOTHING like spell slots, except that you can do lots of it (and, to be fair, much of it is very flashy)--spells are a matter of willpower and can be
created on the fly (Flame of Tar Valon, anyone?), you can burn yourself out, and individuals can be
massively talented at one specific thing and total weaksauce at everything else (e.g. the guy who is
amazing at making portals but barely mediocre at most other magic). Gandalf's magic is only slightly better, and only because it's used in discrete chunks, but it's also very rare, as opposed to the D&D Wizard who is flinging out spells left and right by the time they're at a level remotely like Gandalf's; he's obviously extremely powerful, yet he almost never casts magic! Even Garth Nix's
Old Kingdom doesn't really have D&D-like magic, and the whole "study the Charter Marks, which come in discrete circles of power" thing is one of the closest conceptions I've seen.
If "genre emulation" is what the caster/martial divide is supposed to do, the only "genre" D&D is emulating at this point
is itself. If we're willing to buck other traditions, this one too should be up on the table--particularly when there are dozens of other, *at least* equally-valid, ways of doing magic--that *don't* have to make some characters Simply Weaker than others.
I don't understand why so many have such a problem with it in a game that is supposed to mirror the fantasy genre. Is it because you don't read fantasy books? Or because you prefer a game be balanced?
Before I respond: Questions of the type "do you just not read??" are rather offensive. They make you come across as someone who thinks that, in order to disagree with you, a person must be deficient in some way.
Yes, I read fantasy literature. I also read mythological literature, science-fiction literature, and a number of other genres (including nonfiction, particularly philosophy journals). If "genre emulation" is what we're actually going for, I think D&D is actually pretty crappy at it. It's certainly crappy at representing the vast majority of main characters, and its spellcasters are only the vaguest bit like the vast majority of magical heroes. Gandalf is nothing like a high-level D&D Wizard. Every version of Merlin I've read about is nothing like a high-level D&D Wizard, and others have explained that particular difference well enough. "Artillery-like" (e.g. flashy, single-use, expendable) spells are a very "modern" concept of how magic works. I consider myths, legends, and fables just as much part of the "fantasy" genre as anything from the 50s (Dying Earth, Elric) or even the whole 20th century (e.g. John Carter in the 1910s, Conan in the 30s). Only supporting a "hermetic artillery" conception of magic is a sad loss even compared to that lot, to say nothing of ignoring an enormous swathe of mythic and fable traditions--numerous each from Europe, the Middle East, and both South and East Asia.
What I can say is: I've read basically none of Moorcock or Vance. I see no reason whatsoever to privilege their perspectives on magic above or below any others. I also see plenty of reason to accommodate mythic figures like Cú Chulainn (or Conchobar mac Nessa if you want a non-demigod), Beowulf, Gilgamesh, Agamemnon, Odysseus, Sigurd, Sinbad, Samson, Ali-baba, Harun al-Rashid, Bharata, Karna, Liu Bei, Sun Wukong, and many others (ideally a more gender-mixed group--I suppose I should have added Atalanta, Scheherezade, Ruth, Rahab, Deborah, Scathach, Boudica, Joan of Arc, Wu Zetian, Hua Mulan, and a few more badass ladies.)
And yes, I do value a game which puts all players on a "big picture" level playing field. As well as a game which guarantees that, no matter what preferences you have, those preferences provide baseline choices* which inherently and directly support** participation in all things considered "important" to play†, without depending on general-use items, and without having to "opt into" participating in them††.
*Read: classes and races, mainly; 4e, 5e, 13A, and other games show that a push to add a third "baseline choice," Background/Theme, is growing.
**Ideally with both "active" (aka "declarative") and "passive" (aka "always-on") benefits.
†In 5e, this would be the three pillars, which the designers have explicitly chosen to make important.
††If the designers consider something to be fundamentally important to playing the game, whether you get stuff for it shouldn't be a choice IMHO. I've considered starting a thread specifically to discuss that very topic, since it was...so compellingly questioned in the "Why Does 5e Suck" thread before I bowed out of it.
On magic vs martial, it's slightly more complex than just stronger vs weaker. In 1E, magic-users were dominant combatants, but they had significant weaknesses, especially spell interruption. They needed the martial characters to cover their weaknesses or they were orc meat. In a 1E fight, frequently the most critical issue was to keep the bad guys off your magic-user so he can get his nuke off. In 3E, wizards kept the strengths but lost the weaknesses, so of course they dominated. Spell interruption pretty much went away with the new initiative system and that one feat. In 5E the wizard's strength was reduced moderately, but the weaknesses were not re-introduced. My preference is the 1E approach.
This is what I was trying to say, so thank you, Mishihari.
5e toned down the upper echelons of caster power. It also slightly pushed up the minimum competence of all characters, thus necessarily making characters resting at that minimum better, namely "martial" characters. Clearly, a visible quantity ("some" sounds too small, "many" sounds too big) of people feel these changes were enough. I do not. That does not mean I want Fighters doing every single thing Wizards do (that is obviously silly). I do think SOME caster classes (mostly Cleric/Wizard) are still
slightly too powerful+flexible+adaptable. I also think that Fighters could have been given just, say, two extra (preferably declarative) abilities to call uniquely their own, and would have been brought into "acceptable" territory. I'd still
prefer 4e and the way it definitively solved these issues, that much is sure. But I'd be, at the very least, harder-pressed to complain.
Would I still have complained if this change had been there at release? I don't know. Maybe I would, maybe I wouldn't. Maybe I would have been completely unpleasable with anything that didn't clearly evolve from 4e. Hypothetical aesthetic first-responses to things are not easy to judge. I can, at least, say that there WERE times in the playtest where I was actually excited--the first-round Warlock and Sorcerer, to be specific. And then that excitement was dashed upon the rocks, without even the chance to defend itself.