Prevalence in play seems to be more of a player and table issue. Don't want casters? Limit them. Some people like martial classes (I do). Some don't. Find players who like non-casters.
It's not quite that simple, since a party needs contributions that come primarily from spellcasters. Non-casters, all 5 of 'em in the PH, all contribute DPR well enough, 3 are tough, 2 have particularly good skill checks. That's about it.
But I do also agree that everyone casting the same spells the same way is a problem, but it is probably superior to having a half-dozen different casting systems.
Radically different casting systems can be problematic, yes. The other alternative is for each class to have an entirely unique spell list. That requires a lot more design work and eats up column inches.
That, to me, is the key. As has been pointed out, and as is my experience, it is exceptionally difficult to build a non-magic using class in 5e. Which is pretty much the opposite of, say, 1e.
5e re-uses a lot of spells when it creates a new caster, sure, that makes it easier. Same trick worked in 3e. In 2e spells grouped in spheres gave you lots of 'different' clerics very easily. 1e had many classes sharing spells, as well.
Let's be specific.
In 5e, the only main class that has no subclasses (that I am aware of) that doesn't have spellcasting options is the Barbarian.
The totem barbarian can use some spells as rituals.
You have primary spellcasters (Bard, Cleric, Druid, Sorcerer, Warlock, Wizard). You have the secondary spellcasters (Paladin, Ranger). That leaves four (4!) classes. Of those, three (fighter, monk, and rogue) all have spellcasting (or, in the case of monk, pseudo-spellcasting based on ki) options. And, of course, you can always cast spells with the feats. And this is before considering multi-classing.
In short, it's the exception to have a character that cannot cast spells.
Take 1e as a counterpoint. Clerics, Druids, Magic Users, and Illusionists could cast spells. Period.
And Paladins and Rangers at high enough level. And the bard in his appendix in the back of the book. And the very-common-at-low-level multi-classed non/demi-humans.
So, strict non-casters: Fighter, Thief, Assassin, Monk. 1e w/UA: add Thief-Acrobat, Barbarian, Cavalier.
5e, strict non-casters: Champion, Battlemaster, Thief, Assassin, Berserker.
And there were no cantrips,
They were added pretty quickly, though, even if they weren't at will.
The feel, therefore, is very different. Magic, in 1e (to use an example) was almost always a big deal. Magic, in 5e, is a given.
Magic in 1e was obligatory. Couldn't get by without that Band-Aid Cleric. ;P