Really, this problem is a result of the weird vestigial nature of ability scores. Over time, the role ability scores play in the game has changed considerably, making the numbers more and more important, while making the in-universe qualities they allegedly represent less and less reflected by their mechanical function.
In the earliest incarnations of D&D, ability scores served primarily as prerequisites. If you want to play X class, you need Y ability at Z or higher. They also gave small bonuses to certain tasks, though more often than not, your Class itself had a bigger impact than your ability scores. You see this especially in AD&D (both 1e and 2e), where every action has its own subsystem for resolving it, many with different tables to consult depending on your Class, and some actions you have be a certain Class to even attempt.
This left a lot of freedom to customize ability scores for the sake of role playing. As long as you met the prerequisites for your Class, the rest was largely up to you and what kind of person you wanted your character to be. But, there were a lot of problems with having so many separate subsystems. For one thing, you’d have to remember them all, which was a pain. And then what if a player wanted to attempt something that wasn’t covered by a subsystem? Do you re-purpose an existing system? Do you modify it? Do you just make something up? A lot of folks went the “just make something up” route, which is why there’s often a lot of “back in the day, there was more role playing and less roll playing” comes from, because a good chunk of the time there either weren’t rules for what you wanted to do, or the rules that did exist were ignored for being needlessly complicated.
Enter Wizards of the Coast. They set out to fix these issues in 3rd Edition by unifying all the various action resolution subsystems, making them all use the same math as attack rolls, but flipping the math so you’re adding instead of subtracting. Anything you could conceivably want to do, roll a D20, add a bonus determined by your Ability Score, add a bonus for any training you might have in a relavent Skill, and try to get a higher result than the task’s difficulty. This is, broadly speaking, a much better system. More approachable, easier to learn, and easier to improvise with, making it far more flexible. However, it also meant that every conceivable action had to be able to relate to one of the Abilities, and that’s just not something Abilities were originally designed to be able to handle.
This caused some weird things. A lot of tasks that didn’t previously relate to any Ability had to get rolled into one of the 6, and Wisdom and Dexterity did a lot of that work, which is a big part of why Dexterity is now the god stat and Wisdom is this weird amalgam of intuition, perception, empathy, willpower, and spirituality. It also made Ability scores way more important, because instead of giving small, nice-to-have bonuses on a few tasks but otherwise mainly serving as prerequisites and roleplauing aids, they suddenly became the backbone of the universal task resolution system.
4th Edition recognized that ability scores no longer served the same function, and attempted to fully embrace their function as raw numerical bonuses to various tasks. They wanted each Class to be able to use whatever ability it wanted for its attack rolls, and to a certain extent that made sense. Barbarians are supposed to be tough and do lots of damage, so why not let them use the stat that gives them more HP as the stat they add to their attack and damage rolls? It’s all just numbers anyway, why not swing that sword with Constitution? But, as with so much of 4th Edition, it failed to reconcile this new approach with player expectations. A Constitution Score might just be a number as far as the rules are concerned, but to the players it still represented a character’s hardiness. How could being tougher make you swing a seord harder? Without completely getting rid of the baggage of what these numbers represented in character, this numbers-first approach wasn’t going to be satisfying.
5th Edition, for better or worse, has been hugely influenced by the reaction to 4th edition, and a lot of the choices it makes are effectively apologies for 4th edition’s missteps. So, now we’re back to 3rd edition’s style of handling Ability Scores, where each Score does very specific things, regardless of Class. There’s a tiny bit more flexibility when it comes to magic, as different Classes have different Spellcasting Abilities, but you’re not going to be swinging any swords with Constitution this edition. However, with the addition of Bounded Accuracy (itself partly a response to the enormous numbers creep of 4th Edition), Skill training doesn’t outshine Ability Modifier nearly as much as it did in 3e. That means Ability Scores are even more important in this edition because not only do they tie into every conceivable action, they also account for a much larger proportion of your bonus to any given action. And with base attack bonus being combined with skill proficiency bonus, that goes for attacks as well.
Naturally, with attack rolls and spell saves being the most frequent rolls made in D&D, and with ability modifier playing a pretty big role in your chances of success (or your spell target’s chances of failure), plus your ability adding to damage... it’s onlu natural that every martial class will make either strength or dexterity their first priority, and every spellcaster will make their spellcasting ability their first priority. And as long as the mechanical element that makes a character good at casting wizard spells is the same as the one that represents how smart a character is, Wizards will always have Intelligence scores within a similar range.
