I don't know if it could be D&D without ability scores, because i don't feel that strongly about sacred cows than others do. For some, it sure is going to be a deal-breaker.
I think fundamentally, ability scores can help shape an image of what kind of person a character is. It's easier to look at 6 ability scores and get a feeling then looking at two dozen skill ratings to get a feeling (at least if ability scores are relevant modifiers, if not, it could be utterly misleading. Say, you have a high Dex but Stealth, Acrobatics, Sleight of Hand are still abysmal compared to your Lifting and Jumping scores because you got+23 from skill rating abilities in them them but 0 in the other, and modifiers from ability are only from -4 to +4).
It can also help make people kinda "consistent" with a general character idea. So a smart character that forgot to pick Mathematics might still have a good baseline simply because the +4 from his Int is doing a lot. On the other hand, if you want to play a math savant that however has no clue about physics or magic, that +4 from Int might do too much on the other skills and doesn't feel right.
I could easily see a system that works actually the other way around - your Dex score is based on how good you are at a group of skills, and maybe your Dex score represents the floor of your abilities - if you got Stealth at 15 ranks but Acrobatics and Sleight of Hand at 0, maybe those 15 ranks gives Dex +5 and you could default to Dex for your Acrobatics and Sleight of Hand instead of your skill modifier.
I think from a certain point of view - with a mix of ASIs and point buy, why even bother setting an Intelligence Score for the Wizard? It's going to be 18 or 20 eventually, so just settle with the starting modifier value you want and increase the bonus at the rate that suits your system.
Remove the vestige of the Intelligence score and say Wizards have +5 to Arcane Spell Attacks (and their Spell DC starts at 15) and Knowledge skill checks and +3 to their other class skills. Fighters can choose between +5 to Melee Attacks and +3 to Ranged Attacks or vice versa, and +5 to 3 skills from the fighter class skill list and +3 to the remaining class skills.