Could it work? Absolutely, lots of systems exist that work fine without them. Would it be D&D? Not at first, but give it twenty years and it would probably get normalized into "That's what D&D is."
I will say, the staggering saturation of ability scores across so many games, both table and...