Because the skills aren't primarily assigned to abilities from a balance perspective, they're assigned based on what ability the designers anticipated them being paired with most often. Free-form matching of skill with the most suitable ability for the occasion is presented as an optional rule, but it is clearly how the game was designed to be played, they just backed off from enforcing it. If they arbitrarily made Deception Intelligence based, or whatever, people would still just call for Charisma-based Deception checks.
I don't buy that. You want me to believe that they took that rule, and then explicitly labelled it
Variant under it's own special heading. Meaning, not the assumed, default rule. Then, they took each skill and placed them explicitly under a specific attribute. Not just on the summary chart of all skills available, but
the entirety of the skill descriptions themselves.
If they wanted the DM to pick the attribute, then they presented both attributes and skills exactly incorrectly. They did it so overwhelmingly incorrectly that it's not really credible to believe that they honestly thought the variant would or even should be a common occurrence in the game, never mind the default assumption. I simply don't believe that the D&D design team is
that incompetent at writing books.
It seems clear to me that what they did was decide, after playtesting and surveying, that skills should stay under one attribute essentially always. However, in limited circumstances, it can make sense to use different attributes entirely. In other words, they had a design idea,
it just never came up, and so they walked it back.
Persuasion and Deception are separate skills because either one is a top tier skill on its own.
No, they really aren't. There's almost nothing at a conceptual level that one can do that the other cannot. There's almost no situation where one is valid and the other is not. You have to manufacture limitations for one to apply and the other to be unusable. They may have different long-term consequences, but in nearly every situation where Persuasion comes up, you can simply substitute Deception to accomplish an immediate goal or vice-versa. In my mind that does not really justify them being two different skills under the same attribute.
The fact that you can persuade someone that a lie is the truth, or deceive someone into doing what you want and the game says that both times you should assume you're using Charisma just doesn't pass the design smell test.
Now, if Deception were instead a use of cunning and falsehood to trick someone into trusting you, then
that seems like a different skill. But that's not actually how Deception is described, and putting it under Charisma doesn't really reinforce that they're different skills, either.
That's why they put Investigation under Int and Perception under Wis. That's why they Nature under Int and Survival under Wis. That's why they put Persuasion under Cha, but Animal Handling is under Wis. These are all already skills that
do the same thing but do so with different ability scores.