Think about the half-elf, then, which was already unfair to almost any other build that also benefits from a high Charisma score. If mechanics are not supposed to inviabilize some character concepts, the solution is something more akin to all races being equal, not all classes getting the same...