Ancalagon
Dusty Dragon
I'm very disappointed in you with not agreeing with my premise.I don't agree with your premise but I did want to mention that I've long thought the Warlock should be intelligence based rather than charisma based. There are far too many charisma casters in this game, and you know the folks clever enough to gain power from an otherworldly being are likely to be the smarties in the room but not necessarily the charmers.

That being said, you have a point, but wizards can be wizards because they are so smart they can understand the magical laws of the universe and manipulate them. If you can do that, you don't need a pact with another otherworldly being...
A campaign setting where wizards don't exist but warlocks do? yeah, int-based warlocks make a lot of sense.