I would just dump Agonizing Blast, it's the real degenerate case that causes the intra-class balance problems. Getting +stat mod to cantrip damage is supposed to be something you get at mid levels, and never more than once per cast.
As a warlock player I can tell you this is not a requirement to making a fun, useful, warlock. Nor is EB blast itself.
Wow. What order to unscramble this in?
A) Simple blaster Warlocks are already a thing. And it doesn't get any easier. Step 1: Choose Warlock, Step 2: Select EB. Step 3: Select the EB modifying invocations.
Congrats, you've created the most stale, cookie cutter, damage machine you can with the class.
B) Why isn't EB a class ability? It is. It's an option you can take under the cantrip heading. You have to CHOOSE it because there's all types of warlocks out there, not just damage dealing ones.
Why aren't EB modifiers built right in? Again, choice. And clearly the designers intended you to only have access to a handful of spaces in wich to customize your warlocks power set. Wich brings me to....
C) Those invocations you think should come with a price? They DO. A heavy one. Because if you take one of them you're not doing something else cool. So choose carefully.
Of course this does depend upon how you look at a class. If all you see when you read the entries is x damage? Then you'll never understand how cool something like beast speech can be.
At my table, every player who has picked warlock started with wanting to do Pact of the Blade and then changing their mind after looking closely at it. That's where I'm coming from there. I'm not concerned about modules, as I never run them unmodified. As I'm houseruling, I'm not worried about Adventure League either.
I've strongly considered severely altering EB. I don't like how it scales with character level, but Extra Attack doesn't scale or stack. But I wouldn't want to take away what I consider to be a core ability. And I recognize that it's an option to not take it, but that simply feels weird to me.

(Dungeons & Dragons)
Rulebook featuring "high magic" options, including a host of new spells.