OK, what if Mystic Arcana is removed from the invocations, rather it replaces "Pact Magic" as a class ability, and is how you get the escalating number of spell slots that you can upcast into, for bigger booms that a half-caster wouldn't normally get at the same level.
The number of Invocations you get over 20 levels would then be refigured to scale better, balanced against the increased casting power.
Also, There is nothing wrong with adding a Mystic Arcana at Level 3 in this design. There is nothing saying you can't get a base-class ability at the same level you get your subclass abilities.
I think that's getting closer to something reasonable.
I also think that you could just have normal progression of spell levels, but not have all your slots be at top level, instead having 2 at top level, 2 at that level -1, and 1 and that level -2, or something like that.
So, at level 5, you'd have 2 level 3 slots, 2 level 2 slots, and 1 level 1 slot, per day. From there, we can figure out some "regain x spell levels worth of spell slots y per day" mechanic, and go. Being able to cast 5 spells in a rush is plenty, especially when you also have invocations that give a spell 1/day, or at-will.
Edit: After level 9, you could very slowly gain a couple more slots of the levels you have, and gain mystic arcanum 1 cast per day uses of 6-9th level spells.
I'd also, if the Tome is supposed to be the focus on spellcasting warlock, it should have spells that it lets you cast as a ritual that normally don't have the ritual tag.
