Valid considerations of corner cases that could have big implications.
Most combats start with initiative and go for only a few rounds in which case the Voltron model might work pretty well most of the time. It would even have a pseudo old school vibe of magic users start off weak but then get really strong, although on the scale of rounds instead of levels.
I would generally say that you should not be able to hold a charge waiting for an ambush or walk around with short term 1 minute stuff always on with continual recasting outside of combat, it would work better if everything short term starts when initiative gets rolled. Maybe concentration being a big one spell limit would already make that not an issue.
A standard archmage can cast a meteor swarm at an army round 1 when they get in range right now so that would not be a change. The big deal would be the archmage multiple meteor swarms round after round in a months long siege.
The build up model might work better on a spell level per encounter basis so a first level caster gets one first level slot for one and only one big bang each encounter then cantrips instead of unlimited slots for their biggest bang in a fight, but that is outside the scope of the at will model.