
Maybe you need glasses. I jest..
In all fairness, I think in practice caster players often held back, and that was easier to do at higher levels as they had soo many options and resources. Their goal usually wasn't use all their biggest guns as fast as possible so they ended the day with the most damage (or otherwise most of anything), it was to find the lowest level spell they could cast that would be highly efficient and use those.
What makes casters so much more powerful (at high levels) isn't what they typically do at high levels, it's what they can do. Because it's not unusual for them to finish a day with more than half their slots unused - including their largest.
*Probably should move this discussion to one of the many caster vs martial threads already in existence.