Leatherhead
Possibly a Idiot.
What's wrong with that, then? If you only have 3 levels of wizard, you would cast fireballs as a 3rd-level wizard. If you want to get better at being a wizard, you'll have to gain more levels as a wizard. I don't see the problem.
The problem is Scaling, and that problem only gets worse when you consider action economy.
Your level three fireball is dealing 3d6 damage. How good is that at level 20? If it isn't any good at level 20, then having three levels of wizard is worse than being completely useless. It is a detriment to your character, because you could have had three more levels of your other class for actual benefit.