Agreed.
Sure, I alluded to the same. I may have missed it's usefulness by a hit die or two. Fireball should be useful creatures a lot higher than 4 hd.
A 6 hd creature has a 35% chance of making it's save. Even if it fails, it's still up.
I never said that. I'd just like to see it useful long after level 7 or so.
Because burning a 4th level spell for one or two d6 is almost always a bad decision. 2d6 damage but I lost my only charm monster or polymorph spell is not a wise use of spells.
Stop putting words in my mouth. Pre 3e, fireballs, lightning bolts, etc were staples of every mage and useful even at very high levels. Capping them at 5d6 will make them tools of the vermin exterminator, nothing else.
I apologize if you felt that I was putting words in your mouth. That wasn't my intent; I was just expressing that I feel that you're going too far in what you expect it to do.
Let's take a further look at how Fireball worked. Fireball capped out at 10d6 (an average of 35 damage). Assuming that no save was made, that would only outright kill a 7 (or maybe 8) HD creature. That was at 10th level. Remember that since saves were tied to HD, an 8 HD creature had a noticeably better chance of making it's save than a 4 HD creature. It didn't scale all that amazingly even back in the day, in part because HP scaled faster.
+2d6 for a higher level slot was just a guess based on 1e/2e. For all we know, it could be +3d6, +4d6, or even +5d20. We simply don't know. I'm fairly sure, however, that there is some amount of bonus damage that you'd at least consider trading your Charm Monster spell for. If they make you consider that trade-off carefully, then they've probably nailed it.
In the end, I don't think that fireball should be capable of taking out an equal level creature on an average damage roll. That's overkill. It should definitely be able to eradicate "vermin", and soften up tougher targets, but it shouldn't be able to end a typical encounter of your level. Keep in mind that if things scale as slowly as it seems they might, those "vermin" will be a very real threat to the party.
They're balancing the wizard against the fighter by giving the fighter better normal damage, but the wizard much higher spike damage. AoE damage, however, should always be lower than single target damage spells (otherwise, why bother with the inferior single target spell). If an AoE spell can wipe out an entire encounter by itself, not only is the fighter's advantage negated (because you can't benefit from higher average damage if the wizard ends the encounter on round 1), but the single target spell now has to be powerful enough to take out a creature well above the party's level. Not something I expect many people to find desirable. Fireball needs to leave enough space for the rest of the party, as well as the wizard's other spells, to contribute.
As for it not scaling automatically, I expect that's part of the whole system scaling much more slowly. We're already heard reports that hp and BA are lower / scale at a very moderate pace. If fireball scales automatically, and the wizard also gets new spell slots every level, it's quite possible that he'd outpace the leveling in the system. I think that this soft cap is in place, in part, to allow the wizard to gain new spell slots without completely outpacing the rest of the system.