The reason there's a "sweet spot" in the current game is that it's the approximate range of levels where, purely by coincidence, the math of the system actually works. In those levels, PCs don't drop after one hit, and they don't take a dozen hits to wear down. In those levels, characters miss monsters occasionally, but less than half the time, and monsters miss characters only slightly more often. It's pure chance, really, but it means the game is fun. Outside of those levels, the math doesn't work that way, and the game stops being fun.
In Fourth Edition, we've totally revamped the math behind the system, and that's a big part of the way that we've extended the sweet spot across the whole level range. When PCs fight monsters of their level, they'll find that the math of the system is more or less the same at level 30 as it is at level 1. There will always be variation with different PCs and different monsters, but that variation won't be so great that monsters are either too deadly or too weak.
Of course, there's more to the sweet spot problem than just the math. The proliferation of save-or-die effects and adventure-breaking effects like etherealness and scrying also makes high-level adventuring more difficult to pull off, and we've addressed those issues as well.
Fundamentally, this has meant we've had to abandon some things that might have seemed like sacred cows—fireball spells don't do 1d6/level any more, for example—but it's all in the interest of a far superior play experience.
