Because it makes things vastly easier to DM for a new DM. And it allows much better narrative pacing than to have to be cautious with my monsters lest I accidently kill a PC. Balance is information. And 4e is the best game I have ever played for new DMs - partly because the information presented on what the bad guys can do and how to keep the story feeling hard but actually manageable is so good.
And add to that everything AbdulAlhazred just said.
Having just read the whole topic

, let me second that and add to it. There is really not much on this front that an experienced group couldn't eventually get with any version of D&D. My group pretty much got to that point with 1E, for example, after several years play. I could run low to mid level 3E that way, now.
But besides making it easier for beginners, the other aspect of that is how the balance changes over levels in earlier versions. We didn't play high level 1E much--because it all changed, and we would have had to relearn how to manage it. We had problems with high level 3E for the same reason. Sure, if we kept at it long enough, we would have eventually come to some understanding of how to make it work for us.
Thing is, though, that I don't want to do that, and neither does the group. We want some system mastery competence to matter, but we don't want to spend a couple of years mastering it before we can settle down with telling stories with it. And if I try to shield the players from the complexity, then that is just trading GM burnout issues for system mastery issues.
From a strictly personal preference, it is not simply "amount" of system master, either. It is when and how it is introduced. Fantasy Hero and Burning Wheel are
extremely front-loaded, but once you get them, there isn't much else to learn. To a lesser degree, RuneQuest is too. (Setting mastery is the thing in RQ.) D&D has traditionally added complexity as the game progressed. 4E fits that basic model, more or less, but it is more front-loaded, with less of a curve as it goes, than other versions of D&D, for the players, while being significantly less front-loaded and difficult overall for the GM.
I don't really have much experience dealing with people who are incremental learners
by choice. In software development, as with many complex disciplines, we are forced to by the sheer immensity of the problem. In games, including roleplaying games, everyone I knows wants to master the game options as quickly as possible, to then let whatever emergent gameplay is there to emerge. I have no idea how widespread that attitude is, though.