First, let me link to
this, so people know that even if I may seem critical of 4e in this post, there's a great deal that I love about it. Also, let me state that while the below is written in straight declarative statements without qualifiers, all of it simply represents my personal take on the situation -- it's all "I think" and "IMO".
A lotta folks, particularly 3e fans, point to the marketing for 4e. But this was absolutely out of the 3e playbook. They did the same exact thing when they introduced 3e. Worse, even. To my knowledge, they never introduced t-shirts listing how 4e was better than 3e, which they did for the introduction of 3e. Far be it from me to tell anyone if they should be offended or annoyed with something. But it does then strike me that this is probably not really a 4e issue.
For the moment, let's ignore the big picture (we'll come back to it later), and focus on the relatively small sample of the internet community. In particular, there's something that has always bugged me in these discussions. Manbearcat said,
[sblock]
1) prolific indie design elements (there are tons, from system elements to GMing role and GMing principles)
2) transparency and focus of its architecture (outcome-based design rather than process-base)
3) unified class resource scheduling which focused on the "genre tropes and the tactical resolution of the conflict-charged scene (see 1)" rather than the "strategic murderhoboing of a sandbox setting/dungeon crawl"
4) presentation including actual formatting, the artwork, and the facepalm editorializing in the initial DMG and PHB - "skip the guards and get to the fun (!)"
5) lack of fealty to canon and aggressively establishing its own backstory
6) aggressively coherent design framework that pushes play toward consistently churning out high-octane action for mythical fantasy as default (rather than - say - FFV or GoT)
7) aggressively taking a stand on historically polarizing elements such as HP
8) 4e "basic" (the initial core books) didn't include the Barbarian, Druid, Sorcerer, and Monk. The prior 3 only had to wait 7 months for PHB2 but Monk had to wait 19 months (ouch) for PHB3
brought out the worst tribal instincts in the D&D fanbase.
[/sblock]
I don't mean to pick on Manbearcat, but the above quote nicely encapsulates a common thread of argument. One of the worst bombs to throw into a 4e discussion is "It's not D&D." But I'm sitting here looking at this list of how 4e made major, fundamental breaks from D&Ds of the past, and then when people vehemently reject these changes, it's chalked up to the "worst tribal instincts of the D&D fanbase." Excepting, of course, for the obvious existence of trolls and jerks in almost any internet discussion, I like to assume good faith on both sides of this issue. WotC made drastic changes to the game, fundamentally changing it. It is, for all intents and purposes, a different game that plays differently. They did the same with 3e, but it would seem to a lesser degree. I think it's entirely fair to say WotC changed the game too much to be accepted by a significant number of fans. That doesn't mean "WotC tried to make it into an MMO," is a particularly good argument, but nor does it mean that fans were being unreasonable in expecting a certain fundamental consistency in the gameplay and fluff. I greatly enjoy playing 4e, and it mechanically supports the tropes we liked back in my early BECMI days. But everything Manbearcat mentions in the above quote? Not what I expect or look for in D&D. (For that matter, neither is 3e's obsessive focus on world simulation through play mechanics.)
But back to the big picture! While the above matters for the most part to our the serious RPG gaming community, I do
not think it was the problem with 4e on the whole. And that's because I firmly believe that WotC has never cared about "bringing along the base" with a new edition. I've talked about
before, new editions are all about bringing in
new players. And those players don't really give a crap about previous iterations of the game (at least, not at first). So, while I can sympathize with older players who felt 4e was a change too far, I don't really think that hurt 4e or WotC. They got a lot of new players. They retained some players who liked the changes, and some players who didn't mind the changes. Enough that they could prove the viability of the DDI subscription model, which gave them both gobs of money and gobs of market data.
But I think that early on they found that they were not meeting their projections for new players. Because new editions are in many ways reactions to old editions, you can perceive what the designers/managers saw as issues of old editions by looking at the new ones. 2nd Edition was in many ways a reaction to the baroqueness and family unfriendly presentation of 1st Edition. With 3e, they wanted the mechanical universality and consistency that 2nd Ed. didn't have. With 4th edition, they wanted to address the balance and GM issues of 3e, while bringing in design elements from popular indie, online, and board games.
Mearls has said (I wish I could find the interview to link to it) that one of the things their data showed was that a lot of people were buying the 4e starter sets, but they weren't moving on to the rest of the game. And you can see this hanging over much of 4e and the transition to 5e. 4e had
two starter sets in three years. They tried to lower the barrier to entry with the Red Box starter set and the Essentials line, but ultimately it was no go. Over and over throughout the playtest Mearls has stressed making the core game easy and simple for new players to pick up and get right into gaming. To the point that they've made big changes to their sales model. Instead of 3 Core Books as the game and an introductory set to get you interested, they have a cheap quick-start starter set, that transitions to the free Basic game, that transitions to any one of the Big Three, depending on your particular needs.
Regarding the indie elements of 4e and how much Mearls understood it, I think I'll keep this message from getting any longer than it already is by linking to these
previous posts.