Given the topic, I guess I will post what I have previously posted when I attempted to do a deep dive to try and understand this topic and the history. I will also add that apparently Ben Riggs is releasing a history that I am sure will cause this website to explode in frumious commentary when it was released.
TLDR- I think 4e's "failure" was due to a number of issues- from the economy to the marketing to the corporate expectations that could not be matched to the 3PP issues (OGL etc.) to the choices of the design team that prioritized design over the brand despite warning signs. I also think that it, like the Apple Newton, made design choices that continue to be debated, yet also influence the game today. I think that 4e did interesting things, and has passionate fans, and while it is not my favorite, I am very glad it existed.
I look forward to the time when we can talk about 4e without all the hurt and pain and anger people have on all sides. But given it's a decade later and this hasn't happened yet, I am not holding my breath.
-----
A not-very-brief history of 4e's issues and why it wasn't a success and was killed off:
A. At GenCon in August 2007, WoTC botched the rollout of 4e, causing many in the audience to (incorrectly) believe that a computer was
required to play the game. This was the start of misconceptions about this edition that the powers that be never really addressed.
B. June 6, 2008- the release of 4e. Do you know what else happened between the announcement of the product and the release? The Great Recession. Not the best time to release a new product (especially when you were hoping for sweet recurring subscriber revenue).
C. It was hoped that 4e would have MMO licensing, computer games, and more. But the timeframe was not favorable. Moreover, we can forget how ambitious this was for the time; the idea of "always on" internet was still novel, and services such as Roll20, twitch, and so on weren't around yet. Heck, the original (very slow!) iPhone had just been released. Yes, the D&D audience was more tech-savvy than regular consumers, but the rosy projections did not match the reality.
D. Building on (C), there exist players who view D&D as a mostly tech-free time. A respite from screens and technology. Sure, they might be luddites, and they might be a very small part of the market now, but they exist. Which also goes back to B, and the botched rollout- computers weren't required, but WoTC chose to emphasize it.
E. Other than that, how was the play Mrs. Lincoln? Yeah, sure, the announcement was botched, and the timing was horrible, but they also had terrible, terrible luck! The 4e designers acknowledged that the final push was rushed by directives from the top, which caused them to make the classes too "samey" and left further differentiation on the cutting-room floor. So many parts that could go wrong, did go wrong- key parts of the computer component that was supposed to be rolled out were entrusted to a developer, and that person was unstable and it ended in a horrific tragedy (and also meant no product). The projections for the product, which were too optimistic, combined with the lack of immediate success, resulted in Hasbro immediately slashing funding. But the time Essentials was rolled out in 2010, 4e was already dead internally and they were debating what to do.
F. Within the 4e community, there has been some debate about whether Essentials was a necessary correction that would have appealed to the mass market, or a betrayal of the essential ethos of 4e.
G. Going back to (B), the concept of subscription services and "Everything is Core" (repeated releases of core books each year) is an idea that was, at best, ahead of its time- we are all about subscriptions now, but it wasn't that common then. At that time, it came off as more of a cash grab, especially given the economy.
H. The design team was too insular and wasn't aware that the reception wouldn't be great, and therefore didn't do enough to "sell" the product. When they had 3PP come and playtest 4e, Jason Buhlman of Paizo saw what was going on and that provided Paizo the confidence to continue on with Pathfinder. In other words, outside playtesters realized it would be divisive to some of the core consumers.
I. Building on (H), we now know that there were issues flagged by members of the design team, but were ignored. In addition, there were some inexplicable errors (such as the monster hit points).
J. One more thing- while the internet wasn't "always on" enough for the ambitions of some aspects of 4e, this was the first edition launch that had many D&D players (I assume, I don't have stats for this) have easy access to some form of the internet, which enabled extreme and intense opinions to both form, spread, and become much more noticeable and toxic.
K. Finally, this history has to be measured in terms of what is a "flagship" product. It's not enough for a D&D product to be "good" or "better" than other editions or other products- it's not sufficient or even necessary that it has great design. It has to be broadly and widely popular so that it continues to dominate the TTRPG marketplace. That is the raison d'être for D&D. People can, and do, argue endlessly about what makes D&D better or worse or good or not, but in terms of a product, D&D must always be #1. Starbucks coffee might not taste the best, but they have to careful changing it ... if you know what I mean.
Now, why write this history? Certainly not to rubbish 4e. I think it's an interesting, but essentially unanswerable, question as to whether or not it would have succeeded if the stars had not been aligned against it. The product was already essentially dead internally two years after the launch, yet aspects of the system itself were incorporated into 5e, and it was never as unpopular as its detractors say - just not popular enough given expectations and the brand.
Please remember that people have very strong opinions about the transitions in D&D that occurred from oe to 1e, 1e to 2e, 2e to 3e, 3e to 4e, and 4e to 5e, and you are unlikely to change those opinions. You are, however, likely to anger the blood of other people on all sides- especially w/r/t 4e. No matter how good or clever your point is, it has definitely been said before and it will not change any opinions. But what you choose to write is up to you, your God(s) (or lack thereof), and the forum rules and the moderators that enforce those rules.