...and it seems bizarre to me that there is little effort spent on the elephant in the room; trying to come to grips with what lessons might be learned about why a particular game is so popular- is it truly just path dependency? Or are there additional reasons. That seems like a salient discussion. And it might also shed light on why D&D could never embrace some of the innovations we see in indie games.
I think listing what 5e does well or successfully is definitely useful, but I feel like this might still wind up being a trap, because one of the quickest ways to start a war here is to try to define D&D's strengths and incentivized playstyles in anything but the broadest and most universal terms.
But I'll stop building this army of straw men and get on with it. And I'm going to avoid dropping caveats or throwing secret elbows, even though a lot of what I think 5e's design does well is also more specific and D&D-ish than some care to admit.
-Levels are fun: Video games fully appropriated leveling up, and then somehow the horrible influencers and self-help types followed suit, but leveling is a clear way to do progression.
-Everyone can fight: Though some fight better than others, and there are always those people obsessed with optimization, every class can fight. This is a crucial design element for D&D because of the game's emphasis on combat.
-Feats are cool: I wish you never had to decide between a stat increase and a feat, but feats are great.
-It's hard to die: I get why this is a huge downside for some people, but I'm not a fan of random and frequent PC death, so to me "easy mode" is fine, so long as the DM makes the game challenging in other ways and about more than combat. Fights can be easy (if only in the sense that they're survivable) while larger objectives are not.
And I think that's it, for me. But my criticism of D&D generally and 5e specifically has never really been about a huge number of design elements or mechanics that I think it does "wrong," but rather everything it doesn't try to do. I don't think it mechanically feels like any specific genre or narrative or activity other than "fighting fantasy enemies on a map" (which can obviously be a lot of fun), but I also don't think it provides the tools to do much more or different than that.
So as a tactical combat game that's simple enough to pick up quickly but with enough depth and progression mechanics to support combat-first play for a longer campaign, it cooks. And the system reflects the brand that it's maintained, and the tropes it's injected into pop culture, especially the ones that don't appear in any thing else (like forgetting spells). But as has been discussed to death in other threads, there are so many other systems that do other genres and
types of narratives better. And the fact that a lot of people stick with D&D to do those other kinds of narratives is not, to me, a sign of 5e's design choices or excellence. I think that's about everything else mentioned by others in this thread (opportunity cost, name recognition, etc.). I think, to hopelessly scramble something you mentioned above, they're trying to hire Dane Cook to star in everything, including a period-accurate Mandarin-language wuxia epic...and not in a winking, ironic way.