A few stray thoughts...
I think there are many reasons that D&D has maintained its popularity and top dog status for almost five decades - some of which are intrinsic to the game itself, some not. It is impossible to divide its "popularity share" between the two, intrinsic and extrinsic, but my main point is that we have to consider BOTH, that to focus on only intrinsic elements--what the game itself does--ignores the broader context of RPG history, just as focusing only on the extrinsic elements--the fact of name recognition and such--ignores what the game actually offers as a game.
I mean, there are so many factors we can consider. In terms of extrinsic factors, for instance, we can use monetary wealth as as an analogy: it takes money to make money, and people with lots of money usually don't end up with no money. On the other hand, if you have no or little money, it is hard to get more money. So wealth begets itself, and rich people stay rich (and often get richer) and poor people stay poor (and often get poorer).
So we cannot ignore D&D's central place in RPG history, and the fact that it was first to the top of the hill. If you play King of the Hill, it is far easier to stay on top then get to the top.
On the other hand, we should not cynically reduce D&D's sustained success to just being the first, or equate 5E with a rich and pampered heir who lives off daddy's wealth. For one, 5E is a much larger success than any edition that came before, or at least since 1E, so we have to consider what it brought to the table to build that hill up higher. But even then we cannot ignore extrinsic factors; for instance, Stranger Things is a huge part of why 5E is so popular now (among others, but I'm thinking it is probably the biggest single factor).
I'll share my own personal experience, because in the end I can only speak authoritatively about my own experience. Like lots of old-timers on this board, I started playing D&D in the early 80s. At the time, it was (almost) the only show in town. There were other games, but as a kid still shy of middle school, I didn't know about Runequest or Tunnels & Trolls. I vaguely remember seeing them in a hobby shop, but people mostly played D&D.
By the late 80s I was reading Dragon regularly so started hearing about other games. "No Elves!" declared Talislanta and I thought, "But I like elves!" But then I flipped through the book and bought and loved it. Did I play it? Only once or twice, because, well, most people I knew were playing D&D. At one point a friend brought in a copy of Ars Magica to our D&D group (around 1990ish, I think), and when he explained the magic system, I thought, "Wow, that's how magic should be!" Did we play it? No, because, well, we knew D&D.
Throughout the 90s and into the 00s I explored the world of RPGs beyond D&D, even dabbling in this or that game as the opportunity arose. Over the decades I bought dozens, even hundreds of different games, most of which remain(ed) curiosities on my game shelf. I always came back to D&D for several reasons, but most centrally two:
One, everyone played it. Sure, I'd encounter groups playing other games, and during the mid-90s when I lived in Burlington VT, there was a massive White Wolf LARP going on run by an acquaintance who was a barista at my favorite coffee shop - people would come in and touch base with him, the Storyteller, as he made lattes. But for the most part, most people I knew played D&D.
Two, it felt like home. I grew up with it--so yes, imprinted with it--and it felt familiar and comfortable.
Those are the largest two, but not that far behind is a third: And that is that the game itself works for an enjoyable experience. If it were just the first two, I probably would have moved on. But I like using all the polyhedral dice. I mean, Platonic Solids! I like silly but familiar rules like "saving throws" and "hit points" and weird monsters like the catoblepas and displacer beast. Meaning, crunch and fluff that is intrinsic to D&D.
I do think that there are many fantasy games that are "better" designed, or at least that I find more aesthetically pleasing. I think Ars Magica's magic system makes D&D's Vancian magic look anachronistically primitive. I love the simple d20 system of Talislanta, as well as the richly psychedelic world. It is hard to beat the flexibility and ease-of-use of Savage Worlds. Etc, etc. But for a wide range of factors, the melange that is D&D is just pleasing to me, so I come back to it, again and again.
Oh, one truly final note: I think the editions are a feature and not a flaw of what makes D&D what it is. If it is was just one version over 50 decades, it would grow stale. But a new edition allows us to experience the game afresh. There's a sweet-spot, of course, and the inherent problem of leaving people behind for whom the latest version isn't their cup o tea. But I also like seeing new versions of the same game - and it is the same game, for ultimately the most essential elements of D&D remain intact, edition after edition. It is the secondary elements that we argue over.
That said, I think there are other ways they could handle editions, but that's a different conversation. But in short, I still like the idea of the "complexity dial" and "modular options" that was talked about circa 2013, but has been left behind (for the most part), presumably because it is easier to talk about in the abstract than actually design and publish. But still, I can imagine a theoretically D&D with a simple core rule set and many different modular options that can be used to customize the game as desired by each group. One can dream...