nedjer
Adventurer
Part of it is similar to the reason that very few independently produced records or books will end up gaining popularity; if you don't have a big record company or publisher distributing and advertising for you, it is hard to get the traction needed to make it big. But this is not the only or even main reason that Mechanical Dream is not a popular RPG; I would say that it has to do with particularity, specificity, and a kind of arcane quality that a lot of indie games have: They are created less for playability and game-table enjoyment and more as a kind of artistic rendering or snapshot of RPG potentiality. A game like Tribe 8, for instance, is very focused and flavorful thematically, but it has both limited appeal and scope, and potential for ongoing games.
I would say that one of the main reasons that D&D (and its largest child, Pathfinder) is so popular, year after year--aside from the big publishing house factor, which is significant--is that its play style, from OD&D to Essentials, is particularly conducive to a kind of ongoing, neverending, adventure game feel that you just don't get with many games. Whether we're talking about the sandbox or a tightly crafted epic campaign; there is a sense that the D&D Universe, in all its variations, from the published settings to the thousands of homebrews, is eternal, it exists and goes on. So even if your epic campaign ends with an apocalyptic bang, a new world can arise. To quote Merlin in Excalibur, "There are other worlds, this one is done with me."
There are other D&D worlds, countless of them, yet they are all part of one vast, populated, eternal mythos. Yet there is only one Mechanical Dream or Legends of Alyria, and it is self-contained, a creation of one or two minds; even though D&D was originally the creation of only a few, it has become the ongoing creation of millions. When you play a game like Mechanical Dream you are exploring a foreign land, a place you go to for a time but eventually come home. When you play D&D you are exploring your own world, your own land, and discovering new things about it. Exploring new regions, yes, but also exploring with fresh eyes, especially when you take into account that many of today's D&D players have played during different phases of their lives, from the "Golden Age" of childhood and middle school, to the "Silver Age" of high school, to the "Bronze Age" of college, to the Iron or Dark Age of early to mid-20s to the early 30s when many gamers leave aside such "childish things", to a potential revival and new Golden Age in one's mid-30s and on.
But I've rambled.
Much the same was said of the Indie music scenes in the 80s and 90s, until they became the new standard. You seem to be suggesting no one has ever or will ever want to move on from (the TRPG equivalents of) Phil Collins and Barry Manilow.

