Paizo is having its best year ever.
I think part of the problem is that folks who view, say, D&D or Pathfinder or a licensed game as "too derivative" are looking for a kind of tabletop gaming industry playing field in which oddball ideas without much commercial interest in the first place perform as well as brands that have multiple decades of fan interest, or that are based on genres that are very popular.
I contend that that sort of atmosphere has _never_ existed in the marketplace. It's not that the sort of market folks like Malcolm and Gareth hope for has disappeared, it's that it never really existed the way they envision it in the first place.
PS: That's not to say that there aren't some real threats to the industry, but there is still a ton of money to be made my people at all levels of the game industry.
You just have to make something that people actually want.
This is a good point. Over the last few decades hundreds of different RPGs have arisen, yet most gamers stick with D&D and a few other games; many of those that branch out and try something new end up coming back to D&D. There is only a relatively small segment of the gamer population that seems "mobile" in terms of its gaming choices. It also seems that a large percentage of indie and "oddball" games end up with very few players, no matter how critically received. They might be played by a small group of people for a short period of time, but those people tend to be fickle anyways and move onto the next new clever thing. A lot of the most cutting edge/avante garde games in terms of design end up being museum pieces in collections; sure, they're clever and well-designed, but why aren't people sticking with these games, and why do people always come back to (or stick with) D&D? Does anyone play
Legends of Alyria or
Sorcerer & Sword or
Mechanical Dream behind their creators and a few friends?
This isn't to harp on such games, but to point out that for various reasons they just don't stick. Part of it is similar to the reason that very few independently produced records or books will end up gaining popularity; if you don't have a big record company or publisher distributing and advertising for you, it is hard to get the traction needed to make it big. But this is not the only or even main reason that
Mechanical Dream is not a popular RPG; I would say that it has to do with particularity, specificity, and a kind of arcane quality that a lot of indie games have: They are created less for playability and game-table enjoyment and more as a kind of artistic rendering or snapshot of RPG potentiality. A game like
Tribe 8, for instance, is very focused and flavorful thematically, but it has both limited appeal and scope, and potential for ongoing games.
I would say that one of the main reasons that D&D (and its largest child, Pathfinder) is so popular, year after year--aside from the big publishing house factor, which is significant--is that its play style, from OD&D to Essentials, is particularly conducive to a kind of ongoing, neverending, adventure game feel that you just don't get with many games. Whether we're talking about the sandbox or a tightly crafted epic campaign; there is a sense that the D&D Universe, in all its variations, from the published settings to the thousands of homebrews, is eternal, it exists and goes on. So even if your epic campaign ends with an apocalyptic bang, a new world can arise. To quote Merlin in
Excalibur, "There are other worlds, this one is done with me."
There are other D&D worlds, countless of them, yet they are all part of one vast, populated, eternal mythos. Yet there is only one
Mechanical Dream or
Legends of Alyria, and it is self-contained, a creation of one or two minds; even though D&D was originally the creation of only a few, it has become the ongoing creation of millions. When you play a game like
Mechanical Dream you are exploring a foreign land, a place you go to for a time but eventually come home. When you play D&D you are exploring your own world, your own land, and discovering new things about it. Exploring new regions, yes, but also exploring with fresh eyes, especially when you take into account that many of today's D&D players have played during different phases of their lives, from the "Golden Age" of childhood and middle school, to the "Silver Age" of high school, to the "Bronze Age" of college, to the Iron or Dark Age of early to mid-20s to the early 30s when many gamers leave aside such "childish things", to a potential revival and new Golden Age in one's mid-30s and on.
But I've rambled.