RPGs & D&D have new reality to conquer
When I started D&D, I never saw the need for a 'basic set' I read and reread every book that came out (and there were few enough books you could do that). Computer games barely existed. Wizardry by Sirtech was amazing but couldn't compete with RPGs -- not with Runequest, Powers & Perils, Travellor, AD&D, Paladium, Call of Cthulu. Perhaps this is only my feeling but, game systems seemed to offer a complete system and then leave you to go from there -- all my adventures were mostly home brew.
Now each book is really expensive! Computer games seem to be setting the flavour and level of realism necessary for role playing games. I have players who want to make sure that all their characters get the same treatment they can expect in Neverwinter Nights. The rules are so complicated it seems you need a lawyer to balance them and making dungeons Wow what a hassle!
Everything is a character -- I refuse to calculate feats, prestige classes, & all the administrivia for every dimly observed figure in a storyline.
Characters are all treated soo much better than the common person of the world. I think player characters should be heroes but I think the current system is so rich as to deny an evolutionary link between the common person and the players. Conversely, you are into situation where the every figure in your world needs a computer program to fully generate quickly.
Adventures become so complicated to make that they are not as much fun to write from scratch. I think this is the source of RPG atrophy. If I buy a book or PDF it is an effort to learn it and DM it -- I am less likely to convince people to join my game if it represents work I have to do. If the game has fascinated me so much that I wrote the dungeon myself I am more eager to get players because I am showing off work I already have done and I 'love' my creation. I really like some of the writing for this game (especially Monte Cook's stuff btw) but internalizing it to share it with the players is far more work than it used to be.
As soon as I am buying all my adventures to serve to my players I don't see that I get enough out of it as the Dungeon Master. My creativity deserves an outlet too. If the game is too complicated to play well (without worrying about whining players) to hell with it. I figure the DM's Job is about twice as hard as all the difficulties of all the characters combined.
The other issue that faces all RPGs is that computer games and console games consume a whole lot of the available free time for prospective players. This is simply a new reality after the rise of AD&D in the first place. RPGs seemed like the only game in town when I started them and now players get frustrated with little things way too quickly or seek to make it more like Morrowind or Neverwinter Nights. I think its worse for RPG's since the online game allowed players to get together on the web without leaving their homes. This is certainly not a death blow but it is a competing activity for prospective players both older (with difficult schedules) and younger with twitchier thumbs. There will always be players who don't want to play computer games -- this is not a terminal condition (no pun intended) but I think it cuts of the air supply for paper RPG's to make the profits they might have in the past.
Thats my .02.
I think this is especially difficult, not withstanding that some of the material for all RPGs and the breadth of that material has never been better. I think WOTC was brilliant with their open source game license that has allowed brighter creativity to flourish in other publishing houses and dominated the RPG industry -- Chapters near me has no non D20 games. Some of the perceived stagnation may be the success of that license. I miss the variety of games from my youth. The success and dominance of WOTC has very much warped the RPG industry. (It is ironic that WOTC might be too big to survive on the success they have had).
I think that WOTC have to restructure the advanced rules and publish them with a computer program to help with world creation. Something better and cheaper than Etools.
I think the greatest hope for table top RPG's is coming from the various simplified approaches. There is a maturing process where players and DM's are willing to give up the bells and whistles for easier game play but I think that is happenning now.
I remember being in love with ICE's Rolemaster game structure but slowly giving it up because it was too much work -- slowly I think that will happen to D20 as soon as people tire of the great content to realize the game itself is not workable.
It seems entirely possible that there is a correction coming in the game industry that favours small publishers with lower costs, cottage industries and vanity publications. Table top games might not be a great place to make money -- this does not mean they will ever die however.
I hope other games are released Open Source with even less restrictive licenses. I would rather write adventures and give them away in an environment that encouraged - even inforced this - than watch my little creations get sucked up into an industry where everyone made money from my ideas but me.
Sigurd