Warning: Fat, multi-quoted post follows:
My opinion is simply that a new edition over decade or so isn't a bad idea. It's changing the system every edition thats the problem. There really is no need for it. As Chainsaw Mage has said, there is really little real difference between 1st and 2nd edition. 3rd edition was a vast improvement and probably nessecary from WotC's point of view to differentiate between their D&D and TSR's. Was there a need to wipe the slate clean and create yet another rules system for 4th? No.
What the GAME needs and what WotC needs are two distinctly different things which don't always taste great together.
IMO the game does need to change over time. I agree that it does NOT need to be revolutionized and/or reinvented from the ground up in ANY edition. Yet many of the changes that were made for 3E were overdue from 1E, much less 2E. 2E changed bupkus as far as the meat of the rules is really concerned. 3E, however, changed too much and WotC - IMO - took it down a road that it was NOT meant to go down. They are now seeing more and more that making "Rules Mastery" a built-in cornerstone of their vision of the D&D rules has downsides.
One thing to keep in mind is that the complexity of Settles of Catan is much lower than any edition of Dungeons and Dragons. For example, it does not require a referee in order to play. Nor is the rulebook (for settlers of Catan)on the order of hundreds of pages (as far as I know) whereas all editions of D&D have tended to require a lot of reading and rules mastery.
One of the advantages of the older rulesets is that while "rules mastery" was a viable approach to play it was NOT intended to BE the play. Instead of roleplaying with "Rules Mastery" as a fun sideline it was changed to "Rules Mastery" with roleplaying as a fun sideline. That's overstating it, of course, but while it may not be true - it's
accurate.
D&D, and RPG's in general, need to figure out how to market themselves to be much, much easier to play so that they CAN break out of their niche market.
The appeal of D&D is still a niche appeal regardless of whether the rules are simple or complex. D&D rules began as very simple rules but IMMEDIATELY were set upon by the people using them and made more complex. Even today ONE FLAVOR of D&D rules is not the flavor that everyone wants. Some want a simple set of rules to play freely by the seat of their imaginative pants; some want the libarary of intricate and infinite possibilities and focus their enjoyment around crunching the numbers as WotC said they should. The spectrum from simple to complex will only become more evenly spread with the population of the player base as time goes on. No one edition will EVER again capture them all.
Yes, early D&D was a lot more simple and was hugely popular. But, it was also the corner on the market. It was THE game and was also very complex in its own right. But, as the market matured more complex games became successful because they offered what old D&D offered and even more.
And yet, insofar as I know, D&D has never, ever been dethroned as the most popular, most played RPG. Other games did offer the same thing that D&D offered - but they were NOT D&D and when it came down to it fewer people wanted to leave D&D to play something else than continued to play D&D while wanting it to be/making it be something more/different than it was.
There are new D&D players born everyday and they want a modern version of the rules, not some old, crappy system that wasn't that great in the first place. D&D has done an excellent job with changing with the times, and it is sad that some people can't see that. My kids will probably look at 4th edition and laugh at how silly it was, just like I laugh every time I pull out an old character sheet and see ThacO written on it that I used in middle school.
The effectiveness and value of the degree of change that D&D has made with the times is endlessly debatable. D&D not great in the first place? Yeah, I'd agree that that's true - maybe even that in many ways it was truly crappy. But OLDER editions continue to draw ever more players over time. Again, no single edition is the be-all/end-all for the entire spectrum of players. Even if you believe that 4E is two steps forward it is also one step back - there are elements/aspects of all older versions of the rules that were foolishly eliminated or altered - again IMO.
Original Dungeons & Dragons, 1974. Arguably, without that small RPG's success, we would not have any of these big 5,000 pages RPGs to play with. And yes, I do think this is a very relevant example, mostly because it is a game that is still incredibly fun to play, extremely light on the page count (requires an understanding of Chainmail, though), with a fantastic potential for emergent complexity and customizability. This, to me, is the winning game design that should be emulated.
Well, I'd say "admired" is a better choice of word than "emulated". Again, not everyone wants the simpler set of rules to play by - or they WOULD be playing by them. It is with 1E and 2E that D&D grew outside of the very exclusive wargamer community to a wider audience. D&D would either not be where it is now, or would not have gotten here as quickly as it did if those simple rules had not actually been expanded upon and complicated as they were.
I can't say why, and won't even try, but the fact of the matter is that a great majority of the population have no interest in sitting around a table playing pretend to be an elf. Yes, obviously they ARE willing to sit at a computer and pretend to be an elf for hours on end. But there are massive differences. Not the least of which is MMOs don't really require even a hint of roleplaying. In my experience the "RP" realms for WOW tend to be less popular, are frequently mocked by players on other realms in the same cliche manner table top gamers are mocked in meat space, and don't tend to resemble D&D anyway. MMOs tend to be more about personal empowerment by avatar/proxy and virtually nothing about being "in character".
In point of fact, calling MMO's "roleplaying" games is very much a misnomer. They simply cannot incorporate roleplaying in the way that D&D actually does because the environment that the character/avatar exists in can ONLY react in pre-programmed ways to the pre-approved and limited possibilities for action perpetrated by the player. Simply being able to talk at other PLAYERS by Skype or by text is the palest shadow of actual roleplaying in a game of D&D where interaction with every aspect of the environment and EVERY pc and npc alike is possible.
As regards Settlers of Catan or any board game there simply is little, if anything for comparison to D&D and other RPG's. That there are some who DO play SOC on a weekly or other regular basis doesn't alter the fact that SOC, being a board game, is intended to be a single, isolated, one-off event that is weeks, or really months apart from the last, always has a definitive start, a similarly definitive end, and above all - a
winner. D&D, while it can be played in one-off sessions, is designed and intended to be played regularly, recurrently, with no definitive end and above all - no way to win, only a goal of ongoing enjoyment of play.
Settlers of Catan is NOT D&D, much the same way that Pictionary is NOT Monopoly.