Brother MacLaren said:
Are the rules for all versions of D&D just that horrible that people only play until they realize the flaws (such as at high levels)? Are gamers some breed that get bored with *mechanics* to the point where not even new and cool stories, settings, and adventures can engage them?
Yes; I wouldn't say 'bored' but that I can only take bad mechanics for so long a time befire I have to play something else.
A sufficently good GM can make you ignore or overlook bad mechanics for quite some time, but those GMs are relatively rare. In the hands of an average GM, the system matters a great deal more because it's more obvious. (Also, I've noticed that the really good GMs also ignore or change parts of the system themselves, in a transparent way that you don't always realize is happening unless you're very familiar with the rules yourself).
For many years, I'll say that I only played D&D because it was everyone's second choice. Everyone, including me, had another game in mind as their first choice of a game system to use but none of us could agree on what that was so we all played D&D as a second or third choice. Campaigns wouldn't last all that long, though, and we'd eventually have to change to some other game when we couldn't take D&D's rules anymore.
The rules were badly dated, and were not getting any better while we had a number of fourth and fifth generation RPGs that were doing well and doing so with very nicely-designed rules systems. They had moved away from being 'alternatives' to D&D to being better than D&D.
Now, 3E solved a lot of those problems. In fact, it solved the majority of them. After 3E came out, our consumption of other RPG systems dropped to virtually nothing. I could, if I needed, play 3E another seven years, easily. There are still a lot of things I'd like to fix, and a lot of things I'd like to change, but I could live with 3E as it is now. I'm excited by 4E, though, because it promises to fix some of those problems (such as high level play).
There are some things I still won't like, I'm sure. One thing I don't like, and one thing I think hurts D&D more and more as time goes on, are the sheer number of rules. I think we're seeing the end of the 'huge books of rules' era of game design simply because a lot of people just are not going to put up with learning that huge pile of text. My druthers would have been a D&D that took up a single 200-page book, myself.
Brother MacLaren said:
Why are RPGs so flawed in this manner that they need perpetual revision in a way that other games do not?
Because other games have a definate sharp focus, along with a defined beginning, end, and goal. RPGs don't have those things, so they are very open-ended in what you can add. They also have a greater number of rules than any other kind of game. With more rules, you'll eventually come to the realization that there are better ways of doing things, and you implement those things in a new revision.
Brother MacLaren said:
And going forward, 4e and onward... will this always be the business model? New rulesets, continual tinkering, and so on? Is it destined to be a game that its players ALWAYS look at and say "It's not good enough yet"?
I would say that it always has been like this; almost all other RPGs are (Call of Cthulhu is the only game that comes to mind that has not had - or needed - a significant revision during it's lifespan) like this. Remember that the first 20 years of D&D's life is an abberation brought about by a company that didn't listen to it's customers. People tried for many, many years to tell TSR 'change this, change that', and they might as well have been blowing in the wind.
It's not a 'business model', per se, it's the normal way of doing things: eventually, you come up with better ways of doing what you were doing before, and implement those changes.