My perspective is that the creators of D&D sort of stumbled into ideas that weren't originally well thought out or well formed initially.
I think you are absolutely right.
The subsequent editions have done a good job of finding the core of those ideas.
An obvious example: they didn't work out the difference between a race and a class until AD&D. This is less a rules INVENTION in AD&D than taking something that was half-formed in Basic and better realizing it.
Again, I would agree.
I would be opposed to completely changing the central mechanics of Dungeons & Dragons, but I'm all for evolving them closer to their Platonic ideal, and I feel like we have a long way to go in that regard. To the extent that I am uncomfortable with some of the changes in 4e, it's that I feel like a lot of the changes that were made *weren't* refinements, but in fact were changing the style of play into something that was less true to the game's roots...
This resonates with me. I would never argue that the mechanics of D&D were currently or ever had been the best any game could be but they inform what the definition of the D&D game is, to me. If someone convinces me that this set of rules over here provides a more satisfying (however you want to define that) FRPG than anything D&D is doing or has done, that's great and I'm interested. But it won't be necessarily be D&D. That's not a bad thing. It's just a thing. And I may want to play D&D, even if it's the equivalent of driving a car with no power steering or adaptive breaking or any of the other advances in technology that come with other more recent advances in automotive technology.
When I go to see Shakespeare, I don't want someone rewriting the plot or dialogue because we've learned a lot (thanks to Bill) about what we can do with staged drama. I want the unadulterated original, even though they might be difficult to follow and even though there are more recent dramas that have built on the old dog's work and carved out new territory and new opportunities that I also want to experience from time to time.
What I'm getting at is why can't we have both? Why not preserve the original game for what it was? We can and do have other games we respect and enjoy that build upon and change the assumptions and mechanisms of D&D but what's wrong with preserving a game for posterity? If I can at least partly address my own question, I think some of the difficulty here is that D&D has never been, in any incarnation, something that one might consider entirely complete. Indeed the further you go back in time, the less complete the rules were and the more they extolled the notion that you should build upon and change what you were given. And that's what we've done, both as players and as game publishers.
The logical consequence of pursuing opportunities to refine and improve - however subjectively - what some of us consider to be important about the game is that we inevitably arrive at questions about changing something many players might consider fundamental to the identity of the game they love.
Why not then, for any future edition of D&D, start with a freely available (but still copyright) electronic edition of the original (or Basic c.1978), albeit re-edited for clarity and accessibility, and amended with a chapter on the game's historical evolution into different editions? You could give everyone a context for why the latest paid-for evolution and future experimentation and editions are inevitable. Current and future players might then be more understanding of and amenable to changes so profound that they transform whatever someone's currently accepted idea of the game is.
And we can all refer back to something we identify as a core game of D&D.
In the interests of full disclosure, apart from the obvious subjectively ideal wishful thinking I'm indulging in here, I am a known proponent of so-called 3E, for all its faults. The reason I now make the argument above is that I never want someone's default understanding of D&D to be so far away from what I used to call D&D thirty-odd years ago, that we really don't have much in common. If we both know and understand that D&D character generation rules were for a long time predicated upon a set of PC characteristics derived from six-sided dice rolls, for example, then we have a lot more in common when we explore a brave new game of something that doesn't use those rules. We have some common ground we can call D&D, regardless of any personal preferences about the popular mechanics of the day.
Also, I have been drinking.