Holy crap this is long
It's a lengthy question that I can't do full justice to in this post.
But for me, when I'm talking about D&D (as opposed to some other fantasy roleplaying game) I'm talking about a commonality of experience which can be primarily analyzed from two perspectives:
(1) As a player, do the core classes play in a fundamentally similar fashion?
(2) As a DM, if I set up an identical situation will it result in fundamentally similar gameplay?
/snip examples though they match my experiences
I thought about something after my last post before I went to sleep and didn't want to say anything until I had thought it through. I will try and make my point but I think I will need to take the long way around, so this might be kinda longish. And interestingly enough it made me question a long-held belief of mine that I didn't quite expect. For the poster somewhere that is thinking, "why do we keep having threads that could rekindle the EWs," this is for you.
I have played all editions of D&D except the LBBs--commonly referred to as 0e--and the two basic editions before BECMI. I started with BECMI, joined 1e at its tail end, and grew up with 2e--especially liking the customization that kits offered. I grew tired of D&D in the late 90's and started drifting off to other systems.
[tangent]
That's when we started with Earthdawn. It was an abysmal failure and didn't last. Why, because we tried to play it like D&D. Talk about trying to hammer a nail with a screwdriver. Thinking back on this experience leads me to believe that there is some essence that is D&D.
[/tangent]
When I had heard about 3e on the horizon I was elated, much like the rest of the community. Granted there were some that I have heard about since--and maybe a little back then, since I was here on EN World before 2002 and the great rebuild--that did not welcome 3e, but I was looking forward to it. When it came out I jumped in and didn't look back until 2006 or so. I loved all of the new character options: skills, feats, and removal of demi-human class restrictions; all of its new combat options. And most importantly the one feature that exemplified its elegance, 1d20 to rule them all.
But...there had to be a but, something happened which I can only describe by telling another quick story. A group I played with in the early to mid 2000's had originally played with a kitbash of 1e, 2e, and BD&D. I pestered them to play 3e, look at all of the new character options I said. So when their DM wanted to take a break and be a player himself, I took the reigns. And we played 3e. But very early on we realized all of those fiddly bits didn't add anything to their experience. As an example, AoO were too complicated for our wine-addled minds. The rules got in the way of our collective story. By the time my wife--of course she played too--and I moved, we were essentially playing 1e/2e with 3e characters. And that was the most fun I had with D&D. Let me note here that in 3e I made her characters by asking her to describe her character and me picking the appropriate feats and skills. She didn't want to be bothered with all the different accounting and such.
[important tangent]
While I had been playing D&D for years, my wife didn't start playing D&D until the late 90's when I bought her the red BECMI box set while she was at Penn State. She wanted to learn why I liked D&D so much; why she had to share me with this hobby. I felt that she needed to learn as I did; that the red box set would give her the best chance to see why I love D&D. She enjoyed it. She eventually graduated to 2e when she moved up to Alaska while I was in the Army. I have all of her characters--I think--from that time and you know what, she didn't play anything but fighters, a paladin, and a ranger. Know why? I'll paraphrase her response from so many years ago, "I just want to roll my d20 and kill things. I don't care about all of that other stuff." My wife just wants to play Conan, how cool is that.
[/important tangent]
It was eventually my wife that metaphorically pulled me aside and said that everyone would have more fun if I didn't make the game so complicated. Talk about getting hit upside the head with a sledgehammer. But she was right. While I loved setting up 20-level character progressions and designing challenging NPCs and encounters, everyone else just wanted to play. It was on me as DM to prevent "the Christmas Tree effect"; challenge them appropriately when we deviated from expected rules.
By the time we moved I was tired of 3e; we never did embrace 3.5e and 4e was not yet on the horizon. I was looking for newer systems that were more simple. I found WHFRPG1/2e--arguably not more simple but I loved the gritty realism, lethality, and character growth when starting as a Ratcatcher or a Barber/surgeon. I found Savage Worlds. I found Dragon Age. To me SW and DA scratched my D&D itch, but to my wife none of them were D&D. Why? Do I get to roll my d20 and kill things?
But dear, with SW you get to roll the die that corresponds to your ability and all you have to do is roll a 4 or higher. How simple is that? Do I get to roll my d20 and kill things? No. Where are my hitpoints? Well it doesn't use hit points, there's a damage track. Then it ain't D&D.
But dear, with DA you get to roll 3d6. We can play anywhere that we can buy a set of dice if we forget ours. I don't want to learn new rules. If i can't just roll a d20 and kill things, then it ain't D&D.
So, would she consider 4e as D&D. If like 3e I took out all of the fiddly bits and chose her powers for her. Yes. If all she has to do is roll a d20 and kill things, it's D&D.
Now this is a simplistic anecdote but one that illustrates my point. All she wanted to do is roll a d20 and kills things--though admittedly loved taking on more specialized roles like the paladin and ranger. She can tell you stories like how her paladin took the side of a PC cleric against my Bard/Shadowdancer and others when we explored an island like "the Isle of Dread" and found a meteorite venerated by the native population and some of us wanted to take it to make better weapons and armor. Or like how her first character--a fighter named Ruamil--lost her new friend, a cleric named Aleena, to a powerful magic-user's "glowing bolt of arcane energy" and how she hired a couple of other fighters and a magic-user to hunt "that f--ker Bargle" down and serve justice. Was she playing D&D? Yep. Could she do this again in 4e. Yep. With no more changes than I needed for her to do it in any other edition of D&D.
Does she get the essence of D&D? I think so.
D&D is no more than a toolbox to facilitate the growth of a communal story; a way for like-minded people to pass the time in a manner that matches the group's style. All editions arrive at that goal in different ways.
Afterword
This is why the sandbox I am working on now is using as its base the earliest editions of D&D. Because that's what makes my wife and I happiest. Essentially leave most of the challenges to descriptions and adjudication, and leave the whacking of monsters to the d20. YMMV
[last tangent]
If we could really define love, would we really have needed thousands of years of poetry talking about it?
[/last tangent]
Holy crap that was long.