WotC conducted surveys where most of the participants were people who already played D&D and who self-selected themselves for participation. That's not terribly conducive to figuring out what most people think of when they think of fantasy.
True. But the purpose of the surveys was to figure out what most people who play D&D think of when they think of D&D.
My take on this whole thing (which I've been happy with), is that they recognized that D&D itself has a history, feel, and design that is uniquely D&D. Vancian magic, a Tolkienesque standard for the 'generic' game, the focus on the d20 as the core element whether it makes sense or not, AC and hp, etc.
One of the big things that they learned in the 4th edition is that there were a whole lot of people playing earlier editions of D&D, and of course there were also a bunch that switched to Pathfinder. They are in the business of selling a game. Who are your best customers? The ones that already play the game. Getting new customers is much harder.
So, how do you get those old customers to buy a new game? You make them a game they want. You survey them, and you listen. What actually makes D&D D&D?
D&D is based on the fantasy of Tolkien, Vance, Leiber, Howard and others. It was so unique that it spawned a great many modern video games, other RPGs, etc. The concept of fighter, thief, cleric, and wizard, with humans, dwarves, elves, and halflings, were near universal for a long time. Outside of the gaming community, others probably never knew how much of an influence D&D had as the primary source material. Sure, it's derivative itself, but ended up becoming something in and of itself. One thing I've heard a lot of people say is that the 4th edition was a revolution in game design, and a great game, it just wasn't D&D. If they had marketed it as a new game, it would have been praised, although probably wouldn't have sold as much.
There is a sizable group that doesn't want the 'exotics' like dragonborn and tieflings. I'm one of them, in part. I'm fine with them being more subtle and able to integrate in society, but I don't care for them as they've evolved in the game. At least not in the Forgotten Realms.
Outside of the Forgotten Realms (which is the current core campaign), my preference is for something
more exotic. I loved Dark Sun and Ravenloft because they had an entirely different feel and approach. One of my complaints about things like the dragonborn as presented as they are essentially humans in a dragon skin. They don't have a unique society, culture, or appearance. I'd love to see a campaign world where they dominate and have evolved in unique and non-human ways.
The starting point for the game is the Starter Set + the free Basic D&D which only has the core races and classes. It's a throwback AD&D and the original basic set. It's also a good starting point because you don't have a lot of options yet.
One advantage of a known world such as Middle Earth, or the Star Wars universe is that people already have a good sense of what to expect, how things work. The D&D of elves, dwarves, humans, and halfling fighter, thief, wizard and cleric is that starting point that people know, even those who haven't played the game.
Sure, they could present things differently, maybe word things a little better. But it seems to me that the purpose of this edition was to get back to the core of what D&D is, or at least what the majority of people view as D&D. So it is tailored to the mass market. I prefer that, because I was not happy about how the many, many additions in the 3rd, and especially the 4th edition drastically changed not just the core, but was rolled into the Forgotten Realms in particular. The total overhaul of the deities, the known planes, and insertion of every race and class option as 'core' and largely common dramatically changed the feel of the campaign.
This time they've gotten a little smarter and realized that aside from the game itself, their settings are a valuable asset. Specifically the differences and variations from one to another. Since the initial 'core' campaign is the Forgotten Realms, they've built the rest of the presentation around that concept, which is much closer to what was originally presented. Aside from the surveys and playtesting, I think the continued high sales position of Ed Greenwood Presents Elminster's Forgotten Realms also played a part in how they decided to frame the primary presentation. And that initial presentation is based on the idea that an earlier version of the Forgotten Realms is the closest thing to 'generic' D&D.
Remember that one of the main thrusts of WotC (and really Hasbro) is to build a multi-media brand. With the TTRPG, video games, potential movies, and tons of novels, the Forgotten Realms is the center of building that D&D brand identity. Other settings that are noticeably different like Ravenloft also have their place. But for the core rules, everything is built around what the mass market view of what D&D is, and presented as such.
Ilbranteloth