What do we actually know about WotC's market research?

[Lanefan]

The one thing about the 1999-2000 market research that glaringly stood out for me is this: they ignored any and all submissions from everyone born before (if my math and memory are working: 2000 - 35 = 1965, so) 1966. (might be off by a year or two - if anyone knows better, please advise...)

I rather suspect this includes a large percentage of EnWorlders, along with all those who, like me, were college-age in the early '80's when 1e really hit its stride; never mind the 0e and earlier types. In other words, the people who in general built the game and made it what it is...or was, then.

I also suspect that had this grognard-level info been accepted, their results would have been significantly different for length of campaign, number of players, and number of PC's; just going by what I've read on EnWorld tells me that. But, they'd pre-determined their market was going to be the younger (i.e. under 35) player, and geared their research to that...and 3e, for better or worse, is what we got.

Lane-"born before 1965"-fan

 

[Storm Raven]

I also suspect that had this grognard-level info been accepted, their results would have been significantly different for length of campaign, number of players, and number of PC's; just going by what I've read on EnWorld tells me that. But, they'd pre-determined their market was going to be the younger (i.e. under 35) player, and geared their research to that...and 3e, for better or worse, is what we got.

I suspect the results would not have differed much from what they got. While it is possible that your speculation concerning the gaming habits of pre-1966 birth date gamers is correct, I suspect that that category of gamers makes up a very small percentage of the gaming market, and thus, the impact of including them in the statistical sample would have been slight.

 

[Garnfellow]

In some ways I think grognards are the single worst group to consult in doing this kind of research because, despite their vast experience, they tend to be (1) tremendous outliers and (2) extremely resistant to any kind of change.

Many of the grognards (and in the interest of full disclosure, I'm pretty dang close to one myself) have completely atypical games that are not very representative of an average D&D experience. My last campaign was in a low-magic homebrew setting with many houserules; it ran for 11+ years with a core group of 10 guys. We had a blast, and I'm very proud of our game, but I'd be the first to admit it would not be very insightful for someone looking to define a typical D&D experience.

Likewise, many grognards (certainly not all, though!) are extremely negative or resistant toward anything new or different, like the Comic Book Guy from the Simpsons. "Worst. Module. Ever." These folks might flood the internet with their strong opinions, but they don't speak for most gamers.

Finally, since many grognards tend to either be completists ("This supplement completely sucks, so I only bought 3 shrink-wrapped copies to add to my collection") or total luddites ("I haven't bought anything new from T$R since 1985, when Gygax hosed me with that munchin piece of crap, Unearthed Arcana"), their dollars are very nearly sure things -- these grognards are either going to automatically buy or not buy new materials. Since there's probably no changing their minds, why waste your marketing dollars on this group? You want to focus on trying to win over all those undecided folks in the middle, who represent potential (but not guaranteed) sales.

 

[Raven Crowking]

Still, what made you so suspicious of the research? I mean, it's not exactly rocket science, even though it's not simple either.

It isn't a matter of suspicion. Rather, on several threads the notion of WotC's market research indicating X or Y has come up, in some cases in direct contrast to my experience. Therefore, I thought I ought to educate myself about WotC's market research......If I am suspicious of anything, I am suspicious about the statements from some that it indicates X or Y.

The polling of 1,000 respondents is a larger (though, one would imagine, less in depth) sample size than my own DMing/playing experiences provide, but I had really expected the survey group to be larger.

I am more familiar with sampling in terms of biological sciences than market research, and I have to admit that I have some doubts about the accuracy of that as well. I understand, however, that it is the best tool one has, so long as one remembers that the results may not be totally accurate.


RC

 

[Lanefan]

I suspect the results would not have differed much from what they got. While it is possible that your speculation concerning the gaming habits of pre-1966 birth date gamers is correct, I suspect that that category of gamers makes up a very small percentage of the gaming market, and thus, the impact of including them in the statistical sample would have been slight.

I'm not so sure about that, at least based on EnWorld. I ran a poll a few months ago asking when people started playing D+D, expecting there to be a broad range with spikes around 1980 (for 1e), 1989-ish (2e) and 2000 (3e); what came back was a huge trend toward about 1977-82, a tiny spike around 2000, and an odd unexplainable spike in 1992. Most people who started in 1982 or earlier would probably have been born before 1966...all that data thrown out for no good reason. Makes no sense to me...

Lanefan

 

[Glyfair]

I've heard from time to time both that the baseline setup in 3e was based upon the statistics gleaned (evidence for which does not appear in the published reports) and that it was determined as a consequence of how 3e playtested. Anyone know which of these conflicting reports is accurate?

What conflict? The baseline of the D&D game had elements determined from the survey (things like average length of campaign and average number of players) and elements determined from playtesting.

Unfortunately, a lot of the information that was posted was lost in the various upgrades to Eric's messageboards. A lot of the information that was "leaked" was posted there, although a few bits might be found by going through his news archive.

 

[Warbringer]

A wise man once said that there are three kinds of lies: lies, damn lies, and statistics.

Actually, he was a politican, so forget the wise part

 

[Glyfair]

A wise man once said that there are three kinds of lies: lies, damn lies, and statistics.

Remember, though, that this quote is about using statistics to make your point. For example, a politician might say that in his area the average income is $1,000,000 per year in his district. However, 99 in 100 people make $20,000 a year and 1 in 100 make $100,000,000 a year so the median income is actually $20,000 a year.

In WotC's case the people taking the statistics were trying to be as accurate as possible, because WotC wants accurate statistics. Most of the information we had were mostly explanations from Ryan Dancey and a few others about certain choices made about the new system.

 

[Numion]

Rather, on several threads the notion of WotC's market research indicating X or Y has come up, in some cases in direct contrast to my experience.

Well, statistics don't apply to individual.

 

[Aus_Snow]

I rather suspect this includes a large percentage of EnWorlders, along with all those who, like me, were college-age in the early '80's when 1e really hit its stride; never mind the 0e and earlier types.

I seem to recall that a lot of ENworlders were born in *one particular year* in the 70s, actually. Weird, but (as I recall it) true. I can't remember the distribution past that, however. Could have gone either way.

Where is that thread/poll again. . .