D&D General RPG Theory and D&D...and that WotC Survey

[AbdulAlhazred]

If they did, I missed them; and I was paying at least some attention during that time.

I live a couple miles from WotC headquarters. I can just say "yes, they do market research." Think about it, D&D is now considered to be one of around 5 or so major product groups that put the meat on the table at HASBRO, not just WotC. Of course they do market research. This one survey, done at a much more amateur point in the business development of WotC, has LONG LONG SINCE been vastly surpassed. It may be that they haven't gone out and surveyed the whole market at large as they did in '99, but they are constantly researching what does and doesn't work, what people want, etc. Their understanding of the customer base is probably 100x more sophisticated than it was 23 years ago.

 

[hawkeyefan]

I’m no marketing whiz, but I wouldn’t take the fact that we the public have no evidence of market research to mean that none has been done. They’re under no obligation to share such with us, and may very well have compelling reasons not to.

 

[Parmandur]

If they did, I missed them; and I was paying at least some attention during that time.

They do big surveys all the time, and seeing this model laid out now, I think it still informs hiw they approach the market snd design.

 

[Parmandur]

I’m no marketing whiz, but I wouldn’t take the fact that we the public have no evidence of market research to mean that none has been done. They’re under no obligation to share such with us, and may very well have compelling reasons not to.

I've taken part in major surveys from them on s regular basis for years: they put them out as aggressively as possible.

 

[Umbran]

Or is it? That history I linked above made reference to a survey WotC themselves did about types of gaming. It took some further digging to find a working (archive!) link, and it seems that survey has very nearly vanished from gamer memory.

Unfortunately, yes. But, there are some explanations as to why:

1) First and foremost, WotC never made the underlying data available. Nobody outside WotC could replicate the results, or critique the methodology, or follow up with further study - surveying on the necessary scale is expensive, and doing it properly really requires a few data scientists who know what they are doing.

2) While SKR presented this information to us, he didn't present a charismatic front for theory work.

3) Indeed, this study is empirical, data-based, rather than theory. And... I'm afraid I cannot put this delicately, but the past couple of years have demonstrated to us that data will often be rejected if it does not fit into one's preconceived notions.

 

[Umbran]

Seeing as it's a survey about D&D done by a company attenpting at the time to redesign D&D, I think it's fair to allow them to assume a D&D-like structure.

Or data manipulation.

For example, all survey responses from people over a certain age (35, I think) were tossed*; meaning most of those who had started in the early days had no voice in the data.

If we are going to talk about the statistics, we should get our terms straight.

That wasn't "data manipulation", with its nefarious connotation of altering the data to get the answer you want. That was a possible source of selection bias in the result, and they give the reason they made that choice.

That potential selection bias means the results might not generalize to the population as a whole. That would imply that older gamers are, in some way, different from others. There are several modes of how that could be, but they all point to a basic result - no static model of gamers would be accurate. Not GNS, not the Big Model, none of them could ever represent gamers except as a snapshot in time. Gamers at a particular time would follow the model, but it could not be expected to be representative in the future, or the past.

 

[Lanefan]

If we are going to talk about the statistics, we should get our terms straight.

That wasn't "data manipulation", with its nefarious connotation of altering the data to get the answer you want.

Personally, I've maintained all along that that's exactly what it was. They knew going in that they'd get answers* from the older players that they didn't want to hear, and so they made sure not to hear them.

* - including tendencies toward longer and more stable campaigns, greater prevalence of kitbashing and rule/system tweaking, fewer purchases of anything other than adventure modules, slower levelling, larger parties and-or more players, etc.

That was a possible source of selection bias in the result, and they give the reason they made that choice.

That potential selection bias means the results might not generalize to the population as a whole. That would imply that older gamers are, in some way, different from others.

That's still no reason to exclude their voices.

Imagine the squawks had it gone the other way, that responses from anyone under 35 were tossed.

There are several modes of how that could be, but they all point to a basic result - no static model of gamers would be accurate. Not GNS, not the Big Model, none of them could ever represent gamers except as a snapshot in time. Gamers at a particular time would follow the model, but it could not be expected to be representative in the future, or the past.

Which while true is still no excuse for failing to make the snapshot-in-time as representative of the complete demographic as possible.

 

[Parmandur]

Personally, I've maintained all along that that's exactly what it was. They knew going in that they'd get answers* from the older players that they didn't want to hear, and so they made sure not to hear them.

* - including tendencies toward longer and more stable campaigns, greater prevalence of kitbashing and rule/system tweaking, fewer purchases of anything other than adventure modules, slower levelling, larger parties and-or more players, etc.

That's still no reason to exclude their voices.

Imagine the squawks had it gone the other way, that responses from anyone under 35 were tossed.

Which while true is still no excuse for failing to make the snapshot-in-time as representative of the complete demographic as possible.

How many of the people playing in 1999 could have even been over 35? That's 1964, and most people who had ever even played at all were probably born in the 70's, 80's, and even 90's by then. And in terms of this profiling scheme, it may not have made any difference at any rate.

 

[Reynard]

It doesn't make any sense for a company to manipulate it's market research data. It borders on unhinged conspiracy theory to suggest WotC dramatically manipulated results. What possible benefit does that provide? The only reason to do market research is to find out what to sell to whom. Ignoring or manipulating it is literally throwing away money.

 

[niklinna]

It doesn't make any sense for a company to manipulate it's market research data. It borders on unhinged conspiracy theory to suggest WotC dramatically manipulated results. What possible benefit does that provide? The only reason to do market research is to find out what to sell to whom. Ignoring or manipulating it is literally throwing away money.

Ah, but there's a difference between manipulating the data you use for your own business strategy, and manipulating the data you present in public, for your competition to see....

Not they they did that.

And not that they share any such data publicly, any more.