D&D 4E Ben Riggs' "What the Heck Happened with 4th Edition?" seminar at Gen Con 2023

[Parmandur]

Unfortunately not realistic for a company in the position WotC has put themselves in.

But that is precisely what theybare doing? Theybare gathering the data of what people like or don't like and then designing with thwt as a metric.

 

[Thomas Shey]

Unfortunately not realistic for a company in the position WotC has put themselves in.

Possibly so, but I'm unconvinced "Don't use blindtesters at all" is an improvement.

 

[Micah Sweet]

But that is precisely what theybare doing? Theybare gathering the data of what people like or don't like and then designing with thwt as a metric.

Not exactly. They're not ignoring the subset @Thomas Shey mentioned, but instead trying to please everyone.

 

[Thomas Shey]

But that is precisely what theybare doing? Theybare gathering the data of what people like or don't like and then designing with thwt as a metric.

The question is, how much are they paying attention to relatively small, but loud subsets in their feedback? Its largely impossible to know from the outside (and in some cases, may be difficult to tell from the inside).

 

[Parmandur]

The question is, how much are they paying attention to relatively small, but loud subsets in their feedback? Its largely impossible to know from the outside (and in some cases, may be difficult to tell from the inside).

They seem to be doing a pretty good job, that's why they have their high threshold of satisfaction for each option. That accomplishes specifically rhe goal of helping to filter out the vocal minority. That's why forum and Reddit responses to UA seem to have nothing to do with the results: they are filtering out the noise.

 

[Thomas Shey]

They seem to be doing a pretty good job, that's why they have their high threshold of satisfaction for each option. That accomplishes specifically rhe goal of helping to filter out the vocal minority. That's why forum and Reddit responses to UA seem to have nothing to do with the results: they are filtering out the noise.

This assumes the respondents they're getting are representative in the first place. That may or may not be true.

 

[Alzrius]

They seem to be doing a pretty good job, that's why they have their high threshold of satisfaction for each option. That accomplishes specifically rhe goal of helping to filter out the vocal minority. That's why forum and Reddit responses to UA seem to have nothing to do with the results: they are filtering out the noise.

In truth, I'm dubious as to the wisdom of relying on feedback surveys to the degree that WotC is (or at least, seem to be). It seems to feed far too much into selection bias, insofar as who actually 1) knows about the surveys in the first place, and 2) actually fills them out.

A lot of people forget that (one of) the first use(s) of the phrase "vox populi, vox Dei" was in a warning against treating that as a guiding principle.

 

[Parmandur]

This assumes the respondents they're getting are representative in the first place. That may or may not be true.

Based on the numbers involved, they have a more than representative sample, yes.

 

[Parmandur]

In truth, I'm dubious as to the wisdom of relying on feedback surveys to the degree that WotC is (or at least, seem to be). It seems to feed far too much into selection bias, insofar as who actually 1) knows about the surveys in the first place, and 2) actually fills them out.

A lot of people forget that (one of) the first use(s) of the phrase "vox populi, vox Dei" was in a warning against treating that as a guiding principle.

Whe. Trying to sell people a game to make them happy, figuring out through testing and reaea4ch what makes people happy makes sense. How else could the designers build a better game, wirhout a metric to work towards?

 

[Alzrius]

Whe. Trying to sell people a game to make them happy, figuring out through testing and reaea4ch what makes people happy makes sense. How else could the designers build a better game, wirhout a metric to work towards?

It's not the principle I'm unsure of, it's the execution.