[UPDATED] Most D&D Players Prefer Humans - Without Feats!

prosfilaes said:

Yes, yes, I can know. If you claim that "Most D&D players prefer..." then "an optimally provisioned statistical sample" can only come from a random sample of D&D players, including groups that are playing in home games. The only way to get that random sample is to get a random sample of people and survey the D&D players; otherwise, whatever source you have is going to be a biased sample. Unless WotC is using polling firms, there's no way they could possibly reach a random sample of D&D players.

Click to expand...

That’s not knowing. That’s speculating. You have no information whatsoever about this data, or how it was gathered. You don’t even have a hint.

Morrus said:

That’s not knowing. That’s speculating. You have no information whatsoever about this data, or how it was gathered. You don’t even have a hint.

Click to expand...

Well we know some of it comes from DnD Beyond.

jgsugden said:

You don't need a huge pool to validate data is flawed if you get massively inconsistent data.

Look at it this way:

A new player sits down at your table to play D&D. You're having fun. But a bit into the game, one of the other players says, "Wow, Newbie. You're lucky. You've rolled a d20 only 8 times, and the results were all 11 or above. That is real lucky." You think about it and realize all 8 rolls have been 11 or above. There is a 1 in 250 chance (roughly) that would happen.

A lot of people would have worries that the player was cheating... likely with loaded die or faked results.

Is it a certainty? No. Do similar streaks happen where no funny business is going On? Yes... about 1 in 250 times it occurs and there arecca lot of opportunities for that to happen. However, that doesn't matter. I'm focused on my situation. I see something occuring that should be very unlikely. When does it become more likely there is cheating than that our situation was the oddity of 1 in 250.

When is it more likely their statistic is wrong than that my experiences were so uncommon.... especially when you complicate the matter by introducing the common expectation that a significant majority of players use human variants?

If all of their sampling and statistics are perfect, therevvwould still be a number of people with my experiences. However, the rational position for them to take, without additional information, is that I experienced either really unlikely results, or the stats are fun at. You could also challenge my underlying elements (we're the group's totally independent, etc...), but there are arguments both sides there.

My experiences are enough to say, "Those results look shady. Check yourself." Even with only 8 draws from the hat.

Click to expand...

Like I said, you don’t understand how statistical samples work.

Morrus said:

Like I said, you don’t understand how statistical samples work.

Click to expand...

I encourage you to take a moment and rethink. Really look at what I'm saying rather than dismiss it instantly.

Morrus said:

That’s not knowing. That’s speculating. You have no information whatsoever about this data, or how it was gathered. You don’t even have a hint.

Click to expand...

prosfilaes said:

Unless WotC is using polling firms, there's no way they could possibly reach a random sample of D&D players.

Click to expand...

Do you want to propose some other way they could have reached a random sample of D&D players? Or are you claiming they did use a polling firm, which I did not rule out, even if I consider it highly unlikely? For a random sample of D&D players, they need some way to reach the D&D player who does not interact with WotC online or through Adventure League. Since the PHB doesn't have a registration card, they have no way of knowing who might be playing D&D except by random polling.

jgsugden said:

You don't need a huge pool to validate data is flawed if you get massively inconsistent data.

Look at it this way:

A new player sits down at your table to play D&D. You're having fun. But a bit into the game, one of the other players says, "Wow, Newbie. You're lucky. You've rolled a d20 only 8 times, and the results were all 11 or above. That is real lucky." You think about it and realize all 8 rolls have been 11 or above. There is a 1 in 250 chance (roughly) that would happen.

A lot of people would have worries that the player was cheating... likely with loaded die or faked results.

Is it a certainty? No. Do similar streaks happen where no funny business is going On? Yes... about 1 in 250 times it occurs and there are a lot of opportunities for that to happen. However, that doesn't matter. I'm focused on my situation. I see something occuring that should be very unlikely. When does it become more likely there is cheating than that our situation was the oddity of 1 in 250.

When is it more likely their statistic is wrong than that my experiences were so uncommon.... especially when you complicate the matter by introducing the common expectation that a significant majority of players use human variants?

If all of their sampling and statistics are perfect, there would still be a number of people with my experiences. However, the rational position for them to take, without additional information, is that I experienced either really unlikely results, or the stats are funny. You could also challenge my underlying elements (we're the group's totally independent, etc...), but there are arguments both sides there.

My experiences are enough to say, "Those results look shady. Check yourself." Even with only 8 draws from the hat.

Click to expand...

Hm. So I'll take one last shot at this?

1) Your observation that you found 8 groups in which some players used feats is irrelevant. Jeremy is stating that the majority of players don't use feats, not that the majority of groups don't. I suspect if you did the analysis at the level of individuals you would find that the composition of your sample isn't all that anomalous, even if the majority in your sample happened to use feats.

2) Even if Jeremy had been making a claim about groups, the fact that your eight happen to use feats would not, from the perspective of the population, be that out of the ordinary.

3) To put it another way, using your example of a player that rolled a bunch of numbers above 11 several times in a row. Sure, you might ask-- Is he cheating? Are his dice weighted? Then you might say, 'Let's commission a study to find out by getting a much larger sample.' At the end of that study, if you found that the larger sample indicated that the player wasn't cheating and the dice weren't weighted, you would conclude that the streak was just something that happened by chance. You would certainly NOT decide that your larger sample must be 'shady' because they don't conform to the results of your much, much smaller sample. Similarly, if you discovered that a study already existed (say the Crawford Study) that already looked at your question, you would again conclude that the small sample result was a matter of chance. You would not conclude that the larger study must be 'shady'.

And on a side note-- In random samples, streaks are quite common. If you look at two sets of numbers drawn on a chalkboard-- one pulled at random and one concocted by a person trying to create a random set of numbers-- the quickest way to tell which group of numbers is random is that the random group of numbers will have streaks. People tend to think that 'random' actually means uniform. So, when they try to generate random numbers, they try not to have streaks. Which gives them away.

So if you gave me a list of numbers rolled on a 20 sided and it didn't have streaks where most of the rolls were above 11, I'd suspect you had just made the list of numbers up.

AD

lkj said:

So, I hesitate to get embroiled in this-- But 8 non-independent samples would be considered a very, very small sample size from which to make inferences. Put another way, even if only 40% of groups didn't use feats, it would not be hard to find 8 such groups of the presumably thousands of existing groups.

Click to expand...

millions...

prosfilaes said:

Do you want to propose some other way they could have reached a random sample of D&D players? Or are you claiming they did use a polling firm, which I did not rule out, even if I consider it highly unlikely? For a random sample of D&D players, they need some way to reach the D&D player who does not interact with WotC online or through Adventure League. Since the PHB doesn't have a registration card, they have no way of knowing who might be playing D&D except by random polling.

Click to expand...

And since I'm on a crazy posting kick-- There are ways to model and control for sampling bias. I obviously don't know how they do their analyses or what ancillary data they take to detect bias (none of us do). But I'd be very surprised if a company doing market research would just ignore such bias. It's against their interest to do so.

AD

[UPDATED] Most D&D Players Prefer Humans - Without Feats!

jgsugden said:

I encourage you to take a moment and rethink. Really look at what I'm saying rather than dismiss it instantly.

Click to expand...

I keep reading what you’re saying. You’re saying that your tiny sample of 8 groups is statistically relevant, to the point where a sample orders of magnitude larger must be flawed if it doesn’t agree with yours.

What I and others keep trying to say is that that an anecdote regarding just 8 groups is statistically insignificant.

When I say that you don’t understand how statistical samples work, it’s because you knew rejecting the above two things.

But I’ve said it several times now. I’m not going to keep doing so. What was it Einstein said about doing the same thing over and over and expecting different results?

And, I suppose, that’s fine. I know you’ll reject it again; I don’t expect anything else. I’ll just let you do so; I’m not getting into the Argument Clinic sketch over it.

And in the end, does it even matter?

Morrus said:

And in the end, does it even matter?

Click to expand...

Nothing really matters,
Anyone can see.