D&D 5E (2024) A critical analysis of 2024's revised classes

FrogReaver said:

By definition it wasn’t a representative survey.

Click to expand...

I no longer believe this due to the new information that came to light.

That link someone posted by Mike Mearls above (the only one on this thread who would actually know) indicates it was not dedicated players that took the survey; something I was originally willing to accept at face value but no longer am willing to accept after reading Mearls post.

FrogReaver said:

Being a non-representative survey does mean it’s a flawed to draw any conclusions about the whole population from it.

Click to expand...

This statement would be true if it was a survey that did not represent the population, which I did agree with but no longer do agree with.

However, even if I still accepted this premise - saying it is flawed to draw any conclusions from it is fundamentally different than saying the results are skewed or incorrect.

Sulicius said:

Whatever they did, it was probably already better research than any of us did.

Who here has done actual surveys to come to their conclusion?

Click to expand...

Fortunately, I don’t need a survey to know that I know better than everyone else.

Sulicius said:

Whatever they did, it was probably already better research than any of us did.

Who here has done actual surveys to come to their conclusion?

Click to expand...

This. I dont think the surveys are perfect. I dont expect them to be.

They are good at figuring out what people hate. That's all they need to know.

Sulicius said:

Whatever they did, it was probably already better research than any of us did.

Click to expand...

Given the methodology used, I'd say their 'research' was not much better than a coin toss at identifying improvements

Sulicius said:

Who here has done actual surveys to come to their conclusion?

Click to expand...

Given the accuracy of their surveys (see above), I do not need one to be about as accurate

mamba said:

Given the methodology used, I'd say their 'research' was not much better than a coin toss at identifying improvements


Given the accuracy of their surveys (see above), I do not need one to be about as accurate

Click to expand...

Yeah I don’t know how to deal with this level of pessimism and willful ignorance.

Or are you being sarcastic?

Sulicius said:

Yeah I don’t know how to deal with this level of pessimism and willful ignorance.

Or are you being sarcastic?

Click to expand...

Self selected surveys like the ones they did are so notoriously likely to be skewed by an overrepresented unusually passionate minority among possible respondents that they teach it in college courses related to polling and statistics. All of that is before you consider the impact of statistical insanity like the time Crawford told us all that when they saw things like a class getting high marks on individual features but poor in overall class grade as a whole they assumed respondents forgot what they voted and used the higher value. Another great example was the time that dndshorts put out a "there's a problem with the new warlock" video Esther hysterically calling for survey brigading with a video that got more upvotes than Crawford later bragged about that survey getting as reason for changing course

tetrasodium said:

Self selected surveys like the ones they did are so notoriously likely to be skewed by an overrepresented unusually passionate minority among possible respondents that they teach it in college courses related to polling and statistics. All of that is before you consider the impact of statistical insanity like the time Crawford told us all that when they saw things like a class getting high marks on individual features but poor in overall class grade as a whole they assumed respondents forgot what they voted and used the higher value. Another great example was the time that dndshorts put out a "there's a problem with the new warlock" video Esther hysterically calling for survey brigading with a video that got more upvotes than Crawford later bragged about that survey getting as reason for changing course

Click to expand...

What is your proof that they are self selected? What do you belief? And why do you believe it?

I never heard anything about Crawford saying players forgot what they voted, or maybe that’s from before I played.

I just shared first-hand experience of Mike Mearls about this, and I see nothing but refutals without proof.

Sulicius said:

What is your proof that they are self selected? What do you belief? And why do you believe it?

Click to expand...

It's literally the type of survey they did with the public surveys. Here's a very simple article about self selected surveys

Sulicius said:

I never heard anything about Crawford saying players forgot what they voted, or maybe that’s from before I played.

Click to expand...

He brought it up in one of the onednd videos

Sulicius said:

I just shared first-hand experience of Mike Mearls about this, and I see nothing but refutals without proof.

Click to expand...

Sulicius said:

Yeah I don’t know how to deal with this level of pessimism and willful ignorance.

Or are you being sarcastic?

Click to expand...

no, that is my assessment of their methodology and their handling of feedback. Their approach is about as good at identifying improvements and then sticking to working on them as tossing a coin would be.

Where it considerably beats a coin toss is at identifying total duds and discarding them. It is very reliable there, but pretty useless for anything else. I have been saying this during the UA phase too.

I am honestly surprised they stuck with such a flawed methodology for so long. The only conclusion I can draw is that they do not care about identifying improvements, only about weeding out complete duds, and that the UA mostly is about marketing / engagement / raising awareness about the new product, rather than actually improving the game

tetrasodium said:

All of that is before you consider the impact of statistical insanity like the time Crawford told us all that when they saw things like a class getting high marks on individual features but poor in overall class grade as a whole they assumed respondents forgot what they voted and used the higher value.

Click to expand...

and don’t forget the time where they dropped something that got more than the 70% threshold vote regardless of that, because some comments really did not like the idea

The main issue is even deeper, it is that there is no good way to actually communicate intent (outside of always leaving a comment), so they do not know what me voting 2 actually means and I have no idea what I would have had to vote for them to react in the way I would have wanted them to.

Add to that the 70% threshold for whether people supposedly like a feature on its own merits when in reality no one is evaluating it on its own but comparing it against what exists today and that the feedback is supposed to ignore balancing when the vast majority doesn’t and you get meaningless numbers that are being interpreted randomly, and yet somehow expect that this leads to improvements

I agree that it identifies what is overwhelmingly unpopular, for anything else it is meaningless