D&D (2024) Class spell lists and pact magic are back!

[tetrasodium]

All groups have their voice amplified to the same degree. These munchkins (as you call them - I think it's a terrible and inherently biased description) have the same voice as anyone else.

No that's not how the weighting works. Only groups voting to against a change are amplified by the weighting you pointed out. Groups that can be reliably expected to vote for a veto in a given topic* are going to have their voice amplified in that expected topic* and some effort should be made to normalize it.

*such as a group that cares so strongly about their own power they will vote against anything reducing or restricting their power for any reason no matter what is gained or improved by that change. Since that is quite the mouthful I've been calling it by a descriptive moniker.

 

[Amrûnril]

Zooming out to some of this thread's broader questions, I think that there's certainly room to argue about whether about whether the developers are making good decisions about the game, but I have yet to see a compelling argument that they're making good decisions about survey design and interpretation.

The developers are collecting data in one form (a 5 point approval scale), treating it as if its a different form of data (a binary approval rating), and then applying arbitrary cutoffs to resolve a question survey participants were never asked: Should an unsatisfactory idea be revised or discarded? Would the majority of playtesters have liked to see a different version of template-based wildshapes? I have absolutely no idea, but I know it's not a question that can be answered with any confidence using the data the developers are citing.

It's also worth noting that none of these questions of data interpretation are actually relevant to the issue of spell lists, since the surveys never actually asked about shared spell lists- the developers seem to have been responding to written feedback on less specific related questions. Personally, I'm thrilled with this news but perplexed by the focus on the Wizard- the one class that has essentially the same spell list under both systems. Shared spell lists were a mistake not because they somehow hurt the Wizard, but because they take away the possibility of distinctive spell lists for Sorcerers, Bards and Warlocks, while also failing to achieve any of their purported benefits.

 

[Mistwell]

No that's not how the weighting works. Only groups voting to against a change are amplified by the weighting you pointed out.

No? They MIGHT (I am not sure) have increased the threshold a bit to adjust for people who directly expressed that they liked anything WOTC puts out and don't really care. That's it. If you are for some things and against others, that's not you. If you are for 99% of things and against 1% of things, that's not you. ALL voices except those who literally don't care get the same treatment.

You keep using the word veto. It does not mean what you think it means. There is no veto power with the surveys.

 

[tetrasodium]

No? They MIGHT (I am not sure) have increased the threshold a bit to adjust for people who directly expressed that they liked anything WOTC puts out and don't really care. That's it. If you are for some things and against others, that's not you. If you are for 99% of things and against 1% of things, that's not you. ALL voices except those who literally don't care get the same treatment.

You keep using the word veto. It does not mean what you think it means. There is no veto power with the surveys.

It doesn't matter why wotc weighted it, only that they did weight it. The weighting is

Spoiler: here

 

[Chaosmancer]

yes you did, the rating attached to your answer results in that, not in a revision

Which is, again, the same as saying that by voting "Yes" on an issue that fails, I really voted No. Because you are equating the RESULT of the vote with what I individually voted for. That isn't how this works.

You did not do it alone, but your vote also was in favor of throwing it out again, even if you were not

No, my vote was not in favor of throwing it out. You ASSUME that it was, because you ASSUME you know how these things are working. But you do not know that.

no I am not, it is very obviously a line that does not exist.

Then why do you keep claiming that I voted for something I did not vote for, as proven because the thing I didn't vote for happened?

how can they possibly still use it if they are aware?


I am just that good

On a more serious note, I would like an actual explanation for why it is not broken, not just blind faith in their abilities


it doesn’t take a majority to lead to wrong results


can you prove that they accurately understand them? I’d say I gave good reasons for why there is a disconnect, to the point that you voted wrong given what you wanted to happen.

Without access to WoTC's internal data and the back-end workings of the survey results? No, I can't. But I don't NEED to prove that I understand them. You need to prove that, because you are making an extraordinary claim here. I am claiming that the company which has been using a proccess for 10 years, reported to us on that process, and has seemingly accurately navigated these surveys for a decade... understands those surveys.

Your claim is that because things you don't like have happened, and you feel the survey options aren't accurate enough, that the survey is fundamentally flawed. You are making the claim, so you need the evidence. Just like if you claimed that Cleopatra suffered from Autism, it isn't on me the person saying "there is no evidence of that, and there is good reasoning to assume that lack of evidence is proof you are incorrect" to then definitively prove whether or not she had autism.

Because, again, if they were aware of this flaw, it would be corrected because it would be devastating to their understanding of the game. And how could they possibly be unaware after a decade of using it to accurately navigate these waters?

If they considered this, they would be better off with different options instead of trying to correct for avoidable mistakes

And maybe they tested those different options and they created EVEN WORSE problems. After all, every piece of language is fundamentally flawed and unable to perfectly convey all intent and clarity. That's WHY text-based communication suffers so much compared to in-person communication. And even in-person communication creates "noise" that obfuscates understanding.

Demonstrating "someone might have misunderstood" is the equivalent of demonstrating energy exists. It doesn't actually PROVE anything about the situation.

so the exact same system that lead to subclasses rated in the 20s, despite the goal to get everything over 70?

Saying ‘we have always done it this way’ is not a good reason to keep doing it…

The same process that led to subclasses in the 80's as well. Just because something does not always lead to success does not mean you abandon it as fundamentally flawed.

Also, to my knowledge, nothing in One DnD has been rated in the 20's. The only things rated in the 20's were from Next, and that playtest had a few mitigating circumstances, like none of those things actually being playtested before print. I am around 90% certain none of us saw the Berserker with exhaustion before the PHB.

 

[Chaosmancer]

If you can't tell how what people mean when they vote, it's flawed by design.

And why are we assuming they can't? Because one person can say "But someone might have a different understanding"?

People can literally argue over what the word "car" means. They could argue the sentence "I left" is ambiguous and confusing. You have no EVIDENCE that the survey method is fundamentally flawed.

Their survey methods being shoddy/flawed has been brought up for years. And yes, this discussion has very clearly shown that WotC is unable to tell what someone means by somewhat approve. Most would think that it means that you like it as is. You'd have us believe that you don't like it as is, but want it to be changed. It can't mean both with only a single descriptor like "somewhat approve."

Approval of any sort will put it into the bucket that adds up to 80% approval, and that's NOT where someone who doesn't like it and as is and wants it changed wants to be. WotC should view somewhat disapprove as needs a different iteration, because it's not complete disapproval.

Anyone who wants something to be changes should never vote approval of any sort.

The only people who ever complain the survey results are flawed are the people who didn't get the result they wanted, or who just like declaring that Wizards of the Coast is ignorant/malicious/incompetent or some fun combination of the three.

You can draw lines in the sand and declare "Anyone who thinks X should vote the way I say they should vote and nothing else makes sense!" but you weren't hired by the company to make these survey format. In fact, you have to my knowledge zero credentials in this subject matter. All you are doing is declaring that you must be right, because anyone would agree with your vision.

And therefore, since the survey isn't giving you what you want, it must be flawed, because people must agree with you.

 

[mamba]

Which is, again, the same as saying that by voting "Yes" on an issue that fails, I really voted No. Because you are equating the RESULT of the vote with what I individually voted for. That isn't how this works.

no, I am saying your vote was understood as ‘no’ by WotC.

You do not have to try to explain to me that the survey result is not the same as your vote, I am aware of that… I was already aware the last two times too

No, my vote was not in favor of throwing it out. You ASSUME that it was, because you ASSUME you know how these things are working. But you do not know that.

I would say I am very confident in it, yes. Now show me why you think it did not get counted as a ‘no’, I gave you my reasoning and so far you have shown nothing to contradict it. Just saying ‘I disagree’ won’t change my mind.

Without access to WoTC's internal data and the back-end workings of the survey results? No, I can't. But I don't NEED to prove that I understand them. You need to prove that, because you are making an extraordinary claim here.

well, I have shown the logic, as far as I am concerned it is absolutely sound.

Is your only counter ‘you do not know what WotC does’? Then we can end it right here. Neither do you, and out of the two possibilities I consider mine far more likely. If you cannot even address my concerns by doing more than say ‘you cannot be certain’, then that will not change.

Also, if you make a claim, which you do, you too NEED to show proof, just like you are demanding it from me. Saying you do not need to does not change that. At most you can say you have confidence in them, or are not convinced by my arguments. But if you say that there is no issue and WotC understands the votes correctly, you have a burden of proof just like me. It is not just ‘extraordinary’ claims that require evidence, all claims do.

You are making the claim, so you need the evidence.

I made my case, repeatedly, now refute it, saying ‘WotC smart’ is not even close to that

The same process that led to subclasses in the 80's as well. Just because something does not always lead to success does not mean you abandon it as fundamentally flawed.

if a process that should always lead to >70 leads to both 80 and 20, it very much is flawed.

Also, I am not saying they should abandon the playtest altogether, I am saying they should ask better questions. Not sure how that turned into ‘abandon as fundamentally flawed’…

Also, to my knowledge, nothing in One DnD has been rated in the 20's.

even if it were it would not matter, the playtest is not over. It only needs to prevent the final result from being below 70, not everything along the way. That is not how any of this works

 

[Mistwell]

It doesn't matter why wotc weighted it, only that they did weight it. The weighting is

Yes thank you for re-posting the graphic I posted. Which doesn't show the thing we're talking about (they show no increase in the thresholds nor why there would be an increase or any mention of discounting). And yes it matters. It matters because if they're doing that, they're doing that to realize the opinion that's being expressed, which is the purpose of the surveys.

You seem to be under the impression there is something bad about being an optimizer. I think that is a bad opinion.

 

[Chaosmancer]

You still do not understand samples and quality control. If the survey worked, there would not be such 'noise'.

No? Like, not even a little bit? Like, this shows a disturbing lack of knowledge of how statistics work?

I did not say I was, I said it is quality control. I already said from the start that the sample is too small to extrapolate from. It is not too small to indicate an issue that should be investigated before ruling out a larger problem.

A sample size may be too small to detect the issue, but we already found it in a size of 10, so....

Again... no? Like, let us say I walk into a room and out of five people, one of them is ill. Does this mean I should begin investigating for a global pandemic? After all, in such a small sample size as 5, I found 1 person who is ill. Therefore we should investigate, right?

Except... no. We shouldn't. One person in a single group of five indicates NOTHING about a population of hundreds, let alone thousands or millions. Just because you happened to find one person in this group of ten who was confused does not mean that an investigation needs to take place.

 

[Chaosmancer]

Zooming out to some of this thread's broader questions, I think that there's certainly room to argue about whether about whether the developers are making good decisions about the game, but I have yet to see a compelling argument that they're making good decisions about survey design and interpretation.

The developers are collecting data in one form (a 5 point approval scale), treating it as if its a different form of data (a binary approval rating), and then applying arbitrary cutoffs to resolve a question survey participants were never asked: Should an unsatisfactory idea be revised or discarded?

Pause.

Why are you saying the cut-off is arbitrary? 70% approval is not arbitrary. It is seeking more than 2/3rds of the approval. It was chosen very specifically and very carefully.

Are they treating a 5-point scale like a binary? No, actually they aren't. Because it ISN'T a binary. 80%? Keep as is. 70% probably keep as is, but maybe tweak. 60%? Needs a bit of work, but we can salvage it. 50%, probably should scrap it. That is not binary. That is a range.

So, at this point, you have created two out-right falsehoods. And you will now use this false base as a jumping off point to declare something else, but your foundation is too weak to support anything.

Would the majority of playtesters have liked to see a different version of template-based wildshapes? I have absolutely no idea, but I know it's not a question that can be answered with any confidence using the data the developers are citing.

It's also worth noting that none of these questions of data interpretation are actually relevant to the issue of spell lists, since the surveys never actually asked about shared spell lists- the developers seem to have been responding to written feedback on less specific related questions. Personally, I'm thrilled with this news but perplexed by the focus on the Wizard- the one class that has essentially the same spell list under both systems. Shared spell lists were a mistake not because they somehow hurt the Wizard, but because they take away the possibility of distinctive spell lists for Sorcerers, Bards and Warlocks, while also failing to achieve any of their purported benefits.

Now, I do want to say, I do agree that they did not ask about the spell lists, and I think they should have before scrapping them. But that is fundamentally ASKING FOR MORE SURVEY, not declaring the survey broken.

I vehemently disagree that the addition of the best spells to Sorcerers and Warlocks, while giving Bards a unique spell-positioning, did any harm whatsoever to those classes, and state instead it was a massive boon for them.