D&D (2024) WotC On One D&D Playtest Survey Results: Nearly Everything Scored 80%+!

In a 40-minute video, WotC's Jeremy Crawford discussed the survey feedback to the 'Character Origins' playtest document. Over 40,000 engaged with the survey, and 39,000 completed it. I've summarised the content of the video below.

High Scorers

• The highest scoring thing with almost 90% was getting a first level feat in your background. This is an example of an experimental thing -- like advantage and disadvantage in the original 5E playtests.
• Almost everything also scored 80%+.

About The Scoring System

• 70% or higher is their passing grade. In the 70s is a thumbs up but tinkering need. 80% means the community wants exactly that and WotC treads carefully not to change it too much.
• In the 60s it's salvageable but it really needs reworking. Below 60% means that there's a good chance they'll drop it, and in the 40s or below it's gone. Nothing was in the 50s or below.

Low Scorers

Only 3 things dipped into the 60s --

• the d20 Test rule in the Rules Glossary (experimental, no surprise)
• the ardling
• the dragonborn

The next UA had a different version of the d20 Test rule, and they expect a very different score when those survey resuts come in.

It was surprising that the dragonborn scored lower than the ardling. The next UA will include new versions of both. The main complaints were:

• the dragonborn's breath weapon, and confusion between the relationship between that dragonborn and the one in Fizban's Treasury of Dragons.
• the ardling was trying to do too much (aasimar-like and beast-person).

The ardling does not replace the aasimar. The next version will have a clearer identity.

Everything else scored in the 70s or 80s.

Some more scores:

• new human 83%
• dwarf, orc, tiefling, elf tied at 80-81%
• gnome, halfling tied at 78%

Future installments of Unearthed Arcana

• The next one will have new ardling and dragonborn, a surprise 'guest', and a new cleric. It will be a shorter document than the previous ones, and the one after that is bigger again. Various class groups.
• Warrior group digs into something teased in a previous UA sidebar -- new weapon options for certain types of characters. Whole new ways to use weapons.
• New rules on managing your character's home base. A new subsystem. Create bases with NPCs connected with them, implementing downtime rules. They're calling it the "Bastion System".
• There will be a total of 48 subclasses in the playtest process.
• New encounter building rules, monster customization options.
• New versions of things which appear in the playtest after feedback.

Other Notes

• Playtests are a version of something with the assumption that if something isn't in the playtest, it's still in the game (eg eldritch blast has not been removed from the game). The mage Unearthed Arcana will feature that.
• Use an object and other actions are still as defined in the current Player's Handbook. The playtest material is stuff that has changed.
• Thief subclass's cunning action does not interact with use an object; this is intentional. Removed because the original version is a 'Mother may I?" mechanic - something that only works if the DM cooperates with you. In general mechanics which require DM permission are unsatisfying. The use an object action might go away, but that decision will be a made via the playtest process.
• The ranger's 1st-level features also relied too heavily on DM buy-in, also wild magic will be addressed.
• If you have a class feature you should be able to use it in the way you expect.
• If something is removed from the game, they will say so.
• Great Weapon Fighting and Sharpshooter were changed because the penalty to the attack roll was not big enough to justify the damage bonus, plus they want warrior classes to be able to rely on their class features (including new weapon options) for main damage output. They don't want any feats to feel mandatory to deal satisfying damage. Feats which are 'must haves' violate their design goals.
• Light Weapon property amped up by removing the bonus action requirement because requiring light weapon users to use their bonus action meant there were a lot of bad combinations with features and spells which require bonus actions. It felt like a tax on light weapon use.
• Class spell lists are still an open question. Focus on getting used to the three big spell lists. Feedback was that it would be nice to still have a class list to summarize what can be picked from the 'master lists'. For the bard that would be useful, for the cleric and wizard not necessary as they can choose from the whole divine or arcane list.

The playtest process will continue for a year.

 

[billd91]

To me, the single most important thing player-focused change they can make is to the Action Economy. The bonus action has to go. I still have players every session confuse that actions can't be traded for other actions, that bonus action spells can only be cast alongside a cantrip, that the game hits a brick wall regularly when players pause to search their options for a bonus action (that they probably don't have).

I think a lot of this really has to come down to player style. I've got players for whom this is not a problem at all, and a couple for whom it is. But those players with problems are more likely to be the ones looking to squeeze the most out of every combat round's actions anyway no matter what the action economy or D&D edition. That's the way they play every game and they'd be doing it whether or not they could swap action types around.

 

[Vaalingrade]

It sure is… I swear, it feels like November was four days long. Is it just because I’m getting older that time is compressing like this, or do you think there’s a social/cultural element to it? Like, I certainly felt it before 2020 but it seems significantly more extreme since.

Must be you...

_>

<_<

~fondly pets The Device~

 

[Blue]

Still no packet WotC! Some of us night owls have not only stayed up late, but gotten up early to get kids ready for school. Tick tock!

NEXT PACKET IS UP!

Cleric and Revised Species (Dragonborn, Ardling and Goliath), plus Rules Summary

Claim Unearthed Arcana now! - D&D Beyond

www.dndbeyond.com

 

[Remathilis]

I love that these debate ends up the same point and counterpoint.

"WotC doesn't listen to us!"
Did you use the survey to voice your opinion?
"No, even if I did, they'd just ignore it!"
Did you watch the response video where they talked about the results?
"Yeah. It's all spin. It's an echo chamber!"
Maybe it's because you didn't engage with them when they requested feedback?
"No. I think they are out of touch."

 

[Azzy]

after four elf types with unique racial traits, the 1D elf combined the three subraces into "lineages" which only differ by the free spells known is a regression. Worse, the wood elf became another spellcasting elf type rather than a more physical one. Ideally, I'd like to see three unique sets of racial traits. If that's not doable, then (sigh) combine the wood and high elf and keep drow unique separate.

I loathe what they did to most of the races, but I think the wood elf is the biggest offender IMO. Getting Mask of the Wild replaced by crappy spells is just insulting.

 

[Reynard]

That is people for you.

Actually I feel like they're mostly against me.

 

[Ruin Explorer]

Over 40,000 engaged with the survey, and 39,000 completed it.

Gulps nervously I guess I'm in the 1000 who started it but didn't finish. Feeling a bit bad given how many did!

Hopefully the new Aardling focuses on the beast-person part of its identity. D&D is surprisingly light on beast-people, except cats and birds. We don't need more Aasimar-types though, the Aasimar is fine.

Sad to hear the 70% passing grade is back. WotC management hasn't improved much, then, but I suspect the attitude is "If it ain't broke...", which will backfire eventually (it does 100% of the time), but eventually can be a long time (or a very short and surprising one).

 

[Micah Sweet]

Gulps nervously I guess I'm in the 1000 who started it but didn't finish. Feeling a bit bad given how many did!

Hopefully the new Aardling focuses on the beast-person part of its identity. D&D is surprisingly light on beast-people, except cats and birds. We don't need more Aasimar-types though, the Aasimar is fine.

Sad to hear the 70% passing grade is back. WotC management hasn't improved much, then, but I suspect the attitude is "If it ain't broke...", which will backfire eventually (it does 100% of the time), but eventually can be a long time (or a very short and surprising one).

Sadly to me the ardling still hits the celestial side pretty hard.

 

[Morrus]

Gulps nervously I guess I'm in the 1000 who started it but didn't finish. Feeling a bit bad given how many did!

The weird thing? That's not a long way off the number of people who playtested Level Up. That number surprises me.

 

[EzekielRaiden]

You do realize that they aren't talking about their methodology right?

You don't need to talk about your methodology to give actually meaningful statistics.

They're distilling it down for those of us who aren't statististions and really, really, don't care about the math.

You can distill it down while still not leaving out the information needed to draw useful conclusions. In fact, you should do so. This is like saying that a graph with an unlabelled shortened vertical axis is perfectly acceptable. It isn't. It may be drilling down on the difference, but making it look like thing X is massively smaller than thing Y when they're actually only five percentage points apart is a classic form of manipulating the perception of data.

Why would you assume that not only are they doing it in-house, but also they should lay every point out in the open so that armchair statisticians should be able to second guess every single thing they say?

Because actual statisticians would never make the types of errors they're making, and anyone who cared about making surveys that collect useful data rather than simply reinforcing the intended results would ABSOLUTELY NEVER make the kinds of surveys and polls from the D&D Next playtest--and I've seen nothing from the surveys since which suggest anything else.

Should you have even the slightest bit of evidence to the contrary, I will immediately retract all such assertions and profusely apologize. I strongly suspect you do not have such evidence.

FFS, take it at face value. Presume, just for a moment, that a multi-million dollar project that has this enormous load of information, just maybe hires a firm that knows what they're doing?

It is difficult to make that presumption when I know, both from personal training and from working with someone whose literal doctoral field of expertise is this specific subject, that these surveys are very poorly designed. If WotC is hiring an "expert," they absolutely aren't getting what they're paying for.

Why the automatic presumption that they are lying or being deceptive?

Lying? I didn't say lying. I said "deceptive statistics." Because it is EXTREMELY EASY to be deceptive with statistics. It is extremely easy to do so WITHOUT ANY ILL INTENT, because statistics is an extremely difficult field that involves frequently counter-intuitive thinking. E.g. when Gerd Gigerenzer gave a series of lectures on the subject, he asked medical doctors to evaluate a hypothetical situation. Without any further info: otherwise asymptomatic woman, whose age cohort has a 1% incidence of breast cancer, takes a breast cancer screening, which has a 10% false negative rate and a ~9% false positive rate. She gets a positive result. Based only on this information, what are the patient's chances of having breast cancer? The correct answer is approximately 1 in 11; in at least one lecture, over half of the gynecologists present incorrectly selected "9 in 10" from the four choices presented. They themselves would be deceiving with statistics by telling the woman that result, despite being actively interested in giving her correct information.

Statistics are incredibly slippery, difficult things. They need to be handled with care. Presenting statistical information offhandedly is one of the greatest ways to bamboozle someone--and it often happens even when people very much, very explicitly, want to be truthful, accurate, and supportive.

They're telling you flat out that with certain bands they will react in particular ways. Seems pretty straightforward.

Oh, and, they have to be able to distill that huge amount of information in a couple of months. Yeah, I think I'll settle for taking what they say at pretty much face value thanks. Endlessly kvetching about their methodology when you aren't actually privy to it is a bit too far into conspiracy theory territory for me.

An actual statistician can prepare that kind of information much, much, much, MUCH faster than "in a couple of months." The kinds of straightforward, relatively ordinary statistical analysis here barely even requires training to make use of. We aren't even talking ANOVA, this is literally just summary statistics, stuff a computer can literally spit out for you in milliseconds once all the data is entered. That's yet another reason why I'm convinced they aren't making use of anyone on staff nor any consultants, because this kind of analysis isn't that difficult, and yet we're still seeing evidence that it's not happening, that the statistics are handled in a fashion that wouldn't pass muster in a second-term statistics class, let alone with a professional statistician.