D&D General Is D&D Survey Feedback Read? [UPDATED!]

If you watch a lot of YouTube videos, you may be aware that there's a narrative going around, with 'anonymous' sources that contain Machievellian quotes about how WotC ignores survey feedback, and uses it as some kind of trap to keep discussion off the internet.

We're all unhappy with WotC and its approach to the current licensing situation, and we're all concerned about the fate of the third-party D&D publishing industry which supports hundreds, if not thousands, of creators and small publishers. I'm worried, and afraid for the fate of my little company and those who rely on me to pay their rent, bills, and mortgages.

But it's important to stay factual.

Ray Winninger, who ran D&D until late 2022, said "I left after the first OneD&D feedback was arriving. I know for certain UA feedback is still read."

He went on to say "This is simply false. Before I left WotC, I personally read UA feedback. So did several others. Many, many changes were made based on UA feedback, both quantitative and written. The entire OneD&D design schedule was built around how and when we could collect feedback."

Winninger previously spoke out in support of the OGL movement, after WotC announced their plans in December.

Another WotC employee tweeted, too -- "I read nearly half a million UA comments my first year working on D&D. I was not the only one reading them. I understand the desire to share information as you get it, but this just feels like muckraking."

It's important to stay on the right side of this OGL issue -- and make no mistake, any attempt to de-authorise the OGL is ethically and legally wrong -- but just making stuff up doesn't help anybody.

Benn Riggs, author of Slaying the Dragon: A Secret History of Dungeons & Dragons, chimed in with his own suspicions.

Here is why I am growing more and more suspicious of @DnD_Shorts and their purported source in WoTC. Let's call that source "The Rogue."

1) Getting a source on the record takes time. DnD Shorts is getting commentary incredibly quickly. WoTC's statement came out this morning, and by this afternoon, we know "The Rogue's" thoughts. The statement talks about a survey? "The Rogue" tells us no one will read what we write to the company.

Then there is the logistics. Is "The Rogue" contacting DnD Shorts from WoTC HQ? Doing it from the bathroom? On their lunch break? All while knowing they'd be fired if found out? They don't at least wait to contact DnD Shorts from home?

2) The info provided by "The Rogue" is simply too good. They have mentioned where they work in the company, and directly quoted powerful people within the company. All that means that within WoTC, tracking down "The Rogue" and firing them should take about two hours. Frankly, if "The Rogue" exists, the best proof of it will be when they are fired.

I'm upset about the OGL too, and it's easy to cast doubt on anonymous sources. People have done it to me. So I will say upfront I could be totally wrong about this and if DnD Shorts reads this and curses me for a bastard because they're honest & good & true and I am besmirching them, well I'm sorry.

But something here just feels wrong, and I cannot keep my peace.

And of course, all this fracturing of the 'resistance' only weakens the position of those who are working against the de-authorization of the OGL. The more click-bait nonsense out there, the less seriously anybody takes the real issues which affect real people.

UPDATES! WotC designer Makenzie De Armas has weighed in to describe the survey process:

Hi, actual #WotCStaff and D&D Designer here. I am credited on several UA releases—and I’ve made edits to that content based on both qualitative and quantitative survey results. Let’s walk through what happens behind the scenes of a UA, shall we?

1. We design player-facing mechanical elements that we hope to include in a future product. We then place those mechanical elements into a UA document and release it, to see what our player base at large thinks of it.

2. We release a survey about the UA.

3. The survey information is collated by members of the team. It’s broken down into two parts: quantitative satisfaction expressed as a percentage, and a summary of qualitative feedback trends noticed in the comments.

4. That summary is reported back to the product teams. The designers on the product teams then make edits to the mechanical elements based on the feedback summary.

5. If satisfaction doesn’t meet our quality standards, we’ll rerelease mechanical content in a followup UA.

This is a proven process. Take for example the Mages of Strixhaven UA, where we tried to create subclasses that could be taken by multiple classes. (Fun fact: that was my first UA.) Did we, as studio designers, want that to work? Yes! But it didn’t.

And we learned that it didn’t BECAUSE of the UA process. We learned that it wasn’t something a majority of our players wanted; we also learned what small elements of that design DID bring joy. We salvaged those elements, redesigned them, and put that changed design in the book.

If we didn’t read or listen to feedback, we would have put those polyclass subclasses into the final book, and the product would have been worse for it. Yes, of course we want to know if you like something—we’re game designers! We’re creating something that is meant to be FUN!

And yes, sometimes we get frustrated when people tell us how to do our jobs, or use those feedback opportunities to belittle us; we’re human. But despite all that, we’re still going to listen and always strive to improve. That’s the truth.

They went on to say:

When I say ALL the comments, I mean it in the most literal sense. We have team members who have dedicated WEEKS to diligently reading through feedback. It’s honestly incredible, and I applaud my team members’ work!

Gamehome Con director Alex Kammer added:

Hey everyone. I personally know the guy at Wizards whose job in part is to read and organize all the comments from their surveys. Reasonable OGL talk and demanding action is great. Fallacious hit pieces only cause harm.
 
Last edited:

log in or register to remove this ad

lvl20dm

Explorer
Does anyone else find it to be an incredibly bad idea for them to show the actual reply they got?

I think it may have been the wrong move to show the actual reply they got back. In addition to the person's writing style, we know they've seen the reports based off of One D&D feedback, but they're only "aware" of (as opposed having seen) how previous surveys were handled. This is information that can help Wizards figure out who's been talking.

I know my company has found leakers with less to go on.
I thought the same thing. The leaker also seems to be getting stuff wrong; I'm inclined to believe that this is a hoaxer.
 

log in or register to remove this ad

Haplo781

Legend
Does anyone else find it to be an incredibly bad idea for them to show the actual reply they got?

I think it may have been the wrong move to show the actual reply they got back. In addition to the person's writing style, we know they've seen the reports based off of One D&D feedback, but they're only "aware" of (as opposed having seen) how previous surveys were handled. This is information that can help Wizards figure out who's been talking.

I know my company has found leakers with less to go on.
He didn't. The tweet specifically says it's a paraphrase.
 


ThorinTeague

Creative/Father/Professor
For me (consumer - not publisher) there is really only one issue left - WotC must not attempt to "de-authorize" OGL 1.0a and agree they will never do so.

The issue with a capital I is definitely the deauthorization of 1.0a. Everything else is just muddying the waters.

Which might be a little bit by design on wizards' part.
 


Juomari Veren

Adventurer
It's been pretty common knowledge that UA surveys are an imperfect system, Mike Mearls (back before he got rightly stuffed in a drawer in a cubicle somewhere in the offices never to see the light of day again) once said in a tweet that while Surveys are invaluable for gathering data, the way the data is formatted when it arrives to them makes it impossible to truly read every single word of a response, be it a few characters or really pushing the limits of the 2/500 word limit that they normally impose. I'm sure it's a very sensationalist take that these surveys aren't being read; As someone who routinely maximizes the comments they make on UA surveys every time knowing for a fact that they are never read in full, and having never seen an opinion I've had matched up with what the community at large has been saying (apparently I was the only person who wasn't picking up the majority of what the Dragonlance UA was putting down), I know that it's probably an ineffectual way to be seen, heard, and known, but that's sort of the reality of creating a game that thrives on this kind of feedback and has an active playerbase that's probably more than 7 digits long at this point (er, was, before the OGL business broke out). Do I wish they had one where my feedback was read in full and iterated upon? Not really, that seems like a burden to try and get the game that's already pretty perfect in its design and appeal to me to be even more bespoke than it already is.

But I've never once questioned whether or not it was worth it to do a survey because I know that 1) the amount of players that actually do surveys must be abyssmally low compared to those that use the UA or just play the game, this has to do a lot with player enfranchisement (something WoTC apparently has super strong numbers on, as Mark Rosewater regularly comments on data gathered from MTG players that really highlights the echo chamber-esque nature of online communities not unlike our own vs. what the average consumer thinks about the game). It makes reasonable sense that someday, my feedback will actually be vital, and that I'm happy to spend 15 minutes of time that I would otherwise use doomscrolling social media to give my two cents on my favorite all-consuming hobby, and 2) It's more important that I know how I feel than how my feelings are interpreted by the company on any given subject, and these surveys let me sit down and better analyze what every little piece of content means to me. I fill those surveys out so I know what I want out of a product, and when it doesn't have it, I know what I want to do with that product (Spoiler: If I ever run Shadow of the Dragon Queen, my campaign is gonna be a lot more boring, like deathly so). It's pretty evil of them to hang the fate of any meaningful OGL change that isn't familiar to what we've already had (or been offered in the case of this most recent post, where they seemingly say "actually not a thing we wanted before is going to be in the final version", which also kind of raises the question as to why they felt they need to do a UA-style Survey on it if they're just taking the OGL 1.0 and adding a "do not make NFTs or we will find where you live and sh** down your throat" clause to it), but I've mostly sat the experience of declining in interest to play/run D&D over the last two weeks because I 100% expected this to be their solution from day one, and I knew that until I had something that I could actually think over, straight from the source and complete with survey for me to hmm and haw over what I really think and feel, any opinion myself or others is going to form isn't going to be what I really feel until I'm more or less forced to make a choice on where I stand on a lot of these things.

Does that mean I want the utter downfall of all things 3rd party? No, of course not, and OGL 1.1/2.0 were blatantly evil and had no hope of being received warmly unless they invented a new language with completely alien grammatical structure and drafted the entire document in it (which I'm frankly surprised they didn't do). But we're in the third act of a song and dance that anyone who's ever been familiar with corporate meddling has seen before and could've called from a mile away. And while it's never fun to get to this point, it's the one I've been waiting for, if only so I can firmly say I should start dusting off that d20-based system I was working on...

But I think this hullaballoo with someone somewhere supposedly knowing the score about how UA is handled is ridiculous, anyone could've guessed "maybe they don't read every comment ever????????" and been right without having to twist the narrative, and I hope that this person's laurels that we've seemingly been resting on for a while rot underneath their temples for thinking that attacking the employeees (who, mind you, are not the shareholders or execs, nor have they seemingly been able to speak on the subject at all and I'd hope that was apparent from almost a week of radio silence from most of their Twitters) is at all a fair shake in a game where people's livelihoods (theirs and ours alike) are at stake.
 


Clint_L

Hero
Ben Riggs' arguments should be taken with a grain of salt, IMO. He writes books. Yeah, no kidding this is all moving more quickly than you do, Ben.
Ben Riggs is a very experienced writer of books and articles for a variety of publications. His work that I've read is carefully sourced and fact-checked. As far as I can find, he has an excellent reputation. In particular, he has done investigative writing on Wizards of the Coast and TSR on Slaying the Dragon and elsewhere, so he has some expertise on this particular subject.

As opposed to a YouTuber whose channel I usually enjoy, but who does not seem to have any journalistic credentials, and who already seems to be walking back his comments.
 



Remove ads

Remove ads

AD6_gamerati_skyscraper

Remove ads

Upcoming Releases

Top