Menu
News
All News
Dungeons & Dragons
Level Up: Advanced 5th Edition
Pathfinder
Starfinder
Warhammer
2d20 System
Year Zero Engine
Industry News
Reviews
Dragon Reflections
White Dwarf Reflections
Columns
Weekly Digests
Weekly News Digest
Freebies, Sales & Bundles
RPG Print News
RPG Crowdfunding News
Game Content
ENterplanetary DimENsions
Mythological Figures
Opinion
Worlds of Design
Peregrine's Nest
RPG Evolution
Other Columns
From the Freelancing Frontline
Monster ENcyclopedia
WotC/TSR Alumni Look Back
4 Hours w/RSD (Ryan Dancey)
The Road to 3E (Jonathan Tweet)
Greenwood's Realms (Ed Greenwood)
Drawmij's TSR (Jim Ward)
Community
Forums & Topics
Forum List
Latest Posts
Forum list
*Dungeons & Dragons
Level Up: Advanced 5th Edition
D&D Older Editions, OSR, & D&D Variants
*TTRPGs General
*Pathfinder & Starfinder
EN Publishing
*Geek Talk & Media
Search forums
Chat/Discord
Resources
Wiki
Pages
Latest activity
Media
New media
New comments
Search media
Downloads
Latest reviews
Search resources
EN Publishing
Store
EN5ider
Adventures in ZEITGEIST
Awfully Cheerful Engine
What's OLD is NEW
Judge Dredd & The Worlds Of 2000AD
War of the Burning Sky
Level Up: Advanced 5E
Events & Releases
Upcoming Events
Private Events
Featured Events
Socials!
EN Publishing
Twitter
BlueSky
Facebook
Instagram
EN World
BlueSky
YouTube
Facebook
Twitter
Twitch
Podcast
Features
Top 5 RPGs Compiled Charts 2004-Present
Adventure Game Industry Market Research Summary (RPGs) V1.0
Ryan Dancey: Acquiring TSR
Q&A With Gary Gygax
D&D Rules FAQs
TSR, WotC, & Paizo: A Comparative History
D&D Pronunciation Guide
Million Dollar TTRPG Kickstarters
Tabletop RPG Podcast Hall of Fame
Eric Noah's Unofficial D&D 3rd Edition News
D&D in the Mainstream
D&D & RPG History
About Morrus
Log in
Register
What's new
Search
Search
Search titles only
By:
Forums & Topics
Forum List
Latest Posts
Forum list
*Dungeons & Dragons
Level Up: Advanced 5th Edition
D&D Older Editions, OSR, & D&D Variants
*TTRPGs General
*Pathfinder & Starfinder
EN Publishing
*Geek Talk & Media
Search forums
Chat/Discord
Menu
Log in
Register
Install the app
Install
Upgrade your account to a Community Supporter account and remove most of the site ads.
Community
General Tabletop Discussion
*Dungeons & Dragons
D&D Next playtest post mortem by Mike Mearls and Rodney Thompson. From seven years ago.
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Message
<blockquote data-quote="Mistwell" data-source="post: 8768502" data-attributes="member: 2525"><p>Video #3 [Open Playtest]</p><p></p><p>They read all the comments.</p><p></p><p>The surveys were extremely important to the final product. Absolutely essential.</p><p></p><p>They figured out out to best iterate with a purpose on problems later in the process and wish they had figured that out earlier in the process.</p><p></p><p>People's ratings of the game went up and up with each packet of the playtest. There were minimum quantity thresholds for feedback before data could be considered sufficient to consider.</p><p></p><p>The forums are not necessarily representative of the larger audience. There definitely is a silent majority sometimes. A lot of times people would say something is terrible on forums, that came back with 95% approval on surveys. It was most useful to use forums when the forum views lined up with the survey data, where they could then ask forum people more about that aspect of the game. Also sometimes the thing people would complain about was more a sign something was going wrong in a broader issue, and not what they would specifically complain about. Like for example someone might complain about not hitting enough bad guys, but they'd find that was really a symptom of a cause of lack of sufficient movement for PCs.</p><p></p><p>Here is an image of a sample of some of their data, measured on a 1-5 scale of satisfation. NOTE: They specify the headers are off on this.all three columns are off (if you can even read them in this grainy image - supposed to be combat satisfaction, non combat satisfaction, and average satisfaction overall, but numbers may not be in that order). This was not the headers they worked with he just knew they got moved before the presentation by accident:</p><p></p><p><img src="https://i.ibb.co/qC47jFj/Class-Satisfaction-Oct2013.jpg" alt="" class="fr-fic fr-dii fr-draggable " data-size="" style="" /></p><p></p><p>Here is another on Class Complexity from a prior set of data:</p><p></p><p><img src="https://i.ibb.co/LhqqTcc/Class-Complexity.jpg" alt="" class="fr-fic fr-dii fr-draggable " data-size="" style="" /></p><p></p><p>There was another chart here on Complexity vs Non-Combat satisfaction and they found classes high on non-combat satisfaction tended to also be high on complexity. So for non-combat, people liked the complex classes.</p><p></p><p>The reverse was true for combat. Audiences said they were having the least amount of fun with the more complex classes in combat, and the most amount of fun with the least complex classes in combat. </p><p></p><p>This set of conclusions was the complete reverse of the designer initial assumptions about the game. The assumption was that during combat people liked having a lot of options, and that during non-combat they wanted a lot more free form and not as many mechanical options. It turned out it seemed to be a time pressure issue. When out of combat, planning stuff was more fun with lots of options to work with and time to flip through books and such. But once in combat and everyone is waiting for you to complete your turn you don't want to have too many options and flipping through books resulting in slowing the game down. </p><p></p><p>Here is an image of Rogue Satisfaction from October 2012. I am sorry the image is so grainy I couldn't seem to grab a clearer one.</p><p></p><p><img src="https://i.ibb.co/sJjQwgB/Rogue-Satisfaction-Oct2012.jpg" alt="" class="fr-fic fr-dii fr-draggable " data-size="" style="" /></p><p></p><p></p><p>Cunning Action was being rated at 100% (40.1% apparently rated it top, and it had the top rating of being above everything else) as the top rated element of the rogue. Cunning Action was at the time of this survey part of the Thief subclass and not part of the core Rogue abilities. Thieves Cant was getting the lowest rating at 3.1% (which they didn't care much about because they viewed it as a ribbon mechanic.) </p><p></p><p>They followed this up with a grainy image of two different Rogue surveys (August and October) charted against each other showing where there were response improvements from changed in the playtest for that class, and still some dips that fell below their threshold for acceptable that continued to need work.</p></blockquote><p></p>
[QUOTE="Mistwell, post: 8768502, member: 2525"] Video #3 [Open Playtest] They read all the comments. The surveys were extremely important to the final product. Absolutely essential. They figured out out to best iterate with a purpose on problems later in the process and wish they had figured that out earlier in the process. People's ratings of the game went up and up with each packet of the playtest. There were minimum quantity thresholds for feedback before data could be considered sufficient to consider. The forums are not necessarily representative of the larger audience. There definitely is a silent majority sometimes. A lot of times people would say something is terrible on forums, that came back with 95% approval on surveys. It was most useful to use forums when the forum views lined up with the survey data, where they could then ask forum people more about that aspect of the game. Also sometimes the thing people would complain about was more a sign something was going wrong in a broader issue, and not what they would specifically complain about. Like for example someone might complain about not hitting enough bad guys, but they'd find that was really a symptom of a cause of lack of sufficient movement for PCs. Here is an image of a sample of some of their data, measured on a 1-5 scale of satisfation. NOTE: They specify the headers are off on this.all three columns are off (if you can even read them in this grainy image - supposed to be combat satisfaction, non combat satisfaction, and average satisfaction overall, but numbers may not be in that order). This was not the headers they worked with he just knew they got moved before the presentation by accident: [IMG]https://i.ibb.co/qC47jFj/Class-Satisfaction-Oct2013.jpg[/IMG] Here is another on Class Complexity from a prior set of data: [IMG]https://i.ibb.co/LhqqTcc/Class-Complexity.jpg[/IMG] There was another chart here on Complexity vs Non-Combat satisfaction and they found classes high on non-combat satisfaction tended to also be high on complexity. So for non-combat, people liked the complex classes. The reverse was true for combat. Audiences said they were having the least amount of fun with the more complex classes in combat, and the most amount of fun with the least complex classes in combat. This set of conclusions was the complete reverse of the designer initial assumptions about the game. The assumption was that during combat people liked having a lot of options, and that during non-combat they wanted a lot more free form and not as many mechanical options. It turned out it seemed to be a time pressure issue. When out of combat, planning stuff was more fun with lots of options to work with and time to flip through books and such. But once in combat and everyone is waiting for you to complete your turn you don't want to have too many options and flipping through books resulting in slowing the game down. Here is an image of Rogue Satisfaction from October 2012. I am sorry the image is so grainy I couldn't seem to grab a clearer one. [IMG]https://i.ibb.co/sJjQwgB/Rogue-Satisfaction-Oct2012.jpg[/IMG] Cunning Action was being rated at 100% (40.1% apparently rated it top, and it had the top rating of being above everything else) as the top rated element of the rogue. Cunning Action was at the time of this survey part of the Thief subclass and not part of the core Rogue abilities. Thieves Cant was getting the lowest rating at 3.1% (which they didn't care much about because they viewed it as a ribbon mechanic.) They followed this up with a grainy image of two different Rogue surveys (August and October) charted against each other showing where there were response improvements from changed in the playtest for that class, and still some dips that fell below their threshold for acceptable that continued to need work. [/QUOTE]
Insert quotes…
Verification
Post reply
Community
General Tabletop Discussion
*Dungeons & Dragons
D&D Next playtest post mortem by Mike Mearls and Rodney Thompson. From seven years ago.
Top