Menu
News
All News
Dungeons & Dragons
Level Up: Advanced 5th Edition
Pathfinder
Starfinder
Warhammer
2d20 System
Year Zero Engine
Industry News
Reviews
Dragon Reflections
Columns
Weekly Digests
Weekly News Digest
Freebies, Sales & Bundles
RPG Print News
RPG Crowdfunding News
Game Content
ENterplanetary DimENsions
Mythological Figures
Opinion
Worlds of Design
Peregrine's Next
RPG Evolution
Other Columns
From the Freelancing Frontline
Monster ENcyclopedia
WotC/TSR Alumni Look Back
4 Hours w/RSD (Ryan Dancey)
The Road to 3E (Jonathan Tweet)
Greenwood's Realms (Ed Greenwood)
Drawmij's TSR (Jim Ward)
Community
Forums & Topics
Forum List
Latest Posts
Forum list
*Dungeons & Dragons
Level Up: Advanced 5th Edition
D&D Older Editions
*TTRPGs General
*Pathfinder & Starfinder
EN Publishing
*Geek Talk & Media
Search forums
Chat/Discord
Resources
Wiki
Pages
Latest activity
Media
New media
New comments
Search media
Downloads
Latest reviews
Search resources
EN Publishing
Store
EN5ider
Adventures in ZEITGEIST
Awfully Cheerful Engine
What's OLD is NEW
Judge Dredd & The Worlds Of 2000AD
War of the Burning Sky
Level Up: Advanced 5E
Events & Releases
Upcoming Events
Private Events
Featured Events
Socials!
Twitch
YouTube
Facebook (EN Publishing)
Facebook (EN World)
Twitter
Instagram
TikTok
Podcast
Features
Top 5 RPGs Compiled Charts 2004-Present
Adventure Game Industry Market Research Summary (RPGs) V1.0
Ryan Dancey: Acquiring TSR
Q&A With Gary Gygax
D&D Rules FAQs
TSR, WotC, & Paizo: A Comparative History
D&D Pronunciation Guide
Million Dollar TTRPG Kickstarters
Tabletop RPG Podcast Hall of Fame
Eric Noah's Unofficial D&D 3rd Edition News
D&D in the Mainstream
D&D & RPG History
About Morrus
Log in
Register
What's new
Search
Search
Search titles only
By:
Forums & Topics
Forum List
Latest Posts
Forum list
*Dungeons & Dragons
Level Up: Advanced 5th Edition
D&D Older Editions
*TTRPGs General
*Pathfinder & Starfinder
EN Publishing
*Geek Talk & Media
Search forums
Chat/Discord
Menu
Log in
Register
Install the app
Install
Community
General Tabletop Discussion
*Dungeons & Dragons
Requesting permission to have something cool
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Message
<blockquote data-quote="EzekielRaiden" data-source="post: 9179638" data-attributes="member: 6790260"><p>To be clear, I don't mean looking at <em>player</em> data. I mean using it as an in-house virtual testbed.</p><p></p><p>Full-time in-house playtesters can only do a handful of encounters per day. A machine running BG3 code (without actually showing the 3D models) could run <em>hundreds</em> of combats an hour easily, perhaps far more. Different setups would yield different kinds of data. E.g. create monsters that you are very confident do exactly the kinds of damage they're supposed to, on average, during a fight (various "yardstick" encounters, so to speak.) Then test different group compositions against these yardsticks. Run the same scenario ten thousand times each for two groups that are identical except for having a Champion vs a Battle Master, and then do a couple hypothesis tests to see if the two distributions are consistent with one another or not (e.g. a goodness-of-fit test). Iterate this over various different setups so you can get actual data on the performance of various characters in combat, and make appropriate tweaks as needed to ameliorate unintended weaknesses or strengths (every class and subclass should <em>have</em> weaknesses and strengths, but unintended ones are likely a problem.) Out-of-combat stuff, as stated, requires actual human brains behind it--but now you can have your playtesters <em>focus on</em> that stuff, rather than spreading their efforts around.</p><p></p><p>Then, once that phase of testing is done, aka once you've shifted from <em>asking</em> how strong PCs are to <em>knowing</em> how strong they are, set up some standard party slates (say 10 parties with widely varying composition), and then use <em>those</em> as yardsticks for testing more interesting and diverse monsters. Again, some things cannot be tested in a code environment, they require human judgment to work correctly, and those are the exact things that you sic your live playtesters on. The stuff that <em>doesn't</em> critically depend on human judgment, however, can be run through the simulation a zillion times to check that the spread of results it generates fits the intended results. E.g., ten thousand runs each of parties A-J showing that ghouls are massively more deadly than they should be (aka, what actually bit the 5.0 design team in the ass when they tried some live playtesting) would mean that ghouls either need to be toned down a bit, or need to be marked as higher CR, or some other thing.</p><p></p><p>As stated, if we can get some randomization thrown into the mix as well, beyond just the dice roller (e.g. something that can generate reasonable random encounter maps, so it's not just five-a-side on a featureless flat plane), you could go even further. Use standardized monsters AND standardized parties, <em>and then add terrain effects</em>. Hazards, traps, muck, ice, kaiju stomach acid, the works. Check how these things compare to the exact same maps <em>without</em> said hazards, and boom--you can generate a mathematical representation of how much tougher features can make a fight. Naturally, as with all these numbers, it would be an approximate and statistical fit, not a diamond-perfect solution (in math jargon, it is a "numeric" solution rather than an "analytic" solution.)</p><p></p><p>And, to reiterate, <em>this is not perfect</em>. It cannot completely replace human playtesting, and it should never be expected to. Human judgment, preferences, needs, and interests will always be central to the process of making a good game. But an automated virtual testing environment does <em>wonders</em> for the parts of the game that really are just a lot of random-number generation and mathematical calculation, and combat balance is <em>precisely</em> that. Then, precious live-playtester time can be focused on the stuff that benefits most from live playtesters. What could have been months or even <em>years</em> of public playtesting can potentially be reduced to days, even <em>hours</em> of virtual playtesting.</p></blockquote><p></p>
[QUOTE="EzekielRaiden, post: 9179638, member: 6790260"] To be clear, I don't mean looking at [I]player[/I] data. I mean using it as an in-house virtual testbed. Full-time in-house playtesters can only do a handful of encounters per day. A machine running BG3 code (without actually showing the 3D models) could run [I]hundreds[/I] of combats an hour easily, perhaps far more. Different setups would yield different kinds of data. E.g. create monsters that you are very confident do exactly the kinds of damage they're supposed to, on average, during a fight (various "yardstick" encounters, so to speak.) Then test different group compositions against these yardsticks. Run the same scenario ten thousand times each for two groups that are identical except for having a Champion vs a Battle Master, and then do a couple hypothesis tests to see if the two distributions are consistent with one another or not (e.g. a goodness-of-fit test). Iterate this over various different setups so you can get actual data on the performance of various characters in combat, and make appropriate tweaks as needed to ameliorate unintended weaknesses or strengths (every class and subclass should [I]have[/I] weaknesses and strengths, but unintended ones are likely a problem.) Out-of-combat stuff, as stated, requires actual human brains behind it--but now you can have your playtesters [I]focus on[/I] that stuff, rather than spreading their efforts around. Then, once that phase of testing is done, aka once you've shifted from [I]asking[/I] how strong PCs are to [I]knowing[/I] how strong they are, set up some standard party slates (say 10 parties with widely varying composition), and then use [I]those[/I] as yardsticks for testing more interesting and diverse monsters. Again, some things cannot be tested in a code environment, they require human judgment to work correctly, and those are the exact things that you sic your live playtesters on. The stuff that [I]doesn't[/I] critically depend on human judgment, however, can be run through the simulation a zillion times to check that the spread of results it generates fits the intended results. E.g., ten thousand runs each of parties A-J showing that ghouls are massively more deadly than they should be (aka, what actually bit the 5.0 design team in the ass when they tried some live playtesting) would mean that ghouls either need to be toned down a bit, or need to be marked as higher CR, or some other thing. As stated, if we can get some randomization thrown into the mix as well, beyond just the dice roller (e.g. something that can generate reasonable random encounter maps, so it's not just five-a-side on a featureless flat plane), you could go even further. Use standardized monsters AND standardized parties, [I]and then add terrain effects[/I]. Hazards, traps, muck, ice, kaiju stomach acid, the works. Check how these things compare to the exact same maps [I]without[/I] said hazards, and boom--you can generate a mathematical representation of how much tougher features can make a fight. Naturally, as with all these numbers, it would be an approximate and statistical fit, not a diamond-perfect solution (in math jargon, it is a "numeric" solution rather than an "analytic" solution.) And, to reiterate, [I]this is not perfect[/I]. It cannot completely replace human playtesting, and it should never be expected to. Human judgment, preferences, needs, and interests will always be central to the process of making a good game. But an automated virtual testing environment does [I]wonders[/I] for the parts of the game that really are just a lot of random-number generation and mathematical calculation, and combat balance is [I]precisely[/I] that. Then, precious live-playtester time can be focused on the stuff that benefits most from live playtesters. What could have been months or even [I]years[/I] of public playtesting can potentially be reduced to days, even [I]hours[/I] of virtual playtesting. [/QUOTE]
Insert quotes…
Verification
Post reply
Community
General Tabletop Discussion
*Dungeons & Dragons
Requesting permission to have something cool
Top