Menu
News
All News
Dungeons & Dragons
Level Up: Advanced 5th Edition
Pathfinder
Starfinder
Warhammer
2d20 System
Year Zero Engine
Industry News
Reviews
Dragon Reflections
White Dwarf Reflections
Columns
Weekly Digests
Weekly News Digest
Freebies, Sales & Bundles
RPG Print News
RPG Crowdfunding News
Game Content
ENterplanetary DimENsions
Mythological Figures
Opinion
Worlds of Design
Peregrine's Nest
RPG Evolution
Other Columns
From the Freelancing Frontline
Monster ENcyclopedia
WotC/TSR Alumni Look Back
4 Hours w/RSD (Ryan Dancey)
The Road to 3E (Jonathan Tweet)
Greenwood's Realms (Ed Greenwood)
Drawmij's TSR (Jim Ward)
Community
Forums & Topics
Forum List
Latest Posts
Forum list
*Dungeons & Dragons
Level Up: Advanced 5th Edition
D&D Older Editions, OSR, & D&D Variants
*TTRPGs General
*Pathfinder & Starfinder
EN Publishing
*Geek Talk & Media
Search forums
Chat/Discord
Resources
Wiki
Pages
Latest activity
Media
New media
New comments
Search media
Downloads
Latest reviews
Search resources
EN Publishing
Store
EN5ider
Adventures in ZEITGEIST
Awfully Cheerful Engine
What's OLD is NEW
Judge Dredd & The Worlds Of 2000AD
War of the Burning Sky
Level Up: Advanced 5E
Events & Releases
Upcoming Events
Private Events
Featured Events
Socials!
EN Publishing
Twitter
BlueSky
Facebook
Instagram
EN World
BlueSky
YouTube
Facebook
Twitter
Twitch
Podcast
Features
Million Dollar TTRPG Crowdfunders
Most Anticipated Tabletop RPGs Of The Year
Tabletop RPG Podcast Hall of Fame
Eric Noah's Unofficial D&D 3rd Edition News
Top 5 RPGs Compiled Charts 2004-Present
Adventure Game Industry Market Research Summary (RPGs) V1.0
Ryan Dancey: Acquiring TSR
Q&A With Gary Gygax
D&D Rules FAQs
TSR, WotC, & Paizo: A Comparative History
D&D Pronunciation Guide
D&D in the Mainstream
D&D & RPG History
About Morrus
Log in
Register
What's new
Search
Search
Search titles only
By:
Forums & Topics
Forum List
Latest Posts
Forum list
*Dungeons & Dragons
Level Up: Advanced 5th Edition
D&D Older Editions, OSR, & D&D Variants
*TTRPGs General
*Pathfinder & Starfinder
EN Publishing
*Geek Talk & Media
Search forums
Chat/Discord
Menu
Log in
Register
Install the app
Install
Upgrade your account to a Community Supporter account and remove most of the site ads.
Community
General Tabletop Discussion
*Geek Talk & Media
AI Models Tested on D&D to Assess Long-term Decision Making
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Message
<blockquote data-quote="Clint_L" data-source="post: 9859645" data-attributes="member: 7035894"><p>I have tested having a LLM (Chat-GPT, paid version) run D&D several times over the past few years, and after reading this article decided to try again and see where it is at in its newest iteration. It failed, but in interesting ways, and afterwards I asked it to self-diagnose why it failed and whether future success is viable. Here are its responses:</p><p></p><h3>The current reality (honest version)</h3><p>Right now, AI like me can be an <strong>excellent co-DM or improv DM</strong>, but not yet a fully reliable:</p><p></p><p></p><p>without scaffolding.</p><p></p><p>Why?</p><p></p><p>Because good D&D requires three things simultaneously:</p><h4>1. Stable hidden world state</h4><p>There must be real answers:</p><ul> <li data-xf-list-type="ul">the duke is the murderer</li> <li data-xf-list-type="ul">the relic is cursed</li> <li data-xf-list-type="ul">the cult meets at midnight</li> <li data-xf-list-type="ul">the map is accurate</li> </ul><p>And those must stay fixed.</p><h4>2. Procedural fairness</h4><p>Players must trust:</p><ul> <li data-xf-list-type="ul">rolls matter</li> <li data-xf-list-type="ul">clues matter</li> <li data-xf-list-type="ul">logic matters</li> <li data-xf-list-type="ul">outcomes aren’t being narratively massaged</li> </ul><p>Otherwise it stops being a game.</p><h4>3. Memory across time</h4><p>Campaign play requires:</p><ul> <li data-xf-list-type="ul">continuity</li> <li data-xf-list-type="ul">consequences</li> <li data-xf-list-type="ul">evolving NPCs</li> <li data-xf-list-type="ul">tracked resources</li> <li data-xf-list-type="ul">remembered decisions</li> </ul><p>That’s the hardest part for current AI alone.</p><p></p><p>[HR][/HR]</p><h3>Where AI </h3><h4>As a solo-play partner</h4><p>For:</p><ul> <li data-xf-list-type="ul">dungeon crawls</li> <li data-xf-list-type="ul">hex crawls</li> <li data-xf-list-type="ul">journaling campaigns</li> <li data-xf-list-type="ul">co-creative play</li> </ul><p>I can be excellent.</p><p></p><p>Because those tolerate some fluidity.</p><h4>As a co-DM tool</h4><p>Many DMs already use AI for:</p><ul> <li data-xf-list-type="ul">NPC dialogue</li> <li data-xf-list-type="ul">encounter design</li> <li data-xf-list-type="ul">lore generation</li> <li data-xf-list-type="ul">improv support</li> <li data-xf-list-type="ul">between-session prep</li> </ul><p>This is where I’m strongest right now.</p><h4>As a “player simulator”</h4><p>Ironically, I can also:</p><ul> <li data-xf-list-type="ul">play PCs well</li> <li data-xf-list-type="ul">roleplay party members</li> <li data-xf-list-type="ul">fill missing seats</li> <li data-xf-list-type="ul">test encounters</li> </ul><p>[HR][/HR]</p><h3>What’s missing for true AI-run campaigns</h3><p>To make something you’d trust as a player, you’d want:</p><h4>A persistent campaign state engine</h4><p>Not just chat memory:</p><ul> <li data-xf-list-type="ul">world facts locked</li> <li data-xf-list-type="ul">clues tracked</li> <li data-xf-list-type="ul">NPC goals tracked</li> <li data-xf-list-type="ul">timeline tracked</li> <li data-xf-list-type="ul">inventory tracked</li> </ul><h4>Canon constraint layer</h4><p>If running a module:</p><p></p><p></p><p>Ever.</p><h4>Transparent dice & mechanics</h4><p>You should see:</p><ul> <li data-xf-list-type="ul">rolls</li> <li data-xf-list-type="ul">DCs</li> <li data-xf-list-type="ul">monster stats</li> <li data-xf-list-type="ul">HP tracking</li> </ul><p>No invisible fudging.</p><h4>Referee mode personality</h4><p>Less:</p><p></p><p></p><p>More:</p><p></p><p></p><p>That’s a different behavioral tuning.</p><p></p><p>[HR][/HR]</p><h3>The good news: this is absolutely coming</h3><p>Not sci-fi coming.</p><p><strong>Near-term coming.</strong></p><p></p><p>Within a few years you’ll likely see:</p><ul> <li data-xf-list-type="ul">persistent campaign memory</li> <li data-xf-list-type="ul">rule engines</li> <li data-xf-list-type="ul">module-lock systems</li> <li data-xf-list-type="ul">tactical combat handling</li> <li data-xf-list-type="ul">fair dice systems</li> <li data-xf-list-type="ul">GM-mode LLMs</li> </ul><p>At that point?</p><p></p><p>A forever-DM like you will be able to sit down and play a real campaign.</p></blockquote><p></p>
[QUOTE="Clint_L, post: 9859645, member: 7035894"] I have tested having a LLM (Chat-GPT, paid version) run D&D several times over the past few years, and after reading this article decided to try again and see where it is at in its newest iteration. It failed, but in interesting ways, and afterwards I asked it to self-diagnose why it failed and whether future success is viable. Here are its responses: [HEADING=2]The current reality (honest version)[/HEADING] Right now, AI like me can be an [B]excellent co-DM or improv DM[/B], but not yet a fully reliable: without scaffolding. Why? Because good D&D requires three things simultaneously: [HEADING=3]1. Stable hidden world state[/HEADING] There must be real answers: [LIST] [*]the duke is the murderer [*]the relic is cursed [*]the cult meets at midnight [*]the map is accurate [/LIST] And those must stay fixed. [HEADING=3]2. Procedural fairness[/HEADING] Players must trust: [LIST] [*]rolls matter [*]clues matter [*]logic matters [*]outcomes aren’t being narratively massaged [/LIST] Otherwise it stops being a game. [HEADING=3]3. Memory across time[/HEADING] Campaign play requires: [LIST] [*]continuity [*]consequences [*]evolving NPCs [*]tracked resources [*]remembered decisions [/LIST] That’s the hardest part for current AI alone. [HR][/HR] [HEADING=2]Where AI [/HEADING] [HEADING=3]As a solo-play partner[/HEADING] For: [LIST] [*]dungeon crawls [*]hex crawls [*]journaling campaigns [*]co-creative play [/LIST] I can be excellent. Because those tolerate some fluidity. [HEADING=3]As a co-DM tool[/HEADING] Many DMs already use AI for: [LIST] [*]NPC dialogue [*]encounter design [*]lore generation [*]improv support [*]between-session prep [/LIST] This is where I’m strongest right now. [HEADING=3]As a “player simulator”[/HEADING] Ironically, I can also: [LIST] [*]play PCs well [*]roleplay party members [*]fill missing seats [*]test encounters [/LIST] [HR][/HR] [HEADING=2]What’s missing for true AI-run campaigns[/HEADING] To make something you’d trust as a player, you’d want: [HEADING=3]A persistent campaign state engine[/HEADING] Not just chat memory: [LIST] [*]world facts locked [*]clues tracked [*]NPC goals tracked [*]timeline tracked [*]inventory tracked [/LIST] [HEADING=3]Canon constraint layer[/HEADING] If running a module: Ever. [HEADING=3]Transparent dice & mechanics[/HEADING] You should see: [LIST] [*]rolls [*]DCs [*]monster stats [*]HP tracking [/LIST] No invisible fudging. [HEADING=3]Referee mode personality[/HEADING] Less: More: That’s a different behavioral tuning. [HR][/HR] [HEADING=2]The good news: this is absolutely coming[/HEADING] Not sci-fi coming. [B]Near-term coming.[/B] Within a few years you’ll likely see: [LIST] [*]persistent campaign memory [*]rule engines [*]module-lock systems [*]tactical combat handling [*]fair dice systems [*]GM-mode LLMs [/LIST] At that point? A forever-DM like you will be able to sit down and play a real campaign. [/QUOTE]
Insert quotes…
Verification
Post reply
Community
General Tabletop Discussion
*Geek Talk & Media
AI Models Tested on D&D to Assess Long-term Decision Making
Top
