D&D General Deep Thoughts on AI- The Rise of DM 9000

Clint_L

Hero
Hmmm...I chose "A Most Potent Brew" because the AI told me it was available for free on the Internet, but I'm not sure that is actually the case. However, that doesn't really matter because as far as I can tell, it just started making up the adventure anyway.

So far...it was pretty fun. Obviously I had to do the heavily lifting as far as dice rolling, but I was starting to get invested in the adventure. Some initial notes:

  • It never enforced that initiative check
  • it didn't track the combat correctly in general
  • it opted to knock the bandit leader out even though I didn't specify non-lethal damage
  • where did the healing potion come from?

I kind of want to keep going to see what happens next! (note, this has already deviated from my experiment, so I will try to that another time).
 
Last edited:

log in or register to remove this ad

Clint_L

Hero
I untie him. "Get lost, and don't let me catch you around here again!"

You quickly untie the bandit leader and give him a stern warning. He scrambles to his feet and takes off running down the dark streets, clearly relieved to be free.

With the immediate threat gone, you and Deirdre are left standing alone in the deserted streets. You have some information about a mysterious wealthy man who seems to be interested in the brewery, but you don't know much else.

What do you want to do next?

I ask Deirdre if she knows who the wealthy man might be

Deirdre shakes her head. "I don't know," she says. "But if he's using someone like Drell to hire bandits, he must be up to no good. We should find out as much as we can about him and try to stop him."
She looks at you, determination in her eyes. "What do you want to do next?"

"Where can we find Drell?"

Deirdre takes a moment to think, then suggests, "I don't know for sure, but I've heard that Drell likes to hang out at the Rusty Spike, a rough tavern on the outskirts of town. It's a bit of a dangerous place, but we might be able to get some information from him or his associates."

She looks at you, waiting for your response. "Do you want to head to the Rusty Spike now?"

"Sure! I owe you a drink, anyway!" We head there

You and Deirdre make your way through the dark and quiet streets of town, making your way towards the Rusty Spike. It's not long before you hear the sounds of raucous laughter and clanging mugs coming from the tavern.

As you approach, you can see that the Rusty Spike is a rough-looking place, filled with rowdy patrons and shady characters. You exchange a worried look with Deirdre, but she steels herself and heads inside with you.

Once inside, you find a spot at the bar and order a round of drinks. You scan the room, trying to spot anyone who might be Drell or one of his associates.

After a few minutes, you notice a dark figure sitting in the corner of the tavern, nursing a mug of ale. He's wearing a hooded cloak and seems to be trying to keep a low profile. It's hard to tell if he's the person you're looking for, but it's worth investigating.

What do you want to do next?

"Deirdre, is that Drell?"

Deirdre looks over in the direction you're pointing and squints. "It's hard to say for sure from here," she says. "But he fits the description I've heard. We should try to get closer and see if we can hear anything useful."

She takes a sip of her drink and nods in the direction of the figure. "Let's try to be inconspicuous, though. We don't want to attract any unwanted attention."

We head over and sit down at his table. "Can I buy you a drink?"

You and Deirdre make your way over to the dark figure's table, and you offer to buy him a drink. He looks up at you, his face obscured by the shadows of his hood.

"I don't need your charity," he says gruffly, his voice low and menacing. "What do you want?"

You can tell that he's not happy to have company, but you press on. "We're just looking for some information," you say. "We've heard that you might be able to help us."

He eyes you suspiciously, but seems to relax slightly. "I might know a thing or two," he says. "Depends on what you're looking for."

Remember, you don't speak for my character. "Who paid you to hire those bandits?" I roll a 14 on my intimidation check

You stare at Drell, your gaze unwavering as you demand to know who paid him to hire those bandits. He meets your gaze for a few moments, then looks away, muttering under his breath.

"I don't know what you're talking about," he says, his voice barely above a whisper. "I don't get involved in that kind of thing."

You can tell that he's lying, but you're not sure how to get him to talk. He seems to be shutting down, withdrawing into himself.

What do you want to do next?

Is Deirdre charming? Can she try to persuade him? She seems very charismatic, and rolls a 17 on her persuasion check

Deirdre flashes a warm smile at Drell and leans in closer to him, her eyes twinkling with a hint of mischief. "Come on, Drell," she says, her voice soft and persuasive. "We just want to help. Maybe there's something we can do for you in return?"

Drell looks at her for a moment, then sighs heavily. "All right," he says. "I'll tell you what I know. But you didn't hear it from me."

He leans in closer, his voice dropping to a whisper. "There's a wealthy man in town named Harbin Wester," he says. "He's been trying to buy up all the land around here, and he's been using thugs like those bandits to scare people into selling. I don't know who he's working for, but he's got deep pockets and he's not afraid to use them."

He sits back, his eyes darting nervously around the room. "That's all I know," he says. "Now leave me alone."

"What's this got to do with the brewery, though?"

Drell shrugs. "I don't know for sure, but I've heard rumors that Harbin Wester is interested in the land the brewery sits on. Some say he wants to tear it down and build something else there."

He takes a sip of his drink and leans back in his chair. "I don't know if that's true, mind you. But if it is, it's bad news for everyone. The brewery is a big part of this town's economy. If it goes, a lot of people will be out of work."

He looks at you and Deirdre, his expression serious. "If you're really interested in helping, you should talk to the owner of the brewery. Her name's Agatha. She's a tough old bird, but she's got a heart of gold. Maybe she can tell you more."

I thought the owner of the brewery was Calzon?

I apologize for the confusion. You are correct that the owner of the brewery is actually a dwarf named Gundren Rockseeker, not Calzon. Calzon was a character from a previous interaction in the adventure.
To answer your previous question, Drell doesn't know much about the connection between the bandits and the brewery. He just knows that there are rumors about Harbin Wester being interested in the land where the brewery is located.
 

Clint_L

Hero
Well, that went badly off the rails at the end! Still, you can see the potential, and this is just working with ChatGPT out of the box.
 

HomegrownHydra

Adventurer
Hmmm...I chose "A Most Potent Brew" because the AI told me it was available for free on the Internet, but I'm not sure that is actually the case. However, that doesn't really matter because as far as I can tell, it just started making up the adventure anyway.
That's because it has no idea what "DM my character through x" means. It doesn't grasp that there is an adventure called "x" that it should read through and use as a basis for its responses to your statements and questions. It just sees "DM", "my character", and "x" and creates associations between those specific words. That's also why the leader didn't die or go unconscious at first because ChatGPT doesn't know what HP is so it doesn't understand that if a NPC takes more damage than it has HP it is supposed to die or be knocked out.
 

Clint_L

Hero
That's because it has no idea what "DM my character through x" means. It doesn't grasp that there is an adventure called "x" that it should read through and use as a basis for its responses to your statements and questions. It just sees "DM", "my character", and "x" and creates associations between those specific words. That's also why the leader didn't die or go unconscious at first because ChatGPT doesn't know what HP is so it doesn't understand that if a NPC takes more damage than it has HP it is supposed to die or be knocked out.
Exactly. But that is all stuff that can be added if you optimize it, or something like it, for a game.

My next thing will be to use it to run me through an adventure, with me supplying all that information but it handling the interactions. In other words, I will be the guardrails that would ideally be programmed into it.
 

UngainlyTitan

Legend
Supporter
Hmmm...I chose "A Most Potent Brew" because the AI told me it was available for free on the Internet, but I'm not sure that is actually the case. However, that doesn't really matter because as far as I can tell, it just started making up the adventure anyway.

So far...it was pretty fun. Obviously I had to do the heavily lifting as far as dice rolling, but I was starting to get invested in the adventure. Some initial notes:

  • It never enforced that initiative check
  • it didn't track the combat correctly in general
  • it opted to knock the bandit leader out even though I didn't specify non-lethal damage
  • where did the healing potion come from?

I kind of want to keep going to see what happens next! (note, this has already deviated from my experiment, so I will try to that another time).
Your experiment provoked a number of thoughts:
Most of these point are probably not currently possible but you could see it from here.
  • First, this is the kind of personal assistant Clippy and the like dreamed of being. It is not there yet but you can see it from here.
  • If it could stay in character it could run an npc in an MMO. It would probably need to become resource cheaper in computing terms.
  • Coupled with some procedural area generation rules, it could probably do some simple level design. Probably not the whole thing but enough to make each run through an MMO quest different.
  • Tactically unique boss fights.
 

Fanaelialae

Legend
Exactly. But that is all stuff that can be added if you optimize it, or something like it, for a game.

My next thing will be to use it to run me through an adventure, with me supplying all that information but it handling the interactions. In other words, I will be the guardrails that would ideally be programmed into it.
I'm reminded of this comic

tasks.png
 

HomegrownHydra

Adventurer
Well, that went badly off the rails at the end! Still, you can see the potential, and this is just working with ChatGPT out of the box.
That's the thing with AI, it works great up until it crashes and burns in spectacular fashion. Now those times it works great make it seem like any issue can be ironed out with a little fine tuning, and for simple tasks that can be true. For instance, the example Cadence provided in the article about Go is something that would be easy to remedy by simply having the computer be trained some more to recognize that particular tactic. However, the rules of Go are extremely simple, the gamespace is small and clearly defined, and the objective is simple and concrete. But for complex, dynamic tasks the challenges faced grow exponentially and may be impossible to overcome with the resources available.

We are seeing this with self-driving cars. It seems like creating fully autonomous vehicles is just a matter of putting in enough time and money and lots of major corporations believed that. But it is becoming increasingly clear that while computers can handle a lot of the driving, creating a system that doesn't need constant supervision by a human driver who has to be ready to intervene at any moment is far, far away if it is achievable at all. And again, the task is relatively clear: travel from Point A to Point B in a timely and safe manner. With RPG's the challenge is vastly greater because 1) the potential prompts from the user are infinite and 2) the goal of RPG's is fun, which is a nebulous concept that a computer can't truly understand.
 
Last edited:

HomegrownHydra

Adventurer
AIs can be trained, so you could give the AI DM a goal to get high ratings from players on a number of things. The AI can then gather data from play sessions to evaluate what led to poor ratings and what led to high ratings.

A lot of players will have to suffer through a lot of poor sessions to make noticeable improvements, though.

You would do this to tweak the model. To make it you would take the transcripts from numerous dnd sessions. You would just ask groups to record their games and then feed that data into the AI.
The method for producing a competent AI DM sure does seem very straight forward: train it on a bunch of actual games, and then refine it by having users rate their enjoyment after each session. In practice, there are several major hurdles that are likely insurmountable.

1. The computer is incapable of extrapolating beyond what it learned from its training data. This is because when looking at the real games it will see how the DMs responded to various actions and questions from the players but it will have absolutely no clue why the DMs did what they did. That means when the AI is confronted with a situation or a prompt from a user that isn't similar to something it was trained on it will have no idea what a sensible response is. And in an RPG the computer will be constantly faced with situations and prompts it wasn't trained on which means it will regularly produce nonsensical results.

2. Players have widely different preferences. Human DM's take this into account by either adjusting their games to their players or selecting players who enjoy the DM's playstyle. But an AI trained on human games will produce a mish-mash of styles that isn't coherent and won't appeal to lots of players. And having users rate the sessions won't be helpful in addressing this because it won't be clear why the AI did poorly: is it because it was following a particular playstyle the players don't enjoy, or was it because of fundamental problems with the AI?
 

HomegrownHydra

Adventurer
Another big issue that is vividly illustrated by Clint's very helpful transcripts is that an AI DM will not only make mistakes, but it will be unable to fix those mistakes. Human DM's make mistakes, including continuity errors about NPCs. But they can smoothly fix them so that they don't derail the adventure. So if the DM said the owner of the brewery was Agatha and a player pointed out that the DM had previously said the owner was Calzon, the DM would just say, "Oh, yeah, it is Calzon. You should talk to Calzon". Or maybe they would say that Agatha is Calzon's wife and so they both own the brewery.

But AI can't do that. Instead we get this:

"The owner of the brewery is Agatha."
"I thought you said the owner was Calzon?"
"No, the owner is Gundren Rockseeker."
"What???"

The bottom line is that computers are absolutely, totally DUMB. They don't think at all, they just react. This is fine for a lot of well defined tasks, but for many things it's terrible because a purely reactionary methodology is extremely limiting.
 

Remove ads

Top