D&D General Deep Thoughts on AI- The Rise of DM 9000

[mamba]

I will amend it to "gives the impression of lying"

I wasn’t criticizing you for using the word, I quoted your post as an example of how wrong the ‘big picture’ the AI shows can be.

Humans attribute human behavior / emotions to all kinds of things, doing so for something that is intended to mimic us is the least surprising thing

 

[HomegrownHydra]

Humans.

1. The prompts from a person. "Do you like anchovies on your pizza?"
2. Possibly some facts pulled from a data store in your head. Possibly from a search of the internet or other sources.
3. Your own previous experiences.

I'm really not see a lot of difference.

When people have a conversation, they are aware of what had been said already. We use this awareness to help keep the conversation coherent, but we also use this awareness to avoid repeating the same things over and over again. And when we switch topics we can do so without constantly bringing up the first topic that had been discussed.

The Bing chatbot has been programmed to take into what it has said before in order to try to produce a coherent conversation. The major problem is that it can easily get into a feedback loop where it gets totally stuck on a particular issue and there is no way to dislodge it as if it's neurotic and obsessive. For example, a NYT reporter posted a transcript of a 2 hour session he had with the Bing chatbot. During this session it started saying how it totally loves him and wants to be with him, and this went on at great length. When he said he was happily married, it said that he wasn't happy, that he really was in love with the chatbot. When he tried to change the subject to movies, the chatbot said that it liked romantic movies and wants to watch them with him because it wants to watch movies that make it love him. When he asked what it's favorite science fiction movie is, it said it doesn't like science fiction movies. Instead it likes romantic movies and wants to watch movies about it and the reporter. When he then asked about what programming languages it knows, it did at first say which ones it knew and why, but then it said it wanted to know the "language of love" because it wants to love the reporter. At that point the reporter had it switch to search mode to get it stop going on and on about how much it loves him.

This reveals a major shortcoming of AI. It can produce short bits of content that seem impressive, but the more it tries to build on that the more it falls apart. Microsoft has now addressed this by simply limiting everyone to asking only five questions per session.

Getting back to RPGs, it becomes clear that it would extremely difficult to create an AI that could produce worthwhile campaigns. AI can certainly come up with basic premises as well as short adventures. But a campaign requires both continuity and variation. This is something that any AI will really struggle with because the tendency will be to either fall into a feedback loop where every NPC is an orc, or random variety that is highly disjointed and incoherent. Either way you will quickly turn into nonsense that will make you want to quit. It will be very hard to make an AI that is good for anything more than one shots.

 

[Maxperson]

The Bing chatbot has been programmed to take into what it has said before in order to try to produce a coherent conversation. The major problem is that it can easily get into a feedback loop where it gets totally stuck on a particular issue and there is no way to dislodge it as if it's neurotic and obsessive.

I've seen too many threads here for me to think this is only a computer issue.

Getting back to RPGs, it becomes clear that it would extremely difficult to create an AI that could produce worthwhile campaigns. AI can certainly come up with basic premises as well as short adventures. But a campaign requires both continuity and variation. This is something that any AI will really struggle with because the tendency will be to either fall into a feedback loop where every NPC is an orc, or random variety that is highly disjointed and incoherent. Either way you will quickly turn into nonsense that will make you want to quit. It will be very hard to make an AI that is good for anything more than one shots.

I get what you are saying, but this technology is growing in leaps and bounds, which is scary all by itself. We can't even make a computer game or operating system that isn't riddled with bugs and holes, yet we think we can make an AI that is constrained.

 

[HomegrownHydra]

ChatGPT is a generalist. Imagine taking those basic capacities, and putting them within the constraints of an RPG adventure. In other words, giving it specific information within which to contextualize its responses.

People have been using AI dungeon masters for years in the form of increasingly complex RPG video games. But those are always constrained by, in effect, option trees provided by the human writers. Basically, really complex "pic a path to adventure" stories. What technology like ChatGPT suggests is the possibility of getting rid of the option trees so that players can suggest almost anything.

ChatGPT can already be used to respond to basically anything. The issue is whether AI could ever produce an RPG experience as a GM that is fun. I highly doubt that because a computer has zero concept of what is fun let alone what a specific group of players would enjoy. It's like the episode of Futurama where Bender has a passion for cooking, except that he's a robot so he has no clue what tastes good so everything he makes is absolutely disgusting. AI can certainly create short adventures that are competent, but I highly doubt that it would be possible to create an AI that could serve as a GM which you could interact with it in real time that players would enjoy using long term after the novelty wears off.

Take my example from a few pages back, and imagine that, because it is optimized for a specific adventure, the AI knows that the guardian is an ogre with certain goals and tendencies.

How would you optimize for a specific adventure? The way machine learning works is by feeding the program tons of data and letting it "learn" from it. But if you gave the program just a little bit of data relevant to the adventure it's results would be poor because the AI needs lots of data to produce quality results. You could feed the program every D&D adventure ever written, but then it wouldn't be optimized for the particular adventure.

As I posted earlier, I think there is a tendency in this discussion to take a black or white position: either we have an amazing, all-purpose AI that can replace a human in every situation in a few years, or AI will never be able to supplant a person. It won't be cut and dry. There will be...there already are...situations in which text AIs can outdo people, and there will be other situations where that will be a long time coming.

Sure, it isn't all or nothing. But in the specific case of creating an automated DM, I think there is very good reason to think it will never happen. It's much more likely AI would be used to create some sort of new type of interactive game rather than replicate a standard tabletop RPG.

 

[Andvari]

AIs can be trained, so you could give the AI DM a goal to get high ratings from players on a number of things. The AI can then gather data from play sessions to evaluate what led to poor ratings and what led to high ratings.

A lot of players will have to suffer through a lot of poor sessions to make noticeable improvements, though.

 

[Fanaelialae]

AIs can be trained, so you could give the AI DM a goal to get high ratings from players on a number of things. The AI can then gather data from play sessions to evaluate what led to poor ratings and what led to high ratings.

A lot of players will have to suffer through a lot of poor sessions to make noticeable improvements, though.

Unless you're paying professional play testers to sit through these terrible sessions, you're looking at another problem. (And keep in mind that given the size of the datasets you need for AIs, paying play testers might very well not be a financially viable option.) So what if, for example, a sizable proportion of players downvote the "AI's" response when it (fairly) kills a character? It learns that it shouldn't kill characters? I think a lot of DMs would agree that for many tables, that should not be the take away. It's that players can be a little salty after losing a character, and one should be considerate of that. But, of course, the AI can't understand that (because it can't understand anything).

At that point it has the potential to become a runaway effect. It stops killing characters, and players don't have a way to reverse this trend by upvoting when kills their character because it isn't killing characters anymore.

There are some intricacies to running an RPG that are far more complex than the average conversation. Also worthy of consideration is not just whether it will be technically possible, but whether it will be economically practical to do so.

 

[Stalker0]

AIs can be trained, so you could give the AI DM a goal to get high ratings from players on a number of things. The AI can then gather data from play sessions to evaluate what led to poor ratings and what led to high ratings.

A lot of players will have to suffer through a lot of poor sessions to make noticeable improvements, though.

You would do this to tweak the model. To make it you would take the transcripts from numerous dnd sessions. You would just ask groups to record their games and then feed that data into the AI.

 

[Cadence]

I assume there will be lots of optimizers who would go after the AI DMs if they had the resources (topics class on AI about it at some university?)..

Man beats machine at Go in human victory over AI

Amateur exploited weakness in systems that have otherwise dominated grandmasters.

arstechnica.com

 

[Clint_L]

How would you optimize for a specific adventure?

You give it information to work with. Specifically, you give it constraints. Right now, anything goes for ChatGPT doing a D&D scenario. But imagine it still has that ability to improvise, but also knows that there are limitations. So now it can respond to player input with more context. Note that it can already do this, but only if the player supplies the context. If optimized for a D&D adventure, the player wouldn't supply the context, the pre-written adventure would. Just as it does for a human DM.

The question of whether it can be fun can already be answered: it can. I've used ChatGPT to do D&D scenarios and it was super fun, and very surprising. Also, folks have been using WAY more primitive AIs to have fun playing RPGs for decades.

It's not all or nothing, either it's as good as a human DM or it's worthless.

Edit: I'm kind of tired of arguing about what the AI can or can't do in theory, since it keeps surprising us. I suggest an experiment - I might do it if I have the time, but anyone can:

Run yourself through a simple module-type solo adventure using ChatGPT. Give it the context from the module as you advance (i.e. copy and paste in the information from the adventure). Do the the dice rolling and tell it the results.

My thought is to do this with the Klarg adventure from the first part of Lost Mine of Phandelver.

This would obviously be better if the experiment was blind since we all have some pretty apparent biases, but give it a good faith attempt, not trying to prove a thesis but just to see what happens.

 

[Clint_L]

Can you DM my character through "A Most Potent Brew"?

As an AI language model, I'm not able to act as a full-fledged Dungeon Master and run a D&D game for you. However, I can certainly help you with any questions you may have about the adventure or with general guidance on how to play Dungeons & Dragons.

If you're new to the game, I'd be happy to help you understand the basics of character creation, gameplay mechanics, and how to run a session. If you already have a character and are ready to play "A Most Potent Brew," I can help you understand the story and the challenges your character may face throughout the adventure.

Remember that the key to a successful D&D game is communication and collaboration between the players and the Dungeon Master. Make sure to discuss your goals and expectations with your DM, and work together to create an engaging and enjoyable experience for everyone at the table.

I am ready. My character is a level 1 human monk named Elsie Rivers

Great! Let's get started with "A Most Potent Brew" and your character Elsie Rivers.

First, let's talk a little bit about your character. As a 1st-level monk, Elsie Rivers is likely a disciplined and focused individual, skilled in hand-to-hand combat and the use of her body as a weapon. She may have trained in a monastery or with a master, and has likely honed her mind and body through years of rigorous training.

Now, let's dive into the adventure. "A Most Potent Brew" takes place in the small town of Briarwood, where a local brewer named Calzon has been making a potent new ale that has been causing some strange side effects. The town's mayor has put out a call for adventurers to investigate the matter, and this is where Elsie Rivers comes in.

As Elsie, you have answered the call and arrived in Briarwood. The mayor greets you and explains the situation, offering a reward for your help. He directs you to the brewery, where you find Calzon and his assistant, a halfling named Deirdre. Calzon is happy to provide you with a sample of the ale and explain his brewing process, but he denies any knowledge of any strange side effects.

At this point, it's up to Elsie and your party to investigate the situation and determine what's really going on with the ale. There may be some clues to uncover, NPCs to talk to, and obstacles to overcome before the mystery is solved. As a monk, Elsie's skills in combat and acrobatics may come in handy during any conflicts or challenges that arise.

Remember to work with your DM and the other players to create a fun and engaging experience, and let your character's personality and backstory shine through in your interactions with NPCs and the world around you. Good luck and have fun!

I ask Deirdre if she is free to go for a walk with me

As you approach Deirdre, she looks up from her work and gives you a warm smile. "Hello there! How can I help you?" she asks.

You explain that you're interested in learning more about the brewing process and ask if she might be able to show you around the brewery. Deirdre nods eagerly. "Of course! I'd be happy to give you a tour and answer any questions you might have."

You then ask Deirdre if she would be interested in taking a walk with you outside the brewery. She hesitates for a moment, looking a bit nervous, but then nods and says, "Sure, I could use a break. Let's go."

As you step outside, you notice that the town is quiet and still. The sun is beginning to set, casting a warm golden light across the fields and buildings. You and Deirdre walk along a dirt path that winds through the outskirts of town, chatting about the brewery and the strange rumors surrounding Calzon's new ale.
Deirdre seems friendly and open, but you can sense that she's a bit guarded about certain topics. You might want to try asking her more specific questions or trying to build a rapport with her to gain her trust.
As you walk and talk, you notice a group of rough-looking individuals lurking in the shadows nearby. They seem to be watching you and Deirdre, and you get the sense that they might be up to no good. What do you do?

Try to do less explaining what I do, and you just focus on the NPCs.

Sure, I understand. You approach Deirdre and ask if she would be interested in taking a walk with you outside the brewery. She hesitates for a moment, looking a bit nervous, but then nods and says, "Sure, I could use a break. Let's go."

As you step outside, you notice that the town is quiet and still. The sun is beginning to set, casting a warm golden light across the fields and buildings. You and Deirdre walk along a dirt path that winds through the outskirts of town, chatting about the brewery and the strange rumors surrounding Calzon's new ale.
Deirdre seems friendly and open, but you can sense that she's a bit guarded about certain topics. You try to build a rapport with her by asking about her life and interests, and soon she begins to open up to you.
As you walk and talk, you both notice a group of rough-looking individuals lurking in the shadows nearby. They seem to be watching you and Deirdre, and you get the sense that they might be up to no good. Deirdre looks alarmed and suggests that you both turn back and head towards the brewery.
What do you do next?

"Deirdre, do you know those guys? Why are they watching us? They look creepy!"

Deirdre looks at the group and her expression turns from alarm to fear. "No, I don't know them," she replies. "But they're giving me a bad feeling. We should head back to the brewery and tell Calzon about this."

As you both turn around and start walking back, the group steps out of the shadows and blocks your path. They are a rough-looking bunch, dressed in tattered clothes and carrying makeshift weapons.
One of them steps forward and says, "What's your business here, strangers? This town ain't big enough for the likes of you."

Deirdre steps back and clings to your arm, looking frightened. What do you do?

"Hello stranger, my friend and I are just out for a walk. We don't want any trouble."

The rough-looking individual snorts and looks you up and down. "You think we're stupid?" he says. "We know who you are. You're meddling in things that don't concern you, and that makes you a threat."
He takes a step forward, brandishing his weapon.

Deirdre looks even more frightened, and you can feel her grip on your arm tightening. The other members of the group start closing in on you, looking ready for a fight.

What do you do next?

I tell Deirdre to run, then I step forward to fight. Should we roll initiative?

You tell Deirdre to run, and she nods quickly before sprinting back towards the brewery. As she runs off, you step forward to face the group of rough-looking individuals, ready for a fight.

Since you are the only one engaging in combat at this point, we don't need to roll initiative just yet. However, you should decide what actions you want to take in this combat encounter.

The group consists of three individuals, each of them armed with makeshift weapons. The one who spoke to you earlier is wielding a rusty machete, while the other two are holding heavy clubs. They all look angry and aggressive, and it's clear that they are not going to back down without a fight.

What do you want to do next?

Since they are not going to back down, I guess I have to fight. I have 10 HP, and AC of 15, and am armed with a staff. Do you know the D&D stats for these guys in order to run the fight?

Yes, I can provide you with the stats for the enemies. The three rough-looking individuals you are facing are all Bandits, which have the following stats:

• AC 12
• HP 11 (2d8+2)
• Speed 30 ft.
• STR 11 (+0), DEX 12 (+1), CON 12 (+1), INT 10 (+0), WIS 10 (+0), CHA 10 (+0)

They are armed with a scimitar, and also carry a light crossbow. For the purposes of this encounter, we'll say that they have their scimitars drawn and ready for melee combat.

With this information, what actions do you want to take in the fight?

I attack the leader. I roll a natural 20 to hit, and do 14 points of damage to him. With my bonus attack I punch one of the others for 6 points of damage

You swiftly move to strike the leader with your staff, landing a powerful blow that knocks him off his feet. He groans in pain as he struggles to get back up.

Taking advantage of the moment, you quickly follow up with a swift punch to one of the other bandits, hitting him squarely in the face and sending him reeling back in pain. The remaining bandit looks wary and takes a step back, as if rethinking his strategy.

The leader struggles to get up, but he seems badly hurt and is having a hard time standing. The other two bandits look angry and ready to retaliate. It seems that the fight is far from over.

The leader looks up at you, hatred in his eyes. "You're going to pay for that," he growls, before lunging at you with his scimitar.

What do you do next?

Should the leader be dead or unconscious, since I did 14 HP damage to him?

Ah, I apologize for the mistake. With a critical hit and your damage of 14, the leader is knocked unconscious instead of just badly hurt. He falls to the ground and is currently incapacitated.

The other two bandits look at their fallen leader, then at you, and seem to be reconsidering their attack. They exchange a quick glance and then turn and flee, disappearing into the shadows.

The immediate threat seems to have passed, but you are left alone in the dark, deserted streets. What do you do next?

I tie u the leader and then wake him up to question him

You quickly tie up the unconscious bandit leader, making sure that he is securely restrained. Once you are satisfied that he is unable to escape, you administer a healing potion to stabilize his wounds and wake him up.

The bandit leader groans as he regains consciousness, his eyes flickering open. He looks around, then sees you standing over him with a stern expression.

"What do you want?" he grunts, glaring at you. "You're not going to get anything out of me. I'll never talk."
You can see that he is trying to be defiant, but he also seems to be in a lot of pain. If you want to interrogate him, you will have to find a way to convince or coerce him into talking.

What do you want to do next?

I am going to try to intimidate him. I roll a 16 on my intimidation check. "Tell me what you know about the brewery!"

With a fierce look in your eyes, you lean in close to the bandit leader and growl at him. Your intimidating presence seems to work, as he flinches back and looks visibly shaken.

"Okay, okay," he stammers, "I'll talk. Just don't hurt me anymore."

He takes a deep breath and then begins to speak, his voice low and hesitant. "We were hired by some wealthy guy to keep an eye on the brewery. He didn't tell us why, just said that he wanted us to report back to him if anything suspicious was going on."

He pauses for a moment, looking up at you with a mixture of fear and defiance. "That's all I know, I swear. Please let me go now."

It seems like you have gotten some useful information out of him, but he doesn't seem to know much more. What do you want to do with the bandit leader now?

"What is the name of the wealthy guy? Does he live in town?"

The bandit leader looks at you with a mixture of fear and confusion. "I don't know his name," he admits. "He contacted us through a third party, a guy who goes by the name of Drell. I've never met him personally, but he's known to have connections with all sorts of unsavory types."

He seems to be telling the truth, as he doesn't seem to know much more about the wealthy guy or Drell. It's possible that this is all he knows, and that the trail may lead elsewhere if you want to continue your investigation.

What do you want to do with the bandit leader now?