Will AI replace Dungeon Masters?

[Reynard]

In case anyone missed it, the author of the OP offers just such a service, or aspires to. See the "Source" link at the very bottom of that post.

What, you have never had a fruitful discussion inspired by a commercial?

 

[Cergorach]

This reads like it was written by ChatGPT. Will forum posts be replaced with AI?

They already have been.

 

[Reynard]

They already have been.

I have been training an open source LLM exclusively on the work of @Snarf Zagyg

 

[Celebrim]

I suppose AI could run a prewritten module if it knew all of the PCs' capabilities. Can it improvise when a player tries something off the wall?

Yes. The problem is current generation AI has no memory and the way it fakes memory is highly unreliable. So the problem with playing a game of any sort with a large language model whether it's chess or an RPG is that very quickly the AI loses track of the game state or even that it is a game or what its role in the game is.

 

[Professor Murder]

No, I'm serious. Mods, why is this allowed? This is just a copy/paste from an AI site with no attribution.

 

[Cergorach]

Yes. The problem is current generation AI has no memory and the way it fakes memory is highly unreliable. So the problem with playing a game of any sort with a large language model whether it's chess or an RPG is that very quickly the AI loses track of the game state or even that it is a game or what its role in the game is.

Depends on how you define LLM memory. You either train on it, use RAG, or use the existing context window.

Most of the mainstream LLMs offer context windows around the 128k token size, which should fit about a novel. One of Google's LLMs offerts a million tokens worth of context window (~7.5 novels). And there's one odd duck that offers 100 million tokens context window (~750 novels).

But it's kind of inefficient of logging all the interactions of the users. Humans tend also not remember all the interactions in exact detail, certainly not a week later. You could easily summarize the interactions, something certain LLMs are very proficient in. Thus making a novel a LOT shorter.

You could also log all the interactions in independent files and RAG them, which pretty much allows the LLM to reference the documents without being trained on them.

You could also train the LLM on last sessions interactions for next weeks session.

OR you could do all three....

And while this might take enormous resources if you did this on mainstream generic models, this would be hugely expensive. But keep in mind, that for a specific pnp RPG, the LLM doesn't need all the knowledge in the world/Internet. If you use multiple, highly specific LLMs and chain them together you could get very efficient, especially with what the DeepSeek people have made publicly available.

25 years ago the state of VTT was abysmal compared to what it now is. Some people make LLM out to be the 'ultimate solution' tomorrow, others see it as a useless evil. The reality is somewhere in between and certainly not tomorrow or next week. But people have been building some really interesting stuff with LLM and am very curious how that will evolve in the future.

 

[GreyLord]

AI DMs may replace people or human DMs in the future to a limited degree, but I think you'll always have some pushback against AI being the DM by some people and some groups.

A prime example is the FFG "boardgames" these days that require the use of a phone or tablet in order to play. It is normally a group game with an RPG or tactics based focus (similar to D&D), but instead of a game master controlling the enemy, they have the program on the phone or tablet.

Personally, I hate these programs. Part of the reason to play boardgames is the person to person interaction, and whether it is a player browsing on their phone or being distracted by it, phones just detract in general when at the table. There still needs to be someone who is using their phone to see what the AI is prompting to happen on the board, and a person to move the items on the board, but they also are now interacting with their phone just as much, if not moreso, than they are with everyone else at the table.

I normally have a strict no phones policy when we play. Obviously, with this type of boardgame that is impossible, and as someone has to use their phone so we can play the game in the first place, others start using their phones (sometimes out of boredom as they wait for that one individual to interact with their phone and move the AI's thngs around). It's beyond annoying and these types of games have no charm for me. I actually sort of abhor these types of boardgames.

Someone must find them popular though as I see these FFG games (Legends of the Dark I think is what the more recent one is called) all the time for sale in game shops.

In that light I think there eventually will be AI DMs and they will be immensely popular in some groups. At the same time you'll have those who absolutely hate and detest them such as I probably would. In that light, even if you have those AI DMs, you'll always have those who prefer to have that human interaction rather than a computerized one.

 

[Celebrim]

Most of the mainstream LLMs offer context windows around the 128k token size, which should fit about a novel. One of Google's LLMs offerts a million tokens worth of context window (~7.5 novels). And there's one odd duck that offers 100 million tokens context window (~750 novels).

The problem with my experience is in how it scans back through the context to find the relevant information. It seems to be optimized to reference the most recent plausible context out of the sea of past interactions using relatively simple pattern matching (take context from areas with matching words prompts). And as with all LLMs, it has a tendency to hallucinate if it can't find the right context (fast enough?).

You could easily summarize the interactions, something certain LLMs are very proficient in. Thus making a novel a LOT shorter.

One of AI's biggest limitations I've noticed is that's attempt to summarize frequently inverts the meaning. Like me when I'm writing fast, it often leaves out negators. So it will write something like, "I did not go to the store" as the summary of "I did go to the story". It also is terrible about connecting ideas to the wrong persons. So if you ask it to summarize a hypothetical debate between Keynes and Hayek, don't be too surprised if it misattributes some statements to Keynes that belong to Hayek and visa versa. I don't really trust any AI summary at this point.

I agree that there is some future out there at some point where we have an AI GM, but I think LLM's are going to be just one part of a potential solution and that LLM's are unlikely to get there on their own. And as usual, we're still probably "25 years" away from functional AI.

 

[Cergorach]

The problem with my experience is in how it scans back through the context to find the relevant information.

Not fast enough shouldn't be an issue. Hallucinations are a problem, always! There are some methods to limit that, but they tended to be complex and compute intensive. As relative compute goes down (faster hardware on less heavy LLMs), the people making LLM systems should be able to tackle that.

One of AI's biggest limitations I've noticed is that's attempt to summarize frequently inverts the meaning.

Are you using the right LLM for the job?

And as usual, we're still probably "25 years" away from functional AI.

What kind of AI are you talking about? That robot from the Jetsons? Or just another program/tool/service that does what it does reasonably well? That is already happening, as long as you keep your expectations realistic and don't take your experiences with a few LLMs and project that on all LLM tools/services. 'Perfect' AI, that's probably not happening in 25 years either. And what's good enough for you is somewhere in de middle and that also depends on what is good enough for you.

AI DMs may replace people or human DMs in the future to a limited degree, but I think you'll always have some pushback against AI being the DM by some people and some groups.

That kind of depends on the expectations of the group. Certain people playing pnp RPGs back in the day look at cRPGs in the same way, never going to happen, never going to replace the <insert xyz>. But we've been playing cRPGs for a long time! Even played D&D (Champions of Krynn) 35 years ago on my Amiga 500 and even earlier on my C64 (Pool of Radiance)... Later certain people of a newer generation thought the same about the success of MMORPGs...

For boardgames, the old Descent 1e+2e had one of the players be the 'DM', something not everyone wanted to be. If we wanted to RPG, we would play D&D (or Vampire or Shadowrun, etc.). Having our entire group be players against something else was great! And it might be a mobile app, it doesn't have to be, Gloomhaven works without an app. Depends on the game, depends on the players, depends on the mood, if we would want to play a game with an app. For Gloomhaven (Lion something-something) we used the sound app, so one of us didn't have to read a bunch of text aloud. Was very cool, very atmospheric! When the text reading was over, the phone went away. I also hate people messing on their phone or with anything else when we're playing a game, any game.

I personally use AI/LLM in prep. From making an interesting description of a room from some basic input, to converting it from text to voice, to generating images. That leaves me to do more interesting things and just upping the experience for everyone.

 

[Queer Venger]

"thou shall not create a machine in the likeness of the human mind"

long live the Butlerian jihad...