AI GMs and what players value

[Reynard]

NOTE: This thread is about the potential future and my premise makes some assumptions about what technology can and can't do. Feel free to are with the premise, but be aware my intent is not to discuss what AI GMs might be capable, but rather what players might accept or not from AI GMs.

Over the course of the pandemic, playing online increased dramatically. Unsurprisingly, demand for online GMs also increased. Professional GMing is a real thing (I've done it and have decided it isn't really for me; but I do think I should be paid to GM at cons; it's complicated) and I think LLM driven AI is a potential solution for the demand problem.

What I am interested in here and what I wonder a lot about is whether a AI GM service would be viable and whether some measurable subset of players would be willing to pay for some sort of AI service to run their games (as opposed to hiring a pro GM).

Let's establish a couple ground rules. First, we are going to presume that the AI GM is capable of some level of reactivity. This isn't a video game with just a few established choices, buy a generative AI that can emulate "creativity." Second, we are going to assume that the players involved are people also willing to hire a GM (so there is no expectation of friendship etc). Finally, we are going to assume that the AI GM is running a pre-written module. This last is blatantly fencing the conversation. I don't want to argue about whether the AI GM would "get it" if the players said they were leaving Toril for Krynn by way of Spelljammer.

So, with all that said, do you think players -- especially those willing to otherwise pay for a pro GM to come run their game -- would accept an AI GM so that none of them had to be GM?

I'd not, why not? If so, what do you think is the driving force? In either case, what do you personally think an AI GM would have to be capable of to be a successful service.

 

[aco175]

It seems that over the next few years AI DMs will become better at putting things together and developing a story, dungeon, and monsters to fill it. It will be able to create a broader story than I could and cobble NPCs together that all have quirks and abilities as well as track everything. It appears to be a big help to the real DM.

I'm not sure if AI can read the room so to speak and see who is tuning out and be able to change pace. To look around and think people are bored and throw a group of bandits at them to change pace.

I do see it being a good tool to help the DM. Maybe let the DM be a part time player in the game as well. I'm still waiting for the Ready Player One suit to let me play in the game.

Side note; we can have a poll on which AI voice to have as the DM. I'm torn between "I can not do that, Dave." and "Shall we play a game."

 

[Reynard]

.Side note; we can have a poll on which AI voice to have as the DM. I'm torn between "I can not do that, Dave." and "Shall we play a game."

They would certainly license famous voices. Playing Star Wars? JEJ coming online!

 

[Milieu]

As LLMs exist now, no. If you've tried AI Dungeon, that's sort of what you're talking about, and it can get pretty silly, pretty quickly. It will give responses that contradict previous ones, ignore context, or don't follow logically. It's more like playing D&D in a dream world where things happen that are kind of like things that happen in a fantasy game, but don't really fit together in a sensible way.

LLMs hallucinate all the time. If you ask ChatGPT about a fake historical event that you make up on the spot, it will probably make up an answer. If you ask it to write code, it will make up non-existent libraries to import, and then someone will upload a library with that name containing malicious code. If you ask it about legal cases supporting your client's case, it will make up legal citations to non-existent cases, as a couple of lawyers found out the hard way. If you try to play chess with it, it will break the rules and move its rook on top of its bishop or bring back a piece that was previously captured. This can be mitigated to some extent, but probably never eliminated. So, for example, even though it's trained on the specific module, by prompting it in a certain way, you could probably convince it to give out any magic item you want whether they're in the module or not, including ones that you just made up. Or you could be fighting a dire wolf and it suddenly "forgets" and describes you attacking the wyvern.

I am very skeptical that without a major new advancement (not just feeding it more training data), any LLM-based thing will be worthwhile as a semi-autonomous agent in any use case. As a tool used by a person for brainstorming/creating first drafts, sure.

 

[Reynard]

As LLMs exist now, no. If you've tried AI Dungeon, that's sort of what you're talking about, and it can get pretty silly, pretty quickly. It will give responses that contradict previous ones, ignore context, or don't follow logically. It's more like playing D&D in a dream world where things happen that are kind of like things that happen in a fantasy game, but don't really fit together in a sensible way.

LLMs hallucinate all the time. If you ask ChatGPT about a fake historical event that you make up on the spot, it will probably make up an answer. If you ask it to write code, it will make up non-existent libraries to import, and then someone will upload a library with that name containing malicious code. If you ask it about legal cases supporting your client's case, it will make up legal citations to non-existent cases, as a couple of lawyers found out the hard way. If you try to play chess with it, it will break the rules and move its rook on top of its bishop or bring back a piece that was previously captured. This can be mitigated to some extent, but probably never eliminated. So, for example, even though it's trained on the specific module, by prompting it in a certain way, you could probably convince it to give out any magic item you want whether they're in the module or not, including ones that you just made up. Or you could be fighting a dire wolf and it suddenly "forgets" and describes you attacking the wyvern.

I am very skeptical that without a major new advancement (not just feeding it more training data), any LLM-based thing will be worthwhile as a semi-autonomous agent in any use case. As a tool used by a person for brainstorming/creating first drafts, sure.

Again, per my OP, the point here isn't really to argue about whether this is imminent, but whather assuming it is possible if players will utilize AI GMs.

 

[ichabod]

So, with all that said, do you think players -- especially those willing to otherwise pay for a pro GM to come run their game -- would accept an AI GM so that none of them had to be GM?

Sure. The demand is out there, clearly. If you can make it work, people will pay for it.

 

[Moonmover]

I think such a thing is much more likely to be developed within the video game industry than in the RPG industry. There's more financial resources and more programmer talent there. And, video gamers are already accustomed to things like computer-generated environments (Minecraft), monsters, loot, (Diablo), and even small storylines (Skyrim).

 

[J.Quondam]

So, with all that said, do you think players -- especially those willing to otherwise pay for a pro GM to come run their game -- would accept an AI GM so that none of them had to be GM?

I'd not, why not? If so, what do you think is the driving force? In either case, what do you personally think an AI GM would have to be capable of to be a successful service.

I don't see why not. If the assumption is that the players are already willing to pay for an AI GM and if they already understand its capabilities and limitations, then I imagine they'll try it and either use it or not based on how well it lives up to expectations.
Heck, I could even see a human GM being interested in renting an AI as a co-GM or GM assistant to run a couple AI PCs to round out a small group, or to run some henchmen, certain monsters, less important NPCs, a rival party to the PCs', or whatnot.

I really think the question just boils down to the AI's capability. If it's "good enough", then people predisposed to pay for it will pay for it.

 

[Haiku Elvis]

Side note; we can have a poll on which AI voice to have as the DM. I'm torn between "I can not do that, Dave." and "Shall we play a game."

Both good choices but the correct answer for any question like this is Brian Blessed.