LLMs as a GM

[Scott Christian]

Thanks, Scott. I think we’re very much aligned on what the tools can do—though I’d caution you on one point. Summaries aren’t a solution to the memory problem. They’re a workaround—and only a partial one.

Adding more detail to summaries doesn’t preserve more information. It just increases the cognitive and processing burden the LLM has to carry in-session. The model doesn’t “retain” the summary—it reinterprets it at runtime, consuming tokens and model capacity to make sense of it on the fly. The more verbose or intricate the input, the more pressure it puts on that process. Something has to give.

The trick isn’t more detail. It’s compression with clarity. Summaries need to be:

• High-signal, low-noise
• Prioritized by what the model actually needs to re-reference mid-play
• Consistently reinforced through structured prompts or schema

Otherwise, you’re just shuffling memory around and hoping the system doesn’t drop anything important. It will. LLMs don’t have a memory problem—they have a token economy. You’re renting attention with every message. Spend wisely.

That said, all is not lost. One of their best traits is its flexibility—when it forgets, you can just remind it. It doesn’t argue. It doesn’t resist. It adapts in real time and moves forward. You can always steer the conversation and reinforce what matters, and it will follow. That kind of responsiveness isn’t perfect, but it’s powerful—and more than enough to make it work.

Thanks for that information. It's excellent to learn from someone who has more experience than I do. I feel like the token problem is actually a good reason to keep the sessions size limited and the number of sessions limited. At least, that was what I was thinking while reading you answer. Thanks again for the help. Greatly appreciated.

 

[Scott Christian]

@Jacob Lewis While you are on this thread, any chance you have a good format to use for the frontload documents. This is the format I have been using for the frontload document of the characters.
# Characters for Campaign

## First Adventurer

### Overview: Name: Emme Barnes. An Allorian who follows the Way of Elements. Hit Points = 10. She is 5’4”, 125 pounds, 22 years old, has hazel eyes, and brown skin. She was born to a family of professors, who encourages her intellectual curiosity. She was surrounded by books. Unfortunately, at age 15, her mother had an affair, and the family split apart. The affair was a product and scheme of the Central Kingdom’s thieves’ guild to try and blackmail her for money.

### Skills

• Break/Lift/Jump Simple 5; Hard 10; Epic 15
• Swim/Climb Simple 5; Hard 10; Epic 15
• Sneak/Balance Simple 2; Hard 7; Epic 12
• Resist Simple 5; Hard 10; Epic 15
• History/Religion Simple 2; Hard 7; Epic 12
• Creatures/Nature Simple 5; Hard 10; Epic 15
• Mechanics/Perception Simple 5; Hard 10; Epic 15
• Persuasion/Performance Simple 5; Hard 10; Epic 15
• Arcana Simple 1; Hard 6; Epic 11

### Potions (Emme has one of each.)

• Spider Climb - Auto succeed on any climb skill for one minute
• Hag’s Blood - Breathe underwater for 1 hour; rolls all swim challenges with boon
• Cat’s Eye - Target is able to see in the dark for 1 hour; rolls all perception challenges in the daylight with bane while the potion is in effect

### Experience Exploits

• Warp: When casting a spell, you may change the token type. For example, you may change “simple defense roll” to “simple damage roll.” You may not change the level of difficulty, just the token type.
• Transmutation: When casting a spell, you may change its elemental type. For example, you can change fire to lightning or wind to earth.

### Emme’s Card Summaries (Key Cards in Play):

• Arc Lightning: Type: Elemental. Base Defense:0, Luck:0, Movement:30, Damage:1. Effect: On a successful simple damage roll, you may choose to apply the new tiers damage to a separate target. (First target takes 2, second target takes 2, third target takes 3.)
• Earthen Shield: Type: Elemental. Base Defense:1, Luck:0, Movement:30, Damage:0. Effect: On a successful simple defense roll, increase the defense bonus of the hard tier by +2.
• Tornadic Fury: Type: Elemental. Base Defense:1, Luck:0, Movement:30, Damage:0. Effect: On a successful simple defense roll, increase the base movement of this card by 10 feet. This movement counts as flying. If the movement ends while in the air, falling immediately occurs and falling damage is taken. (1 point of damage per 10 feet fallen.)
• Spitting Flames: Type: Elemental. Base Defense:0, Luck:0, Movement:30, Damage:1. Effect: On a successful simple damage roll, the target must succeed on a forced hard nature challenge or be Stunned.
• Counterspell: Type: Elemental. Base Defense:0, Luck:1, Movement:30, Damage:0. Effect: On a successful simple luck roll, you may attempt a single arcana skill challenge.

### Equipment

• Silver and gray robe
• small wand
• Pouch with chalk
• Tan boots
• A baguette stuffed with cheese in a wrapped clean cloth
• A small pouch
• Three potions (see above) each wrapped in thick cloth
• 8 golden suns, 30 frostmarks

## Second Adventurer

### Overview: Name: Elias. A Vling who follows the Way of Guile. Hit points = 10. He is 5’5”, 145 pounds, 24 years old, has gold eyes, and milky white skin. He was born into a family of tinkerers. The parents made a good living until the Central Kingdom’s thieves’ guild stole several parts they had been working on and replaced them with faulty replicas. This caused one of the bells at the church of Airee to collapse, killing several people. Since then they have been shunned. Elias has since dedicated himself to thieving arts, hoping to one day get a chance at revenge.

### Skills

• Break/Lift/Jump Simple 5; Hard 10; Epic 15
• Swim/Climb Simple 3; Hard 8; Epic 13
• Sneak/Balance Simple 0; Hard 5; Epic 10
• Resist Simple 5; Hard 10; Epic 15
• History/Religion Simple 5; Hard 10; Epic 15
• Creatures/Nature Simple 5; Hard 10; Epic 15
• Mechanics/Perception Simple 2; Hard 7; Epic 12
• Persuasion/Performance Simple 5; Hard 10; Epic 15
• Arcana Simple 5; Hard 10; Epic 15

### Tools (Emme has one of each.)
* Thieves’ Tools - Allows Elias to roll with boon on any mechanics/perception challenge

### Experience Exploits

• Target: When attacking, add one of the following effects: vulnerable, stunned, hobbled, paralyzed, or weakened.
• Haste: At the beginning of combat, you take the first initiative.

### Elias’s Card Summaries (Key Cards in Play):

• Leather Life: Type: Light Armor. Base Defense:1, Luck:0, Movement:30, Damage:0. Effect: On a successful simple defense roll, increase the defense bonus of the hard tier by 1. If you succeed on an epic defense roll, you gain 2 luck.
• Better Lucky Than Good: Type: Light Weapon. Base Defense:0, Luck:1, Movement:30, Damage:1. Effect: On a successful hard luck roll, increase the luck bonus of each tier by +1.
• The Acrobat: Type: Light Weapon. Base Defense:0, Luck:0, Movement:30, Dam:0. Effect: On a successful simple movement roll, you may attempt a single break/lift/jump or swim/climb challenge.
• Shadow Strike: Type: Light Weapon. Base Defense:0, Luck:0, Movement:30, Damage:1. Effect: On a successful simple damage roll, the target must succeed on a forced hard perception challenge or be distracted.
• Lucky Shot: Type: Thrown. Base Defense:0, Luck:1, Movement:30, Damage:1. Effect: For each successful luck roll, you may choose to split up your luck between the same number of allies. The allies must be within 60 feet of you.

### Equipment

• Leather armor
• Rapier
• Three throwing daggers
• Gray cloak, gray shirt, tan pants, and black boots
• Thieves’ tools
• Flint and steel
• Bullseye lantern
• 25 feet of string
• 3 tiny bells the size of a finger
• 22 golden suns, 12 frostmarks

Any thoughts? Does this model work or is there a better way to structure it for readability purposes? I originally placed it in a spreadsheet, but it got quite confused. This has worked a lot better, and in fact, I haven't really had any problems with it (yet).

 

[Theory of Games]

For a long time now folks have derided videogames as not being able to GM an RPG the way a human can. As in adapt to choices of a player and create from whole-cloth new material A.I. might be able to change that perception.

Okay but what choices are missing from games like the Elden Ring or Baldur's Gate or Civilization series? What don't those games allow you to do that you feel should bee in the game?

I do think this is a little different than a typical computer RPG. It is really a new frontier. I think that is what drove me to do it. To, you know, see the hype. I also wanted to see if I could teach it to think within certain parameters. In this case, game mechanics and the rules associated with them. That interests me from a work standpoint, as I might be able to translate some of this skill to my job. Lastly, as much fun as I have creating lore and running games, it is interesting to see what another person's take on my lore is. I get the same insight from having this use my lore as I have watching another gamemaster use my world. It is kind of neat and fun to see what they craft with it.

You say "new frontier" but an LLM is a computer program so ... you're trying to play a video game. Nothing new really. Also, based on the feedback from posters here who've tried "LLM as GM", it seems more of a step backwards from trad video games since your LLM programs are forgetting the data you're feeding them.

But, I get the curiosity that drives people to LLMs

 

[Jacob Lewis]

@Jacob Lewis While you are on this thread, any chance you have a good format to use for the frontload documents. This is the format I have been using for the frontload document of the characters.

<snip>

Any thoughts? Does this model work or is there a better way to structure it for readability purposes? I originally placed it in a spreadsheet, but it got quite confused. This has worked a lot better, and in fact, I haven't really had any problems with it (yet).

You're definitely on the right track. Honestly, if what you're doing is working for you now, then you're where you need to be.

That said, I’ve taken it a few steps further by formatting information in a way that more closely resembles machine-readable structures—something like JSON. It’s not code, but it follows the same principles: clean labels, consistent structure, and tightly scoped data. The reason this works so well is because LLMs are trained on structured data formats like this. Parsing patterns, fields, and schemas is core to how they function.

Think of it this way: we’re not writing for a human reader anymore. We’re writing for the model itself. So speaking in its “native language”—structured, minimal, predictable—makes it easier for it to recall and use the information correctly.

If you're curious what that looks like, I converted your character sheet into a format I typically use when prepping for LLM-driven sessions. Just be aware, my LLMs are already trained to understand and work with this kind of format. They know it's meant for them, not me. So don’t expect magic if you just paste it in without context—but I'd genuinely be curious to see how yours handles it without additional instruction.

 

[LordEntrails]

Did you do a dedicated dataset outside the LLM for this? After a while, it will forget your original upload, at least that has been my experience. It might take a while, but it will happen unless you have a dedicated dataset for it to utilize a retrieval system for. (I think that is called RAG.) I only ask because I made the mistake of walking through about a hundred creatures with it. It offered great advice, but when we got to the end, it couldn't tell me anything about the first twenty creatures we discussed. And, after covering lore and cultures, it forgot the creatures all together. So you might want to keep adding to the Notebook for that purpose. Just a thought.

Yep, so any info that you want to keep you have to copy and paste and add it to the LLM notebook. Otherwise yes it will forget what it told you awhile ago (like most AIs). I have just gotten Gemini Pro and with it you can tell it to save specific information. But not sure how that works yet and it's only been a few days and I have not gone back to try and have it recall what it saved for me.

 

[LordEntrails]

Consistently reinforced through structured prompts or schema

Could you elaborate and provide some examples please?

 

[Scott Christian]

You're definitely on the right track. Honestly, if what you're doing is working for you now, then you're where you need to be.

That said, I’ve taken it a few steps further by formatting information in a way that more closely resembles machine-readable structures—something like JSON. It’s not code, but it follows the same principles: clean labels, consistent structure, and tightly scoped data. The reason this works so well is because LLMs are trained on structured data formats like this. Parsing patterns, fields, and schemas is core to how they function.

Think of it this way: we’re not writing for a human reader anymore. We’re writing for the model itself. So speaking in its “native language”—structured, minimal, predictable—makes it easier for it to recall and use the information correctly.

If you're curious what that looks like, I converted your character sheet into a format I typically use when prepping for LLM-driven sessions. Just be aware, my LLMs are already trained to understand and work with this kind of format. They know it's meant for them, not me. So don’t expect magic if you just paste it in without context—but I'd genuinely be curious to see how yours handles it without additional instruction.

That is a great example. Thank you so much!

 

[MNblockhead]

I've played around with ChatGPT project to have it run D&D5e and WFRP4e games for me and three artificial players (personas). It was surprising good and fun and a good way to familiarize yourself with new rulesets. I have no interest in using it much for actually playing. First, I don't enjoy text based gaming, whether ZORK, text-based MUDs or play by post, or live chat. But eventually we'll have well performing voice and video interaction. Second, it is the interaction and co-creation with other real people that is the most important aspect of TTRPGs for me.

Two areas where AI can enhance my games without detracting from what I most like are:

1. Rules Look Up and generally finding and summarizing rules or content quickly.
2. Session summaries. I've been tempted to move from Discord to Google Meet for my oneline games so that I could have the transcript recorded. Not that I would ever want to read over the transcript but then I could have Gemini create a session summary.

I have used AI to generate NPC art, which is handy, but I don't use a lot of art in my games otherwise. When it comes to maps and battlemaps, I find it take longer to try to create anything useful versus finding existing maps.

I also use ChatGPT to help me create macros and random name generators.

 

[LordEntrails]

Session summaries. I've been tempted to move from Discord to Google Meet for my oneline games so that I could have the transcript recorded. Not that I would ever want to read over the transcript but then I could have Gemini create a session summary.

Check out www.sessioneer.cc
I haven't started using it yet, but do have it linked to my Discord.

 

[Jd Smith1]

You say "new frontier" but an LLM is a computer program so ... you're trying to play a video game. Nothing new really. Also, based on the feedback from posters here who've tried "LLM as GM", it seems more of a step backwards from trad video games since your LLM programs are forgetting the data you're feeding them.

But, I get the curiosity that drives people to LLMs

LLM/AI is at the stage video games were in 1990. They were fun, but not a serious challenger to TTRPGs or traditional war games played F2F.

35 years later, video games are huge and still growing, and online gaming is steadily gaining traction.