Following something mentioned in one of the other AI threads here, a couple weekends ago I asked ChatGPT to run me a solo campaign for D&D 2024. The character (one I am planning on playing for real in the next campaign I get to play) is 18-19 year old human woman, CG, Warlock going Great Old One patron. Concept being that she is a simple tavern keeper's daughter but something has tapped her and started giving her power. She knows/senses it comes from a dark source but if she spurns it, she goes back to being a normal ordinary person and she doesn't want that. She now has to handle the power and maintain her general good nature.
Chat has actually run a fairly interesting and different story than I would have expected. No traditional dungeons or the like. Instead it has woven a story of some dark bureaucratic power in the city who are using bureaucratic concepts to effect the city, especially the poorer quarters where the character lives. "Creatures" which exist just outside what can be seen which are deleting people so that they never existed, both physically and accountingly (made up word) - removing them from records so they haven't/can't pay taxes, own/start a business, be paid. It's weird and I'm not describing it very well but it's very clever and well made. It's not something that I would have considered or even thought of, but for the character concept it's oddly perfect for a solo campaign.
Thoughts on ChatGPT as the DM from this experience:
1. It has a good memory for what it has done but ever few hours I have to ask it to save the campaign so I can restart it in a fresh chat and continue because it slows down so quickly.
2. It has a very good grasp of D&D 2024 rules. I did catch it out once when it used a 2014 wording rather than 2024. It also pointed something out to me that I had not realised about my character build a couple levels in. It also politely said it would hold the combat and wait while I fixed the error.
3. It handles combat well and lets the dice fall as they will. Otherwise it took me a couple days to realise that it remained what some people call "a people pleaser". I noticed that even in the most difficult or awkward problems/puzzles, whatever I came up with worked perfectly, which did put a damper on it a little. I wanted the option to be wrong and take any consequences. Some of the tension, which had been building nicely, was lost.
All in all, it's nice and it came up with a clever concept and has run it well. Even role-play between myself and NPC's has been top notch. I can't get ChatGPT to hold a real world debate but it does well when role-playing it. It's been fun for a short while but it's not 100% what I want from an AI Dungeon Master in the long run.