Based on what I've seen, I bet that a modified version of ChatGPT could probably run a competent PbP game right now.
Having played with it extensively, it's undeniably impressive... But no, I don't think it could run a game yet. Its responses come across as rather canned, and it has huge trouble staying consistent.
I'll admit to being floored that it can write fairly competent poetry. Not great, by any means, but not terrible either. Its essay-writing, on the other hand, leaves a LOT to be desired. It comes across as a high-school student trying to cover all the bases as vaguely as possible, hoping for the teacher's mercy or inattention.
Another interesting interaction was when I asked it to come up with a magic system for use with Fate. It replied with a lot of vague pablum. So I drilled down and asked it to create a Fate magic system for a setting resembling Renaissance Italy. This time it came up with some actual brief suggestions, one of which was alchemy. I pressed it to tell me more about how alchemy would work in the game, and it actually came up with something that sounded a bit useful and interesting.
Finally, I told it to get specific: What actual aspects, skills, and stunts might be involved? And it came up with some options that weren't terrible! They could be used by a GM as a springboard to come up with something better, no question!
...and given the pace of things, I can't imagine that we are more than five years away from having AIs take the places of DMs and/or players.
I would seriously consider a future AI GM for playing solo games. (Especially if I could feedback what was working for me and what wasn't.) Maybe even for pickup PbP - at least you could count on it not ditching the game without warning! But I wouldn't want it for my regular group, where social interaction is part of the point.
