D&D 5E (2024) University project on D&D spells

Hiteshd

Villager
Hi,
I am conducting a survey for my university research project that requires participants who play or are familiar with D&D.
Will you be interested in contributing to it?

It's an anonymous survey where you can try to guess the source of the spells.


PS - If this post goes against any community guidelines, please let me know and I’ll take it down.
 

log in or register to remove this ad

You will be shown a series of spells. These spells come from three different sources:

A Human Player: An experienced Dungeons & Dragons player.
GPT: A general-purpose large language model from OpenAI.
LLaMA: A large language model from Meta that has been specially fine-tuned to generate D&D spells.

Some of the spells were ok and some felt obvious like AI.

I do not really know what the difference is between GPT and LLaMA and it seems like arguing between 5e and 4e where they are all D&D. I was just guessing on some between the two not really knowing.
 



you really need an option of "I have no opinion," I am not a fan of answering questions without knowing which to choose, and you're just introducing noise into the results. If you want me to always pick B when I don't know that's what I'll do.
 

Cool project. Typical LLM text I feel confident identifying, but that was not the case here. I haven't seen too many LLM spells.

Will you update us with the results? It would be fun to see.
 



This was fascinating because the spells I got, the vast majority of them were just very very much too high level. Like, level 3 spells with level 1 or even cantrip-tier effects. A level 7 spell that would have been bad at level 5. A level 5 spell that was kind of objectively worse than upcasting the L1 spell it mimicked (which gains more damage than it had and does basically the same thing more efficiently).

My experience of people making spells is that 90% of the time, spells are either the right level, or a bit overpowered for that level. They're almost never too high level. But there were loads of spells here which were objectively inferior to Tasha's Hideous Laughter, but level 3 or higher - and I do mean objectively lol.

The spells were also not at all creative. That's always been true with player-made spells, but it was clearly true with GPT and LLaMa as well. Almost every spell was basically "like this other spell, only inexplicably higher level, and worse". Sometimes there was a weird rider but those also tended to be pretty bad.

In the end the only spell I saw that would be even be worth casting if you had access to the "real" versions was a beefed-up version of Blur, which I assume was human because it was slightly better than Blur, but the same level (certainly most spells I've seen over the last 30 years where people say "Please tell me if my homebrew spell is okay" have been along those lines).

The mistakes were interesting too - like one spell said "Pick three effects from the below list" and just didn't have a list.

I did enjoy the truly awful L3 spell which was functionally "Tell one person a believable lie, they act on it for a few rounds ", which like, my brother in christ, because they get a saving throw, this is actually worse than having the Rogue or Warlock or whoever roll Deception, and it's level 3, Concentration, short-ranged, and has a maximum duration of 10 rounds. So it wouldn't ever work if there was more than one person.

I also enjoyed the spell which was a level 3 AOE damage spell with a really short range, small radius, and 4d6 damage. LOL what. You could choose the damage type but even if they were vulnerable, you're still doing the same damage as a fireball at absolute best! And it claimed to create a field of fire/water/etc. but only explained what the electricity one did.

Oh and not a single spell had any info on upcasting - that was pretty sus in some cases, though I forgot to mention it. But the vast majority of the spells I saw were terrible CC spells so plausibly might not have had an upcast.
 
Last edited:

This was fascinating because the spells I got, the vast majority of them were just very very much too high level. Like, level 3 spells with level 1 or even cantrip-tier effects. A level 7 spell that would have been bad at level 5. A level 5 spell that was kind of objectively worse than upcasting the L1 spell it mimicked (which gains more damage than it had and does basically the same thing more efficiently).

My experience of people making spells is that 90% of the time, spells are either the right level, or a bit overpowered for that level. They're almost never too high level. But there were loads of spells here which were objectively inferior to Tasha's Hideous Laughter, but level 3 or higher - and I do mean objectively lol.
I agree. It all seemed soulless somehow. As if a writer was being paid by the spell for content with no trace of underlying enthusiasm for creating spells, and a fear of breaking the game.

My conspiracy theorists brain is saying "almost all human trying to look like a new LLM"
 

Remove ads

Top