D&D General D&D and AI Text-to-Image Generation

Byron.the.Bard

Villager
Byron the Bard has had fun with D&D inspired text-to-image generation!

Here's a little bit of context:

Machine learning models for generating images from text prompts have significantly improved over the last few years. These models process a large data of images with captions, and encode knowledge about the world and language statistically. Once trained, these models may be presented with a text prompt and they will create an original image on demand. Here we have some fun by trying to elict how much knowledge of the world of roleplaying games, and specifically Dungeons and Dragons, one of these models can capture.

Take a look for instance at the beholders generated by an AI:

a-band-of-kobolds-fight-against-a-beholder.png
a picture of a beholder by Picasso.png



To read more and see more images, check it at: D&D and Text-to-Image Generation
 
Last edited:

log in or register to remove this ad

EzekielRaiden

Follower of the Way
There's a fascinating (and kind of scary) new generative neural network that can create whole images purely from text prompts, images that are potentially photorealistic and modifiable (e.g. asking the computer to add a sofa or remove a plant). It can even work in different styles or artistic flourishes.

I immediately saw the potential for use in generating character art in D&D. It's not directly accessible by the public, sadly, but there are exciting developments on this front.

Here's a link.

Edit: hah, serves me right for not going to your link first! This is just the more advanced version of the thing you've discussed.
 




G

Guest 7034872

Guest
Yeah...Artflow doesn't handle non-human faces very well, does it? Like, not even tieflings, to say nothing of dragonborn.

Everything non-human looks like an elf or an orc.
All true. I'm still surprised by just how plausible the human faces are, though. I had no idea the software had come this far.
 

J.Quondam

CR 1/8
All true. I'm still surprised by just how plausible the human faces are, though. I had no idea the software had come this far.
AI does photorealistic faces, too:


On that site, there are no inputs or prompts or anything; just hit reload to get a new picture. Usually you have to pore over the image pretty closely to spot the little discrepancies that show it's artificially generated. (Though once you spot some of the weirder ones, they can't be unseen, heh!)

It's a little shocking how good the tech has become-- and how fast it continues to evolve.
 


EzekielRaiden

Follower of the Way
Eventually all formerly creative media will be automatically-generated by AI, probably.
Unlikely. Statistical models can become quite adept at generating patterns with which they have some familiarity. E.g. the one I linked that allows people to alter the style of an image, or request the same painting from a different angle. But there remains a key problem with such approaches, one that is unlikely to be resolved quickly or easily.

These models do not contain semantic content.

Remember when that new GPT model was announced, and the people making it made ominous promises not to release it for others to use because they were afraid that it was too dangerous? Yeah that can generate maybe a couple paragraphs of text without (much) need for humans fixing the problems. As shown with the "unicorns that speak perfect English" fake article though, it breaks down really badly once you get past three or four paragraphs, becoming wildly unhinged and self-contradictory. That's because it contains no semantic content, only statistically observed syntactic content.

Neither DALL-E nor GPT3 has the smallest scrap of semantic content, of grasping the meaning and significance of a piece. A great demonstration of this comes up with trying to create an AI that generates classical music. You can often train one that can generate some really beautiful and surprising  passages of, say, baroque music in the style of Bach, but it will struggle mightily with introductions and cadences, because the AI doesn't understand anything; it cannot "see" that a piece needs a satisfying ending, to say nothing of knowing what "satisfying ending" means. Much of the time it will produce a bizarre infinitely-continuing piece, where chunks of beautiful Bach-like filigree are embedded in a sea of random noise notes. Better training can reduce the amount of noise, but you need a whole different approach to write music that will actually be pleasant to listen to.

For relatively simple things, like still images, simple and short videos that only change small parts (like faces) of extant video, etc. then yes, there is the possibility that these tools could someday replace human effort for all but minor finishing touches. For novels or cartoons or performance music? No, not really, not any time soon. We're pushing the boundaries of what's possible, and we'll be running into limits of data storage and usability soon.

Plus...the instant you throw a genuine unknown at one of these things it chokes. Sometimes badly. As noted with my "wow this thing can't even do tiefling" thing; it just spits out humans, elves, and orcs. Maybe dwarves and halflings too, haven't tested them yet.

So yeah. The much more realistic concern is these things being used to automate parts of entertainment that currently do require humans, turning certain aspects of some types of creative work into mere editing. But even writing a news article is gonna be tough when you need to actually reference real names, quotes, etc. and not fictive made up ones.

As long as AI deal only in long-scale statistical associations between things, that is, only in syntactic content, then no matter how complex they do it, they will never completely replace human effort in creative media. You need semantic content, understanding how the meanings and purposes relate to one another, to produce most creative works of any meaningful length.
 
Last edited:

Thanks for that, it makes me feel a bit better. "AI making artists obsolete" is one of those ideas that gives me the willies (although a lot of the AI generated images I've seen give me a very strong "uncanny valley" feeling).

I did find it amusing that even this AI thinks that Demogorgon is that creature from Stranger Things.
 

Remove ads

Top