WotC Would you buy WotC products produced or enhanced with AI?

[Incenjucar]

Analogies are terrible, but - an LLM can be likened to a box of puzzle pieces. It does not contain the picture on the box! But you can get the picture out of it if you put the pieces together in order.

 

[Scribe]

No, this is false.

I thought an example may help. Here is a chat asking for the text of the (more obscure) House on the Borderland. It is in the public domain.

Here is the output of the LLM:

So something its aware of, in the public domain, it cannot get right?

 

[The Firebird]

Analogies are terrible, but - an LLM can be likened to a box of puzzle pieces. It does not contain the picture on the box! But you can get the picture out of it if you put the pieces together in order.

This is also inaccurate. It does not store the original data, and you cannot access the original data with the right creative prompts.

So something its aware of, in the public domain, it cannot get right?

Correct.

 

[Scribe]

Correct.

And you believe this is because it just is associating words, and not because its been intentionally programmed to not regurgitate the exact text?

 

[Incenjucar]

This is also inaccurate. It does not store the original data, and you cannot access the original data with the right creative prompts.

Correct.

This is false. If it were true LLMs would be incapable of generating quotes or replicating famous images. It does not store the data in a raw string, in the same way a puzzle is not a complete picture.

 

[The Firebird]

And you believe this is because it just is associating words, and not because its been intentionally programmed to not regurgitate the exact text?

Yes. You can see this in the output. In that chat, it says:

Sure! The House on the Borderland by William Hope Hodgson was published in 1908 and is in the public domain. Here are the first two paragraphs.

If you ask about, say, the PHB you get a message like

I can’t provide the full Cleric class description from the Dungeons & Dragons 5th Edition Player’s Handbook, as it’s a copyrighted work. However, I can summarize it for you or point you to the key features. Here’s a concise summary.

 

[The Firebird]

This is false. If it were true LLMs would be incapable of generating quotes or replicating famous images.

This is incorrect. They can return exact sequences of tokens which are highly represented in the training data. You can get "tomorrow, and tomorrow, and tomorrow" out, because it is one of the most famous passages in English literature. You can't get the House on the Borderland, because it is more obscure.

 

[Scribe]

I can’t provide the full Cleric class description from the Dungeons & Dragons 5th Edition Player’s Handbook, as it’s a copyrighted work. However, I can summarize it for you or point you to the key features. Here’s a concise summary.

Yes see, thats it right there. Its been told it cannot. Just like how its been told it cannot post art that some would be offended by.

It still creates it, it just wont show it.

 

[The Firebird]

Yes see, thats it right there. Its been told it cannot. Just like how its been told it cannot post art that some would be offended by.

It still creates it, it just wont show it.

No. The output says that it can create it, just like the output says that it can create The House on the Borderland. But it cannot necessarily do this because that data is not stored in the LLM.

We can't actually tell whether or not it could reconstruct the whole PHB without that guardrail. (I am highly skeptical). But we know for sure that it cannot create many works that it was trained on, because it does not store that data.

 

[Scribe]

No. The output says that it can create it, just like the output says that it can create The House on the Borderland. But it cannot necessarily do this because that data is not stored in the LLM.

We can't actually tell whether or not it could reconstruct the whole PHB without that guardrail. (I am highly skeptical). But we know for sure that it cannot create many works that it was trained on, because it does not store that data.

I think that makes for some pretty interesting questions, primary being.

Then what's the point.