D&D General Deep Thoughts on AI- The Rise of DM 9000

[pogre]

Most of my graded writing is in-class, so it does not "scare" me as a teacher.

Later this semester we are doing some debates and I plan on working on helping students create quality questions for AI and instruct them on how to drill down to get better responses.

Anybody who does not think that Chatgpt is not capable of writing decent high school essays has either not worked much with Chatgpt much or has not had to read high school level essays lately. If the latter, I am highly jealous.

 

[aco175]

I thought this Chat thing was in test mode right now. I'm guessing that Hasbro will place it behind the Chat1.1 paywall before it is ready to take over the world, or at least be able to DM.

 

[Umbran]

..........sometimes, Umbran, you make it TOO easy.

What, just because I handed you a setup in which a human acts just like ChatGPT, cutting and pasting some old standards?

 

[p_johnston]

So the thing that ChatGpt brings to my mind, particularly in regards to essays, is math. Most people on these boards are probably old enough to remember when a teacher told you "you won't always have a calculator in your pocket." Now days most people not only have a calculator they have a mini super computer with access to any almost any mathematical equation you could ever need in their pocket. Not a perfect comparison but the one that I think of. If ChatGpt and other AI programs are here to stay then, at least in terms of education, we need to adjust how we teach to take them into account and possibly how to use them (once they become more advanced). Just going "they're cheating don't use them" seems short sighted.

 

[TaranTheWanderer]

I'll just put this here:

One To One - Penny Arcade

Videogaming-related online strip by Mike Krahulik and Jerry Holkins. Includes news and commentary.

www.penny-arcade.com

I love penny arcade.

But to quote an excerpt of the article below the comic:

"These are completely unregulated information weapons and we're making toys out of them. These devices have mined a subset of our species' intellectual wealth, they dug too greedily and too deep, and it's everything their frenzied summoners can do to maintain the circle of salt and fragile symbols that keep these things from acting just like real people do online."

 

[Dausuul]

That is the EXACT OPPOSITE of what has been demonstrated so far lol. These text-based language-logic AIs are aggressively intellectually dishonest, completely lack humility, and merely reflect human biases, or invent whole new biases of their own.

This is anthropomorphizing. A large language model is not "aggressively" anything. Nor is it dishonest -- it has no conception of truth and falsehood; it is incapable of honesty or dishonesty. It's true that it lacks humility, but in the way that a chair lacks humility.

ChatGPT is merely a device for synthesizing text which resembles what a human would write. It's very, very good at that. Its problem is that it isn't hooked up to anything else. It's like the language centers of a brain, floating disembodied in the void, without access to the parts of the brain that make and store memories, process sensory experience, make moral judgements, etc.

Except that in this case, it's the language centers of an absolutely gigantic brain which has been trained on every question ever posed on the Internet, and has practiced and practiced and practiced the answers that people made to those questions. So it has "reflexive" responses to the vast majority of things you might say to it, allowing it to seem conscious and intelligent. But when your question is one that hasn't been asked a lot, then its reflexes produce very odd results.

If you feed ChatGPT a set of facts ("Assume the following statements are true") and then ask it questions relating to those facts, its responses -- at least from my admittedly limited testing -- are consistent with the facts it was given. I suspect the next big advance in AI will come from figuring out how to take a prompt, identify what factual information is relevant to that prompt, and go find those facts in some giant database.

 

[Clint_L]

I am going to push back on this, slightly. The fact that ChatGPT can currently write a decent high school essay doesn't mean that it wasn't a special skill - just like the fact that AI Art programs can make cool art doesn't mean that making cool art ... isn't a special skill.

Mastering the basics of clear communication in the form of high school writing is difficult, and not every student does that. Doing it at the college level? Even harder.

In my own experience, I deal with people that, theoretically, have gone through high school AND college AND have learned to write in law school as well (at a minimum, the IRAC method), and I can assure you that there are a ton of bad writers out there. Many of them I wouldn't trust to write a clear high school essay.

All of which is to say- I think we might be misunderstanding what is transformative about this. It's not just writing a high school essay (although that is what is getting the current coverage). Coding. Drink recipes. Creating adventures. Sure, it's based on what people have done before ... but that's what people do too.

At a fundamental level, I think we might not be fully appreciating this, because we keep viewing this in terms of the past. "Oh, I guess chess isn't as special as we assumed because a computer can beat us at it."

Let's try this out- imagine that you believe that, "Writing prompts that make cool stuff come forth from an AI," is, in fact, the "special skill" that humans have. Now, imagine we have ten years of people writing prompts to AIs, and we use that as the corpus to train an AI to ... write prompts for AIs.


What is the special skill? Anyway, not to get dystopian (at all!), because I think this is amazingly cool, but I truly think we are going to see some transformative effects in the next decade that we have trouble imagining, of the type that makes the changes from the internet look like small potatoes.

So, I'm going to push back against your push back a bit, because I largely agree with what you are writing, but as a lawyer you appreciate the importance of nuance. I didn't say that essay writing is not a special skill, but that maybe it "was never as special a skill as we had assumed." There's a difference.

What I am saying is that if a strange alien with tentacles for eyes came to earth and studied human high school or college curriculum, they would have to assume that our species held the essay out as the pinnacle of human achievement, given its prominence in our teaching and assessment. But this happened not because essays are the perfect mode of teaching, but because they are relatively efficient to assess. We use essays, or variations on essays (oral commentary, etc.), for everything in two of my three subjects.

But a lot of essay writing is basically filler. The reason ChatGPT can pull it off so successfully is that good essays are pretty predictable. So maybe going forward we continue to teach essays so that students understand how to structure and build an effective argument, but we allow them to take advantage of AI for a lot of the writing drudgery. I cannot tell you how often students write lousy essays because they have good ideas but twist themselves into knots trying to transpose them into the expected rhetorical style. My ongoing refrain to students is "just use plain language."

I completely agree that writing AIs are going to unleash a new wave of creativity. But it's scary because those of us doing this for a living have no idea what that is going to be. We are just starting to wrestle with the fact that a lot what we had been doing for generations is probably obsolete. The good news is that it has probably been obsolete for, like, a century, so ChatGPT forcing us to finally make drastic changes to education will be a good thing.

 

[Clint_L]

Except that in this case, it's the language centers of an absolutely gigantic brain which has been trained on every question ever posed on the Internet, and has practiced and practiced and practiced the answers that people made to those questions. So it has "reflexive" responses to the vast majority of things you might say to it, allowing it to seem conscious and intelligent. But when your question is one that hasn't been asked a lot, then its reflexes produce very odd results.

If you feed ChatGPT a set of facts ("Assume the following statements are true") and then ask it questions relating to those facts, its responses -- at least from my admittedly limited testing -- are consistent with the facts it was given. I suspect the next big advance in AI will come from figuring out how to take a prompt, identify what factual information is relevant to that prompt, and go find those facts in some giant database.

I have spent quite a few hours over the past few months trying to get it to write blank verse. It can repeat the definition of blank verse, but it just cannot seem to write it. It keeps giving me rhyme. Its iambic pentameter is hit or miss, depending on the prompt.

My personal sign of the singularity will be when it cracks the blank verse problem.

 

[The Shadow]

Anybody who does not think that Chatgpt is not capable of writing decent high school essays has either not worked much with Chatgpt much or has not had to read high school level essays lately. If the latter, I am highly jealous.

That's a sad commentary on today's yout'!

I asked it to write an essay on the 2016 election, and it thought that the Democratic nominee was... Bernie Sanders.

I assume that wouldn't get an A in your class.

 

[Dausuul]

I have spent quite a few hours over the past few months trying to get it to write blank verse. It can repeat the definition of blank verse, but it just cannot seem to write it. It keeps giving me rhyme. Its iambic pentameter is hit or miss, depending on the prompt.

My personal sign of the singularity will be when it cracks the blank verse problem.

It sure does want to include rhymes when you ask for poetry, doesn't it? I finally got it to mostly quit with this prompt: "Write a poem in blank verse about ice cream. Use iambic pentameter. Do not have lines that rhyme with each other." A few rhymes did creep in toward the end, but it was generally rhyme-free.

As for the meter, it is indeed rather shaky. It would not be hard to train a specialized PoetryGPT to address these technical problems, though -- I don't think you even need AI to determine whether a line of poetry complies with a particular metrical scheme. You just need a dictionary of English words with emphasis marked. Almost always, if you can write a simple script to compute "this is right" and "this is wrong," you can train an AI to produce results that meet the requirement.

Training this hypothetical PoetryGPT to produce good poetry is a harder job, though.