ChatGPT lies then gaslights reporter with fake transcript

[Scribe]

IT workers are suddenly feeling real called out.

I am one, for decades!

 

[CleverNickName]

The best I have heard that AI does is to create summaries, such as with notes from a meeting.

This keeps coming up as a "no big deal" talking point for AI, but let me tell ya: this is a big concern in the world of engineering. We have a lot of meetings, and whoever controls the meeting notes controls the narrative.

Let's say we have a design meeting, where we are discussing the different design considerations for building a bridge. Everyone brings their concerns and solutions to the table, we hash it out, and we approve the design. Fast-forward five years: the bridge is under construction, and a design flaw is discovered! Construction is halted, lawyers are called, and fingers get pointed.

Nobody remembers that design meeting, or what was discussed, or what was decided. But the engineer took the notes at the meeting, and those notes say that the engineer saw the flaw, asked for it to be addressed, but everyone else ignored it. Unless someone else has documentation to contradict it, that's what the official record will say, and that's how the courts will decide.

I wouldn't trust a meeting summary to AI, any more than I'd trust a bridge design.

 

[Morrus]

asking ChatGTP for the answer with sources gives me the Wikipedia link and the correct answer, much quicker than googling for it and reading the whole article.

The problem with that is that once you have removed the actual websites from the interaction, you remove any incentive for the website to exist because it gets no traffic. Then, once all the actual human-created websites have been driven away, where does ChatGPT get its information from? It's not online, because there's no incentive for anybody to put it online.

I assume that at some point websites will be forced to develop the ability to stop ChatGPT scraping them through a combination of technology and legislation. And at that point, ChatGPT becomes less useful than using a reputable website directly. Things go full-circle and we're back to the early internet again.

 

[dragoner]

This keeps coming up as a "no big deal" talking point for AI, but let me tell ya: this is a big concern in the world of engineering. We have a lot of meetings, and whoever controls the meeting notes controls the narrative.

Let's say we have a design meeting, where we are discussing the different design considerations for building a bridge. Everyone brings their concerns and solutions to the table, we hash it out, and we approve the design. Fast-forward five years: the bridge is under construction, and a design flaw is discovered! Construction is halted, lawyers are called, and fingers get pointed.

Nobody remembers that design meeting, or what was discussed, or what was decided. But the engineer took the notes at the meeting, and the engineer said the design flaw needed to be addressed but everyone else ignored it. So that's what the official record says, and that's how the courts will decide.

I wouldn't trust a meeting summary to AI, any more than I'd trust a bridge design.

Being an engineer, I understand that. Though we also have a thousand pointless meetings about timelines, schedules, HR etc.. The bridge design flaw would be on the contractor, even though the plan would have been submitted through the city. Ultimately you get used to being named in lawsuits after big projects, when they "sue everybody." Lawyer is usually $300. Then you sit there and the case against you is dismissed. Alot of times it is about cost overruns, and if they can declare bankruptcy, the firm that contracted it's construction, can go under, and the real developer buys it from the bank and below market value etc.. Other than like with boilerplate, I don't see AI being trusted with any of this.

 

[Scribe]

I wouldn't trust a meeting summary to AI, any more than I'd trust a bridge design.

"Then Adam said that Susan should be fired."

 

[Jfdlsjfd]

The problem with that is that once you have removed the actual websites from the interaction, you remove any incentive for the website to exist because it gets no traffic. Then, once all the actual human-created websites have been driven away, where does ChatGPT get its information from? It's not online, because there's no incentive for anybody to put it online.

I'd say that institutional websites (the one where lots of actual accurate information is dispensed), fan sites and non-profit sites (like wikipedia) will keep providing a large backbone of informations. True, it might put a dent in the commercial Internet, though, the one relying on ads. It might be a return to the Internet of yore. Isn't the ad-based model more threatened by the very effective adblockers we get now?

Edit: if anything, LLMs can initiate an Internet connection to simulate loading the page, generating the necessary clicks. It would be an intelligent ad-blocker, reading a website, summarizing the pertinent information, and removing unwanted overload.

 

[Ryujin]

Really? I am pretty sure they do try, but in my professional and personal experience, I wasn't exposed to such claims, but to more moderate ones. In my job we've been pitched a dedicated legal AI to boost productivity by searching databases from precedents and extensive private legal articles from reputable law journal, and that sounded quite focused. And we're doing an evaluation of it, to see if it's worth paying for it, not firing half the legal assistants before knowing what the actual benefits are. I'd say that my environment is more sane than most, given the reported experiences here. Maybe it because t was an AI tool peddled by the legal database maintainer, not an Ai tool made by an AI company (well, it is certainly chatgpt under the hood, but not directly). I agree with you that marketing types tend to exagerate the qualities of their products (they are producting human slop?).

A 30 second internet search will show the range of claims.

 

[MoonSong]

That's where the disconnect comes from. I very well knew that Paul Deschanel is the 10th president of the Third Republic. But I might have a lapse of memory and asking ChatGPT will give me a quicker result than going to the library or checking websites for that information. Do I need to check it? No, it is just recalling common knowledge, the recollection is enough. For more specific knowledge, it might be that I don't know exactly when he fell from his train ; asking ChatGTP for the answer with sources gives me the Wikipedia link and the correct answer, much quicker than googling for it and reading the whole article. Research time + checking time is quicker than research time without using this tool. It saved me a few seconds to a minute -- not great, but still.. Of course, you're free to think I am untrustworthy and incompent because my experience differs from yours. Such wouldn't be true or false, it would be a judgement of value of yours. What would be pertinent for assessing the usefulness of AI isn't what we "feel" about things but conducting extensive analysis on productivity gains. Anecdotes like the one in the original post or mine have no bearing on the answer. Pretending they have by virtue of generalization will be met with pushback. Not from people who might think otherwise, but also by people who value logical thinking.

Why not just have wikipedia as a search engine on yout phone?

 

[Morrus]

non-profit sites (like wikipedia)

Non-profit doesn't mean it doesn't need revenue to run. They're constantly asking for donations. If nobody is visiting Wikipedia, nobody is seeing the donation requests. Wikipedia will go away. ChatGPT will not be able to scrape it because it's not there. The whole 'death of the internet' theory starts to become reality--it's just AI eating its own tail with no human content to feed upon.

True, it might put a dent in the commercial Internet, though, the one relying on ads. It might be a return to the Internet of yore. Isn't the ad-based model more threatened by the very effective adblockers we get now?

Ads are down, but not dead yet. It's more that as more ad blockers are used, sites are forced to use more intrusive ad measures that they originally would not have considered but which make more money. It's a vicious circle. But yeah, eventually ads will run out of steam unless the technology changes drastically. Which it probably will--there is a lot of incentive for comapnies with very deep pockets to ensure that ads continue to work. Ironically, probably using AI.

A lot of independent fan/creators (whether sites, youtubers, whatever) end up leveraging that traffic into selling something of their own--just look how many D&D YouTubers have Kickstarters. Again, that won't work if nobody is visiting their sites or watching their videos. So yeah, most of those will eventually fall by the wayside.

So as you say, you're left with the 'institutional' sites. You'll see official info, press releases, that sort of thing, but that's all. You won't get sites like this one, for example (although this one should be OK in the medium term simply because it is funded by a print publishing arm, which ChatGPT can't yet mimic--at least until the synthesis of print-on-demand and AI happens, which doesn't seem like a large technological leap to me).

 

[trappedslider]

When I was writing my RPG Solis People of the Sun/Kosmic my gf convinced me to try AI and it was: wrong, plagiarizing, or boring. I understand that I am very niche in being near future, hard sf, and solarpunk with probably the most accurate star maps. Other science fiction writers complimented my maps as a valuable resource.

The best I have heard that AI does is to create summaries, such as with notes from a meeting.

I know that author John Birmingham made a substack post about using AI: Prediction, Not Retrieval for his Story Bible and summed it up as : But you have to treat them as what they are: unreliable narrators, it's great for data management but not much else.

I am one, for decades!

And that's why you have such a cherry out look right?