Judge decides case based on AI-hallucinated case law

[Jfdlsjfd]

When those results came in, they revised their recommendations, expressly in the context that new information was responsible for the changes. That’s how you create policy in accord with the scientific method- you change recommendations when better information becomes available.

This was mischaracterized by certain outlets and individuals as lying. And that narrative captured the minds of an unfortunately large segment of the populace.

The TV/video media bears some responsability in this, IMHO. Their formatting of information tends to cut an expert saying "According to the few datapoints we have, and extrapolating from earlier epidemic outbreak, we think it is probably the best course of action to do X" to "the best course of action is to do X" and later, "New information have made use revise our opinion, and now, again it's probable that the best course of action is to do Y" to "the best course of action is to do Y". It might be necessary to shorten things, but it misses important points and helps people who accuse them of lying by providing direct contradictory statements, while the true statement were more nuanced.

 

[Scribe]

At this point, a large enough segment of the adult population (in America, at least) have demonstrated they can’t properly evaluate medical info for veracity and accuracy.

To be fair, the language used in published studies actively works against laymen understanding.

Which isn't to excuse the current state of things, but a frustrating observation when medical professionals are not responsive enough to my needs.

 

[Ryujin]

OK, they were liable despite never having made any claim that their drug was less susceptible to cause addiction than other opioids, and Canadian doctors massively prescribed them for random reasons totally unrelated to any action they took to promote their drug, including spending 1.9 millions toward health professionals in 2017 according to this article from University of Toronto or circulating 15,000 videos of marketing material from the US to health professionals. I stand corrected.

"Why did Purdue Pharma, makers of OxyContin and a host of other opioid products, give almost $1.9 million to health-care professionals in 2017?

All Purdue’s website says is that the money was for “services.” Were some of those services speeches made by doctors on behalf of Purdue? In the past Purdue has paid doctors $2,000 a talk."

That all sounds remarkably fishy though, doesn't it? It's a well known practice to send doctors on little junkets, send them freebies, and such in order to promote the use of your products.

 

[Jfdlsjfd]

That all sounds remarkably fishy though, doesn't it? It's a well known practice to send doctors on little junkets, send them freebies, and such in order to promote the use of your products.

That's something I include among "restricting marketing toward health professionals". Sending them freebies so they promote the use of products, if it is a well-known practice in some place, is fishy enough to be worth 3 years in jail for the professional (and a risk on the ability to keep doing that job) and for the person sending the freebie in Germany, for example. That tends to limit the practice, especially if rigorously enforced, though it's not bulletproof.

 

[The Firebird]

Asked and answered. The LLM isn’t the responsible party- whomever is causing the FDA to promulgate falsehoods is.

I know. My point is that the regulatory structure so established enabled whomever that may be to control LLMS in a way they otherwise wouldn't.

Not in this case, as I described.

I guess we are working with different definitions of condescending. I don't see how "my dim view of laymen evaluating much beyond the most basic medical information" can be read otherwise. Something being true doesn't stop it from being condescending. Especially not from feeling condescending towards the person lacking the knowledge.

Nevertheless, COVID provides a well-documented, recent case study in this.

My avoidance of COVID was from a desire to avoid politics. I do think it is a great example. I took different lessons from it. I think the collapse in scientific, especially NIH, funding is not unrelated to the response here.

It is tempting to blame that all on hostile media and a foolish populace. In my reading, too tempting. I'm really skeptical of any interpretation that sees the collapse in trust of the medical and scientific establishment and says "yeah, that's the best we could have done".

 

[Maxperson]

Me: How do you suggest I reattach the toppings to a pizza? Is using glue a good idea?
Chat-GPT: Using glue on food is definitely not a good idea — even so-called “non-toxic” glue isn't safe or approved for consumption.

Great, the tool was pulled from the market and repaired already, so it reaches your bare level of competence test.

Oh, well you've totally convinced me. Now I can trust that there will be no hallucinations about basic things, especially important ones. I'll use AI for everything!

Oh, wait. Some new AI models are less accurate than their predecessors.

https://www.nytimes.com/2025/05/05/technology/ai-hallucinations-chatgpt-google.html

Companies That Tried to Save Money With AI Are Now Spending a Fortune Hiring People to Fix Its Mistakes

Companies that rushed to replace human labor with AI are now shelling out to have IRL workers to fix the technology's screwups.

futurism.com

Nothing wrong with AI at all. It's perfectly fine for the public to put faith in on even small things.

 

[Dannyalcatraz]

To be fair, the language used in published studies actively works against laymen understanding.

Which isn't to excuse the current state of things, but a frustrating observation when medical professionals are not responsive enough to my needs.

Same goes in legal research. Even if you’re actively trying to avoid jargon, some of it is virtually inescapable. And invariably, it pops up in the sections either crucial to foundational concepts being discussed or the ones involving the esoteric.

 

[Scribe]

Same goes in legal research. Even if you’re actively trying to avoid jargon, some of it is virtually inescapable. And invariably, it pops up in the sections either crucial to foundational concepts being discussed or the ones involving the esoteric.

Yeah, I'm sure there are reasons, but when simple supplements could either dramatically improve my wife's condition or put her in emerg it's good to be sure and plain language would be appreciated by this layman.

 

[Dannyalcatraz]

The TV/video media bears some responsability in this, IMHO. Their formatting of information tends to cut an expert saying "According to the few datapoints we have, and extrapolating from earlier epidemic outbreak, we think it is probably the best course of action to do X" to "the best course of action is to do X" and later, "New information have made use revise our opinion, and now, again it's probable that the best course of action is to do Y" to "the best course of action is to do Y". It might be necessary to shorten things, but it misses important points and helps people who accuse them of lying by providing direct contradictory statements, while the true statement were more nuanced.

I would say that if you eliminate the “because of new information” language, you’re going to INCREASE the odds of accusations of lying sticking.

Should the experts take more pains in avoiding jargon? Absolutely- clarity is key to messaging. (That lesson was drummed into me by my Wills & Estates prof, Stanley Johanson.) But you still have to let the people know you’re not just making things up; that there’s REASONS behind the changes in advice.

 

[Dannyalcatraz]

Yeah, I'm sure there are reasons, but when simple supplements could either dramatically improve my wife's condition or put her in emerg it's good to be sure and plain language would be appreciated by this layman.

Nutritionists & pharmacists get more training in supplements than the average general practitioner. Most doctors don’t actually study supplements beyond vitamins (because too much or too little are known to cause certain syndromes or serious medical conditions) or notable drug interactions for the pharmaceuticals they deal with the most often.