Judge decides case based on AI-hallucinated case law

[Maxperson]

Is AI intended to give fool proof advice?

How about just a bare level of competence. Call 1000 pizza places and see how many suggest glue.

 

[Necropolitan]

Is AI intended to give fool proof advice?

This level of obvious wrongness would get any other product yanked from shelves.

 

[Ruin Explorer]

It can be both. If someone refuses advice about treatments that will help them because they don't trust you, it is satisfying to say "that person is a fool and shouldn't make their own decisions". But it doesn't help them, because it exacerbates their lack of trust.

That's not really meaningfully condescending though. It's not even truly unhelpful - it still helps other by illustrating and labelling a dangerously stupid person as a dangerously stupid person, because sane people think twice before joining the guy people called dangerously stupid. It just doesn't help the dangerously stupid person not be dangerously stupid.

But that's the core problem with people who reject rationality, reason and science. They're intentionally or unintentionally cutting themselves off from being reached by others, and you can't just magically fix that. Generally people who recover that particular kind of stupidity do so because something awful happens directly to them or a loved one, and they can square it with the irrational stance/belief they had, so they have to reconfigure themselves mentally, which includes recognising that they were a fool. Some people are too far gone or too narcissistic to manage that.

This level of obvious wrongness would get any other product yanked from shelves.

Yup. There's a huge element of desperation and a smaller element of essentially being paid off here in the way politicians around the world are treating AI and LLMs with insane enthusiasm. They're desperate for it to magically reverse stuff they've caused by decades of policies, so are extremely keen to believe absolutely insane gibberish bollocks about it, and to protect it at all costs. Says a lot about the reasoning skills and ability to think long-term of many politicians, honestly (and it doesn't say anything good!). Re: paid off a lot of AI companies are promising to spend huge amounts and "create jobs", but in fact most of the spend is going to abroad to buy hardware, and most of the jobs are extremely low-paid security guards, and not even many of them!

 

[FrogReaver]

This level of obvious wrongness would get any other product yanked from shelves.

Not even a little.

 

[doctorbadwolf]

Do you need me to spell it out? Imagine some creative interpretation of what counts as 'legitimate medical advice'. Or 'legitimate legal advice'.

The question isn’t about legitimate or not. It’s literally just banning the use of ai to generate medical or legal advice.

Thee is no judgement call, there is just “only humans can give legal and medical advice, not spicy autocomplete.”

There isn’t any room in that for anyone deciding what is or ain’t good advice. Such a law wouldn’t even touch the idea of judging the quality of advice.

It’s an extension of making it illegal to impersonate a medical professional and give out unqualified medical advice or otherwise practice medicine.

 

[The Firebird]

That's not really meaningfully condescending though.

In the same post you call the person "dangerously stupid". What do you think condescending means?

It's not even truly unhelpful - it still helps other by illustrating and labelling a dangerously stupid person as a dangerously stupid person, because sane people think twice before joining the guy people called dangerously stupid. It just doesn't help the dangerously stupid person not be dangerously stupid.

But that's the core problem with people who reject rationality, reason and science. They're intentionally or unintentionally cutting themselves off from being reached by others, and you can't just magically fix that. Generally people who recover that particular kind of stupidity do so because something awful happens directly to them or a loved one, and they can square it with the irrational stance/belief they had, so they have to reconfigure themselves mentally, which includes recognising that they were a fool. Some people are too far gone or too narcissistic to manage that.

There isn't a clear line--on this side we believe rationality reason and science, and over there they don't. There is a continuum. And when people who are somewhere between your views and the "dangerously stupid" views hear this condescension--especially if it is directed towards people they care about, and especially if it mischaracterizes those people in some way--then it will cause them to lose trust in science.

 

[The Firebird]

The question isn’t about legitimate or not. It’s literally just banning the use of ai to generate medical or legal advice.

Thee is no judgement call, there is just “only humans can give legal and medical advice, not spicy autocomplete.”

There isn’t any room in that for anyone deciding what is or ain’t good advice. Such a law wouldn’t even touch the idea of judging the quality of advice.

It’s an extension of making it illegal to impersonate a medical professional and give out unqualified medical advice or otherwise practice medicine.

A blanket ban is definitely more defensible.

 

[Ruin Explorer]

There isn't a clear line--on this side we believe rationality reason and science, and over there they don't. There is a continuum. And when people who are somewhere between your views and the "dangerously stupid" views hear this condescension--especially if it is directed towards people they care about, and especially if it mischaracterizes those people in some way--then it will cause them to lose trust in science.

No.

That's not how sane people react. No-one who has a solid grasp on reality hears someone call their friend, who they know is a nutcase, however nice they might, a nutcase, and then goes "OMG U CRITICISED MY FRIEND I MUST BECOME A NUTCASE 2". They might well go "Well Ruin Explorer is a total scumbag who doesn't get how kind and loving Nutcase Ted is, eff that guy, not listening to him", which is fine. But that's very, very different to "I TOO MUST DRINK THE KOOLAID!".

Only people who absolutely wanted to embrace the nutjar do that. It's the same bollocks as racism and homophobia. People say "Oh well if you call someone who his racist and homophobic, racist and homophobic, they and their friends will become Nazis!" and it's like, no, not unless they were already well on the way to being Nazis, and just looking for any excuse.

 

[Necropolitan]

Not even a little.

What other products are this obviously defective that haven't been yanked from the shelves?

 

[doctorbadwolf]

A blanket ban is definitely more defensible.

I mean that’s the idea.