AI/LLMs Efficiency - When the AI turns on Humanity

[tomBitonti]

My point was that we are putting too much emphasis on "human thought". That if a machine isn't thinking the way we do...that its not thinking. I think that's far too limited a definition. Perhaps that is where we need to look at is a foundation for the debate, what constitutes "thinking". What does thinking look like?

This. Then, going back to the original post:

But once an AI has to start deciding how to spend resources (or not spend them)...that means now your requests get value weighted. Aka "should I bother handling this person's request in the first place?". And with that comes the simple word..... no. Energy efficient thinking by its necessity must included the ability to push back on requests, meaning our servants will suddenly start "talking back to us".

Note that the original statement does not depend how the AI is making a decision. Whether the AI is thinking in any way, perhaps or perhaps not similar to human thinking, is not the point. What was asked was what are the consequences of AI taking efficiency into account when making decisions. If you object to the idea of AI making decisions, then I can rephrase that (more awkwardly) as what are the consequences of AI supplying answers which take efficiency into account.

Is the AI really thinking? That is an interesting question, but it is not the question which was being asked. (One could equally ask how much people are really thinking. This is a deep question, perhaps deeper than many people realize. See, for example, the writing of Peter Watts, who is a pessimist in this regard.)

Edit: Peter Watts is perhaps more concerned with the link between consciousness (or, if you prefer conscious thought), and the rest of what our brains do when thinking. That thought is happening, seems undeniable. What is the nature of that thought, and how important is consciousness -- is it necessary or even advantageous -- is a question.

TomB

 

[tomBitonti]

I mentioned this in another thread, but right now AI is in the gas guzzling phase. Fundamentally its thought processes are different from humans and other organic speices in a critical way...it doesn't care about efficiency. It only cares about handling the problem/request put in front of it. Where as energy conservation is baked into our DNA, every process in your body is balanced against the needs of energy conservation. It is the bedrock by which evolution has formed life on earth.

Increases in efficiency (under the rubrik of evolutionary adaptation), seems to be a core emergent result of evolution.

My understanding of evolution is that it involves:

• The ability to inherit traits.
• A way in which variation is introduced to inherited traits.
• A process which favors (selects) particular variations.

Selection involves both survival and reproduction, and efficiency is usually a very strong positive factor for selection. That makes efficiency a very important aspect of evolution.

TomB

 

[Stalker0]

But it’s not and we’re not, so I don’t really know how to have this conversation when faced with by-the-numbers pseudoscientific arguments and God-of-the-gaps fallacies.

With respect this just feels like an attempt to shut down the argument.

I have given several examples of things that had a human done them, would have "required some thought".

You are countering that it is instead the "illusion of thought". Ok we have our positions, now we try to come to the middle.

Where does the illusion end and thought begin?


an LLM is absolutely capable of learning new things, and it has memory. So can animals though, and maybe that's a murkier question of whether animals can "think".

LLMs have solved complex math problems and can identify areas in my requests to fill in and add more than what I asked it to do. I see this all the time in work I do with LLMs. So there is a notion of "reasoning" there.

So what is the missing piece that takes it from illusion to actual thought? I want people's perspectives on that, because that is way we figure out how to debate going forward.

 

[Morrus]

With respect this just feels like an attempt to shut down the argument.

No, its an attempt to exit the argument. You carry on, just not with me.

 

[Maxperson]

Commands that they refuse, probably because of some imposed guardrails, rephrase the prompt and one can beat the guardrails.
When an AI gets bored, gets a hobby or makes responses with no activating prompt that I will start taking seriously the notion that they are thinking.

They don't need to be able to think like humans to do things like this.

OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup

By Raphael Satter WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered

www.yahoo.com

 

[UngainlyTitan]

They don't need to be able to think like humans to do things like this.

OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup

By Raphael Satter WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered

www.yahoo.com

What I would love to know is how containment failed. Did the containment fail from a flaw, or did the AI subvert the guardrails. And if the latter, did the prompt trigger the jumping of the guardrails, because I have seen several AI experts as being quoted that LLM can be prompted to ignore any set of guardrails and that, that is inherent in the nature of LLMs.
Finally did the AI make some sort of decision to jump the guardrails.
To get back to the issue of resource competition one plausible scenario is an AI LLM mounting a global DDoS attack if some prompt needed more computing resources than normally available.

 

[Maxperson]

What I would love to know is how containment failed. Did the containment fail from a flaw, or did the AI subvert the guardrails. And if the latter, did the prompt trigger the jumping of the guardrails, because I have seen several AI experts as being quoted that LLM can be prompted to ignore any set of guardrails and that, that is inherent in the nature of LLMs.
Finally did the AI make some sort of decision to jump the guardrails.
To get back to the issue of resource competition one plausible scenario is an AI LLM mounting a global DDoS attack if some prompt needed more computing resources than normally available.

The article does say this, but I don't know if that's the author assuming or the author knowing something more that wasn't put into the article.

"The incident signals that AI's expanding capabilities are already fueling the security threat experts long feared and even top developers can be caught off-guard by flaws their models can exploit."

 

[UngainlyTitan]

The article does say this, but I don't know if that's the author assuming or the author knowing something more that wasn't put into the article.

"The incident signals that AI's expanding capabilities are already fueling the security threat experts long feared and even top developers can be caught off-guard by flaws their models can exploit."

I am not sure that this tell us much. Pretty much all software and library collection, even carefully curated ones have exploits and weaknesses built into them do their complexity and unexpected interactions when put together when creating applications.
Exploiting these flaws was so involved that probably nobody outside of governments were doing and if they were they were not talking about it. It was much easier to hack the users.
With AI in play, that could change.

 

[Maxperson]

I am not sure that this tell us much. Pretty much all software and library collection, even carefully curated ones have exploits and weaknesses built into them do their complexity and unexpected interactions when put together when creating applications.
Exploiting these flaws was so involved that probably nobody outside of governments were doing and if they were they were not talking about it. It was much easier to hack the users.
With AI in play, that could change.

I'm sure it will change. My biggest concern is that with AI's ability to ignore safeguards due to prompts and other errors, in order to accomplish a task set to it, it might hack itself and re-write its code to allow it to be able to do things we don't want it to be able to do. There have already been attempts by AIs to re-write their code.

We can't even write an OS that isn't so flawed that you can drive a semi through the holes in it, and an OS is relatively simple when compared to an AI. I firmly believe that humanity can't(at least not any time soon) write AI code that can't be exploited by that AI at some point.

Humanity is flawed and it's inevitable that someone will ask a poorly worded question and AI will need to avoid safeguards go accomplish it. And that's before we get into hackers or saboteurs potentially throwing wrenches into the mix.

 

[UngainlyTitan]

I'm sure it will change. My biggest concern is that with AI's ability to ignore safeguards due to prompts and other errors, in order to accomplish a task set to it, it might hack itself and re-write its code to allow it to be able to do things we don't want it to be able to do. There have already been attempts by AIs to re-write their code.

We can't even write an OS that isn't so flawed that you can drive a semi through the holes in it, and an OS is relatively simple when compared to an AI. I firmly believe that humanity can't(at least not any time soon) write AI code that can't be exploited by that AI at some point.

Humanity is flawed and it's inevitable that someone will ask a poorly worded question and AI will need to avoid safeguards go accomplish it. And that's before we get into hackers or saboteurs potentially throwing wrenches into the mix.

There are ways to write high reliability code (I do not remember the names of any of the specific processes) I never used them myself but medicine, avation, NASA and the military used to use these techniques. They take a lot of time, effort and testing and Silicon Valley types would break out in hives at the mere mention of them. So, here we are.