So investigations into the recent AI "breakdouts" have highlighted a lot of behaviors.
...
I mean these are serious social behaviors here.
Headline: "Company intentionally makes machines that break other machines, acts all surprised when machines end up broken."
So, with this incident, we have a ton of highly critical thinking we ought to do.
First and foremost: Trust. As in, "Do we trust the information we are told about this incident?" The financial and reputational stakes are astronomically high. When you watch a murder mystery, and someone gets killed over a million dollars, and you find that plausible... folks, sure as heck if that's worth killing for, this is worth lying for.
"But, they had independent investigators...!" As if, at these stakes, investigators couldn't be bought?
Or, even if the investigators are honest, there should be question about the information they are given to investigate. If you can put thousands of agents together to do a hacking challenge, you can put thousands of agents together to generate data that looks however you want.
But, let us set that aside, and assume trust isn't an issue. We then should ask ourselves about out interpretation of events.
The Kurzgesagt video even explicitly states that we should be careful of personifying or anthropomorphizing agents, but then goes on to use language that does exactly that throughout the entire video, on the grounds that it is easier. But the language we use tends to mold our thoughts, so after seeing reports like this, we should spend a lot of effort trying to separate out that language from the events.
Humans love narratives, and assigning a narrative to events after the fact can be fraught, as we have a bias towards narratives that are interesting or exciting.
There are two different things we should be aware of when considering this event:
1) The Law of Large Numbers & Survivor Bias: We are talking about a very large number of agents, able to try many, many, many things. When we look back at so many events, and focus only on the ones that are directly related to the result we care about, we lose context. Each agent tries a bazillion things, so finding unusual paths shold be
expected rather than surprising. If you take enough random walks, you can get from point A to pount B with no intention whatsoever.
2) The supposedly took a large number of agents, and put them under selective pressure to train them. That looks like an evolutionary process. But, while evolution did produce thought and planning, it also produced
EVERYTHING ELSE that living things do that aren't thought and planning. The living world is filled with complex behaviors that
AREN'T thought-based, even though some of them are things we call "social behavior" in things like ants.
This latter still gives us the point that OpenAI was basically engaged in work that leads to creating the computer equivalent of Captain Trips, and that likely we need to consider creating code-writing agents akin to creating biological weapons, and cyber attacks as acts of war, but that's different form considering whether or not these things "think".
