AI/LLMs Navier-Stokes "Solved" by AI

[Umbran]

I feel it worth noting: there is no proven theft here. There is speculation. There is theoretical opportunity, but no smoking gun - and likely never will be one.

And I feel it worth noting that this is not about criminal law, or the burdens of proof attached to such. We do not need "smoking guns".

This is about attempted theft of a point of reputation - over who actually solved a thing, over who gets credit. The burden or proof is about who is more credible.

And, on that, dude, they built their machines by scraping everybody else's material off the internet, without asking, without licensing, and then trying to sell it back to us. They have zero credibility on not doing the same sort of thing here. ZERO.

Tristan Buckmaster has said:
"I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.

I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer."

We should note that the model not looking up the data still allows that humans may have looked it up and fed it to the model.

When asked about when the prompt that supposedly solved the issue was made, they admitted that it was after the human work on the problem became known to OpenAI. When pressed as to whether the humans work was used in training the AI, they did not answer.

Not a "smoking gun", but not a denial either. If they could truthfully deny it, why would they not do so?

Then, Buckmaster continues:
"I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”

So, when Buckmaster refused to play ball, they threaten his career. That is not a sign of people who'd have scruples about stealing just a small bit of work in his career. This is, like, movie gangster nonsense. "Nice career you got there. It'd be a shame if something were to... happen to it..."

Whether or not the AI is awesome, the people who make it... kinda stink. Maybe you don't want to weigh that into your considerations, but it seems to me that allowing such crummy people to control our technical future is... unwise.

If he has a process where he can generate 258 papers that pass peer review in a year then what's the issue?

That depends - how much credit does he want for those papers? And now much can he speak to the content of those papers if he isn't reading from it at the moment?

That latter being a big one - folks who create content via AI tend to not know it nearly as well, or think about it as holistically, as someone who actually writes their own thoughts. The end result is "generated papers", not actually someone knowing anything.

If you (generic, not anyone here in particular) don't expand human knowledge, you have failed to understand the point and value of the enterprise. Racking up a long list of papers that you can't possibly understand yourself smacks of a childish act of trying to "win" by scoring points on technical grounds, rather than actually being able to perform well.

 

[FrogReaver]

And I feel it worth noting that this is not about criminal law, or the burdens of proof attached to such. We do not need "smoking guns".

This is about attempted theft of a point of reputation - over who actually solved a thing, over who gets credit. The burden or proof is about who is more credible.

And, on that, dude, they built their machines by scraping everybody else's material off the internet, without asking, without licensing, and then trying to sell it back to us. They have zero credibility on not doing the same sort of thing here. ZERO.

Tristan Buckmaster has said:
"I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.

I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer."

We should note that the model not looking up the data still allows that humans may have looked it up and fed it to the model.

When asked about when the prompt that supposedly solved the issue was made, they admitted that it was after the human work on the problem became known to OpenAI. When pressed as to whether the humans work was used in training the AI, they did not answer.

Not a "smoking gun", but not a denial either. If they could truthfully deny it, why would they not do so?

Then, Buckmaster continues:
"I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”

So, when Buckmaster refused to play ball, they threaten his career. That is not a sign of people who'd have scruples about stealing just a small bit of work in his career. This is, like, movie gangster nonsense. "Nice career you got there. It'd be a shame if something were to... happen to it..."

Whether or not the AI is awesome, the people who make it... kinda stink. Maybe you don't want to weigh that into your considerations, but it seems to me that allowing such crummy people to control our technical future is... unwise.



That depends - how much credit does he want for those papers? And now much can he speak to the content of those papers if he isn't reading from it at the moment?

That latter being a big one - folks who create content via AI tend to not know it nearly as well, or think about it as holistically, as someone who actually writes their own thoughts. The end result is "generated papers", not actually someone knowing anything.

If you don't expand human knowledge, you have failed to understand the point and value of the enterprise. Racking up a long list of papers that you can't possibly understand yourself smacks of a childish act of trying to "win" by scoring points on technical grounds, rather than actually being able to perform well.

Please do not reply to me except in your mod duties.

 

[The Firebird]

When asked about when the prompt that supposedly solved the issue was made, they admitted that it was after the human work on the problem became known to OpenAI. When pressed as to whether the humans work was used in training the AI, they did not answer.

Not a "smoking gun", but not a denial either. If they could truthfully deny it, why would they not do so?

This is not accurate. They said it was unlikely in their original statement, then followed up internally and issued a categorical denial. I'm sure that's little comfort to a cynic but it's worth getting the facts straight.

 

[Umbran]

This is not accurate.

It was accurate as of the time of the writing.

And, they came to such a discussion without being able to answer that most obvious question?

They said it was unlikely in their original statement, then followed up internally and issued a categorical denial. I'm sure that's little comfort to a cynic but it's worth getting the facts straight.

"Cynic" is not the proper term, in this case. I do not, in general, feel human decisions are driven by selfishness and self-interest.

A term that fits better here is "skeptic".

 

[The Firebird]

It was accurate as of the time of the writing.

Fair enough. You just omitted known context to make a rhetorical point.

 

[Umbran]

Fair enough. You just omitted known context to make a rhetorical point.

Nope.

I was unaware of any following denials from OpenAI. It would be great of you could provide a link to what they said.

I am sorry you chose to assume that of me.

This is one of the reasons why we ask that folks not make the discussions personal. For some reason, you tried to make this about me, and got it wrong. If it were someone other than me, they'd have good reason to be upset with you - and you'd have ended up with a row on your hands, or gotten reported for insulting someone.

Maybe keep your theories why other do things to yourself in the future. Thanks.

 

[Justice and Rule]

This is not accurate. They said it was unlikely in their original statement, then followed up internally and issued a categorical denial. I'm sure that's little comfort to a cynic but it's worth getting the facts straight.

Eh, issuing a categorical denial after it has become public is a very different thing than not being able to say in private. And given that they can't really say where their "rumor" of someone trying to solve the problem came from (and that no one can seem to identify where it came from on the net), I would say that's probably not all that true. But given how they bargained (and even how one of the former interns described the guy who made the "offer"), I'm just not really convinced. These are also the same people saying we've hit AGI and I'm fairly sure we all know that's false, too.

 

[The Firebird]

Nope.

I was unaware of any following denials from OpenAI. It would be great of you could provide a link to what they said.

I am sorry you chose to assume that of me.

My sincere apologies my post came across that way. I wrote it in a way which, I thought, allowed for for said omission to be inadvertent. ('known' referring to the fact that the point was raised by me in the other LLM thread which had many of the same participants; not that you, Umbran, had read that post). I should have worded it more carefully.

The link is here. The quote is:

The Wednesday evening statement from OpenAI was more emphatic: “We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.”

Maybe keep your theories why other do things to yourself in the future. Thanks.

We are clearly not getting along in this thread, or indeed several others. Were it possible, I would put you on my ignore list, for at least a month or so, to avoid potential misunderstandings. If it is possible for you to ignore me, I would appreciate you doing so. Otherwise, I will second @FrogReaver and ask that you no longer engage with my posts. I will do likewise.

 

[The Firebird]

Eh, issuing a categorical denial after it has become public is a very different thing than not being able to say in private. And given that they can't really say where their "rumor" of someone trying to solve the problem came from (and that no one can seem to identify where it came from on the net), I would say that's probably not all that true. But given how they bargained (and even how one of the former interns described the guy who made the "offer"), I'm just not really convinced. These are also the same people saying we've hit AGI and I'm fairly sure we all know that's false, too.

The timeline that makes sense to me is:

0) Noam Brown reports they have tried Millenium problems without success, but haven't throw much inference at them (Aug 28)
1) OpenAI learns that Buckmaster has some exciting NS results. Reasoning that problems which one LLM can do, multiple can do (which has been shown via previous mathematics results), they start throwing a lot of inference at NS to get a result.
2) Their run succeeds in ~88 hours or so, and they report positive results to Buckmaster. That prompts Buckmaster to question them about the training leakage.
3) They consider it highly unlikely for such an event to occur, but it takes time to verify, so they hedge a bit shy of the full statement.
4) After having the time to look into, they categorically deny it.

I don't think OpenAI comes off well in that story, because it is clearly true they made a big rush to scoop Buckmaster, even in the absence of using data from his chats. And I think the point from the intern is very much worth listening to.

That said it is also distinct from straight reading the chats.

 

[FrogReaver]

The timeline that makes sense to me is:

0) Noam Brown reports they have tried Millenium problems without success, but haven't throw much inference at them (Aug 28)
1) OpenAI learns that Buckmaster has some exciting NS results. Reasoning that problems which one LLM can do, multiple can do (which has been shown via previous mathematics results), they start throwing a lot of inference at NS to get a result.
2) Their run succeeds in ~88 hours or so, and they report positive results to Buckmaster. That prompts Buckmaster to question them about the training leakage.
3) They consider it highly unlikely for such an event to occur, but it takes time to verify, so they hedge a bit shy of the full statement.
4) After having the time to look into, they categorically deny it.

I don't think OpenAI comes off well in that story, because it is clearly true they made a big rush to scoop Buckmaster, even in the absence of using data from his chats. And I think the point from the intern is very much worth listening to.

That said it is also distinct from straight reading the chats.

Thank you! I was just about to ask someone to lay out the timeline.

The questions I have:

• Who is Noam Brown?
• How did OpenAI learn about Buckmaster and the NS results? High level Math can be a small leaky community so I could see some basics getting around.
• Do we have the technical details of exactly how OpenAI set up their agents and prompted them?
• What all did they check to ensure it was highly unlikely? Who performed those checks? What additional checks can be confirmed?

My thoughts: I actually think testing on soon to be proven problems without publicly available proofs is a good testing scenario - assuming solution access really isn't present to the AI directly or indirectly, this kind of exercise says quite alot about the AI's capabilities.

Also, if true the report about threatening Buckmaster if he doesn't make a public statement giving their AI credit makes them really slimy. Lying is generally less evil than threatening and/or blackmail, so if it's true that occurred then lying in general wouldn't be something to put past them. That said, even liars tell the truth sometimes. It just sucks that we will never really know whether the AI solved Navier-Stokes without 'cheating' (assuming it's proof holds up). IMO. It'll always have an asterisk beside that accomplishment in the record books.