AI/LLMs Can LLMs Think? - The Debate

[FrogReaver]

No more than you would like to back up yous I imagine. Calculating the financial benefit of knowledge acquisition has always been terribly complicated because its hard for us to predict what knowledge will actually be useful. Lasers went from pet project fringe sceience to one of the most important underpinnings of modern technology. Meanwhile, we have spent decades on string theory without a lot of commercial application.

So would 176 man-years of mathematican time bore better fruit than the solution to this Navier-Stokes millenium problem? Maybe, maybe not.

Of course.

Yes to my recent point...unfortunately this particular example is not the best when it comes to saying "how much is the AI thinking" because its still hard to parse out what the AI did on its own versus just walked the path already laid before it. Its very unfortunate, so we have to look to other examples for the debate.




Overall, I have seen you bring up this point a few times regarding the "cost" of AI intellligence in terms of money, resources, etc. Your point being that AI is simply too expensive which therefore nullifies it as a viable option for thought expansion. My point is, all new technologies start that way. We are already seeing signs that AI can solve problems that traditional have taken very intelligent very educated people to solve. And the difference is, the AI is getting smarter every year, whereas mathematicians still require decades of schooling to get to the point where they contribute meaningfully to their areas. If AI is capable of doing that work, than it becomes an efficiency problem....we need more intelligence per dollar. That's an engineering and computer science problem, and we have traditionally been very good at taking new technologies and making them more efficient once they hit the threshold of true utility. there is nothing to me that AI is any different in this repect than the hundreds of ground breaking techs that have come before it.

Now I can respect that we have taken such an "inefficienT" technology and expanded it way too quickly before its proven itself, that I could fully respect and agree with. But it doesn't detract from the more fundamental point that the AI IS able to do work that even a few years ago would have been a pipe dream...work that has traditionally been considered very "thoughful" work.

The bigger issue isn’t the monetary expense anyway. There are only so many mathematicians in the world who can legitimately contribute to problems at this level, and allocating their time to one problem means forgoing the work those mathematicians could have done on other problems.

And mathematical progress isn’t necessarily additive in person hours. Many problems are only solvable once someone develops a genuinely new technique or insight. You can’t necessarily compensate for the absence of that insight by throwing more mathematician hours at the problem.

So I think the relevant comparison isn’t simply mathematician expense versus however much compute cost the AI used. It’s also what else could those mathematicians have accomplished with their time on other problems. And that opportunity cost is much harder to quantify.

 

[Cadence]

Agree with this. Ime LLMs are better than humans at analyzing routine scientific datasets, just because they implement the details that most humans would reject as unimportant or not worth the time. They are not yet as good at defining and orchestrating a scientific program

Are they better at numerical data? (Are tokens the way they use them in anyway optimal for it?) Would some specially made algorithm be a lot better? (Do some LLMs call such specially made routines to analyze some types of data for them if they can? Do they maybe claim they didn't even if they did, or vice-versa?).

 

[FrogReaver]

My read is the training data for this model cut off on July 3rd (pre training? RL?) and it hasn't seen new data since.

Since when did training data become the only source of information an LLM works with. Isn’t there a live context windows? System resources it can pull up that it wasn’t directly trained on. Etc?

 

[FrogReaver]

Are they better at numerical data? (Are tokens the way they use them in anyway optimal for it?) Would some specially made algorithm be a lot better? (Do some LLMs call such specially made routines to analyze some types of data for them if they can? Do they maybe claim they didn't even if they did, or vice-versa?).

I guess that depends on if you treat LLM as the end product or just the training and inference layer. I generally associate claims that an ai did X with the end output. But it’s a semantic distinction and not a technical one I think.

 

[The Firebird]

Are they better at numerical data? (Are tokens the way they use them in anyway optimal for it?) Would some specially made algorithm be a lot better? (Do some LLMs call such specially made routines to analyze some types of data for them if they can? Do they maybe claim they didn't even if they did, or vice-versa?).

Better at numerical data? Yes

Optimal token usage? Doubt it

Special algorithms? In principle perhaps, but the benefit of the LLMs is general reasoning ability which allows them to choose appropriate algorithms and correct based on the context. For example, an analyst may see two peaks (e.g., chromatographic, NMR) and decide whether to split them, treat them as the same analyte, or reject the data. Algorithms that deal only with the raw data have a more difficult time doing this, while an LLM can ingest relevant background data about the system that allows it to make an informed choice.

Do LLMs call such routines? Typically, but currently it depends on the availability of the routines. It is best if they are implemented in a publicly available package.

Do they claim they did even if they didn't? Not often ime. The problem is moreso that they can't effectively express what they did in an easy-to-interpret way, so they will say what they did (correctly) and the researcher will misinterpret their words rather than taking the time to understand them.

 

[The Firebird]

Since when did training data become the only source of information an LLM works with. Isn’t there a live context windows? System resources it can pull up that it wasn’t directly trained on. Etc?

Right but I don't believe the models have access to the codex dataset (or w/e) while they are running. My GPT 5.6 Sol instance can't access your chat from yesterday. But it can access the web.

 

[FrogReaver]

Right but I don't believe the models have access to the codex dataset (or w/e) while they are running. My GPT 5.6 Sol instance can't access your chat from yesterday. But it can access the web.

Right. It depends on its access permissions not just on the training data set.

 

[Coroc]

Except it isn't. That is an illusion.

The cost of those features are dumped out on the public (in the form of loss of income for creators, and financial risk to the general public), rather than absorbed by the company.

This is like saying burning fossil fuels is "cheap", by neglecting the billions and billions of dollars of damage incurred to the rest of us via environmental impacts of letting carbon loose in the atmosphere.

True, but also it has become fashion to claim that electric cars do have zero emissions. The emission profile gets neutral at about 150k miles before that it is worse than a modern low emission fossil fuel car. The reason for that is that the manufacturing of an electric car needs much much more energy than a regular car.
Also some people act like they are total emission free, that also is not true unless all the power comes from green energy, which is not the case in most areas of the world. Also so called green energy comes with its own problems for the environment e.g. rotor blades of wind power stations that cannot be reasonably recycled. Of course power from coal is much worse.

Here some additional bummers: ONE big ship running on diesel emits as much as the whole traffic in an average European country.

In China and India there are some coal mines very close to the surface. They are constantly burning, like in this horror movie, the title of which atm does evade me.
The fires cannot be put our reasonably, because poor people do wild mining to earn a little cash, and the openings work like chimneys stoking the fires again.
Each of this burning coal mines does emit as much pollution as the whole fossil traffic worldwide. No one seems to care.

 

[tomBitonti]

I don’t disagree with the sentiment but…

Analysis is a particular field within mathematics. It doesn’t map directly onto analytical thinking any more than another mathematical discipline does. Mathematical analysis has its own specific subject matter and methods. The Navier Stokes problem, for example, is a problem in mathematical analysis, not simply an example of “analytical thinking.”

Yeah, after I wrote, I have similar misgivings about using the term analysis. The term has a strong match to "real analysis", which is not what I was trying convey.
I'm not sure what better term to use. Maybe "logical reasoning"? I run into a problem with that: Saying "people are not good at logical reasoning" comes off as pretentious.
People can to logical reasoning, to a point. People fail doing exhaustive, detailed reasoning.
Thanks!
TomB

 

[tomBitonti]

True, but also it has become fashion to claim that electric cars do have zero emissions. The emission profile gets neutral at about 150k miles before that it is worse than a modern low emission fossil fuel car. The reason for that is that the manufacturing of an electric car needs much much more energy than a regular car.
Also some people act like they are total emission free, that also is not true unless all the power comes from green energy, which is not the case in most areas of the world. Also so called green energy comes with its own problems for the environment e.g. rotor blades of wind power stations that cannot be reasonably recycled. Of course power from coal is much worse.

(Additional text omitted.)
This. The "greenness" of an electric car depends a lot on how the electricity is sourced, on the size of the care, and perhaps on the battery materials and on how recyclable (or reusable) is the battery.
A theme here in the USA is that as efficiencies improve, folks get bigger cars (!)
(For full disclosure: I drive a fully electric car. I wish I could get a similar car in a smaller package here in the USA.)
(And now back to the LLM discussion ...)
Thanks!
TomB