Reply to thread

Message

<blockquote data-quote="FrogReaver" data-source="post: 9078643" data-attributes="member: 6795602">All hype.&nbsp; Give it a few years, maybe 10-20 tops and we will understand the mechanisms at play for in-context learning - which is the phenomenon I assume you are referencing.On in-context learning - (extremely simplified) it's basically giving the AI a categorization list, giving it a word not on the list and asking it to categorize.&nbsp; Behavior is as expected if the categorizations on the list are semantically relevant.&nbsp; However, flip the categorizations and the LLM will pick up on that as well which shouldn't probably shouldn't be possible if it's only looking at word association probabilities for the word you gave it.&nbsp; Until it's proven or stated otherwise I would assume the base process is actually different.&nbsp; Possibly the LLM doesn't actually care about the word you used to categorize something by.&nbsp; Instead it treats each categorization as a variable (an unknown word) and compares to it's data model to find the most likely actual word that maps to what it's treating as an unknown word.&nbsp; Then it just compares the test word to see which of those of those actual words are most associated with it.&nbsp; Final step it uses the known to unknown word map it previously created to translate the final output back to you.&nbsp; And perhaps this there's some threshold for when it does one vs the other.Anyways, the point is that in-context learning need not indicate - generalized intelligence, theory of mind, nor actually exceeding their parameters.Or perhaps, a predictive text generator is actually an even more powerful tool than we initially realized.</blockquote>

[QUOTE="FrogReaver, post: 9078643, member: 6795602"] All hype. Give it a few years, maybe 10-20 tops and we will understand the mechanisms at play for [U]in-context learning[/U] - which is the phenomenon I assume you are referencing. On [U]in-context learning[/U] - (extremely simplified) it's basically giving the AI a categorization list, giving it a word not on the list and asking it to categorize. Behavior is as expected if the categorizations on the list are semantically relevant. However, flip the categorizations and the LLM will pick up on that as well which shouldn't probably shouldn't be possible if it's only looking at word association probabilities for the word you gave it. Until it's proven or stated otherwise I would assume the base process is actually different. Possibly the LLM doesn't actually care about the word you used to categorize something by. Instead it treats each categorization as a variable (an unknown word) and compares to it's data model to find the most likely actual word that maps to what it's treating as an unknown word. Then it just compares the test word to see which of those of those actual words are most associated with it. Final step it uses the known to unknown word map it previously created to translate the final output back to you. And perhaps this there's some threshold for when it does one vs the other. Anyways, the point is that in-context learning need not indicate - generalized intelligence, theory of mind, nor actually exceeding their parameters. Or perhaps, a predictive text generator is actually an even more powerful tool than we initially realized. [/QUOTE]

Verification