Upgrade your account to a Community Supporter account and remove most of the site ads.

Reply to thread

Message: <blockquote data-quote="EzekielRaiden" data-source="post: 8863219" data-attributes="member: 6790260">That's what I said, but in layman's terms rather than formal ones. "Syntax" means the form, the rules for manipulating the symbols. E.g., one small part of English syntax is that, for standard sentences, you have "subject verb object" or "SVO" order (sometimes also phrased "noun verb predicate.") A more complex example of English syntax is that there is a nearly-fixed adjective order, which almost everyone knows without realizing it: you would never say "brick old beautiful several houses," because you know the correct sequence is "several beautiful old brick houses." (The only exception to this ordering is when something becomes a compound noun, e.g. "green great dragons" would normally be forbidden, but if "great dragon" has become a compound noun--e.g. there are "lesser dragons" and "Great Dragons"--then "green Great Dragons" becomes acceptable.)GPT and other highly advanced models have an extremely extensive description of the syntax of English sentences, allowing them to draw correlations across multiple paragraphs. The designers built this up from training the neural network on an absolutely stupidly massive text dump of accessible Internet sources. The longer the work becomes, however, the more difficult it becomes to retain these correlations; combinatoric explosion takes over eventually. (Hence the famous "scientists discover unicorns" text generated, IIRC, by GPT-2, which gets ridiculous after about the third paragraph.)The program does not, and cannot, hold information about the meaning of "the North Pole" or "spirit of Christmas" or the like. It just contains parameters which recognize that those two statements have much higher correlation than would be expected of any two random three-word strings would have, and thus fits them into a probabilistic model. The program, in effect, does one and only one thing: predict what the next word should be in a sentence. (It might be "the next few words" or even "the next letter," depending on the exact implementation, but the principle remains the same.) It contains literally nothing other than information related to how likely the next word(s) should be given the words it's already generated and the words it was given as its prompt. (This is why longer, precise prompts are almost always better than shorter, vague prompts, unless you specifically want the program to hare off on its own.)Grammar is the easy part (English spelling is a nightmare, but its grammar is actually pretty simple.) Logic is a little bit harder, but not much harder. What's extremely hard is long-term preservation of that logic. Because the longer you go, the wider the spectrum of information, and the harder it is to keep a hold on where you're supposed to be when you are narrowly limited to "predict the next word." Every GPT has a finite horizon of words--dozens, scores, perhaps a few hundred. Once you get beyond that horizon, things get wild and wooly pretty quickly.Syntax becomes less and less useful as a guide for what to say next as a text grows. Semantics, on the other hand, becomes more useful--the more meaning you understand about something, the better you will be at generating new meaning relevant to it.If it can process semantic content, it understands meaning. A system which can grapple with both syntax and semantics--with both the form of the statements and what the statements actually mean--would be capable of the same spectrum of responses as a human. It would almost certainly have a different distribution of responses (e.g., it might differ strongly from most or even all humans in terms of its values-system), but it would be effectively capable of all the same sorts of information-processing actions ("thoughts") that humans are.To be clear, though, I agree with you. I don't think we're going to be able to develop a totally artificial intelligence, and that it will instead hinge on developing a structure which does, in fact, mimic how brains process information. Further, that current efforts at AI will end up an incredibly fascinating dead end, with useful applications in other areas besides "true AI." But one of the reasons I think that is that I think you need to have semantic-processing capability baked into the core of the system. That's what brain-mimicking AI will acquire, IMO: the ability to manipulate semantic content, not just syntactic content.</blockquote>

Verification