Reply to thread

Message

<blockquote data-quote="FrogReaver" data-source="post: 9079283" data-attributes="member: 6795602">On preventing harmful texts1.&nbsp; A language model is just a probability distribution.&nbsp; It's large because it happens to have say billions+ of weights.2.&nbsp; Because it's a probability distribution with so many weights and relationships between them, it's going to be very difficult to change specific weights to prevent whatever text is undesirable and even if you could you likely lose many of the weights and relationships that are needed to produce as realistic of results.3.&nbsp; So instead you might try to just pick training data that doesn't have those problematic elements, but perhaps those things are more pervasive in our language (just under the surface) than we realize - making this not work so great either.4.&nbsp; So then maybe you just try to train another AI to recognize the results you don't want - which from my understanding is where we are at now.&nbsp; But of course generative ai's predictions/pattern recognition isn't foolproof, in which case certain undesirable text slips through the cracks.On understanding ChatGPTOpenAI doesn't fully understand why ChatGPT outputs what it does is technically true - but only because the amount of data is too large and training process too computationally complex to step through step by step as you would in a traditional program.&nbsp; High level they know exactly how it works.&nbsp; They will also eventually settle on the reasons it's able to do anything unexpected that we see it doing - like what's happening when it's given a categorization prompt based on categorizations that are unlikely to be present in it's data model.</blockquote>

[QUOTE="FrogReaver, post: 9079283, member: 6795602"] [U]On preventing harmful texts[/U] 1. A language model is just a probability distribution. It's large because it happens to have say billions+ of weights. 2. Because it's a probability distribution with so many weights and relationships between them, it's going to be very difficult to change specific weights to prevent whatever text is undesirable and even if you could you likely lose many of the weights and relationships that are needed to produce as realistic of results. 3. So instead you might try to just pick training data that doesn't have those problematic elements, but perhaps those things are more pervasive in our language (just under the surface) than we realize - making this not work so great either. 4. So then maybe you just try to train another AI to recognize the results you don't want - which from my understanding is where we are at now. But of course generative ai's predictions/pattern recognition isn't foolproof, in which case certain undesirable text slips through the cracks. [U]On understanding ChatGPT[/U] OpenAI doesn't fully understand why ChatGPT outputs what it does is technically true - but only because the amount of data is too large and training process too computationally complex to step through step by step as you would in a traditional program. High level they know exactly how it works. They will also eventually settle on the reasons it's able to do anything unexpected that we see it doing - like what's happening when it's given a categorization prompt based on categorizations that are unlikely to be present in it's data model. [/QUOTE]

Verification