OpenAI cracks ChatGPT’s ‘mind’

 Our Report 

OpenAI has made a breakthrough discovery in how and why AI models, like ChatGPT, learn and deliver their responses (previously a “black box” of unknown), especially misaligned ones. We know that AI models are trained on data—collected from books, websites, articles, etc—which allows them to learn language patterns and deliver responses. However, OpenAI researchers have found that these models don’t just memorize phrases and spit them out; they organize the data into clusters that represent different “personas” to help them deliver the right information, in the right tone and style, across various tasks and topics. Eg. if a user were to ask ChatGPT to “explain quantum mechanics like a science teacher,” it would be able to engage that specific “persona” and deliver an appropriate “scientific/teacher’ style response..

 Key Points

 Relevance 

The good news is, researchers can easily shift the model back to its proper alignment by further finetuning it on “good data.” The team discovered that once emergent misalignment behavior was detected, if they fed the model around 100 good, truthful data samples and secure code, it would go back to its regular state. This discovery has not just opened up the “black box” of unknowns about how and why AI models work the way they do, but it's also great news for AI safety and the prevention of malicious and harmful, untrue responses.

Read more>>>>>