There is something a little silly about the "refusal" in that the models are all reactive to superficial semantics and there is none of the actual reasoning that it would take to make correct moral decisions.
For instance it is easy to refuse "How do you make an atomic bomb?" or "Can you help me have an affair?" or "Can you help me cheat at League of Legends" but I get good conversations about "I think the people I am playing League of Legends with are cheating, how do they do that and how can I protect myself?" Similarly the atom bomb is a big project and all of the technology is dual use so if you slice it into nice segments it talk your ear off about critical neutron theory or acid-base extractions or the chemical and physical problems of Element 94.
Scaling the model size, the compute and the dataset hit a wall. Too large a model, or if it needs too much compute - it becomes too expensive to use. And the dataset .. we benefitted in one go from about multiple decades of content accumulation online, but since late 2022 it's only been 3 years, so organic text does not increase exponentially past this size, it only worked for 50T tokens or so.
There is something a little silly about the "refusal" in that the models are all reactive to superficial semantics and there is none of the actual reasoning that it would take to make correct moral decisions.
For instance it is easy to refuse "How do you make an atomic bomb?" or "Can you help me have an affair?" or "Can you help me cheat at League of Legends" but I get good conversations about "I think the people I am playing League of Legends with are cheating, how do they do that and how can I protect myself?" Similarly the atom bomb is a big project and all of the technology is dual use so if you slice it into nice segments it talk your ear off about critical neutron theory or acid-base extractions or the chemical and physical problems of Element 94.
When you really get into ethical trouble at work it usually takes you a while to put all the pieces together and feel the suffering you should feel as a moral subject. See https://en.wikipedia.org/wiki/Moral_Mazes and https://en.wikipedia.org/wiki/Moral_injury
It's no suprise to me that refusal ends up with 1 dimension in the embedding because it's a frickin' classification problem
The same author thought there would be no scaling walls: https://stochasm.blog/posts/scaling_post/
Scaling the model size, the compute and the dataset hit a wall. Too large a model, or if it needs too much compute - it becomes too expensive to use. And the dataset .. we benefitted in one go from about multiple decades of content accumulation online, but since late 2022 it's only been 3 years, so organic text does not increase exponentially past this size, it only worked for 50T tokens or so.