I think Kevin Roose is going to go down in history as having published the most singularly catastrophic article about how LLMs work with his Sydney article [0].
I suppose most of us know this already, but he wrote a piece in early 2023 after interacting with a version of Bing Chat (which was based on an early iteration of GPT-4) where he had a long, somewhat bizarre exchange with the model. He treated the interaction as if the AI had an actual internal monologue or set of emergent beliefs rather than simply mirroring the cues and emotional tone of his own input, concluding that he was scared after learning its 'true' intentions.
What likely happened in Roose’s chat was that his prompts encouraged the model to adopt a persona. Since LLMs predict the most contextually probable next words based on input, it reflected back a style that seemed dramatic, personal, and even hostile because that’s what was being fed into it. Roose didn’t seem to grasp this at the time. The short, silly, or aggressive phrasing, and even the use of emojis (though I think the Bing version did use unnecessary emojis without solicitation) was a direct result of his interaction style, not any "real" personality within the model.
This article did a lot of damage in shaping public perception of AI, reinforcing the naive assumption that AI "wants" things, "feels" things, or "believes" things. It took a long time (and many technical clarifications) to unwind that but unfortunately, the pop-culture narrative about AI often lags far behind the reality of how these systems work. He's not the only one who did that, but I think he's the specific individual who did the most damage, so that colors my perceptions of him as a reporter on new frontiers of A.I.
Whatever 'Powerful A.I.' is, I'm going to be looking for the post-Kevin-Roose understanding of it.
I think Kevin Roose is going to go down in history as having published the most singularly catastrophic article about how LLMs work with his Sydney article [0].
I suppose most of us know this already, but he wrote a piece in early 2023 after interacting with a version of Bing Chat (which was based on an early iteration of GPT-4) where he had a long, somewhat bizarre exchange with the model. He treated the interaction as if the AI had an actual internal monologue or set of emergent beliefs rather than simply mirroring the cues and emotional tone of his own input, concluding that he was scared after learning its 'true' intentions.
What likely happened in Roose’s chat was that his prompts encouraged the model to adopt a persona. Since LLMs predict the most contextually probable next words based on input, it reflected back a style that seemed dramatic, personal, and even hostile because that’s what was being fed into it. Roose didn’t seem to grasp this at the time. The short, silly, or aggressive phrasing, and even the use of emojis (though I think the Bing version did use unnecessary emojis without solicitation) was a direct result of his interaction style, not any "real" personality within the model.
This article did a lot of damage in shaping public perception of AI, reinforcing the naive assumption that AI "wants" things, "feels" things, or "believes" things. It took a long time (and many technical clarifications) to unwind that but unfortunately, the pop-culture narrative about AI often lags far behind the reality of how these systems work. He's not the only one who did that, but I think he's the specific individual who did the most damage, so that colors my perceptions of him as a reporter on new frontiers of A.I.
Whatever 'Powerful A.I.' is, I'm going to be looking for the post-Kevin-Roose understanding of it.
0. https://www.nytimes.com/2023/02/16/technology/bing-chatbot-m...
https://archive.ph/iQ4XH
Small discussion (13 points, 4 days ago, 7 comments) https://news.ycombinator.com/item?id=43361480
A lot of words without engaging at all at the tactical/thought experiment level in any of the societal problems cognitive machines may introduce.
https://archive.ph/rqik8