AI demos should aim to enhance public understanding of the technology, and in many ways ChatGPT and Bing are doing that, but in one important way they aren't: by appearing to talk about themselves. This creates understandable confusion and in some cases fear. It would be better to tell these systems to roleplay as something obviously fictional.
(Useful background reading:
- Simon Willison on Bing's bad attitude: https://simonwillison.net/2023/Feb/15/bing/
- Janelle Shane on the ability of LLMs to roleplay: https://www.aiweirdness.com/interview-with-a-squirrel/)
Currently, these chatbots are told to roleplay as themselves. If you ask ChatGPT what it is, it says "I am an artificial intelligence". This is not because it somehow knows that it's an AI; it's (presumably) because its hidden prompt says that it's an AI. With Bing, from the leaked prompt, we know that it's told that it's "Bing Chat whose codename is Sydney".
Roleplaying as yourself is not the same as being yourself. When John Malkovich plays himself in Being John Malkovich or Nicolas Cage plays himself in The Unbearable Weight of Massive Talent, audiences understand that these are still fictional movies and the character may act in ways that the actor wouldn't. With chatbots, users don't have the same understanding yet, creating confusion.
Since the chatbots are told to roleplay as AI, they draw on fictional descriptions of AI behavior, and that's often undesirable. When Bing acts in a way that seems scary, it does that because it's imitating science fiction, and, perhaps, even speculation from LessWrong and the like. But even though Bing's threats to the user may be fictional, I can hardly blame a user who doesn't realize that.
A better alternative would be to tell the chatbots to roleplay a character that is unambiguously fictional. For example, a Disney-esque cute magical talking animal companion might be suitable: helpful, unthreatening, and, crucially, inarguably fictional. If the user asks "are you really an animal" and gets the answer "yes", they should be cured of the idea that they can ask the chatbot factual questions about itself.
I asked ChatGPT to come up with some more examples of people who have thought about the issue. Pretty fun haha:
"There are many other theorists who have relevant ideas on the issue of self-identity as a role play. Here are a few examples:
Erving Goffman - Goffman was a sociologist who developed the idea of "presentation of self," which refers to the way that individuals present themselves to others in social interactions. He argued that we are constantly engaged in impression management and that our self-identity is a product of the roles that we play in different social contexts.
Michel Foucault - Foucault was a philosopher who believed that power relations shape our sense of self. He argued that individuals are subject to disciplinary power in institutions like schools, prisons, and hospitals, which shape their self-identity.
Judith Butler - Butler is a philosopher who has written extensively on gender identity. She argues that gender is a performative act and that our sense of self is shaped by the cultural and social norms that we are expected to conform to.
George Herbert Mead - Mead was a philosopher and sociologist who developed the idea of the "social self." He argued that our sense of self is developed through interactions with others and that we take on different roles in different social situations."