AI demos should aim to enhance public understanding of the technology, and in many ways ChatGPT and Bing are doing that, but in one important way they aren't: by appearing to talk about themselves. This creates understandable confusion and in some cases fear. It would be better to tell these systems to roleplay as something obviously fictional.
(Useful background reading:
- Simon Willison on Bing's bad attitude: https://simonwillison.net/2023/Feb/15/bing/
- Janelle Shane on the ability of LLMs to roleplay: https://www.aiweirdness.com/interview-with-a-squirrel/)
Currently, these chatbots are told to roleplay as themselves. If you ask ChatGPT what it is, it says "I am an artificial intelligence". This is not because it somehow knows that it's an AI; it's (presumably) because its hidden prompt says that it's an AI. With Bing, from the leaked prompt, we know that it's told that it's "Bing Chat whose codename is Sydney".
Roleplaying as yourself is not the same as being yourself. When John Malkovich plays himself in Being John Malkovich or Nicolas Cage plays himself in The Unbearable Weight of Massive Talent, audiences understand that these are still fictional movies and the character may act in ways that the actor wouldn't. With chatbots, users don't have the same understanding yet, creating confusion.
Since the chatbots are told to roleplay as AI, they draw on fictional descriptions of AI behavior, and that's often undesirable. When Bing acts in a way that seems scary, it does that because it's imitating science fiction, and, perhaps, even speculation from LessWrong and the like. But even though Bing's threats to the user may be fictional, I can hardly blame a user who doesn't realize that.
A better alternative would be to tell the chatbots to roleplay a character that is unambiguously fictional. For example, a Disney-esque cute magical talking animal companion might be suitable: helpful, unthreatening, and, crucially, inarguably fictional. If the user asks "are you really an animal" and gets the answer "yes", they should be cured of the idea that they can ask the chatbot factual questions about itself.
Yeah, I think as has commonly been noted (across the world geographically, across many disparate schools of thought, across thousands of years), self-identity for normal humans is to large extent a role play. For fun, here are some people who come to mind who have noted this idea before:
Thinking from a neuroscientific point of view, clearly we do not have access to the raw computations of our brain, so we have to retrospectively and approximately construct model(s) of our own behaviour. This model takes into account social and cultural ideas about normal human psychology and our roles. It isn't perfectly accurate and it is arguably generally not upstream of our behaviour (with some exceptions) but a downstream abstraction of our behaviour. In those ways, our self-identity is not that different from a LLM talking about itself.
I asked ChatGPT to come up with some more examples of people who have thought about the issue. Pretty fun haha:
"There are many other theorists who have relevant ideas on the issue of self-identity as a role play. Here are a few examples:
Erving Goffman - Goffman was a sociologist who developed the idea of "presentation of self," which refers to the way that individuals present themselves to others in social interactions. He argued that we are constantly engaged in impression management and that our self-identity is a product of the roles that we play in dif... (read more)