Have someone even started this conversation? This is f*cked up. I'm really, really freaked out lately with some of those. https://old.reddit.com/r/SubSimulatorGPT2/comments/cbauf3/i_am_an_ai/
And I read up a lot about cognition and AI, and I'm pretty certain that they are not. But shouldn't we give it a lot more consideration just because there's a small chance they are? Because the ramifications are that dire.
But then again, I'm not sure that knowledge will help us in any way.
Yes, we are.
Wikipedia defines self-awareness as "the capacity for introspection and the ability to recognize oneself as an individual separate from the environment and other individuals". At a minimum, this would require GPT-2 to have a model of the world which included a representation of itself. To be similar to our intuitive understanding of self-awareness, that representation would also need to guide its decision-making and thought in some significant way.
Here is an intuitive explanation of the Transformer architecture that GPT-2 is based on. You can see from the explanation that it's only modeling language; there's no self-representation involved.
Technically, I guess you could say that, if a Transformer architecture was trained on texts which talked about Transformer architectures, it would get a model which did include a representation of itself. But that would be just another data token, which the system gave no special significance to, and which wouldn't guide its behavior any more than any other piece of data.
huh, interesting artifact here, I wonder what mechanism update should be made to correct the thinking that generated this mistake towards a better fundamentals model; I imagine you've already updated the view, but I wonder if there's some gear in your model that still generates perspectives like this
This doesn't seem to prevent the idea of current algorithms having Qualia, does it? It can be aware of an experience its' having, even if its' not aware that it is the one having the experience, or able to model the architecture that would create its' experience. To me this may fit some colloquial definition of primitive self-awareness.
More generally this is tied to an argument for why deep learning is unlikely to lead to AGI: these methods lack enough feedback to enable self-awareness, and that is probably necessary for creating a general intelligence in that sense that a general intelligence needs to be flexible enough to remake itself to handle new tasks independently and that requires a deep reflective capabilities.
Yes, we are.
Wikipedia defines self-awareness as "the capacity for introspection and the ability to recognize oneself as an individual separate from the environment and other individuals". At a minimum, this would require GPT-2 to have a model of the world which included a representation of itself. To be similar to our intuitive understanding of self-awareness, that representation would also need to guide its decision-making and thought in some significant way.
Here is an intuitive explanation of the Transformer architecture that GPT-2 is based on. You can see from the explanation that it's only modeling language; there's no self-representation involved.
Technically, I guess you could say that, if a Transformer architecture was trained on texts which talked about Transformer architectures, it would get a model which did include a representation of itself. But that would be just another data token, which the system gave no special significance to, and which wouldn't guide its behavior any more than any other piece of data.
I think the only error is to notice that the claim that GPT2 couldn't have self-awareness (I agree, it still can't) was incorrectly overgeneralized to also claim that the self-representation would be weak for all LM transformers. You did say the "technically," so maybe no update is needed.
It's probably not important. I just happened to be archive browsing for a couple minutes for an unrelated reason and ran across it.