eli_sennesh comments on Confused as to usefulness of 'consciousness' as a concept - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (229)
If we're going the game theory route, there's a natural definition for consciousness: something which is being modeled as a game-theoretic agent is "conscious". We start projecting consciousness the moment we start modelling something as an agent in a game, i.e. predicting that it will choose its actions to achieve some objective in a manner dependent on another agent's actions. In short, "conscious" things are things which can be bargained with.
This has a bunch of interesting/useful ramifications. First, consciousness is inherently a thing which we project. Consciousness is relative: a powerful AI might find humans so simple and mechanistic that there is no need to model them as agents. Consciousness is a useful distinction for developing a sustainable morality, since you can expect conscious things to follow tit-for-tat, make deals, seek retribution, and all those other nice game-theoretical things. I care about the "happiness" of conscious things because I know they'll seek to maximize it, and I can use that. I expect conscious things to care about my own "happiness" for the same reason.
This intersects somewhat with self-awareness. A game-theoretic agent must, at the very least, have a model of their partner(s) in the game(s). The usual game-theoretic model is largely black-box, so the interior complexity of the partner is not important. The partners may have some specific failure modes, but for the most part they're just modeled as maximizing utility (that's why utility is useful in game theory, after all). In particular, since the model is mostly black-box, it should be relatively easy for the agent to model itself this way. Indeed, it would be very difficult for the agent to model itself any other way, since it would have to self-simulate. With a black-box self-model armed with a utility function and a few special cases, the agent can at least check its model against previous decisions easily.
So at this point, we have a thing which can interact with us, make deals and whatnot, and generally try to increase its utility. It has an agent-y model of us, and it can maybe use that same agent-y model for itself. Does this sound like our usual notion of consciousness?
What game-theory route?