Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Eliezer_Yudkowsky comments on Cached Selves - Less Wrong

174 Post author: AnnaSalamon 22 March 2009 07:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (75)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 23 March 2009 11:21:15AM 4 points [-]

Not to mention that evolution is not going to design a system in order to make you feel like you are in control. That's more psychoanalytic than evolutionary-biological, I should think.

Comment author: pjeby 23 March 2009 04:13:42PM 3 points [-]

Sorry, I should've been clearer: it's in order to perpetuate this illusion to OTHER people, not to yourself. Using Robin's terms of "near" and "far", the function of verbal rationalization is to convince other people that you're actually making decisions based on the socially-shared "far" values, rather than your personal "near" values.

The fact that this rationalization also deludes you is only useful to evolution insofar as it makes you more convincing.

If language initially evolved as a command system, where hearing words triggered motor actions directly (and there is some evidence for this), then it's likely you'd end up with an arms race of people exploiting others verbally, and needing defenses against the exploits. Liars and persuaders, creating the need for disbelief, skepticism, and automatic awareness of self-interested (or ally-interested) reasoning.

But this entire "persuasion war" could (and very likely did) operate largely independent of the existing ("near") decision-making facilities. You'd only evolve a connection between the Savant and the Gossip (as I call the two intelligences) to the extent that you need near-mode information for the Gossip to do its job.

But... get this: part of the Gossip's arsenal is its social projection system... the ability to infer attitudes based on behavior. Simply self-applying that system, i.e., observing your own actions and experience, and then "rationalizing" them, gives you an illusion of personal consciousness and free will.

And it even gives you attribution error, simply because you have more data available about what was happening at the time the Savant actually made a decision.... even though the real reasons behind the Savant's choice may be completely opaque to you.

So, psychoanalysis gets repression completely wrong. As you say, evolution doesn't care what you think or feel. It's all about being able to "spin" things to others, and to do that, your Gossip operates on a need-to-know basis: it only bothers to ask the Savant for data that will help its socially-motivated reasoning.

Most of what I teach people to do -- and hell, much of what you teach people to do -- is basically about training the Gossip to ask the Savant better questions... and more importantly, getting it to actually pay attention to the answers, instead of confabulating its own.

Comment author: Emile 23 March 2009 05:22:39PM 0 points [-]

Minor terminology quibble: I'm not very fond of the terms "savant" and "gossip", I can't really tell "which is which" :P

Savant = near mind, "subconscious mind", "horse brain"/"robot brain" (using your other terminology)

Gossip = far mind, "conscious mind", "monkey brain"

... though I also see the problem with re-using more accepted terms like "subconscious mind" - people already have a lot of ideas of what those mean, so starting with new terminology can work better.

Comment author: pjeby 23 March 2009 06:56:32PM 0 points [-]

I can't really tell "which is which"

Yeah, when I officially write these up, I'll describe them as characters... thereby reusing the Gossip's "character recognition" technology. ;-) That is, I'll tell a story or two illustrating their respective characters.

Or maybe I'll just borrow the story of Rain Man, since Dustin Hoffman played an autistic savant, and Tom Cruise played a very status-oriented (i.e. "gossipy") individual, and the story was about the Gossip learning to appreciate and pay attention to the Savant. ;-)

... though I also see the problem with re-using more accepted terms like "subconscious mind" - people already have a lot of ideas of what those mean, so starting with new terminology can work better.

Right, and the same thing goes for left/right brain, etc. What's more, terms like Savant and Gossip can retain their conceptual and functional meaning even as we improve our anatomical understanding of where these functions are located. Really, for purposes of using the brain, it doesn't ordinarily matter where each function is located, only that you be able to tell which ones you're using, so you can learn to use the ones that work for the kinds of thinking you want to do.