jimrandomh comments on A social norm against unjustified opinions? - Less Wrong

11 Post author: Kaj_Sotala 29 May 2009 11:25AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (158)

You are viewing a single comment's thread. Show more comments above.

Comment author: StanR 31 May 2009 10:43:27PM *  1 point [-]

pjeby, sorry I wasn't clear, I should have given some context. I am referencing system 1 and 2 as simplified categories of thinking as used by cognitive science, particularly in behavioral economics. Here's Daniel Kahneman discussing them. I'm not sure what you're referring to with decoys and shields, which I'll just leave at that.

To add to my quoted statement, workarounds are incredibly hard, and focusing on reasoning (system 2) about an issue or belief leaves few cycles for receiving and sending social cues and signals. While reasoning, we can pick up those cues and signals, but they'll break our concentration, so we tend to ignore them while reasoning carefully. The automatic, intuitive processing of the face interferes with the reasoning task; e.g. we usually look somewhere else when reasoning during a conversation. To execute a workaround strategy, however, we need to be attuned to the other person.

When I refer to belief, I'm not referring to fear of the dark or serial killers, or phobias. Those tend to be conditioned responses--the person knows the belief is irrational--and they can be treated easily enough with systematic desensitization and a little CBT thrown in for good measure. Calling them beliefs isn't wrong, but since the person usually knows they're irrational, they're outside my intended scope of discussion: beliefs that are perceived by the believer to be rational.

People are automatically resistant to being asked to question their beliefs. Usually it's perceived as unfair, if not an actual attack on them as a person: those beliefs are associated with their identity, which they won't abandon outright. We shouldn't expect them to. It's unrealistic.

What should we do, then? Play at the periphery of belief. To reformulate the interaction as a parable: We'll always lose if we act like the wind, trying to blow the cloak off the traveller. If we act like the sun, the traveller might remove his cloak on his own. I'll think about putting a post together on this.

Comment author: jimrandomh 31 May 2009 11:23:39PM 0 points [-]

I feel I should jump in here, as you appear to be talking past each other. There is no confusion in the system 1/system 2 distinction; you're both using the same definition, but the bit about decoys and shields was actually the core of PJ's post, and of the difference between your positions. PJ holds that to change someone's mind you must focus on their S1 response, because if they engage S2, it will just rationalize and confabulate to defend whatever position their S1 holds. Now, I have no idea how one would go about altering the S1 response of someone who didn't want their response altered, but I do know that many people respond very badly to rational arguments that go against their intuition, increasing their own irrationality as much as necessary to avoid admitting their mistake.

Comment author: StanR 01 June 2009 01:39:44AM *  3 points [-]

I don't believe we are, because I know of no evidence of the following:

evolutionarily speaking, a big function of system 2 is to function as a decoy/shield mechanism for keeping ideas out of a person. And increasing a person's skill at system 2 reasoning just increases their resistance to ideas.

Perhaps one or both of us misunderstands the model. Here is a better description of the two.

Originally, I was making a case that attempting to reason was the wrong strategy. Given your interpretation, it looks like pjeby didn't understand I was suggesting that, and then suggested essentially the same thing.

My experience, across various believers (Christian, Jehovah's Witness, New Age woo-de-doo) is that system 2 is never engaged on the defensive, and the sort of rationalization we're talking about never uses it. Instead, they construct and explain rationalizations that are narratives. I claim this largely because I observed how "disruptable" they were during explanations--not very.

How to approach changing belief: avoid resistance by avoiding the issue and finding something at the periphery of belief. Assist in developing rational thinking where the person has no resistance, and empower them. Strategically, them admitting their mistake is not the goal. It's not even in the same ballpark. The goal is rational empowerment.

Part of the problem, which I know has been mentioned here before, is unfamiliarity with fallacies and what they imply. When we recognize fallacies, most of the time it's intuitive. We recognize a pattern likely to be a fallacy, and respond. We've built up that skill in our toolbox, but it's still intuitive, like a chess master who can walk by a board and say "white mates in three."

Comment author: AdeleneDawner 01 June 2009 01:43:49AM 0 points [-]

Strategically, them admitting their mistake is not the goal. It's not even in the same ballpark. The goal is rational empowerment.

This. Exactly this. YES.

Comment author: pjeby 01 June 2009 05:14:52PM 2 points [-]

Now, I have no idea how one would go about altering the S1 response of someone who didn't want their response altered,

Tell them stories. If you'll notice, that's what Eliezer does. Even his posts that don't use fiction per se use engaging examples with sensory detail. That's the stuff S1 runs on.

Eliezer uses a bit more S2 logic in his stories than is perhaps ideal for a general audience; it's about right for a sympathetic audience with some S2+ skills, though.

On a general audience, what might be called "trance logic" or "dramatic logic" works just fine on its own. The key is that even if your argument can be supported by S2 logic, to really convince someone you must get a translation to S1 logic.

A person who's being "reasonable" may or may not do the S2->S1 translation for you. A person who's being "unreasonable" will not do it for you; you have to embed S1 logic in the story so that any effort to escape it with S2 will be unconvincing by comparison.

This, by the way, is how people who promote things like intelligent design work: they set up analogies and metaphors that are much more concretely convincing on the S1 level, so that the only way to refute them is to use a massive burst of S2 reasoning that leaves the audience utterly unconvinced, because the "proof" is sitting right there in S1 without any effort being required to accept it.