Belief in Self-Deception

27Eliezer_Yudkowsky05 March 2009 03:20PM

Continuation ofNo, Really, I've Deceived Myself
Followup toDark Side Epistemology

I spoke yesterday of my conversation with a nominally Orthodox Jewish woman who vigorously defended the assertion that she believed in God, while seeming not to actually believe in God at all.

While I was questioning her about the benefits that she thought came from believing in God, I introduced the Litany of Tarski - which is actually an infinite family of litanies, a specific example being:

  If the sky is blue
      I desire to believe "the sky is blue"
  If the sky is not blue
      I desire to believe "the sky is not blue".

"This is not my philosophy," she said to me.

"I didn't think it was," I replied to her.  "I'm just asking - assuming that God does not exist, and this is known, then should you still believe in God?"

She hesitated.  She seemed to really be trying to think about it, which surprised me.

"So it's a counterfactual question..." she said slowly.

I thought at the time that she was having difficulty allowing herself to visualize the world where God does not exist, because of her attachment to a God-containing world.

Now, however, I suspect she was having difficulty visualizing a contrast between the way the world would look if God existed or did not exist, because all her thoughts were about her belief in God, but her causal network modelling the world did not contain God as a node.  So she could easily answer "How would the world look different if I didn't believe in God?", but not "How would the world look different if there was no God?"

She didn't answer that question, at the time.  But she did produce a counterexample to the Litany of Tarski:

She said, "I believe that people are nicer than they really are."

I tried to explain that if you say, "People are bad," that means you believe people are bad, and if you say, "I believe people are nice", that means you believe you believe people are nice.  So saying "People are bad and I believe people are nice" means you believe people are bad but you believe you believe people are nice.

I quoted to her:

  "If there were a verb meaning 'to believe falsely', it would not have any
  significant first person, present indicative."
          -- Ludwig Wittgenstein

She said, smiling, "Yes, I believe people are nicer than, in fact, they are.  I just thought I should put it that way for you."

  "I reckon Granny ought to have a good look at you, Walter," said Nanny.  "I reckon
  your mind's all tangled up like a ball of string what's been dropped."
          -- Terry Pratchett, Maskerade

And I can type out the words, "Well, I guess she didn't believe that her reasoning ought to be consistent under reflection," but I'm still having trouble coming to grips with it.

I can see the pattern in the words coming out of her lips, but I can't understand the mind behind on an empathic level.  I can imagine myself into the shoes of baby-eating aliens and the Lady 3rd Kiritsugu, but I cannot imagine what it is like to be her.  Or maybe I just don't want to?

This is why intelligent people only have a certain amount of time (measured in subjective time spent thinking about religion) to become atheists.  After a certain point, if you're smart, have spent time thinking about and defending your religion, and still haven't escaped the grip of Dark Side Epistemology, the inside of your mind ends up as an Escher painting.

(One of the other few moments that gave her pause - I mention this, in case you have occasion to use it - is when she was talking about how it's good to believe that someone cares whether you do right or wrong - not, of course, talking about how there actually is a God who cares whether you do right or wrong, this proposition is not part of her religion -

And I said, "But I care whether you do right or wrong.  So what you're saying is that this isn't enough, and you also need to believe in something above humanity that cares whether you do right or wrong."  So that stopped her, for a bit, because of course she'd never thought of it in those terms before.  Just a standard application of the nonstandard toolbox.)

Later on, at one point, I was asking her if it would be good to do anything differently if there definitely was no God, and this time, she answered, "No."

"So," I said incredulously, "if God exists or doesn't exist, that has absolutely no effect on how it would be good for people to think or act?  I think even a rabbi would look a little askance at that."

Her religion seems to now consist entirely of the worship of worship.  As the true believers of older times might have believed that an all-seeing father would save them, she now believes that belief in God will save her.

After she said "I believe people are nicer than they are," I asked, "So, are you consistently surprised when people undershoot your expectations?"  There was a long silence, and then, slowly:  "Well... am I surprised when people... undershoot my expectations?"

I didn't understand this pause at the time.  I'd intended it to suggest that if she was constantly disappointed by reality, then this was a downside of believing falsely.   But she seemed, instead, to be taken aback at the implications of not being surprised.

I now realize that the whole essence of her philosophy was her belief that she had deceived herself, and the possibility that her estimates of other people were actually accurate, threatened the Dark Side Epistemology that she had built around beliefs such as "I benefit from believing people are nicer than they actually are."

She has taken the old idol off its throne, and replaced it with an explicit worship of the Dark Side Epistemology that was once invented to defend the idol; she worships her own attempt at self-deception.  The attempt failed, but she is honestly unaware of this.

And so humanity's token guardians of sanity (motto: "pooping your deranged little party since Epicurus") must now fight the active worship of self-deception - the worship of the supposed benefits of faith, in place of God.

This actually explains a fact about myself that I didn't really understand earlier - the reason why I'm annoyed when people talk as if self-deception is easy, and why I write entire blog posts arguing that making a deliberate choice to believe the sky is green, is harder to get away with than people seem to think.

It's because - while you can't just choose to believe the sky is green - if you don't realize this fact, then you actually can fool yourself into believing that you've successfully deceived yourself.

And since you then sincerely expect to receive the benefits that you think come from self-deception, you get the same sort of placebo benefit that would actually come from a successful self-deception.

So by going around explaining how hard self-deception is, I'm actually taking direct aim at the placebo benefits that people get from believing that they've deceived themselves, and targeting the new sort of religion that worships only the worship of God.

Will this battle, I wonder, generate a new list of reasons why, not belief, but belief in belief, is itself a good thing?  Why people derive great benefits from worshipping their worship?  Will we have to do this over again with belief in belief in belief and worship of worship of worship?  Or will intelligent theists finally just give up on that line of argument?

I wish I could believe that no one could possibly believe in belief in belief in belief, but the Zombie World argument in philosophy has gotten even more tangled than this and its proponents still haven't abandoned it.

I await the eager defenses of belief in belief in the comments, but I wonder if anyone would care to jump ahead of the game and defend belief in belief in belief?  Might as well go ahead and get it over with.

Comments (48)

pjeby06 March 2009 04:55:13AM6 points [-]

I'll go one step further and defend belief in belief, infinitely regressed. ;-) As you point out, the placebo effect here is simply the expectation of a positive result -- and it applies equally at any level of recursion here.

Humans only need a convincing argument for predicting a positive result, not a rational proof of that prediction! Once the positive result is expected, we get positive emotions activated every time we think of anything linked to that result, leading to self-fulfilling prophecies on every level.

This being the case, one might question whether it's rational to disbelieve in belief, if you have nothing equally beneficial to replace it with.

When it comes to external results, sure, it makes sense to have greater prediction accuracy. But for interior events -- like confidence, creativity, self-esteem, etc. -- biasing one's predictions positively is a significant advantage, as it stabilizes what would otherwise be an unstable system of runaway feedback loops.

People whose systems are negatively biased, on the other hand, can get seriously stuck. They basically hit one little setback and become paralyzed because of runaway negative self-fulfilling prophecy.

(I've been such a person myself, and I've worked with/on many of them. Indeed, it was noticing that other, far less "rational" and "intelligent" individuals were much more confident, calm, and successful than I was, that led me to start seriously investigating the nature of mind and beliefs in the first place, and to begin noting the distinctions between people I dubbed "naturally successful" and those I considered "naturally struggling".)

Tyrrell_McAllister05 March 2009 10:29:00PM13 points [-]

I don't know how well you know this person, so my advice may be unnecessary. But your post gives me the impression that you need to be much more careful about speculating on how her mind works. I think that it's a red flag when you write first that

I can see the pattern in the words coming out of her lips, but I can't understand the mind behind on an empathic level. I can imagine myself into the shoes of baby-eating aliens and the Lady 3rd Kiritsugu, but I cannot imagine what it is like to be her.

. . . and then proceed to make apparently confident declarations about how her mind works, such as

I now realize that the whole essence of her philosophy was her belief that she had deceived herself, and the possibility that her estimates of other people were actually accurate, threatened the Dark Side Epistemology that she had built around beliefs such as "I benefit from believing people are nicer than they actually are."

She has taken the old idol off its throne, and replaced it with an explicit worship of the Dark Side Epistemology that was once invented to defend the idol; she worships her own attempt at self-deception. The attempt failed, but she is honestly unaware of this.

As you yourself have observed, we largely understand other people by taking a portion of our own black-box mind, plugging in a few explicit settings (such as beliefs or experiences), letting the model run for a bit, and seeing what pops out. In particular, to understand how another person makes judgments, we collect their evinced beliefs, try to twiddle some dials until our model expresses the same beliefs, and then let it run for a bit. We then try to peer into the model as best we can, getting as good a picture of its inner workings as introspection allows us. We then take this picture as our hypothesis about how the other person thinks.

But the first quote above is strong evidence that your mind works differently from hers in some highly relevant respects. Therefore, you should be highly skeptical that what is going on in her mind resembles what it took to make the model of her in your own mind match her utterances. But you give me the impression that you haven't been sufficiently skeptical of the match between her mind and your model of it. I think that this has led you astray on several points.

For example, based on what you've written, I don't think that you're using the right model to understand what was going on in her mind when she said, "I believe that people are nicer than they really are." You were led to this confusion because she was not using the word "believe" in the way that you, and your model of her, do. You are using "belief" to mean a feature of a model of how the world is. But that, I expect, is not what she meant. Thus, your remarks here --

I tried to explain that if you say, "People are bad," that means you believe people are bad, and if you say, "I believe people are nice", that means you believe you believe people are nice. So saying "People are bad and I believe people are nice" means you believe people are bad but you believe you believe people are nice.

-- were irrelevant because they do not apply to the sense of the word "believe" that she was using.

For what it's worth, in my model of her, when she said "I believe that people are nicer than they really are," she meant, "When I reflect on my emotional attitude towards people, I see that this attitude is of the sort that, in the absence of its actual cause, could have been caused by a falsely high belief (in your sense) about peoples' niceness."

The actual cause for her emotional attitude is perhaps her "religion". Or perhaps it is something else. Perhaps she has no idea what the actual cause is, or perhaps she thinks she does, but she doesn't really. But none of this implies that she was attributing to herself the belief that people are nicer than she actually believes them to be (where, here, I'm using "belief" in your sense.)

Her utterance seems analogous to someone who walks out of an optometrist's office after having his pupils dilated and says, "Because of those drops the optometrist gave me, I believe the sun is brighter than it really is." If we heard this, we shouldn't conclude that he believes something contradictory, or that he has incorrect beliefs about his beliefs. His word "belief" in this case probably does not mean "best guess about how things really are." Rather, it's a clumsy way to say that some qualities of his experience of the world are as if he had a certain belief (in the sense normally understood). He does not mean to imply that he has any wrong beliefs (in the conventional sense). It would be a mistake to say that his subjective experience of the light is in any way erroneous. After all, it accurately reflects the fact that he had those drops put in his eyes.

Similarly, your interlocutor's statement that she "believes" that people are nicer than they really are referred to a particular quality of her emotional attitude towards them, not to a belief (in your sense) about how they are. In particular, it didn't imply any expectation about how they would behave. That, I expect, is why she was initially taken aback when you asked, "So, are you consistently surprised when people undershoot your expectations?" The problem wasn't, as you appear to think, that she had prevented her own mind from drawing obvious conclusions. The problem was that you (because of her confusing wording) were speaking of her so-called "belief" as though it were a belief in the normal sense, something that should lead to certain expectations about other peoples' actions. But I expect that it wasn't any such thing, notwithstanding her unfortunate choice of words.

billswift06 March 2009 12:34:13AM2 points [-]

I think the first time Eliezer said he couldn't get into her mind was that he couldn't understand the psychological state she needed to be in to make that statement. The second time - where he was writing about what she believed - he was discussing her apparent epistemological state.

There are significant differences between the two for observers. I can almost never understand someone else's psychological state, but I can often figure out what they are talking about and how they got there epistemologically - that is, what could have caused their stated beliefs.

Eliezer_Yudkowsky06 March 2009 12:02:26AM1 point [-]

For what it's worth, in my model of her, when she said "I believe that people are nicer than they really are," she meant, "When I reflect on my emotional attitude towards people, I see that this attitude is of the sort that, in the absence of its actual cause, could have been caused by a falsely high belief (in your sense) about peoples' niceness."

An interesting hypothesis, Tyrrell; but she explicitly explained to me about how, if you think people are nicer than they really are, then this makes you happier.

Tyrrell_McAllister06 March 2009 09:15:27AM1 point [-]

You're right to call it a mere hypothesis. I hope that I made its tentative nature clear.

But that explanation of hers seems to me to be consistent with my hypothesis. No surprise, because it was part of the data that I was trying to fit when I constructed it.

I would be curious to know more about how she responded when you asked her, "So, are you consistently surprised when people undershoot your expectations?" Did she have anything more to say after repeating the question?

Yasser_Elassal06 March 2009 07:52:47PM2 points [-]

My hypothesis is that she simply meant, "It makes me happy to pretend that people are nicer than they really are."

cleonid06 March 2009 12:21:51AM* 3 points [-]

Voltaire, using rationalist arguments, concluded that “if God did not exist, it would be necessary to invent him”. So could it be that adhering to facts in all situations is essentially an irrational position?

Consider the following statements:

1) Rational humans (unlike rational AI) should aim to be happy.

2) Rational humans should not believe fanciful notions unsupported by empirical evidence.

3) Empirical studies (e.g. http://www.lifesitenews.com/ldn/2008/mar/08031807.html) suggest that humans who believe in such notions are more likely to be happier.

The consequence of the above statements seems to be that a rational human should reject rationality.

Does anyone see flaws in this reasoning?

Psy-Kosh05 March 2009 10:28:46PM4 points [-]

Hrm... While on the one hand I can look at her position and basically react with a "your mind is entirely alien to me", on the other hand, I can actually imagine being in that state.

That does NOT mean, of course, that it is a reasonable state to be in, but it does seem to be the sort of state that my mind can support.

I guess the basic key is that human minds aren't necessarally naturally consistent. So we can end up in actual inconsistent states. Including states a bit confused about consistency itself.

A bit more of a personal example would be a state I sometimes recall having been in in the past, and have certainly seen in others, would be when one might say something like, oh, I dunno, "and scientifically, the universe is about 13.7 billion years old and earth is about 4.5 billion years old" and of course, the world was created about 6000 years ago."

As near as I can tell, happens is that we almost imagine the "scientific world" and the "religious world" as parallel universes that... are actually the same one, so mentally we keep track of it by keeping track of different things.

The way this works is someone might manage to end up in a state that they completely fail to really face the question of "okay, but if you rewind time a bit, will you see 6000 years ago the universe poofing into existence, or can you go farther back, etc? ie, what ACTUALLY happened in ACTUAL REALITY?"

Then, when facing that question, all sorts of Escher mentality stuff starts forming as a defense. But what I think initially happens, at least in part, is sort of mentally tracking those as being about different subjects, rather than contradictory statements about the same thing. So that one will end up, with "science glasses", visualizing prehistoric humans doing stuff tens of thousands of years ago, while etc etc etc...

At least, that's my own, partly introspective model of what's going on here, of how people can end up in these states.

Eliezer_Yudkowsky06 March 2009 12:05:03AM2 points [-]

I think that people who had actual mental models of the world would notice a contradiction that large.

People who profess two different beliefs may not see a contradiction. It's just good to profess one, and also good to profess the other, for different reasons. They aren't visualizing a world that, at one time or another, needs to either poof or go on. They're visualizing that "science" and "religion" both seem like good groups to join.

Psy-Kosh06 March 2009 01:13:03AM4 points [-]

I think that may be part of it, but I'm also thinking back a bit to when I was more religious, and so on, and also thinking about how some people I know seem to talk, and as near as I can tell, there really does seem to be a bit of that.

I'm claiming they're visualizing a world that goes "poof, 'LET THERE BE LIGHT!'". AND visualizing a world that goes farther back, and somehow doing some form of funny doublethink them thinking of those as different worlds that are both in some sense true, while some aspect of them is treating those not as contradictory models, but almost as, well, different worlds. ie, two different "truths" ("but what is truth?" :))

That is, simply holding the contradiction in place, having two "models", not along the lines of two competing models, but that (though they don't actually notice it), they're imagining it more as parallel worlds that, depending on circumstances, they'll consider either one or the other "this world"

They would (usually, see somewhat below) not ever actually say, or even notice that they're thinking that way. In other words, I'd expect if you asked such a person something like "do you believe in a set of parallel realities, one in which the world was spoken into existance ~6000 years ago, and another about 13.7 billion years old or at least certainly older than 6000 years", they'll probably give you funny looks. But I think, without them noticing, something like that is going on in how it's being stored.

And I can speak from personal experience about some of the REALLY weird stuff I used to think in terms of, so it's in part a "pay no attention to the contradiction behind the curtain" situation.

Heck, sometimes when I bring various contradictions up, I'll get responses like "this isn't a debate class" or "this isn't a court room and you're not a lawyer", and basically have it laughed off like that from some family members. (and, of course, the infamous "in your opinion" fully general retort to any position you don't like. :))

I'm not saying this is all of it, but it sure seems to me that something like this is going on in some cases. It may also be what underlies stuff like "I believe people are nicer than they are". That is, statements like that may partly cash out to "I have a couple different models of people, one of which says they're nicer than the other. I hold both of these at the same time, but I call one my belief, and one the actual situation"

At least, when I try to imagine being in a mental state that could provoke me to utter such a statement, ie, when I try to simulate that state on myself, that seems to be what the result "looks like."

Oh, that bit from earlier, well... sometimes it's made a bit explicit.

I've come across some bits of occult philosophy that basically talks about how there can be many histories that are "true" (no, not in the sense a physicist might talk about interference), and they'll explicitly say stuff like the "there's the actual historical history, but that's not the only 'true' one.."

But also just from introspection, well, it does feel to me that in the past I would be in such a state, have multiple models that I wasn't so much treating as competing so much as treating as, well, simply true, in different senses.

The Escher mental tangle can get REALLY strange. :)

Marcello05 March 2009 06:23:36PM* 8 points [-]

If I had been talking to the person you were talking to, I might have said something like this:

<del>Why are you deceiving yourself into believing Orthodox Judaism as opposed to something else? If you, in fact, are deriving a benefit from deceiving yourself, while at the same time being aware that you are deceiving yourself, then why haven't you optimized your deceptions into something other than an off-the-shelf religion by now?</del> Have you ever really asked yourself the question: "What is the set of things that I would derive the most benefit from falsely believing?" Now if you really think you can make your life better by deceiving yourself, and you haven't really thought carefully about what the exact set of things about which you would be better off deceiving yourself is, then it would seem unlikely that you've actually got the optimal set of self-deceptions in your brain. In particular, this means that it's probably a bad idea to deceive yourself into thinking that your present set of self deceptions is optimal, so please don't do that.

OK, now do you agree that finding the optimal set of self deceptions is a good idea? OK, good, but I have to give you one very important warning. If you actually want to have the optimal set of self deceptions, you'd better not deceive yourself at all while you are constructing this set of self deceptions, or you'll probably get it wrong, because if, for example, you are currently sub-optimally deceiving yourself into believing that it is good to believe X, then you may end up deceiving yourself into actually believing X, even if that's a bad idea. So don't self deceive while you're trying to figure out what to deceive yourself of.

Therefore, to the extent that you are in control of your self deceptions, (which you do seem to be) the first step toward getting the best set of self deceptions is to disable them all and begin a process of sincere inquiry as to what beliefs it is a good idea to have.


And hopefully, at the end of the process of sincere inquiry, they discover the best set of self deceptions happens to be empty. And if they don't, if they actually thought it through with the highest epistemic standards, and even considered epistemic arguments such as honesty being one's last defence, slashed tires, and all that.... Well, I'd be pretty surprised, but if I were actually shown that argument, and it actually did conform to the highest epistemic standards.... Maybe, provided it's more likely that the argument was actually that good, as opposed to my just being deceived, I'd even concede.

Disclaimer: I don't actually expect this to work with high confidence, because this sort of person might not actually be able to do a sincere inquiry. Regardless, if this sort of thought got stuck in their head, it could at least increase their cognitive dissonance, which might be a step on the road to recovery.

Roko05 March 2009 06:30:37PM5 points [-]

"Disclaimer: I don't actually expect this to work with high confidence, because this sort of person might not actually be able to do a sincere inquiry."

  • well exactly... If the person were thinking rationally enough to contemplate that argument, they really wouldn't need it.

I have never successfully converted a religious person to atheism, but my ex-girlfriend did. I am a more rational person than her, I know more philosophy, I have earnestly tried many times, she just did this once, etc. How did she do it? The person in question was male and his religion forbade him from sex outside marriage. Most people are mostly ruled by their emotions.

Marcello06 March 2009 12:47:39AM* 3 points [-]

well exactly... If the person were thinking rationally enough to contemplate that argument, they really wouldn't need it.

My working model of this person was that the person has rehearsed emotional and argumentative defenses to protect their belief, or belief in belief, and that the person had the ability to be reasonably rational in other domains where they weren't trying to be irrational. It therefore seemed to me that one strategy (while still dicey) to attempt to unconvince such a person would be to come up with an argument which is both:

  • Solid (Fooling/manipulating them into thinking the truth is bad cognitive citizenship, and won't work anyway because their defenses will find the weakness in the argument.)

  • Not the same shape as the argument their defenses are expecting.

Roko: How is your working model of the person different from mine?

Roko06 March 2009 04:17:46PM* 4 points [-]

My working model of a religious person such as the above is that they assess any argument first and foremost on the basis "will accepting this argument cause me to have to abandon my religious belief?". If yes, execute "search for least implausible counterargument".

As such, no rational argument whose conclusion obviously leads to the abandonment of religion will work. However, rational arguments that can be accepted on the spot without obviously threatening religion, and which lead via hard-to-predict emotional channels to the weakening and defeat of that belief might work. It is my suspicion that persuading someone to change their mind on a really important issue almost always works like this.

Annoyance05 March 2009 07:35:46PM1 point [-]

"she just did this once, etc. How did she do it? "

By appealing to a non-rational or irrational argument that would lead the person to adopt rationality.

Arguing rationally with a person who isn't rational that they should take up the process is a waste of time. If it would work, it wouldn't be necessary. It's easy to say what course should be taken with a rational person, because rational thought is all alike. Irrational thought patterns can be nearly anything, so there's no way to specify an argument that will convince everyone. You'd need to construct an argument that each person is specifically vulnerable to.

billswift06 March 2009 12:49:40AM1 point [-]

The problem is that you often don't know until you actually start arguing with them that they are irrational or just confused and misled.

George H Smith has a pretty good essay about arguing with people to convert them to rationality, " Atheism and the Virtue of Reasonableness". For example, he advocates the "Presumption of Rationality" - you should always presume your adversary is rational until he demostrates otherwise. I don't know if the essay is on-line or not, I read it as the second chapter of "Atheism, Ayn Rand, and Other Heresies."

eirenicon05 March 2009 07:17:39PM-1 points [-]

"Most people are mostly ruled by their emotions."

To be more specific, most men, for a considerable portion of their lives, are mostly ruled by their sex drives.

Eliezer_Yudkowsky05 March 2009 06:54:33PM4 points [-]

To be clear, she never did say, "I am deceiving myself" or "I falsely believe that there is a God".

Marcello06 March 2009 12:16:49AM1 point [-]

I stand corrected. I hereby strike the first two sentences.

pre05 March 2009 08:00:09PM* 2 points [-]

I would expect a reply along the lines of: It's precisely because I can't trust my own reasoning when deciding which false beliefs I should have that I accept these which are handed down. I pick Judaism because it's the oldest and thus has shown through memetic competition that it's the strongest set of false beliefs one could have.

Or ..." I pick Christianity because it's the most popular and has therefore proven itself memetically competitive."

I have a lot of friends who think "it's old therefore it must be good to have survived this long" about Tarot and eastern religions etc.

Personally I'd wanna eliminate the false beliefs even if it cost me my mojo, but that's a different set of priorities I guess.

RobinHanson05 March 2009 04:29:59PM8 points [-]

Consistent consciously intended self-deception may be hard. But our minds are designed to produce self-deceptions all the time without us noticing. Just don't look behind the curtain and "let it be", "go with the flow" etc. and you can be as self-deceived as most folks.

Ryan05 March 2009 11:10:02PM2 points [-]

I know some people who are like the woman you describe, my own folks might be like that to some extent. I became atheist pretty early on. So I'm not sure that adults who believe in belief are likely to be passing that along to their kids, if they even try. In my case, I put on a show for a while, but when I stopped it was no big deal.

If these people are able to agree with a scientific worldview and not be obstructionist on things like stem cell, but simply want to add "and I believe there is a god" to the end of it, fine. Seems like a natural step towards the end of belief in god entirely.

Roko05 March 2009 06:00:49PM5 points [-]

"And so humanity's token guardians of sanity (motto: "pooping your deranged little party since Epicurus") must now fight the active worship of self-deception - the worship of the supposed benefits of faith, in place of God."

  • As I keep saying, helping people to overcome biases (such as the above) is a lot easier if there are psychologically viable places for people to jump to once they've overcome their bias.

You should have spent much more of your time in this debate convincing your tangled friend that, if she were to abandon her religious belief (or belief in belief, or whatever), she would still be able to feel good about herself and good about life; that life would still be a happy meaningful place to be.

Maybe she has a massive internal guilt complex and thinks of herself as a bad person, and she thinks that only religion can help her with this. Maybe she is frightened that atheism will lead to nihilism.

PeteG05 March 2009 06:55:40PM2 points [-]

"You should have spent much more of your time in this debate convincing your tangled friend that, if she were to abandon her religious belief (or belief in belief, or whatever), she would still be able to feel good about herself and good about life; that life would still be a happy meaningful place to be."

I don't think Eliezer cared so much to correct someone's one wrong belief as much as he cared to correct the core that makes many such beliefs persist. Would he really have helped her if all his rational arguments failed, but his emotional one succeeded? My guess is that it wouldn't be a win for him or her.

Roko05 March 2009 07:06:15PM2 points [-]

Well that depends on whether your aim is to make people have correct beliefs, or whether you want to make people have correct beliefs by following the ritual of rational argument... and I think that EY would claim to be aiming for the former.

Annoyance05 March 2009 07:14:57PM2 points [-]

What use is it to have correct beliefs if you don't know they're correct?

If the belief cannot be conveniently tested empirically, or it would be useless to do so, the only way we can know that our belief is correct is by being confident of the methodology through which we reached it.

Comment deleted 05 March 2009 07:45:54PM* [-]
Annoyance05 March 2009 07:52:57PM3 points [-]

"I naturally prefer to have a high level of confidence in my beliefs."

Doesn't that depend on how reliable those beliefs are?

If you're fleeing through the temple pursued by a boulder, you don't want to dither at an intersection, so whichever direction you think you should go at one moment should be constant. But there's no reason why your confidence should be high to avoid dithering; you need merely be stable.

"'ll take the path I belief leads to safety. This will turn out to be a wise choice"

If, and only if, your belief is correct. If your belief is wrong your choice is a disastrous one. Rationality isn't about being right or choosing the best course, it's about knowing that you're right and knowing which is the best course to choose.

Comment deleted 05 March 2009 07:58:09PM[-]
Annoyance05 March 2009 08:10:56PM2 points [-]

Then I think I agree with you, mostly. If time or a similar limited resource makes rigorous justification too expensive, we shouldn't require it. But whatever we do accept should be minimally justified, even if it's just "I have no idea where to go so I'll pick at random".

I wouldn't look at the map if I were running from the boulder. But I would have looked at it before entering the temple, and you can bet I'd be trying very hard to retrace my steps on the way out, unless I thought I could identify a shortcut. Even then I might not take the gamble.

nazgulnarsil05 March 2009 09:25:14PM2 points [-]

This is a perfect example of the web that builds itself around even one confusion of a value statement and a factual statement. I fear we all have these lurking.

thomblake05 March 2009 03:33:44PM4 points [-]

Why destroy placebo effects? According to some stuff Robin Hanson points to, it seems that most of medicine might consist of placebos. Aren't you fighting what wins in favor of the truth?

ciphergoth05 March 2009 10:13:15PM1 point [-]

There is evidence that placebos work even if you know that they contain no active ingredients, so we may be spared this interesting dilemma!

Emile05 March 2009 04:09:22PM2 points [-]

To further illustrate the point that self-deception isn't easy: if believe you're shy, you can't just make yourself believe you're not shy.

Maybe you can make yourself believe that you believe that you're not shy, but I don't think you'll reap many benefits from placebo effect - you'll still get nervous when you want to speak up or go talk to a girl you don't know or whatnot. You can't argue yourself logically into self-confidence.

haig06 March 2009 12:45:30PM0 points [-]

Placebo effects from 'belief in (false) beliefs' only work as long as self-deception is maintainable.

I think the point at which self-deception ceases to work is when you can consciously be aware of it breaking your causal models of the world. Highly intelligent people, or anyone for that matter, cannot continue to deceive themselves into believing in god or unregulated markets, or whatever complex concept take your pick, if you explicitly show how it breaks a model they cannot disagree with. Controversial topics of the day like belief in god, public policy, etc. are not single data points under contention, but tangled balls of causation that must be dealt with in a somewhat parallel fashion--to see the big picture and say, wait a minute that cannot fit unless this, and this, and this, and finally reach a dead end and have to relinquish their starting belief. The more abstract or the more complex a concept is, the easier it is to deceive yourself of its falsehood.

The limits to working memory plays a role here, and if we are to truly be less wrong, we not only have to overcome biases, but we need to amplify our rational intelligence by using tools designed for these specific purposes. What if beliefs such as 'a personal god exists' were as hard to believe in as 'the sky is green'? What if it was explicitly laid out in front of someone that they absolutely could not hold a belief in something because of all the cascading links it breaks in their world model that is confirmed to be 'reality'.

I want to work on such tools.

Olle05 March 2009 09:11:13PM0 points [-]

I believe the following five things.

(1) Barcelona will not win the Champions League.

(2) Manchester U will not win the Champions League.

(3) Chelsea will not win the Champions League.

(4) Liverpool will not win the Champions League.

(5) I falsely believe one of the statements (1), (2), (3) and (4).

This seems to me like a reasonable counterexample to Wittgenstein's doctrine.

Olle06 March 2009 07:30:41AM1 point [-]

topynate: It was only for reasons of space that I listed five events with probability 0.8 each, rather than 1000 events with probability 0.999 each; the modification is obvious.

Eliezer: Point taken.

Eliezer_Yudkowsky06 March 2009 12:07:51AM3 points [-]

You need to work with probabilities, and then make statements about your expected Bayes-score instead of truth or falsity; then you'll be consistent. I have a post on this but I can't remember what it's called.

Z_M_Davis06 March 2009 12:57:01AM4 points [-]
thomblake05 March 2009 09:15:29PM1 point [-]

I think Wittgenstein's point was that you're using 'believe' in a strange way. I have no idea what you meant by the above comment; you're effectively claiming to believe and not believe the same statement simultaneously.

If you're using paraconsitent logic, you should really specify that before making a point, so the rest of us can more efficiently disregard it.

Olle05 March 2009 09:28:51PM1 point [-]

I judge each of the four teams to have probability 0.2 of winning the Champions League. Their victories are mutually exclusive. Hence I judge each of statements (1)-(5) to have probability 0.8.

topynate06 March 2009 12:29:06AM2 points [-]

Hm. Wittgenstein requires that the meaning be "indicative". In English the indicative mood is used to express statements of fact, or which are very probable. They don't necessarily have to be true or probable, of course, but they express beliefs of that nature. You say "I believe X" when you assign a probability of at least 0.8 to X; 0.8 is probable, but not very probable. Would you state baldly "Barcelona will not win the Champions League", given your probabilities? I doubt it. When you say instead "I believe Barcelona will not win the Champions League", you could equally say "Barcelona will probably not win the Champions League." But this isn't in the indicative mood, but rather in something called the potential/tentative mood, which has no special form in English, but does in some other languages, e.g. daro in Japanese (which has quite a complex system for expressing probability). It's better to just say your degree of belief as a numeric probability.