Today's post, Protected From Myself was originally published on 19 October 2008. A summary (taken from the LW wiki):

 

Ethics can protect you from your own mistakes, especially when your mistakes are about really fundamental things.


Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Dark Side Epistemology, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.

New Comment
11 comments, sorted by Click to highlight new comments since:

Don't lie, because what you believe now is probably wrong. if you lie to convince others of what you currently believe, then the same evidence which later causes you to change your belief may not suffice to change theirs.

It's even worse -- if you successfully lie to convince others, their opinions may be the evidence that later convinces you.

In theory this should not happen, because a perfect reasoner should be able to discount the effect of their lies on others. But humans are not like this. If everyone else believes something, there is a strong pressure to join them. (A good rationalization could be: "First I was just joking, but gradually I realized that this is real.")

I would guess that there were many cult leaders who started by lying to people around them, and gradually accepted their own lies when repeated by the others.

Sadly, it's even worse than that... sometimes, the very act of lying serves to gradually change my beliefs, even if nobody else is convinced. (Admittedly, this depends somewhat on how committed I am to my self-image as a liar.)

Given that one of the best ways to tell a deliberate lie is to believe it, I can understand why good liars often end up convincing themselves.

For that matter, it's a technique that I think I've personally used to change my own beliefs, for the saner. (If that's not confusing, you're either not working or have information I haven't shared.)

Well, I understood you to mean that there are beliefs you currently have, that at one time you did not have, and you suspect that the way you managed the transition was by lying about your beliefs and coming to believe your own lies, and you endorse your current beliefs as saner than your previous beliefs.

I did not experience much confusion parsing it, though it is of course possible I misunderstood.

I didn't lie about my beliefs, in the sense of value judgements. I lied about my beliefs regarding matters of fact, like my memories.

(nods) OK.

I agree with Decius. Do you have a wiki account, so you can post your own edit under your own name?

I don't have a wiki account, and I don't feel the need to retain credit.

I might clean it up a bit if I was putting it out for summary rather than for discussion:

'Don't lie or use other black arts to convince others to believe what you believe, because...'

I'm not sure if using this line to convince others that lies are immoral is hypocrisy, meta-hypocrisy, or both. The statement itself is purely rationalist pragmatism.

This seems worth adding to the summary.

The universe isn't set up to reward virtue—so why did my ethics help so much?

Because the ethics are set up to reward following them.