Emile comments on Open Thread: July 2010, Part 2 - Less Wrong

6 Post author: Alicorn 09 July 2010 06:54AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (770)

Tags: open_thread

Comments (770)

Sort By: Best

Show:

You are viewing a single comment's thread. Show more comments above.

Comment author: Emile 23 September 2010 04:02:18PM 5 points [-]

I think you're overestimating your ability to see what exactly is wrong and how to fix it. Humans (westerners?) are biased towards thinking that improvements they propose would indeed make things better. This tendancy is particularly visible in politics, where it causes the most damage.

More generally, humans are probably biased towards thinking their own ideas are particularly good, hence the "not invented here" syndrome, etc. Outside of politics, the level of confidence rarely reaches the level of threatening death and destruction if one's ideas are not accepted.

Comment author: waitingforgodel comment score below threshold [+] (5 children)

Comment author: Emile 24 September 2010 03:25:54PM * 4 points [-]

Yes, I read the whole thread (and the banned doubleplus ungood post by Roko).

I wouldn't mind if you were putting "your money" on the table. What I mind is threatening to take act with the goal of reducing mankind's chances of survival. That's not "your money".

If you had just threatened to stop donating money to SIAI (do you donate?), no problemo. Whether that action has an impact on existential risk is unclear; my problem isn't doing actions that might have increase existential risk, it's doing actions whose purpose is to increase existential risk. Or even brainstorming about those.

Imagine that Omega guy came down from the sky and gave each human on earth a device with a button. The button can only be pushed once, and has a one-in-a-million chance of making the sun go Nova (and the button only works for the rightful owner).

What would you think of someone who publicly threatened to push his device's button If the US elected the wrong president? If the government didn't give him $500 a month? If some website didn't adopt the moderation policy he prefered? Or someone brainstorming about how to build a device like that, with the above uses in mind?

Comment author: kodos96 24 September 2010 06:44:35PM 4 points [-]

What if somebody else had a similar button, but with 1 in 100,000 probability. Would it be ok to threaten to push your 1 in a million button if the other guy pressed his 1 in 100,000 button? If you had reason to believe that the other guy, for some reason, would take your threat seriously, but wasn't taking the threat of his own button seriously?

OK, that got kind of convoluted, but do you see what I'm saying?

Comment author: Emile 24 September 2010 07:22:33PM 1 point [-]

If you were sufficiently certain that the situation is as you describe (least convenient world), yes, it would be OK to threaten (and carry out the threat). If however you obtained the information through a bug-ridden device that is known to be biased towards overconfidence in this kind of situations - then such threats would be immoral. And I think most imaginable real-world situations fall in the second category.

Comment author: kodos96 24 September 2010 07:32:18PM * 1 point [-]

I think the only thing we disagree on then is the word "immoral". I would say that it may very well be incorrect, but not immoral, so long as he is being sincere in his motivations.

ETA: ok after thinking about it some more, I guess I could see how it might be considered immoral (in the least convenient world to the point of view I'm arguing). I guess it kind of depends on the specifics of what's going on inside his head, which I'm of course not privy to.

Comment author: Emile 24 September 2010 08:57:09PM 1 point [-]

I'm not sure it would count as "immoral", guess it also depends of how you define the terms.

I see this as a case of the more general "does the end justify the means?". In principle, the end do justify the means, if you're sufficiently confident that those means will indeed result in that end, and that the end is really valuable. In practice, the human brain is biased towards finding ends to justify means that just happen to bring power or prestige to the human in question. In fact, most people doing anything widely considered "bad" can come up with a good story about how it's kinda justified.

So, part of what makes a human moral is willingness to correct this, to listen to the voice of doubt, or at least to consider that one may be wrong, especially when taking action that might harm others.

Emile comments on Open Thread: July 2010, Part 2 - Less Wrong

Article Navigation

Comments (770)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (770)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login