Eliezer_Yudkowsky comments on If it were morally correct to kill everyone on earth, would you do it? - Less Wrong

-6 Post author: Bundle_Gerbe 30 January 2013 11:58PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (43)

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 31 January 2013 08:52:55AM 3 points [-]
Comment author: ESRogs 01 February 2013 08:29:51AM 1 point [-]

For the lazy, the punchline:

And if an external objective morality does say that the universe should occupy some horrifying state... let's not even ask what you're going to do about that. No, instead I ask: What would you have wished for the external objective morality to be instead? What's the best news you could have gotten, reading that stone tablet?

Go ahead. Indulge your fantasy. Would you want the stone tablet to say people should die of old age, or that people should live as long as they wanted? If you could write the stone tablet yourself, what would it say?

Maybe you should just do that?

I mean... if an external objective morality tells you to kill people, why should you even listen?

Comment author: MugaSofer 01 February 2013 02:55:49PM -1 points [-]

No. Humans are not perfect moral reasoners; if he booted up a Friendly superintelligence and it mentioned black people, should a racist conclude he screwed up?

It's unlikely, to my mind, that the arguments presented by the OP are correct - but they are not trivially false. There are people who already espouse similar views; hell, there are people who believe we should all commit suicide now, to spare future generations the pain of living. To simply say "I don't want to do that" is a fully general counteragument, and if it was properly integrated you would be immune to any moral argument.

(You would also have to conclude that everyone but you has a wildly different utility function, which would rather defeat the purpose of CEV.)