[SEQ RERUN] The Bedrock of Morality: Arbitrary?

MinibearRex

Today's post, The Bedrock of Morality: Arbitrary? was originally published on 14 August 2008. A summary (taken from the LW wiki):

Humans are built in such a way as to do what is right. Other optimization processes may not. So what?

Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Is Fairness Arbitrary?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.

Today's post, The Bedrock of Morality: Arbitrary? was originally published on 14 August 2008. A summary (taken from the LW wiki):

Humans are built in such a way as to do what is right. Other optimization processes may not. So what?

So, can someone summarize why EY thinks that p-morality is inferior (not just h-inferior) to h-morality, which he seems to call a one-place function "morality"? The OP and the following discussion did not make it clear for me.

He doesn't. He only thinks that p-morality is h-inferior. He doesn't believe that there's such a thing as "inferior".

EDIT: Hmmm... I don't really mean that EY doesn't believe that there's such a thing as "inferior". I just mean that when he uses the word "inferior" he means "h-inferior". He doesn't think that there's some universal "inferior" by which we can judge p-morality against h-morality, but of course p-morality is h-inferior to h-morality.

0fubarobfusco14y

My impression of it: H-morality approximates certain (objective, mathematical) truths about things such as achieving well-being and cooperation among agents, just as human counting and adding ability approximates certain truths about natural numbers. P-morality does not approximate truths about well-being and cooperation among agents. A creature that watches sheep passing into a sheepfold and recites, "One, two, seventeen, six, one, two ..." (and imagines the actual numbers that these words refer to) is not doing counting, and a creature whose highest value is prime-numbered pebble piles is not doing morality. Morality, in the sense of "approximating mathematical truths about things such as achieving well-being and cooperation among agents", is not just an arbitrary provincial value; it is a Good Move. And it is a self-catalyzing Good Move: getting prime-numbered piles of pebbles does not make you more able to make more of them, but achieving well-being and cooperation among agents does make you more able to make more of it. (EDIT: I no longer believe the above is the point of the article. Not using the retract button on account of making it hard to read is just silly.)

6

[SEQ RERUN] The Bedrock of Morality: Arbitrary?

6

6

6

[SEQ RERUN] The Bedrock of Morality: Arbitrary?

6

6