P-morality has a different view about well-being of agents. P-well-being consists solely of the universe having more piles of properly sorted pebbles. Hunger of agents is p-irrelevant, except that it might indirectly affect the sorting of pebbles. If a properly sorted pile of pebbles can be scattered to prevent the suffering of an agent, it p-should not be.
Conversely, h-morality considers suffering of agents to be directly h-relevant, and the sorting of piles of pebbles is only indirectly h-relevant. An agent h-should not be tortured to prevent the scattering of any pile of pebbles.
None of this provides a reason why torturing agents is objectively o-worse than scattering pebbles, so does not validate any claim to objective morality. To appeal to objective morality, we first have to accept that everything that is h-right and/or p-right may or may not be o-right. Frankly. I'm scared enough that that is the case that I would rather remain h-right and be ignorant of what is o-right than take the risk that o-right differs significantly from what is h-right. From the subjective point of view, that is even the h-right decision to make. The pebblesorters also agree- it is p-wrong to try to change to o-morality, just like it is p-wrong to change to h-morality.
Today's post, The Bedrock of Morality: Arbitrary? was originally published on 14 August 2008. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Is Fairness Arbitrary?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.