My impression of it:
H-morality approximates certain (objective, mathematical) truths about things such as achieving well-being and cooperation among agents, just as human counting and adding ability approximates certain truths about natural numbers. P-morality does not approximate truths about well-being and cooperation among agents.
A creature that watches sheep passing into a sheepfold and recites, "One, two, seventeen, six, one, two ..." (and imagines the actual numbers that these words refer to) is not doing counting, and a creature whose highest value is prime-numbered pebble piles is not doing morality.
Morality, in the sense of "approximating mathematical truths about things such as achieving well-being and cooperation among agents", is not just an arbitrary provincial value; it is a Good Move. And it is a self-catalyzing Good Move: getting prime-numbered piles of pebbles does not make you more able to make more of them, but achieving well-being and cooperation among agents does make you more able to make more of it.
(EDIT: I no longer believe the above is the point of the article. Not using the retract button on account of making it hard to read is just silly.)
P-morality has a different view about well-being of agents. P-well-being consists solely of the universe having more piles of properly sorted pebbles. Hunger of agents is p-irrelevant, except that it might indirectly affect the sorting of pebbles. If a properly sorted pile of pebbles can be scattered to prevent the suffering of an agent, it p-should not be.
Conversely, h-morality considers suffering of agents to be directly h-relevant, and the sorting of piles of pebbles is only indirectly h-relevant. An agent h-should not be tortured to prevent the scatter...
Today's post, The Bedrock of Morality: Arbitrary? was originally published on 14 August 2008. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Is Fairness Arbitrary?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.