Today's post, Moral Error and Moral Disagreement was originally published on 10 August 2008. A summary (taken from the LW wiki):
How can you make errors about morality?
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Sorting Pebbles Into Correct Heaps, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Okay. Now take step two—try to show that, in fact, a 'should' really cannot be derived from an 'is'.
"Perpetual motion machines cannot be built" can be demonstrated to be true based on empirically-observable facts. If "'Should' statements cannot be made from 'is' statements" is a true 'is' statement, it will also be possible to show it is true based entirely on empirically-observable facts, right?
The usual mistake people make at this point is to claim that various "shoulds" contradict what "is". But what people think should be is not proof of what is. No matter how hard people believe genies should give wishes, it won't bring them into existence. What people believe morality should say doesn't prove what morality is.
(Unless, of course, you argue that morality is whatever people say it should be. But then you're deriving your should - morality - from what is - what people say.)
Nope. Incompleteness shows that there are some statements which are true which cannot be proven to be true.
However, empirically observed facts in the absence of moral imperative do not create a moral imperative. Typically ethics are formed around by a value judgement and then molded and polished by facts. I see that you are trying to trap me by saying that "I believe that this is better" is a fact, rather than allowing the value judgement "This is better" to stand.
Morality is, among other things, subjective. There is no basis in fact t... (read more)