AlexanderRM comments on Unspeakable Morality - Less Wrong

27 Post author: Eliezer_Yudkowsky 04 August 2009 05:57AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (116)

You are viewing a single comment's thread. Show more comments above.

Comment author: woozle 04 August 2009 10:33:37PM *  7 points [-]

The Story of Bob does have an adequate answer in the "vocabulary of harms". The implicit claim that it does not echoes claims of Jonathan Haidt in much of his work on morality, especially his "five pillars" theory and subsequent extrapolations which have been eagerly seized upon by conservative proponents as evidence that liberals are narrow-minded.

It therefore irritates me a great deal when I see such claims going unchallenged, despite their (to me, anyway) obvious inaccuracy.

Here, then, is my "harm/care-based moral system" take on The Sacrifice of Bob:

I'm going to presume that the fictional culture in the story is reasonably happy and prosperous, otherwise we would have been talking about how terrible their culture is even before the sacrifice had taken place.

  • (1) Given that this fictional culture and its president have a good track record at keeping the peace (in diametric opposition to certain other presidents whose similar but much less moral actions are implicitly being referred to), and that Bob's sacrifice probably saved millions of lives, there is nothing wrong with the President's action -- certainly nothing worse than that of a general sending a soldier into a known death trap.
  • (2) Given that this culture seems to achieve goals we value (reasonably happy, prosperous people) while using morals we find questionable (highly authoritarian), I should think that we would want to study them intently to see how they do it. Perhaps we can learn some things -- or perhaps the appearance of happiness and prosperity will turn out to have hidden costs.

Having now played a turn by the rules, I have a bone to pick with the basic concept.

The problem with such examples is that you are basing an implicit conclusion -- "harm-based morality is limited" -- on a fiction, a lie. It's a complex form of circular reasoning: "Imagine a world in which this particular form of morality inexplicably produces positive results. Don't you feel silly trying to defend your morality now?" or even "Imagine a universe in which the earth is a flat disc riding on the back of 4 elephants. How can you stick to your narrow-minded idea that Earth is a spheroid orbiting the sun when in some other universe it might not be?"

All that said, I think I agree with the ground ground rules you propose at the end. I have always said that inability to explain one's reasoning doesn't prove one wrong, and that science needs to pay more attention to intuition (reasoning based on summed data for which complete records were not kept) -- but there do need to be guidelines, because intuition should no more trump science unilaterally than the other way round.

Comment author: AlexanderRM 18 April 2015 10:59:15PM 0 points [-]

My impression of the thought experiment is that there's suppose to be no implication that their side winning the war would be any better than the other side winning. Their side winning is explicitly about maintaining social status and authority. "Keep harm at a low level" might mean "lower than a Hobbesian war of all against all", not necessarily low by our standards. It seems like maybe the thought experiment could be improved by explicitly rephrasing it to make their nation be a pretty terrible place by our standards and winning the war be bad overall. That would rather complicate things though when the point is Bob being tortured and killed. So maybe it should be the country is at peace and "The president feels much more relaxed and it able to work better at crafting his new anti-homosexuality legislation" or something like that?

However, I do on an unrelated note really like your comment about "Imagine a world in which this particular form of morality inexplicably produces positive results. Don't you feel silly trying to defend your morality now?". I've noticed (...although I have trouble thinking of actual examples, but I'm sure I've seen some) that in a fair amount of fiction there's a tendency to have Utilitarian villains with plans that will clearly bring about terrible results, as a result of them having made a very obvious error which the heroes are for some reason able to spot, which when used as an argument against Utilitarianism is pretty much literally "this particular form of morality inexplicably produces negative results". (obviously it's entirely possible for Utilitarians to make mistakes which have horrendous consequences. It's just that as a rule, on average, Utilitarianism will get you better consequences from a Utilitarian standpoint than non-consequential Hollywood Morality. Which is exactly why it's such an appealing argument to use in fiction, because it's a plausible scenario which leads to obviously incorrect conclusions if generalized.)