Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Unspeakable Morality

27 Post author: Eliezer_Yudkowsky 04 August 2009 05:57AM

It is a general and primary principle of rationality, that we should not believe that which there is insufficient reason to believe; likewise, a principle of social morality that we should not enforce upon our fellows a law which there is insufficient justification to enforce.

Nonetheless, I've always felt a bit nervous about demanding that people be able to explain things in words, because, while I happen to be pretty good at that, most people aren't.

"I remember this paper I wrote on existentialism. My teacher gave it back with an F. She’d underlined true and truth wherever it appeared in the essay, probably about twenty times, with a question mark beside each. She wanted to know what I meant by truth." —Danielle Egan (journalist)

This experience permanently traumatized Ms. Egan, by the way.  Because years later, at a WTA conference, one of the speakers said that something was true, and Ms. Egan said "What do you mean, 'true'?", and the speaker gave some incorrect answer or other; and afterward I quickly walked over to Ms. Egan and explained the correspondence theory of truth:  "The sentence 'snow is white' is true if and only if snow is white"; if you're using a bucket of pebbles to count sheep then an empty bucket is true if and only if the pastures are empty.  I don't know if this cured her; I suspect that it didn't.  But up until that point, at any rate, it seems Ms. Egan had been so traumatized by this childhood experience that she believed there was no such thing as truth - that because her teacher had demanded a definition in words, and she hadn't been able to give a good definition in words, that no good definition existed.

Of which I usually say:  "There was a time when no one could define gravity in exquisitely rigorous detail, but if you walked off a cliff, you would fall."

On the other hand - it is a general and primary principle of rationality that when you have no justification, it is very important that there be some way of saying "Oops", losing hope, and just giving up already.  (I really should post, at some point, on how the ability to just give up already is one of the primary distinguishing abilities of a rationalist.)  So, really, if you find yourself totally unable to justify something in words, one possibility is that there is no justification.  To ignore this and just casually stroll along, would not be a good thing.

And with moral questions, this problem is doubled and squared.  For any given person, the meaning of "right" is a huge complicated function, not explicitly believed so much as implicitly embodied.  And if we keep asking "Why?", at some point we end up replying "Because that is just what the term 'right', means; there is no pure essence of rightness that you can abstract away from the specific content of your values."

But if you were allowed to answer this in response to any demand for justification, and have the other bow and walk away - well, you would no longer be computing what we know as morality, where 'right' does mean some things and not others.

Not to mention that in questions of public policy, it ought to require some overlap in values to make a law.  I do think that human values often overlap enough that different people can legitimately use the same word 'right' to refer to that-which-they-compute.  But if someone wants a legal ban on pepperoni pizza because it's inherently wrong, then I may feel impelled to ask, "Why do you think this is part of the overlap in our values?"

Demands for moral justification have their Charybdis and their Scylla:

The traditionally given Charybdis is letting someone say that interracial marriage should be legally banned because it "feels icky" to them.  We could call this "the unwisdom of repugnance" - if you can just say "That feels repugnant" and win a case for public intervention, then you lose all the cases of what we now regard as tremendous moral progress, which made someone feel vaguely icky at the time; women's suffrage, divorces, atheists not being burned at the stake.  Moral progress - which I currently see as an iterative process of learning new facts, processing new arguments, and becoming more the sort of person you wished you were - demands that people go on thinking about morality, for which purpose it is very useful to have people go on arguing about morality.  If saying the word "intuition" is a moral trump card, then people, who, by their natures, are lazy, will just say "intuition!" all the time, believing that no one is allowed to question that or argue with it; and that will be the end of their moral thinking.

And the Scylla, I think, was excellently presented by Silas Barta when... actually this whole comment is just worth quoting directly:

Let's say we're in an alternate world with strong, codified rules about social status and authority, but weak, vague, unspoken norms against harm that nevertheless keep harm at a low level.

Then let's say you present the people of this world with this "dilemma" to make Greene's point:

Say your country is at war with another country that is particularly aggressive and willing to totally demolish your social order and enslave your countrymen. In planning how to best fight off this threat, your President is under a lot of stress. To help him relieve his stress, he orders a citizen, Bob, to be brought before him and tortured and murdered, while the President laughs his head off at the violence.

He feels much more relieved and so is able to craft and motivate a war plan that leads to the unconditional surrender of the enemy. The President promises that this was just a one-time thing he had to do to handle the tremendous pressure he was under to win the war and protect his people. Bob's family, in turn, says that they are honored by the sacrifice Bob has made for his country. Everyone agrees that the President is the legitimate ruler of the country and the Constitution and tradition give him authority to do what he did to Bob.

Was it okay for the President to torture and kill Bob for his personal enjoyment?

Then, because of the deficiency in the vocabulary of "harms", you would get responses like:

"Look, I can't explain why, but obviously, it's wrong to torture and kill someone for enjoyment. No disrespect to the President, of course."

"What? I don't get it. Why would the President order a citizen killed? There would be outrage. He'd feel so much guilt that it wouldn't even relieve the stress you claim it does."

"Yeah, I agree the President has authority to do that, but God, it just burns me up to think about someone getting tortured like that for someone else's enjoyment, even if it is our great President."

Would you draw the same conclusion Greene does about these responses?

Unfortunately, it does happen to be a fact that most people are not good at explaining themselves in words, unless they've already heard the explanation from someone else.  Even if you challenge a professional philosopher who holds a position, to justify it, and they can't... well, frankly, you can't conclude much even from that, in terms of inferring that no good explanation exists.  Philosophers, I've observed, are not much good at this sort of job either.  It's Bayesian evidence, by the law of conservation of evidence; if a good explanation would be a sign that justification exists, then the absence of such explanation must be evidence that justification does not exist.  It's just not very strong evidence, because we don't strongly anticipate that even professional philosophers will be able to put a justification into words, correctly and convincingly, when justification does in fact exist.

Even conditioning on the proposition that there is overlap in what you and others mean by 'right' - the huge function that is what-we-try-to-do - and that the judgment in question is stable when taken to the limits of knowledge, thought, and reflective coherence - well, it's still not sure that you'd be able to put it into words.  You might be able to.  But you might not.

And we also have to allow a certain probability of convincing-sounding complicated verbal justification, in cases where no justification exists.  But then if you use that as an excuse to flush all disliked arguments down the toilet, you shall be left rotting forever in a pit of convenient skepticism, saying, "All that intellekshual stuff could be wrong, after all."

So here are my proposed rules of conduct for arguing morality in words:

  • "Intuition" is not a trump card.  If you had to spell out what your intuition was, and where it came from (evolution? culture?), and whether it has consequences beyond itself, it's possible that we would find it unconvincing in the stark light of reflection; that we would wish to intuit some other intuition than this.  We can't hold up the intuition for reflective judgment unless we know what it is.  So spelling it out, is important; and if you can win arguments by saying "Intuition!" then no one will bother to spell things out any more.  Please try to say what sort of intuition it is.
  • "I can't put it into words" is believable to some extent, but constitutes weak evidence against the existence of valid justification.  If this is a popular debate and no one on your side, politician or philosopher or interested scientist or eloquent blogger, is able to give a convincing justification in words, then that is stronger evidence that no good justification exists.  The longer the failure continues, the stronger the evidence.
  • Still, at the end of the day, we don't really expect people to be very good at verbalizing moral intuitions, especially since most of them have incoherent explicit metaethics.  So if you can give a justification for your political policy that stutters off into incoherence only at the point of explaining why pain is a bad thing - if you can give reasonable arguments for everything else up until that point - that's probably about as much as we can demand of anyone short of a full-fledged master reductionist.
  • But we also expect that people may pass judgments that they would revoke in the light of better information or new arguments; and, especially before passing to that limit, it may be that sociopaths do not overlap with the values shared by most in a society.  So if A says that event B is inherently wrong and awful, and C disagrees on the grounds that it just doesn't seem all that awful to them, then the burden of argument needs to lie on A before any social, legal, public action is brought into play.  We should bear in mind that people of the past would have a lot of icky feelings about things that we, today, think are not only permitted but virtuous or even mandatory - the challenging of these icky feelings for good and sufficient public justification, was a key element of their relinquishment, which we regard as moral progress.

Comments (116)

Comment author: lavalamp 04 August 2009 03:46:13PM 20 points [-]

Just give up already

I cannot say how many arguments I've had where this would have prevented hurt feelings. Often, after the argument, I discover the other person persisted in arguing for about 10 minutes after they realized they were wrong, all the while getting more angry at me for shooting down ever worse rationalizations.

To be fair, the way this happens isn't that the person persists in arguing for something they know to be false; instead, they drop a subtle hint that maybe they might be wrong and we should stop talking about it now (presumably so they can save face). I invariably miss this hint (well, I'm better now that I know to be looking for it, but not a lot) because it's usually in the form of a ridiculous but hard to disprove objection, to which I (because I'm weird) will come up with a medium-good response. This pisses my interlocutor off, because I missed their social cue, and because I've now forced them to defend a belief (their lousy objection) that they don't actually hold.

This behavior is very understandable; once I noticed others doing it, I noticed a tendency in myself. It's surprisingly hard to say, "Oops, I guess I'm wrong," or, "I can't see a good counter argument to what you're saying; maybe I need to reconsider."

Anyway, I'm saying this because the article linked by the quoted phrase wasn't quite what I was hoping for on the subject. :)

Comment author: haig 05 August 2009 08:09:14AM 3 points [-]

Yes, a big problem is the human tendency to associate strongly with beliefs so that they become a part of your identity. When I once got into an argument with a particularly stubborn friend regarding religion, I tried to disassociate arguments as much as possible by writing them down and having an impartial 3rd party check for inconsistencies and biases blindly in a type of scoring system. How'd it turn out? He gave up alright, but still retained his beliefs!

Comment author: lavalamp 05 August 2009 02:34:45PM 0 points [-]

a big problem is the human tendency to associate strongly with beliefs so that they become a part of your identity.

This is true. Another thing that make such arguments difficult/pointless is that it seems the majority of people give rationalizations for what they believe instead of giving the reasons for which they actually obtained those beliefs. This is understandable as people often don't know (or won't admit to) the way they obtained their beliefs. If one doesn't know/won't admit/won't say why they really think what they do, there's no possible way to present a counter-argument against it. I think there's a post saying pretty much this around here somewhere.

Comment author: CronoDAS 05 August 2009 03:10:04PM 1 point [-]

Often, the real reason one believes something is simply "My parents (or other trusted authority figure) told me it's true."

Comment author: John_Maxwell_IV 06 August 2009 09:47:13PM *  1 point [-]

People seem to rarely admit they are wrong, especially on important issues. I tend to think the root cause of this problem is status preservation, not bias. See this. Confirmation bias exacerbates this when people try to flesh out a consistent worldview that incorporates the thing they couldn't lose face by denying.

Comment author: woozle 04 August 2009 10:33:37PM *  7 points [-]

The Story of Bob does have an adequate answer in the "vocabulary of harms". The implicit claim that it does not echoes claims of Jonathan Haidt in much of his work on morality, especially his "five pillars" theory and subsequent extrapolations which have been eagerly seized upon by conservative proponents as evidence that liberals are narrow-minded.

It therefore irritates me a great deal when I see such claims going unchallenged, despite their (to me, anyway) obvious inaccuracy.

Here, then, is my "harm/care-based moral system" take on The Sacrifice of Bob:

I'm going to presume that the fictional culture in the story is reasonably happy and prosperous, otherwise we would have been talking about how terrible their culture is even before the sacrifice had taken place.

  • (1) Given that this fictional culture and its president have a good track record at keeping the peace (in diametric opposition to certain other presidents whose similar but much less moral actions are implicitly being referred to), and that Bob's sacrifice probably saved millions of lives, there is nothing wrong with the President's action -- certainly nothing worse than that of a general sending a soldier into a known death trap.
  • (2) Given that this culture seems to achieve goals we value (reasonably happy, prosperous people) while using morals we find questionable (highly authoritarian), I should think that we would want to study them intently to see how they do it. Perhaps we can learn some things -- or perhaps the appearance of happiness and prosperity will turn out to have hidden costs.

Having now played a turn by the rules, I have a bone to pick with the basic concept.

The problem with such examples is that you are basing an implicit conclusion -- "harm-based morality is limited" -- on a fiction, a lie. It's a complex form of circular reasoning: "Imagine a world in which this particular form of morality inexplicably produces positive results. Don't you feel silly trying to defend your morality now?" or even "Imagine a universe in which the earth is a flat disc riding on the back of 4 elephants. How can you stick to your narrow-minded idea that Earth is a spheroid orbiting the sun when in some other universe it might not be?"

All that said, I think I agree with the ground ground rules you propose at the end. I have always said that inability to explain one's reasoning doesn't prove one wrong, and that science needs to pay more attention to intuition (reasoning based on summed data for which complete records were not kept) -- but there do need to be guidelines, because intuition should no more trump science unilaterally than the other way round.

Comment author: AlexanderRM 18 April 2015 10:59:15PM 0 points [-]

My impression of the thought experiment is that there's suppose to be no implication that their side winning the war would be any better than the other side winning. Their side winning is explicitly about maintaining social status and authority. "Keep harm at a low level" might mean "lower than a Hobbesian war of all against all", not necessarily low by our standards. It seems like maybe the thought experiment could be improved by explicitly rephrasing it to make their nation be a pretty terrible place by our standards and winning the war be bad overall. That would rather complicate things though when the point is Bob being tortured and killed. So maybe it should be the country is at peace and "The president feels much more relaxed and it able to work better at crafting his new anti-homosexuality legislation" or something like that?

However, I do on an unrelated note really like your comment about "Imagine a world in which this particular form of morality inexplicably produces positive results. Don't you feel silly trying to defend your morality now?". I've noticed (...although I have trouble thinking of actual examples, but I'm sure I've seen some) that in a fair amount of fiction there's a tendency to have Utilitarian villains with plans that will clearly bring about terrible results, as a result of them having made a very obvious error which the heroes are for some reason able to spot, which when used as an argument against Utilitarianism is pretty much literally "this particular form of morality inexplicably produces negative results". (obviously it's entirely possible for Utilitarians to make mistakes which have horrendous consequences. It's just that as a rule, on average, Utilitarianism will get you better consequences from a Utilitarian standpoint than non-consequential Hollywood Morality. Which is exactly why it's such an appealing argument to use in fiction, because it's a plausible scenario which leads to obviously incorrect conclusions if generalized.)

Comment author: Elusu 17 March 2014 11:17:16PM 0 points [-]

I would take Bob's deal if either adequately compensated or convinced that the premise was true. I already have done work for pay that was so unpleasant I'd rather be tortured for a short time than do that sort of work again, and time wasted is partial death anyway.

As for our culture in general, this deal is very very common. Many people watch someone from another universe, a 'fictional person' being tortured to death for their entertainment, and there isn't any proof that the characters in, say, Saw, aren't real people somewhere. Now, before we come down hard on horror fans, note that every fan of the Dark Knight movie with Heath Ledger is watching entertainment that killed someone. Every person who relaxes by reading history or war .... everyone who reads the Bible or watches most entertainment based on it. At least Eliezer's example President is honest enough to say that he needs to watch this to refresh his spirit; people (like me!) who go and refresh their spirit by looking at past sufferings of people, animals, etc are at least 'guilty' of encouraging that type of suffering in much the same way that hamburger buyers (in a modern farm economy) are guilty of causing animal suffering.

Note that /I do not think suffering is bad/ in and of itself. Sometimes it /is/ necessary. Bob and the President might just be doing something sensible.

Considering that IRL we have had a series of leaders who make themselves feel better by /torturing people non-consensually/, I'd rather live in Bob's world where it's Bradley Manning, or some random Afghan goat farmer whose neighbor wanted to graze on their land, who is getting tortured, in some cases to death, so that Great Leader can feel better.

Comment author: Jiro 18 March 2014 12:53:40AM *  0 points [-]

Many people watch someone from another universe, a 'fictional person' being tortured to death for their entertainment, and there isn't any proof that the characters in, say, Saw, aren't real people somewhere.

Likewise, if you watch fiction where people are happy, there isn't any proof that the existence of a happy character in your fiction isn't associated with a real person somewhere who is suffering.

Thinking about the possibility that there's a suffering person who corresponds to fiction about a suffering fictional character, but not thinking about the possibility that there's a suffering person who corresponds to a happy fictional character, or for that matter the possibility that there's a suffering person (created by a perverse Omega) who comes into existence whenever you eat a slice of pizza is a form of availability bias. It's easier to imagine the former since your mind is processing the concept of suffering at the time, but there's no actual reason to expect that that pair is any more closely connected than any other arbitrary pair.

Comment author: dumbshow 05 August 2009 01:58:53PM 2 points [-]

since most of them have incoherent explicit metaethics

Is there a coherent metaethical theory specified in a single document somewhere on the Internet? Or does the theory have to be compiled from multiple blog posts? I guess I'm not sure what you're talking about...

Comment author: Jonathan_Graehl 04 August 2009 08:38:07AM *  2 points [-]

I'm going to comment on the presentation (I have no objection to the ideas):

This piece pleased me immensely in the beginning. The "F" grade anecdote was perfect and actually enraged me upon later reflection. In spite of that, I don't feel the persuasion was manipulative.

The reiteration of the linked ideas was good. I especially liked the story of the irrational gambler who could be used as money pump.

The material up to and including the Greene dilemma was good.

Following that, however, is a painfully verbose recapitulation of the opening "I've always felt a bit nervous about demanding that people be able to explain things in words, because, while I happen to be pretty good at that, most people aren't.".

Then we have some clever self-hedging ("shared moral beliefs might suggest an underlying reason, even if there's no satisfactory exposition", "but even unreasonable beliefs sometimes get a seemingly-convincing apology!") that looks like preparation for a debate. Why not let your readers have those thoughts on their own? They just serve to annoy those who are capable of having them, and confuse those who aren't. There's no need to claim priority on such thoughts; let them arise in the comments (post them yourself, even).

Or, in other words, considerations that occurred to you, but don't substantially change the final judgment, can often be omitted or relegated to a comment/footnote.

Finally, the rules. I agree with them. I think it's okay to be as verbose as you are, because there's real danger in having people blindly follow if you're unclear. The arguments for them have already been made (or are obvious). I didn't feel a need to read past the first sentence of each rule, but it's good that detail is there.

The final rule is more hedging like I already complained about. I guess you imagine some of your readers as automatons or idiots :) Or, more likely, you're preparing yourself for expected objections.

I think it's good that you consider the consequences of your ideas, possible misunderstanding of them, and have rebuttals in advance.

I'd like to propose a new form for you:

  1. First, all the great stuff. Really original ideas without repetition, plus particularly clever demonstrations or evocative examples/problems. All the formalization, you have, too. Essentially, everything that would inform and please a not-obnoxious and competent person who has no fundamental disconnect from you.

  2. A separator to be understood as "you can stop now if you understood everything and are impatient".

  3. Now, all the stuff I complained about earlier. Some kind of linking or numbering might be necessary to get the same context as the usual inline form.

Maybe that's too much work. But as a reader, I often find that the required investment to be off-putting. If you don't want "skim and then socialize in the comments" happening, it might be worth the effort, though. (I do think most comments by regulars indicate that they really read the article they comment on).

Comment author: dmfdmf 04 August 2009 10:03:39AM 1 point [-]

Demands for moral justification have their Charybdis and their Scylla:

A rather fancy way of saying the horns of a dilemma. If I were a Bayesian I might say that my prior is to believe that this is a sure sign of a false premise hidden in there somewhere leading to the false alternative. If I where a frequentist I might say 999 times out 1000 such dilemmas are a sure sign of the same. Ethics is full of such horns and dilemmas, handed out like poisoned candy to the kiddies on halloween by the very professors who are suppose to find the error and resolve them. In any case, a B or a F should be motivated to prove the hypothesis or rule it out. Throwing up one's hands and creating ad hoc rules for moral issues seems.... more wrong.

And hopefully the whole frequentist -v- bayesian dichotomy-debate will turn out not to have a false premise behind it. Of this I am not sure.

Comment author: Tiiba 04 August 2009 07:59:32AM 1 point [-]

If you don't mind me asking... How does anyone who believes that pain and/or death is bad NOT conclude that pepperoni pizza is bad?

Comment author: lavalamp 04 August 2009 03:28:45PM 3 points [-]

A google search I did yesterday for a recipe turned up the book "Eat what you want and die like a man"; I haven't read it except for the two pages that the search showed me, but I'd be willing to bet that the author holds exactly that position...

Comment author: Tiiba 04 August 2009 04:13:23PM *  1 point [-]

Apparently, the idea that killing animals for food is evil is so alien to modern people that everyone thought I'm talking about HUMAN pain and death.

When the motivations of others are so different from yours that they can't even understand your questions, you must consider the possibility that maybe, just maybe, the reason is something more interesting than a genetic accident.

Comment author: UnholySmoke 04 August 2009 10:01:51PM 3 points [-]

To be frank, I'm a vegetarian and I didn't pick up on that one.

Comment author: Eliezer_Yudkowsky 04 August 2009 04:32:00PM 3 points [-]

Ah. Actually, I think a reason I didn't get this was that when I hear "X is bad" I tend to look to its consequences before looking to its antecedents. For example, if you said "soap is bad", I would first think "being clean is bad?" before "maybe there's something wrong with the process that manufactured the soap". Utilities flow backward in time, not forward. Unless all this is just a post-facto rationalization, rather than my actually being unusually good at verbalizing the cognitive algorithms behind a thought...

Comment author: Tiiba 04 August 2009 04:58:01PM *  0 points [-]

My mom taught me to look both ways.

When you buy meat, you pay for the next round of butchery. So it does flow forward. So if you have to eat meat, steal it.

Comment author: kess3r 04 August 2009 08:10:33PM 1 point [-]

Hey, could someone explain the logic of vegetarianism to me? I get the part where vegeterianism is supposedly healthier. But I don't get the part about not wanting to eat animals because they get killed. I mean, it's not like cows would live happily ever after if nobody ate them. If all humans suddenly stopped eating cows, there would be no reason to raise cows anymore apart from zoos, and cows are not very good at taking care of themselves in the wild. It seems like vegeterianism would lead to cow extinction or very close to it.

Comment author: Alicorn 04 August 2009 08:19:27PM *  6 points [-]

I value a lack of cow suffering. I do not value the existence of the cow species, except inasmuch as cows are useful towards ends I care about, and since I don't eat them and don't think they're cute or interesting, they are useful to me only for milk and, in limited quantities, skin. (I'll assume you meant to assume that widespread veganism and leather boycott would lead to the extinction of cows.)

Comment author: UnholySmoke 04 August 2009 10:03:52PM 4 points [-]

Sounds very pessimistic to value a lack of cow suffering, but completely discount cow enjoyment.

I mean, if we're quantizing stuff, might as well quantize everything, right?

Comment author: pwno 05 August 2009 05:23:16PM 1 point [-]

Do you think there is morally wrong to eat meat?

I have the same preferences as you when it comes to meat, but I still eat it. Maybe if it was proven that a lot of animal suffering goes into the meat I eat, I might stop. Otherwise, a cow's non-suffering, short-lived existence is more favorable than not existing at all.

Comment author: Alicorn 05 August 2009 06:29:35PM 4 points [-]

I've rehashed this several times, but I'll repeat it for your benefit: I think it is wrong for many people to eat meat. Some people, through circumstances beyond their control, would find their quality of life unacceptably diminished by a lack of meat consumption. I do not think it is morally wrong for those people to eat meat: their quality of life is more important than the lives of the animals they eat. I, and many other people, can be happy vegetarians. People who can be happy vegetarians (or who won't be significantly less happy as vegetarians than as omnivores) should be vegetarians. For those people, it is wrong to eat meat because it is unnecessarily destructive of animal lives, which have non-negligible value even if they aren't more important than human quality of life.

There is an overwhelming amount of gory detail about the suffering undergone by the majority of domesticated meat animals in developed countries. If you are curious about how much suffering your food underwent to arrive at your plate, PETA et. al. will be happy to supply that information and you can find it without my help.

a cow's non-suffering, short-lived existence is more favorable than not existing at all.

I disagree. I think many animals (and people, for that matter) ought not to have been born.

Comment author: ChrisHibbert 09 August 2009 03:50:33AM 0 points [-]

There is an overwhelming amount of gory detail about the suffering undergone by the majority of domesticated meat animals in developed countries. If you are curious about how much suffering your food underwent to arrive at your plate, PETA et. al. will be happy to supply that information and you can find it without my help.

I'm not particularly curious. I have no doubt that I could find plenty of testimony from partisans. Why should I expect that testimony to present the issue in a fair light? Are there any non-partisans trying to find a middle position and present a balanced view of the issue?

Comment author: pwno 05 August 2009 07:04:20PM 0 points [-]

Ok, so there is a cost to eating meat (beyond the price tag) and some people love meat so much, it's worth the cost. You don't think there is a chance that the hidden cost is actually much worse than meat eaters think? That, given the true cost, not even the most meat-loving person would eat meat?

I dont' think the true cost is high enough to warrant all meat-eating bad, but substantially worse than most meat eaters think.

I disagree. I think many animals (and people, for that matter) ought not to have been born.

That's because you're adding other details. Assuming a person or animal contributed to society an equal amount to its cost to society, would living a non-suffering, short-lived existence still be worse than no existence at all?

Comment author: lavalamp 06 August 2009 02:02:22AM *  0 points [-]

I do not value the existence of the cow species, except inasmuch as cows are useful towards ends I care about....

Elsewhere (http://lesswrong.com/lw/14r/unspeakable_morality/10jc) you said:

... I follow that intuition all the way down and think that stuff in general shouldn't be destroyed unnecessarily.

I've been reading your vegetarian comments with interest. Can you please explain how you don't think stuff should be destroyed unnecessarily, yet would not care if an entire species vanished?

Is it that it's somehow ok if something is destroyed as long as it's not intentional? I.e., if a famous painting was about to fall into a fire or something accidentally, it seems to me (if I follow your logic) you would catch it if you could do so without undue danger to your person, even if you didn't particularly like the painting. So how can you be ok with cows (or, let's say pigs, since as far as I know they are not used for leather or milk) going extinct?

Comment author: Alicorn 06 August 2009 02:27:20AM 1 point [-]

Can you please explain how you don't think stuff should be destroyed unnecessarily, yet would not care if an entire species vanished?

I distinguish between taking action to destroy something, and ceasing to take measures to preserve it. The domestic cow species, as well as the domestic pig species, requires continual human support to keep it in existence. I would not have any problem with cows or pigs ceasing to exist if the following conditions were met:

  • No person anywhere just plain likes cows (pigs) and wants them around.
  • Cows (pigs) serve no purpose of any person, directly or indirectly, and are not reasonably expected to do so in the future.
  • The continued existence of the cow (pig) species takes up resources that could be diverted elsewhere, to more useful ends.
  • The extinction of the cow (pig) species does not require active destructive participation on the part of any person.

I would have problems of greater or lesser degree with the extinction of cows (pigs) if any of the above conditions were not fully met, as in fact they are not at this time.

Is it that it's somehow ok if something is destroyed as long as it's not intentional?

I'm usually careful to specify that I think an action can be unethical only if it was intentional or negligent.

Comment author: lavalamp 06 August 2009 03:53:46AM 0 points [-]

I'm usually careful to specify that I think an action can be unethical only if it was intentional or negligent.

...Which is why I didn't use the world "ethical" :)

More to the point,

I distinguish between taking action to destroy something, and ceasing to take measures to preserve it.

So I gather that it is the act of destruction you find bad, and not the loss of the thing destroyed?

(And my follow-up question if you answer in the affirmative: Why, then, is it bad to destroy things?)

(And don't construe this line of questioning as disagreeing or agreeing with you; I'm just trying to understand your point of view)

Comment author: AllanCrossman 04 August 2009 08:12:27PM 2 points [-]

It seems like vegeterianism would lead to cow extinction or very close to it.

It would, but that's an entirely separate issue from animal cruelty.

Comment author: kess3r 04 August 2009 08:20:09PM 0 points [-]

So does that mean vegetarians are ok with eating animals that were treated very humanly or that died of natural causes? Could a vegetarian here explain?

In case there are no vegetarians on this site, how are we driving away or failing to attract vegetarians?

Comment author: Alicorn 04 August 2009 08:24:52PM *  1 point [-]

So does that mean vegetarians are ok with eating animals that were treated very humanly or that died of natural causes? Could a vegetarian here explain?

I'm a pescetarian, but let's assume I count. I wouldn't eat those animals because non-fish meat no longer resembles food to me; because if I resumed eating meat of any kind, it would be more difficult to resist meat of inappropriate provenance; and because humanely-treated meat is hard to come by (and still has to be slaughtered) and naturally-dead meat is of suspect quality.

In case there are no vegetarians on this site, how are we driving away or failing to attract vegetarians?

For an idea of how many vegetarians we have, check out this poll.

Comment author: kess3r 04 August 2009 08:41:04PM 0 points [-]

Do you think it is unethical for humans to eat other animals? If so, what do you suggest?

Comment author: anonym 06 August 2009 04:06:41AM *  0 points [-]

I'm a vegetarian, and if I weren't a bit repulsed by meat, I would have no ethical qualms about eating the flesh of a wild animal (or person) that died of natural causes, assuming my eating it didn't have other negative consequences.

Comment author: Tiiba 04 August 2009 08:36:27PM 0 points [-]

I was, actually, fine with eating free range meat at first. After all, even their deaths might be less horrible than my own. But then I thought that if everyone did that, having so many animals living like people might be more than the Earth can take. It's having trouble with people living like people.

Basically, free-range meat is a move in the right direction, but suboptimal.

Comment author: MichaelBishop 11 August 2009 05:23:11PM 0 points [-]

There are many different logics. See this thread http://lesswrong.com/lw/ei/essayquestion_poll_dietary_choices/ for some of them, including my own.

Comment author: pwno 05 August 2009 05:13:47PM -1 points [-]

If you heard "sweatshops are bad" or "styrophone cups are bad" you would first look for its antecedents. So maybe the cognitive algorithm goes something like this: If X in "X is bad" is not associated with unfavorable antecedents, then examin X's consquences by default.

Comment author: thomblake 04 August 2009 04:16:04PM *  2 points [-]

Pepperoni pizza was an odd example to use if you're talking about the evils of killing animals for food. There isn't even much meat on there. That's just poor communication. How about you reword like this:

If you don't mind me asking... How does anyone who believes that pain and/or death is bad NOT conclude that pepperoni pizza is bad, since killing animals for food is evil?

That seems much clearer, if that's what you intended to communicate.

Comment author: lavalamp 04 August 2009 04:22:16PM 1 point [-]

Ah, that makes your comment make sense. Obviously I know that peperoni is an animal product, but the pizza part threw me off, possibly because peperoni pizza is like the poster-child for foods that are supposedly bad for you.

I have no problem with the idea that modern factory-style living conditions for chickens, etc, are inhumane (and I personally buy cage-free, etc., both for that reason and because they apparently have more nutrients). But you seem to be suggesting that any killing of any animal for food purposes is immoral. So now I have to ask if you think it's evil for a lion to kill and eat an antelope?

Comment author: Tiiba 04 August 2009 04:46:53PM *  3 points [-]

If lions didn't eat antelopes, they would starve, and so would the antelopes when their burgeoning populations run out of food. Bad end. Pain, suffering, death.

Now, if lions could be made vegetarian, and antelopes could be made to use condoms... That'd be the perfect solution.

Comment author: anonym 06 August 2009 03:55:19AM 0 points [-]

I got your point, and my response was that people generally don't think pain and death are bad in all cases and without exceptions. Many people think that the pain and death of their enemies is good, and that the pain and death of creatures sufficiently different from them (according to various criteria such as degree of sentience or sapience or ability to feel pain) is neutral.

Comment author: Eliezer_Yudkowsky 04 August 2009 03:59:40PM 1 point [-]

Pepperoni pizza isn't bad. Human metabolisms are bad.

Comment author: thomblake 04 August 2009 04:01:58PM 0 points [-]

Both pepperoni pizza and human metabolisms are good, and gluttony is bad.

Well, good enough for government work.

Comment author: Eliezer_Yudkowsky 04 August 2009 04:28:23PM 3 points [-]

Pepperoni pizza, human metabolisms, and gluttony are good, and being fat is bad. Thus we have a philosophical paradox!

Comment author: thomblake 04 August 2009 04:35:36PM 0 points [-]

I think considering 'gluttony' good is just abusing the language.

As for your philosophical paradox... drugs to the rescue?

Comment author: anonym 04 August 2009 08:17:30AM 0 points [-]

Does anybody think pain and/or death are unconditionally bad, in all cases, with no exceptions? I've never heard of such a person. But perhaps I misunderstood what you were asking.

Comment author: MBlume 04 August 2009 05:16:34PM 1 point [-]
Comment author: anonym 05 August 2009 05:38:43AM 1 point [-]
Comment author: Bongo 04 August 2009 04:50:47PM 0 points [-]

Does anybody think pain and/or death are unconditionally bad

You don't?

When is pain or death not bad?

Comment author: Kaj_Sotala 05 August 2009 01:13:43PM 2 points [-]

Masochism.

Comment author: anonym 05 August 2009 05:35:19AM 1 point [-]

Pain: when it wakes you up to alert you that you are in mortal danger.

Comment author: Kaj_Sotala 05 August 2009 01:13:24PM 0 points [-]

At the risk of getting into semantics: in that case, pain serves a useful purpose, but that doesn't make pain itself non-bad. Creating an alternative ("upgraded") alert system that served the wake-up function but wasn't painful would be better. If pain in that context wouldn't be bad, then "does the alert system cause pain" would be an irrelevant question and the upgraded alert system wouldn't be considered any better.

Comment author: anonym 06 August 2009 03:59:12AM 0 points [-]

in that case, pain serves a useful purpose

Right, which was exactly my point: not every instance of pain should be classified as bad, and so it doesn't make sense to say the general phenomenon is "unconditionally bad, in all cases, with no exceptions", which is exactly what Bongo implicitly asserted.

Comment author: RobinZ 04 August 2009 05:20:19PM 0 points [-]

To be perfectly fair, the absolute is difficult to assert due to the fuzziness of the concept. I mean, is tearing a piece of paper in half bad, because you killed the paper? What about tearing a virus in half? What about a bacterium? Where does the transition come in from merely irreversible to murderous.

Comment author: Joe 09 September 2009 07:03:23PM 1 point [-]

I'm curious about the opening line: It is a general and primary principle of rationality, that we should not ... enforce upon our fellows a law which there is insufficient justification to enforce.

On my ordinary understanding of the sentence, it seems to imply that acting justly is necessarily part of what Eliezer means by "acting rationally". Is this right?

More explicitly: the implication is that refraining from "enforcing insufficiently justified laws" is a "general and primary" principle of rationality. Perhaps what is meant is some tautology deriving from (Eliezer's) meanings of "fellows", "justification", and "should". Or perhaps it's just a rhetorical flourish?

Comment author: Eliezer_Yudkowsky 09 September 2009 07:42:06PM 0 points [-]

Clarified opening line.

Comment author: Cyan 05 August 2009 02:43:58PM *  1 point [-]

I see a lot of people in the comments to this post talking about divorcing the "damage report" aspect of pain from the "unpleasant experience" aspect. This won't work -- at least, not in naive form. Emotions are what motivate people to do things. If you take away the negative emotional impact of pain without some kind of situational fine-tuning, you might as well remove pain entirely. Those people with pain asymbolia that I mentioned previously are indifferent to the threat of harm.

Comment deleted 05 August 2009 11:57:00PM [-]
Comment author: Alicorn 06 August 2009 12:16:25AM 4 points [-]

This might be placed better in the open thread.

Comment author: teageegeepea 05 August 2009 03:00:08AM *  0 points [-]

and where it came from (evolution? culture?)

Once you know a source, how do you know whether it makes your morals more or less likely to be true? A lazy thinker would just wind up engaging in Bulverism.

only at the point of explaining why pain is a bad thing

Would you adapt that if speaking into the chronophone to a time before negative-utilitarianism?

Comment author: bgrah449 04 August 2009 09:20:30PM 0 points [-]

All moral codes drill down to a rocky core of "ick," though. Suppose A says, "Well it's clearly wrong." And C says, "No, it's not. Make your case." The case is made when A says, "B inevitably leads to D. Does D make you feel icky?" and C says, "It does."

It's true that people in the past had a lot of icky feelings we don't have today. We also have a lot of icky feelings they didn't have. Given that, I would like to see a follow-up article written about, under this framework, how many more letters of the alphabet have to agree with A before A gets to punish C for making him feel icky, depending on the number of letters in agreement, how severe the ick, that kind of thing.

Comment author: JGWeissman 04 August 2009 09:27:45PM 1 point [-]

My reason for objecting to being punched in the face is not that it makes me feel icky, it is that it causes actual damage to me.

Comment author: Jayson_Virissimo 04 August 2009 09:39:17PM 2 points [-]

I guess he could say that you think damage or harm is icky.

Comment author: bgrah449 04 August 2009 11:28:07PM 0 points [-]

I don't think the desire to avoid being punched in the face is a moral code. Maybe you have a strong conviction that you don't deserve to be punched in the face, but I can conceive of a situation where you feel guilty for doing something (such as previously punching a friend in the face in anger) and you actually do want to be punched in the face, to adhere to a moral code, even despite your still-extant avoidance-of-pain code.

Similarly, these two guidelines can be in parallel - a part of you just doesn't want to be punched in the face, and another part of you agrees with that part on account of the fact you totally haven't done anything to deserve being punched in the face.

Comment author: kess3r 04 August 2009 09:39:25PM 0 points [-]

Hey, I very much agree with your explanation. Jonathan Haidt has a very good theory on what makes humans feel this "ick". http://www.ted.com/index.php/talks/jonathan_haidt_on_the_moral_mind.html Don't be turned off by his implication that liberals should be more conservative. Strictly as an empirical model, his theory is quite good.

Comment author: PhilGoetz 04 August 2009 06:02:37PM *  0 points [-]

It is a general and primary principle of rationality, that we should not believe that which there is insufficient reason to believe, nor enforce upon our fellows a law which there is insufficient justification to enforce.

Is that a principle, or a tautology?

Nonetheless, I've always felt a bit nervous about demanding that people be able to explain things in words, because, while I happen to be pretty good at that, most people aren't.

I was thinking just yesterday that this is a problem in communications between men and women. Women know a lot of things that they can't explain. Here's a conversation you might overhear between a man and a woman:

She: "Look at that guy. He just doesn't see that she isn't into him."

He: "Huh?"

She: "Well, look at her. Look at how she looks at him. Look at how at how she's sitting."

He: "What about how she looks at him? How does she look at him?"

She: "Just look. It's obvious."

He thinks that She doesn't know what she's talking about, because She can't explain it. But somehow it often turns out that She is right.

So if A says that event B is inherently wrong and awful, and C disagrees on the grounds that it just doesn't seem all that awful to them, then the burden of argument needs to lie on A before any social, legal, public action is brought into play.

Is that the main point of your post, or was there something more? I'm looking at your 4 bullet points, and I agree with them, but they aren't novel or even controversial.

Comment author: spriteless 05 August 2009 08:05:41AM 1 point [-]

Hmm. So if she said "I'm reading her body language, a skill I developed by socializing with women lots" would that make the guy believe her? I shall have to remember that.

Comment deleted 04 August 2009 08:43:08AM [-]
Comment author: Eliezer_Yudkowsky 04 August 2009 03:58:27PM 3 points [-]

Would you care to take your shot at rewriting at half the length? That only half of my post is useful or necessary for a reader, is a very common phenomenon; the problem is, it's not always the same half for each reader.