Swimmer963 comments on Disguised Queries - Less Wrong

57 Post author: Eliezer_Yudkowsky 09 February 2008 12:05AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (104)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Swimmer963 13 April 2012 11:43:46PM 0 points [-]

Well, my brain is structured in such and such a way as a result of evolution, so I think I'll kill this completely innocent guy over here.

Beliefs don't feel like beliefs, they feel like the way the world is. Likewise with brain structures. If someone is a sociopath (in short, their brain mechanism for empathy is broken) and they decide they want to kill someone for reasons X and Y, are they being any more irrational than someone who volunteers at a soup kitchen because seeing people smile when he hands them their food makes him feel fulfilled?

("I like being happy, and I'm capable of empathy, so I think other people must like being happy too, and since my perfect world would be one where I was happy all the time, the perfect world for everyone would be one with maximum happiness.")

There's a missing inference here from wanting to be happy to wanting other people to be happy. Can you explain how you think this argument gets filled out? As it stands, it's not valid.

Sorry for not being clear. The inference is that "empathy", the ability to step into someone else's shoes and imagine being them, is an innate ability that most humans have, leads you to think that other people are like you...when they feel pleasure, it's like your pleasure, and when they feel pain, it's like your pain, and there's a hypothetical world where you could have been them. I don't think this hypothetical is something that's taught by moral theories, because I remember reasoning with it as a child when I'd had basically no exposure to formal moral theories, only the standard "that wasn't nice, you should apologize." If you could have been them, you want the same things for them that you'd want for yourself.

I think this is immediately obvious for family members and friends...do you want your mother to be happy? Your children?

Comment author: [deleted] 14 April 2012 12:02:01AM *  1 point [-]

Beliefs don't feel like beliefs, they feel like the way the world is.

Perhaps on some level this is right, but the fact that I can assess the truth of my beliefs means that they don't feel like the way the world is in an important respect. They feel like things that are true and false. The way the world is has no truth value. Very small children have problem with this distinction, but so far as I can tell almost all healthy adults do not believe that their beliefs are identical with the world. ETA: That sounded jerky. I didn't intend any covert meanness, and please forgive any appearance of that.

If someone is a sociopath (in short, their brain mechanism for empathy is broken) and they decide they want to kill someone for reasons X and Y, are they being any more irrational than someone who volunteers at a soup kitchen because seeing people smile when he hands them their food makes him feel fulfilled?

I think I really don't understand your question. Could you explain the idea behind this a little better? My objection was that there are reasons to do things, and reasons why we do things, and while all reasons to do things are also reasons why, there are reasons why that are not reasons to do things. For example, having a micro-stroke might be the reason why I drive my car over an embankment, but it's not a reason to drive one's car over an embankment. No rational person could say to themselves "Huh, I just had a micro-stroke. I guess that means I should drive over this embankment."

I think this is immediately obvious for family members and friends...do you want your mother to be happy? Your children?

Sure, but I take myself to have moral reasons for this. I may feel this way because of my biology, but my biology is never itself a reason for me to do anything.

Comment author: Swimmer963 14 April 2012 02:12:33AM 1 point [-]

Beliefs don't feel like beliefs, they feel like the way the world is.

Perhaps on some level this is right, but the fact that I can assess the truth of my beliefs means that they don't feel like the way the world is in an important respect.

OK, let me give you a better example. When you look at something, a lot of very complex hardware packed into your retina, optic nerve, and visual cortex, a lot of hard-won complexity optimized over millions of years, is going all out analyzing the data and presenting you with comprehensible shapes, colour, and movement, as well as helpful recognizing objects for you. When you look at something, are you aware of all that happening? Or do you just see it?

(Disclaimer: if you've read a lot about neuroscience, it's quite possible that sometimes you do think about your visual processing centres while you're looking at something. But the average person wouldn't, and the average person probably doesn't think 'well, there go my empathy centres again' when they see an old lady having trouble with her grocery bag and feel a desire to help her.)

I think I really don't understand your question. Could you explain the idea behind this a little better? My objection was that there are reasons to do things, and reasons why we do things, and while all reasons to do things are also reasons why, there are reasons why that are not reasons to do things.

Okay, let's try to unpack this. In my example, we have a sociopath who wants to murder someone. The reason why he wants to murder someone, when most people don't, is because there's a centre in his brain that's broken and so hasn't learned to see the world from another's perspective, thus hasn't internalized any social morality because it doesn't make sense to him...basically, people are objects to him, so why not kill them. His reason to murder someone is because, let's say, they're dating a girl he wants to date. Most non-sociopaths wouldn't consider that a reason to murder anyone, but the reason why they wouldn't is because they have an innate understanding that other people feel pain, of the concept of fairness, etc, and were thus capable of learning more complex moral rules as well.

Sure, but I take myself to have moral reasons for this. I may feel this way because of my biology, but my biology is never itself a reason for me to do anything.

The way I see it, the biology aspect is both necessary and sufficient for this kind of behaviour. Someone without the requisite biology wouldn't be a good parent or friend because they'd see no reason to make an effort (unless they were deliberately "faking it" to benefit from that person). And an ordinary human being raised with no exposure to moral rules, who isn't taught anything about it explicitly, will still want to make their friends happy and do the best they can raising children. They may not be very good at it, but unless they're downright abused/severely neglected, they won't be evil.

Comment author: [deleted] 14 April 2012 05:26:09PM 0 points [-]

When you look at something, are you aware of all that happening? Or do you just see it?

I just see it. I'm aware on some abstract level, but I never think about this when I see things, and I don't take it into account when I confidently believe what I see.

"His reason to murder someone is because, let's say, they're dating a girl he wants to date. Most non-sociopaths wouldn't consider that a reason to murder anyone"

I guess I'd disagree with the second claim, or at least I'd want to qualify it. Having a broken brain center is an inadmissible reason to kill someone. If that's the only explanation someone could give (or that we could supply them) then we wouldn't even hold them responsible for their actions. But dating your beloved really is a reason to kill someone. It's a very bad reason, all things considered, but it is a reason. In this case, the killer would be held responsible.

"The way I see it, the biology aspect is both necessary and sufficient for this kind of behaviour. "

Necessary, we agree. Sufficient is, I think, too much, especially if we're relying on evolutionary explanations, which should never stand in without qualification for psychological, much less rational explanations. After all, I could come to hate my family if our relationship soured. This happens to many, many people who are not significantly different from me in this biological respect.

An ordinary human being raised with no exposure to moral rules in an extremely strange counterfactual: no person I have ever met, or ever heard of, is like this. I would probably say that there's not really any sense in which they were 'raised' at all. Could they have friends? Is that so morally neutral an idea that one could learn it while leaning nothing of loyalty? I really don't think I can imagine a rational, language-using human adult who hasn't been exposed to moral rules.

So the 'necessity' case is granted. We agree there. The 'sufficiency' case is very problematic. I don't think you could even have learned a first language without being exposed to moral rules, and if you never learn any language, then you're just not really a rational agent.

Comment author: Swimmer963 14 April 2012 05:58:04PM 0 points [-]

An ordinary human being raised with no exposure to moral rules in an extremely strange counterfactual: no person I have ever met, or ever heard of, is like this.

A weak example of this: someone from a society that doesn't have any explicit moral rules, i.e. 'Ten Commandments.' They might follow laws, but but the laws aren't explained 'A is the right thing to do' or 'B is wrong'. Strong version: someone whose parents never told them 'don't do that, that's wrong/mean/bad/etc' or 'you should do this, because it's the right thing/what good people do/etc.' Someone raised in that context would probably be strange, and kind of undisciplined, and probably pretty thoughtless about the consequences of actions, and might include only a small number of people in their 'circle of empathy'...but I don't think they'd be incapable of having friends/being nice.'

Comment author: [deleted] 14 April 2012 07:59:57PM 0 points [-]

A weak example of this: someone from a society that doesn't have any explicit moral rules, i.e. 'Ten Commandments.' They might follow laws, but but the laws aren't explained 'A is the right thing to do' or 'B is wrong'.

I can see a case like this, but morality is a much broader idea than can be captured by a list of divine commands and similar such things. Even Christians, Jews, and Muslims would say that the ten commandments are just a sort of beginning, and not all on their own sufficient to be moral ideas.

Someone raised in that context would probably be strange, and kind of undisciplined, and probably pretty thoughtless about the consequences of actions, and might include only a small number of people in their 'circle of empathy'...but I don't think they'd be incapable of having friends/being nice.'

Huh, we have pretty different intuitions about this: I have a hard time imagining how you'd even get a human being out of that situation. I mean, animals, even really crappy ones like rats, can be empathetic toward one another. But there's no morality in a rat, and we would never think to praise or blame one for its behavior. Empathy itself is necessary for morality, but far from sufficient.