Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Disguised Queries

49 Post author: Eliezer_Yudkowsky 09 February 2008 12:05AM

Followup toThe Cluster Structure of Thingspace

Imagine that you have a peculiar job in a peculiar factory:  Your task is to take objects from a mysterious conveyor belt, and sort the objects into two bins.  When you first arrive, Susan the Senior Sorter explains to you that blue egg-shaped objects are called "bleggs" and go in the "blegg bin", while red cubes are called "rubes" and go in the "rube bin".

Once you start working, you notice that bleggs and rubes differ in ways besides color and shape.  Bleggs have fur on their surface, while rubes are smooth.  Bleggs flex slightly to the touch; rubes are hard.  Bleggs are opaque; the rube's surface slightly translucent.

Soon after you begin working, you encounter a blegg shaded an unusually dark blue—in fact, on closer examination, the color proves to be purple, halfway between red and blue.

Yet wait!  Why are you calling this object a "blegg"?  A "blegg" was originally defined as blue and egg-shaped—the qualification of blueness appears in the very name "blegg", in fact.  This object is not blue.  One of the necessary qualifications is missing; you should call this a "purple egg-shaped object", not a "blegg".

But it so happens that, in addition to being purple and egg-shaped, the object is also furred, flexible, and opaque.  So when you saw the object, you thought, "Oh, a strangely colored blegg."  It certainly isn't a rube... right?

Still, you aren't quite sure what to do next.  So you call over Susan the Senior Sorter.

    "Oh, yes, it's a blegg," Susan says, "you can put it in the blegg bin."
    You start to toss the purple blegg into the blegg bin, but pause for a moment.  "Susan," you say, "how do you know this is a blegg?"
    Susan looks at you oddly.  "Isn't it obvious?  This object may be purple, but it's still egg-shaped, furred, flexible, and opaque, like all the other bleggs.  You've got to expect a few color defects.  Or is this one of those philosophical conundrums, like 'How do you know the world wasn't created five minutes ago complete with false memories?'  In a philosophical sense I'm not absolutely certain that this is a blegg, but it seems like a good guess."
    "No, I mean..."  You pause, searching for words.  "Why is there a blegg bin and a rube bin?  What's the difference between bleggs and rubes?"
    "Bleggs are blue and egg-shaped, rubes are red and cube-shaped," Susan says patiently.  "You got the standard orientation lecture, right?"
    "Why do bleggs and rubes need to be sorted?"
    "Er... because otherwise they'd be all mixed up?" says Susan.  "Because nobody will pay us to sit around all day and not sort bleggs and rubes?"
    "Who originally determined that the first blue egg-shaped object was a 'blegg', and how did they determine that?"
    Susan shrugs.  "I suppose you could just as easily call the red cube-shaped objects 'bleggs' and the blue egg-shaped objects 'rubes', but it seems easier to remember this way."
    You think for a moment.  "Suppose a completely mixed-up object came off the conveyor.  Like, an orange sphere-shaped furred translucent object with writhing green tentacles.  How could I tell whether it was a blegg or a rube?"
    "Wow, no one's ever found an object that mixed up," says Susan, "but I guess we'd take it to the sorting scanner."
    "How does the sorting scanner work?" you inquire.  "X-rays?  Magnetic resonance imaging?  Fast neutron transmission spectroscopy?"
    "I'm told it works by Bayes's Rule, but I don't quite understand how," says Susan.  "I like to say it, though.  Bayes Bayes Bayes Bayes Bayes."
    "What does the sorting scanner tell you?"
    "It tells you whether to put the object into the blegg bin or the rube bin.  That's why it's called a sorting scanner."
    At this point you fall silent.
    "Incidentally," Susan says casually, "it may interest you to know that bleggs contain small nuggets of vanadium ore, and rubes contain shreds of palladium, both of which are useful industrially."
    "Susan, you are pure evil."
    "Thank you."

So now it seems we've discovered the heart and essence of bleggness: a blegg is an object that contains a nugget of vanadium ore.  Surface characteristics, like blue color and furredness, do not determine whether an object is a blegg; surface characteristics only matter because they help you infer whether an object is a blegg, that is, whether the object contains vanadium.

Containing vanadium is a necessary and sufficient definition: all bleggs contain vanadium and everything that contains vanadium is a blegg: "blegg" is just a shorthand way of saying "vanadium-containing object."  Right?

Not so fast, says Susan:  Around 98% of bleggs contain vanadium, but 2% contain palladium instead.  To be precise (Susan continues) around 98% of blue egg-shaped furred flexible opaque objects contain vanadium.  For unusual bleggs, it may be a different percentage: 95% of purple bleggs contain vanadium, 92% of hard bleggs contain vanadium, etc.

Now suppose you find a blue egg-shaped furred flexible opaque object, an ordinary blegg in every visible way, and just for kicks you take it to the sorting scanner, and the scanner says "palladium"—this is one of the rare 2%.  Is it a blegg?

At first you might answer that, since you intend to throw this object in the rube bin, you might as well call it a "rube".  However, it turns out that almost all bleggs, if you switch off the lights, glow faintly in the dark; while almost all rubes do not glow in the dark.  And the percentage of bleggs that glow in the dark is not significantly different for blue egg-shaped furred flexible opaque objects that contain palladium, instead of vanadium.  Thus, if you want to guess whether the object glows like a blegg, or remains dark like a rube, you should guess that it glows like a blegg.

So is the object really a blegg or a rube?

On one hand, you'll throw the object in the rube bin no matter what else you learn.  On the other hand, if there are any unknown characteristics of the object you need to infer, you'll infer them as if the object were a blegg, not a rube—group it into the similarity cluster of blue egg-shaped furred flexible opaque things, and not the similarity cluster of red cube-shaped smooth hard translucent things.

The question "Is this object a blegg?" may stand in for different queries on different occasions.

If it weren't standing in for some query, you'd have no reason to care.

Is atheism a "religion"Is transhumanism a "cult"?  People who argue that atheism is a religion "because it states beliefs about God" are really trying to argue (I think) that the reasoning methods used in atheism are on a par with the reasoning methods used in religion, or that atheism is no safer than religion in terms of the probability of causally engendering violence, etc...  What's really at stake is an atheist's claim of substantial difference and superiority relative to religion, which the religious person is trying to reject by denying the difference rather than the superiority(!)

But that's not the a priori irrational part:  The a priori irrational part is where, in the course of the argument, someone pulls out a dictionary and looks up the definition of "atheism" or "religion".  (And yes, it's just as silly whether an atheist or religionist does it.)  How could a dictionary possibly decide whether an empirical cluster of atheists is really substantially different from an empirical cluster of theologians?  How can reality vary with the meaning of a word?  The points in thingspace don't move around when we redraw a boundary.

But people often don't realize that their argument about where to draw a definitional boundary, is really a dispute over whether to infer a characteristic shared by most things inside an empirical cluster...

Hence the phrase, "disguised query".

 

Part of the sequence A Human's Guide to Words

Next post: "Neural Categories"

Previous post: "The Cluster Structure of Thingspace"

Comments (97)

Sort By: Old
Comment author: Psychohistorian3 09 February 2008 01:03:58AM 18 points [-]

While the advisory against using a dictionary to resolve such arguments are true, a lot of arguments stem from confusion or disagreement over the meaning of words. Based on the work I've done in philosophy, this type of disagreement probably covers 50% of philosophical debates, with about 2% of the participants in such debates admitting that that is what they disagree about.

For example, "Most atheists believe in the divinity of Christ" could be resolved easily without recourse to the empirical world. If I believe that it is possible for someone to be an atheist and believe in the divinity of Christ, then I am using atheist to mean something very different from its actual meaning.

As you wrote earlier, using words invokes connotations regardless of whether a newly assigned definition merits the same connotations. Some on the far left have defined "racism" to mean "is White and lives in the USA." Appealing to a dictionary is useful in an argument with such a person because it prevents them from using a very charged word inappropriately. Similar tricks occur with "fascism," "freedom," "democracy," and many other such words.

Basically, a dictionary doesn't decide if an empirical cluster has a certain property, but it does ensure that the word you are using matches the empirical cluster you are referring to. It is irrational to try to prove an empirical fact with a definition. It is not at all irrational if there is any disagreement over what group is picked out by the word, or whether the group picked out by the word must or must not have a certain property, or else the word would not pick them out. More disagreements center on poorly understood definitions than most people would like to admit.

On a related note, this recent series on definitions is quite brilliantly written, Eliezer, even more so than usual.

Comment author: TGGP4 09 February 2008 02:02:09AM 1 point [-]

Based on the work I've done in philosophy, this type of disagreement probably covers 50% of philosophical debates, with about 2% of the participants in such debates admitting that that is what they disagree about. Someone remind me against why I'm supposed to take philosophy seriously.

Comment author: Caledonian2 09 February 2008 02:19:31AM 1 point [-]

Because if no one takes philosophy seriously, the philosophers will have nothing at all.

Will you take that away from them? They have so little as it is.

Comment author: Richard_Hollerith2 09 February 2008 02:42:56AM 8 points [-]

Is atheism a "religion"? Is transhumanism a "cult"?

My favorite example is, Is a fetus a person?

Comment author: james_gnz 09 April 2011 10:14:55AM *  -2 points [-]

Quote: My favorite example is, Is a fetus a person?

I can answer this one: A foetus is not a person prior to 20 weeks gestation (18 weeks of pregnancy), but may be a person from that point onwards.

A body with one mind is one person. A body with two minds is two people (conjoint twins). A body with three minds would be three people. A heart transplant does not switch a person into a different body. A lung transplant does not switch a person into a different body. A brain transplant (and therefore a mind transplant) would switch a person into a different body. It is minds, not bodies, that defines people.

The mind exists, if at all, in the brain, or more specifically the cerebral cortex. The cerebral cortex begins to develop connections no earlier than 20 weeks gestation, therefore there is not a person before this time (though the body does have reflexes).

'Brain Waves' When??? <http://eileen.250x.com/Main/Einstein/Brain_Waves.htm> Margaret Sykes

Comment author: ec429 16 September 2011 04:13:43PM 8 points [-]

That doesn't answer the question "Is a fœtus a person", it just supplies a definition of "person", which may or may not be relevant to any given query.

Suppose my real query is "Can a fœtus talk?" Now, just because I choose to define "person" in such a way that most "person"s can talk, and in such a way that a fœtus classes as a "person", that doesn't make the probability that a fœtus can talk any different to if I'd defined "person" differently.

The whole point of these examples of disguised queries is that if you find yourself trying to answer them, you're doing it wrong.

Suppose we call the horse's tail a leg.

Comment author: lessdazed 16 September 2011 05:31:10PM 0 points [-]

Hi, welcome!

Comment author: Nick_Tarleton 09 February 2008 02:43:27AM 8 points [-]

People who argue that atheism is a religion "because it states beliefs about God" are really trying to argue (I think) that the reasoning methods used in atheism are on a par with the reasoning methods used in religion, or that atheism is no safer than religion in terms of the probability of causally engendering violence, etc...

Or they're applying a Fully General Counterargument without actually trying to make any substantive point, or realizing that they should be?

Comment author: ignoramus 17 December 2011 12:32:44PM *  2 points [-]

Indeed.
For example:

Eliezer: Religion sucks, because of this and Bayes...
Jesus: Ah, not so fast, chap. You see, atheism is also a religion, because of this and that...

I think that Jesus' response is a non sequitur (a well designed one (by using a technique similar to equivocation), which is why it makes for such good "blocking" technique). So there's no disguised query, since Jesus isn't querying at all, he's just trying to "win" the argument.

Comment author: Tsujigiri 17 December 2011 02:08:14PM -1 points [-]

There is a similarity between Christians and many atheists in their moral philosophy, however. Atheists may not believe in God, but I think they mostly adhere to the 10 commandments.

At least Christians can say they follow their moral philosophy because God told them so. What reason do atheists have?

Comment author: ignoramus 17 December 2011 02:44:24PM *  0 points [-]

I was actually just trying to say that Eliezer gave a bad example of a disguised query.

As for moral philosophy, it can be considered a science. So atheists that believe in morality should value it as any other science (for it's usefulness etc). Well, hm, atheists need not be fans of science. So they can be moral because they enjoy it, or simply because "why the heck not".

Comment author: Tsujigiri 17 December 2011 02:59:55PM 0 points [-]

I wouldn't call moral philosophy a science.

If we both independently invented an imaginary creature, neither would be correct. They are simply the creatures we've arbitrarily created. There is no science of moral philosophy anymore than there is a science of inventing an imaginary creature.

I'd say to be science there needs to be the ability to test whether something is valid. There is no such test for the validity of morals anymore than there is a test for the validity of an imaginary creature.

Comment author: TheOtherDave 17 December 2011 02:49:12PM *  3 points [-]

You have that backwards.

Moral people follow their moral philosophy because they believe it's the right thing to do, whether they are Christian or atheist or neither.

Some moral people also believe God has told them to do certain things, and use those beliefs to help them select a moral philosophy. Those people are moral and religious.
Other moral people don't believe that, and select a moral philosophy without the aid of that belief. Those people are moral and atheist.

Some immoral people believe that God has told them to do certain things. Those people are immoral and religious.
Some immoral people don't believe that. Those people are immoral and atheist.

Incidentally, I know no atheists (whether moral or not) who adhere to the Talmudic version of the first commandment. But then, since you are talking about the ten commandments in a Christian rather than Jewish context, I suppose you don't subscribe to the Talmudic version anyway a. (cf http://en.wikipedia.org/wiki/Ten_Commandments#Two_texts_with_numbering_schemes)

EDIT: I should probably also say explicitly that I don't mean to assert here that nobody follows the ten commandments simply because they believe God told them to... perhaps some people do. But someone who doesn't think the ten commandments are the right thing to do and does them anyway simply because God told them to is not a moral person, but rather a devout or God-fearing person. (e.g., Abraham setting out to sacrifice his son).

Comment author: DanArmak 17 December 2011 03:10:33PM 2 points [-]

Incidentally, I know no atheists (whether moral or not) who adhere to the Talmudic version of the first commandment.

I also know no atheists who adhere to the second commandment (make no graven image), the fourth (no "work" on Shabbath), or the tenth (do not covet).

Comment author: [deleted] 17 December 2011 03:03:14PM 6 points [-]

Atheists may not believe in God, but I think they mostly adhere to the 10 commandments.

I think you're just trying to say that atheists follow moral expectations of modern Christian-influenced culture, but taken literally, the statement's nonsense.

I mean, look at the Ten Commandments:

  1. Thou shalt have no other gods before me.
  2. Thou shalt not make unto thee any graven image (...).
  3. Thou shalt not take the name of the Lord thy God in vain (...).
  4. Remember the sabbath day, to keep it holy. (...)
  5. Honour thy father and thy mother (...).
  6. Thou shalt not kill.
  7. Thou shalt not commit adultery.
  8. Thou shalt not steal.
  9. Thou shalt not bear false witness against thy neighbour.
  10. Thou shalt not covet thy neighbour's house, (...) nor his ass, nor any thing that is thy neighbour's.

The first 4 are blatantly ignored, 6 is famously problematic, 9 and 10 are mostly ignored (via gossip, status seeking, greed and so on) and finally 7 and 8 might be typically obeyed, but minor theft (especial anonymous) is common and adultery has at least 10% base rates.

How is this a "mostly adhered"? (Obviously, Christians and atheists don't really differ in their behavior here.)

Comment author: TheOtherDave 17 December 2011 03:12:28PM 1 point [-]

In the interests of charitable reading, I took them to mean "atheists adhere to the ten commandments about as well as Christians do".

Comment author: wedrifid 17 December 2011 03:27:57PM 1 point [-]

In the interests of charitable reading, I took them to mean "atheists adhere to the ten commandments about as well as Christians do".

I looked through them and I was surprised at how little I break them. 4 is way off, of course and I'll honour my father and mother to the extend they damn well earn it (rather a lot as it turns out). The thing is going by the standards that I actually held for following all those commandments when I was Christian I could have expected to be violating all over the place. I'm particularly disappointed with No. 7. I've been making a damn fine effort to be living in sin as much as conveniently possible but since I have yet to sleep with a married woman I seem to be clean on that one. Going by the actual commandment I'm probably even ok with 3. The "swearing" thing seems to be totally blown out of proportion.

Comment author: TheOtherDave 17 December 2011 03:48:46PM 0 points [-]

Personally I break some of them more often than I'd like, but then again I did so when I identified as an Orthodox Jew as well.

Of course, if I were to take this seriously, I'd get bogged down in definitional issues pretty quickly. For example, I've slept with a married man (married to someone else, I mean), so I guess I've violated #7... or at least, he did. OTOH, given that everyone involved was aware of the situation and OK with it, I don't consider that any of us were doing anything wrong in the process.

But a certain kind of religious person would say that my beliefs about what's right and wrong don't matter. Of course, I would disagree.

Comment author: wedrifid 17 December 2011 04:22:58PM 1 point [-]

I suppose you do technically scrape through in adhering to No. 7 as it is presented in that wikipedia passage based on two technicalities. That it it is only adultery if you sleep with a married woman and that being the partner of the adulterer doesn't qualify. (I'm a little skeptical of that passage actually). Come to think of it you may get a reprieve for a third exception if it is the case that the other guy was married to a guy (ambiguous).

Comment author: TheOtherDave 17 December 2011 04:33:30PM 0 points [-]

The guy in question was married to a woman at the time.

Agreed about the technicalities.

Comment author: wedrifid 17 December 2011 04:24:16PM *  1 point [-]

Personally I break some of them more often than I'd like

It's 6, isn't it! (Dexter has that problem too - I recommend following his example and at least chanelling it into vigilantism.)

Comment author: TheOtherDave 17 December 2011 04:47:10PM 1 point [-]

Well, I kill all the time... most people I know do.

But if we adopt the conventional practice of translating "lo tirtzoch" as "don't murder", and further adopt the conventional practice of not labeling killings we're morally OK with as "murder", then I squeak by here as well... I'm basically OK with all the killing I've done.

I've never actually watched Dexter, but I gather it's about someone compelled to murder people who chooses to murder only people where the world is improved by their death? Hrm. I'm not sure I agree.

Certainly, if I'm going to murder someone, it should be the least valuable person I can find. Which might turn out to be myself. The question for me is how reliable my judgment is on the matter. If I'm not a reliable judge, I should recuse myself from judgement.

Perhaps I should assemble a committee to decide on my victims.

Comment author: wedrifid 17 December 2011 04:57:00PM 0 points [-]

But if we adopt the conventional practice of translating "lo tirtzoch" as "don't murder", and further adopt the conventional practice of not labeling killings we're morally OK with as "murder"

That doesn't sound like a convention that the quite fits with culture or spirit of the holy law in question or of the culture which would create such a law.

Comment author: wedrifid 17 December 2011 05:08:48PM *  0 points [-]

I've never actually watched Dexter, but I gather it's about someone compelled to murder people who chooses to murder only people where the world is improved by their death?

In what I've seen of Dexter the most ethically grey kill was of a pedophile who was stalking his step-daughter (and that's a murder I'd be comfortable committing!). The rest were all murderers who were highly likely to kill again.

For my part I would prefer to live in a world in which other people don't go around being vigilantes and also don't want to be a vigilante myself. Because frankly it isn't my problem and it isn't worth the risk or the effort it would take me.

Comment author: ArisKatsaris 17 December 2011 05:12:04PM 0 points [-]

I've never actually watched Dexter, but I gather it's about someone compelled to murder people who chooses to murder only people where the world is improved by their death?

Slightly more specific and slightly less consequentialistic than that. He chooses to kill only other murderers, and usually only cold-blooded murderers who are unrepentant and likely to murder again, (example: one time he stopped when he realized his selected victim had only murdered the person that had raped him in prison).

But it's not about improving the world really, sometimes he even sabotages the police investigation just so he can have these people to himself.

Comment author: ArisKatsaris 17 December 2011 05:17:35PM 0 points [-]

But if we adopt the conventional practice of translating "lo tirtzoch" as "don't murder", and further adopt the conventional practice of not labeling killings we're morally OK with as "murder", then I squeak by here as well...

I think the general idea is that by "murder" the concept of 'do not kill people without it being prescribed by the law' is meant -- with the rest of Mosaic law indicating in which cases it was okay to kill people nonetheless.

So killing insects doesn't count (because they're not people), nor being a state executioner counts (because it's prescribed by the law).

Comment author: Tsujigiri 17 December 2011 03:15:21PM 2 points [-]

I'll have to concede that atheists moral beliefs don't mostly adhere to the 10 commandments.

The point I wished to make was that many of the moral philosophies of rationalists are very similar to their Christian counterparts. I believe the similarity is mostly due to the culture they were brought up in rather than whether they believe God exists or not. You might even consider God to be irrelevant to the issue.

Comment author: [deleted] 17 December 2011 03:18:28PM 0 points [-]

Agreed. Obligatory Moldbug link (warning: long, and only first in a series) for an interesting derivation of (some) modern morality as atheistic Christianity.

Comment author: TheOtherDave 17 December 2011 03:29:05PM 0 points [-]

I certainly agree that many people's moral beliefs are shaped and constrained by their culture, and that God is irrelevant to this, as is belief in God.

Comment author: juliawise 17 December 2011 06:41:17PM 0 points [-]

Upvoted for the 10th commandment link.

Comment author: raptortech97 13 April 2012 08:41:28PM 0 points [-]

What reason do atheists have?

Maybe because they have decided that a specific moral philosophy would be most useful?

Comment author: Ezekiel 13 April 2012 09:20:27PM *  1 point [-]

Atheists may not believe in God, but I think they mostly adhere to the 10 commandments.

Nitpick: Only half of the Ten Commandments are nice humanitarian commandments like "don't murder". The other half are all about how humans should interact with God, and I don't think most atheists put much weight behind "you will not make for yourself any statue or any picture of the sky above or the earth below or the water that is beneath the earth".

At least Christians can say they follow their moral philosophy because God told them so.

They can say that, but unless they already have a moral philosophy that gives God moral authority (or states that Hell is to be avoided, or justifies gratitude for Creation, or...) that's not actually a reason.

Comment author: [deleted] 13 April 2012 09:28:14PM 1 point [-]

Well, for myself, it's because game theory says the world works better when people aren't dicks to one another, and because empathy (intuitive and rational) allow me to put myself in other peoples' shoes, and to appreciate that it's good to try to help them when I can, since they're very much like myself. I have desires and goals, and so do they, and mine aren't particularly more important simply because they're mine.

Comment author: Swimmer963 13 April 2012 09:37:21PM 0 points [-]

I have desires and goals, and so do they, and mine aren't particularly more important simply because they're mine.

This is the base of my whole moral philosophy, too. And you know what? There are people who actually disagree with it! Responses I've gotten from people in discussions have ranged from "I don't give a shit about other people, they're not me" to "you can't think like that, you need to think selfishly, because otherwise everyone will trample on you."

Comment author: Swimmer963 13 April 2012 09:34:45PM -1 points [-]

What reason do atheists have?

Lots of reasons. It's pretty much built into the human brain that being nice to your friends and neighbours is helpful to long-term survival, so most people get pleasant feelings from doing something they consider 'good', and feel guilty after doing something they consider 'bad'. You don't need the Commandments themselves.

...Oh and the whole idea that it's better to live in a society where everyone follows laws like "don't murder"...even if you personally could benefit from murdering the people who you didn't like, you don't want everyone else murdering people too, and so it makes sense, as a society, to teach children that 'murder is bad'.

Comment author: [deleted] 13 April 2012 10:25:11PM 0 points [-]

It's pretty much built into the human brain that being nice to your friends and neighbours is helpful to long-term survival, so most people get pleasant feelings from doing something they consider 'good', and feel guilty after doing something they consider 'bad'.

Are these reasons to not kill people or steal? Can I propose a test? Suppose that it were built into the human brain that being cruel to your friends and neighbors is helpful to long-term survival (bear with me on the evolutionary implausibility of this), and so must people get pleasant feelings from doing things they consider cruel, and feel guilty after doing nice things.

Suppose all that were true: would you then have good reasons to to be cruel? If not, then how are they reasons to be nice?

Comment author: dlthomas 13 April 2012 10:33:06PM 0 points [-]

You would clearly have reasons; whether they are good reasons depends how you're measuring "good".

Comment author: [deleted] 13 April 2012 10:42:43PM 0 points [-]

We might want to distinguish here between reasons to do something and reasons why one does something. So imagine we discover that the color green makes people want to compromise, so we paint a boardroom green. During a meeting, the chairperson decides to compromise. Even if the chairperson knows about the study, and is being affected by the green walls in a decisive way (such that the greenness of the walls is the reason why he or she compromises), could the chairperson take the greenness of the walls as a reason to compromise?

Comment author: dlthomas 13 April 2012 11:53:53PM 0 points [-]

A reasonable distinction, but I don't think it quite maps onto the issue at hand. You said to suppose "people get pleasant feelings from doing things they consider cruel, and feel guilty after doing nice things". If one has a goal to feel pleasant feelings, and is structured in that manner, then that is reason to be cruel, not just reason why they would be cruel.

Comment author: [deleted] 14 April 2012 12:12:36AM 0 points [-]

If one has a goal to feel pleasant feelings, and is structured in that manner, then that is reason to be cruel, not just reason why they would be cruel.

Agreed, but so much is packed into that 'if'. We all seek pleasure, but not one of us believes it is an unqualified good. The implication of Swimmer's post was that atheists have reasons to obey the ten commandments (well, 4 or 5 of them) comparable in formal terms to the reasons Christians have (God'll burn me if I don't, or whatever). That is, the claim seems to be that atheists can justify their actions. Now, if someone does something nice for me, and I ask her why she did that, she can reply with some facts about evolutionary biology. This might explain her behavior, but it doesn't justify it.

If we imagine someone committing a murder and then telling us something about her (perhaps defective) neurobiology, we might take this to explain their behavior, but never to justify it. We would never say 'Yeah, I guess now that you make those observations about your brain, it was reasonable of you to kill that guy." The point is that the murderer hasn't just given us a bad reason, she hasn't given us a reason at all. We cannot call her rational if this is all she has.

Comment author: Swimmer963 14 April 2012 02:16:23AM *  0 points [-]

The implication of Swimmer's post was that atheists have reasons to obey the ten commandments (well, 4 or 5 of them) comparable in formal terms to the reasons Christians have (God'll burn me if I don't, or whatever).

I didn't claim that, and if I implied it, it was by accident. (Although I do think that a lot of atheists have just as strong if not stronger reasons to obey certain moral rules, the examples I gave weren't those examples.) I was trying to point out that if someone decides one day to stop believing in God, and realizes that this means God won't smite them if they break one of the Ten Commandments, that doesn't mean they'll go out and murder someone. Their moral instincts, and the positive/negative reinforcement to obey them (i.e. pleasure or guilt), keep existing regardless of external laws.

The point is that the murderer hasn't just given us a bad reason, she hasn't given us a reason at all. We cannot call her rational if this is all she has.

So we ask her why, and she says "oh, he took the seat that I wanted on the bus three weeks in a row, and his humming is annoying, and he always copies my exams." Which might not be a good reason to murder someone according to you, with your normal neurobiology–you would content yourself with fuming and making rude comments about him to your friends–but she considers it a good reason, because her mental 'brakes' are off.

Comment author: Swimmer963 13 April 2012 10:53:53PM -1 points [-]

Suppose all that were true: would you then have good reasons to to be cruel?

It's a hard world to visualize, but if cruelty-tendencies evolved because people survived better by being cruel, then cruelty works in that world, and society would be dysfunctional if there were rules against it (imagine our world having rules against being nice, ever!), and to me, something being useful is a good reason to do it.

If we ever came across that species, no doubt we'd be appalled, but the universe isn't appalled. Not unless you believe that morality exists in itself, independently of brains...which I don't.

Suppose that it were built into the human brain that being cruel to your friends and neighbors is helpful to long-term survival (bear with me on the evolutionary implausibility of this), and so must people get pleasant feelings from doing things they consider cruel, and feel guilty after doing nice things.

If there were an entire society built out of people like this, then probably quite a lot of minor day-to-day cruelty would go on, and there would be rationalized Laws, like the Ten Commandments, justifying why being cruel was so important, and there would be social customs and structures and etiquette involved in making sure the right kind of cruelty happened at the right times...

I'm not saying that our brain's evolutionary capacity for empathy is the ultimate perfect moral theory. But I do think that all those moral theories, perfect or ultimate or not, exist because our brains evolved to have the little voice of empathy. Which means that if you take away the Ten Commandments, most people won't stop being nice to people they care about.

(Being nice to strangers or members of an outgroup is a completely different matter...there seems to be a mechanism for turning off empathy towards groups of strangers, and plenty of societies have produced people who were very nice to their friends and neighbors, and barbaric towards everyone else.)

Most atheists don't accept deontological moral theories–i.e. any theory that talks about a set of a priori rules of what's right versus wrong. But morality doesn't go away. If you reason it out starting from what our brains already tell us, you end up with utilitarian theories ("I like being happy, and I'm capable of empathy, so I think other people must like being happy too, and since my perfect world would be one where I was happy all the time, the perfect world for everyone would be one with maximum happiness.")

Alternately you end up with Kantian theories ("I like being treated as an end, not a means, and empathy tells me other people are similar to me, so we should treat everyone as an end in themselves or not a means... Oh, and Action X will make me happy, but if everyone else did Action X too, it would make me unhappy, and empathy tells me everyone else is about like me, so they wouldn't want me to do X, so the best society is one in which no one does X.") Etc.

If you don't reason it out, you get "well, it made me happy when I helped Susan with her homework, and it made me feel bad when I said something mean to Rachel and she cried, so I should help people more and not be mean as much." These feelings aren't perfect, and there are lots of conflicting feelings, so people aren't nice all the time...but the innate brain mechanisms are there even when there aren't any laws, and the fact that they're there is probably the reason why there are laws at all.

Comment author: [deleted] 13 April 2012 11:23:36PM 0 points [-]

These feelings aren't perfect, and there are lots of conflicting feelings, so people aren't nice all the time...but the innate brain mechanisms are there even when there aren't any laws, and the fact that they're there is probably the reason why there are laws at all.

So we agree that one might have a reason to do something because it's recommended by moral theories. What I'm questioning is whether or not you can have a reason to do something on the basis of brain mechanisms or if you can have reason to adopt a moral theory on the basis of brain mechanisms. And I don't mean 'good' reasons, I mean admissible reasons.

Imagine someone thinking to themselves: 'Well, my brain is structured in such and such a way as a result of evolution, so I think I'll kill this completely innocent guy over here.' Is he thinking rationally?

And concerning the adoption of a moral theory:

("I like being happy, and I'm capable of empathy, so I think other people must like being happy too, and since my perfect world would be one where I was happy all the time, the perfect world for everyone would be one with maximum happiness.")

There's a missing inference here from wanting to be happy to wanting other people to be happy. Can you explain how you think this argument gets filled out? As it stands, it's not valid.

Likewise:

"I like being treated as an end, not a means, and empathy tells me other people are similar to me, so we should treat everyone as an end in themselves or not a means...

Why should the fact that other people want something motivate me? It doesn't follow from the fact that my wanting something motivates me, that another person's wanting that thing should motivate me. In both these arguments there's a missing step which, I think, is pertinent to the problem above: the fact that I am motivated to X doesn't even give me reason to X, much less a reason to pursue the desires of other people.

Comment author: Swimmer963 13 April 2012 11:43:46PM 0 points [-]

Well, my brain is structured in such and such a way as a result of evolution, so I think I'll kill this completely innocent guy over here.

Beliefs don't feel like beliefs, they feel like the way the world is. Likewise with brain structures. If someone is a sociopath (in short, their brain mechanism for empathy is broken) and they decide they want to kill someone for reasons X and Y, are they being any more irrational than someone who volunteers at a soup kitchen because seeing people smile when he hands them their food makes him feel fulfilled?

("I like being happy, and I'm capable of empathy, so I think other people must like being happy too, and since my perfect world would be one where I was happy all the time, the perfect world for everyone would be one with maximum happiness.")

There's a missing inference here from wanting to be happy to wanting other people to be happy. Can you explain how you think this argument gets filled out? As it stands, it's not valid.

Sorry for not being clear. The inference is that "empathy", the ability to step into someone else's shoes and imagine being them, is an innate ability that most humans have, leads you to think that other people are like you...when they feel pleasure, it's like your pleasure, and when they feel pain, it's like your pain, and there's a hypothetical world where you could have been them. I don't think this hypothetical is something that's taught by moral theories, because I remember reasoning with it as a child when I'd had basically no exposure to formal moral theories, only the standard "that wasn't nice, you should apologize." If you could have been them, you want the same things for them that you'd want for yourself.

I think this is immediately obvious for family members and friends...do you want your mother to be happy? Your children?

Comment author: [deleted] 14 April 2012 12:02:01AM *  1 point [-]

Beliefs don't feel like beliefs, they feel like the way the world is.

Perhaps on some level this is right, but the fact that I can assess the truth of my beliefs means that they don't feel like the way the world is in an important respect. They feel like things that are true and false. The way the world is has no truth value. Very small children have problem with this distinction, but so far as I can tell almost all healthy adults do not believe that their beliefs are identical with the world. ETA: That sounded jerky. I didn't intend any covert meanness, and please forgive any appearance of that.

If someone is a sociopath (in short, their brain mechanism for empathy is broken) and they decide they want to kill someone for reasons X and Y, are they being any more irrational than someone who volunteers at a soup kitchen because seeing people smile when he hands them their food makes him feel fulfilled?

I think I really don't understand your question. Could you explain the idea behind this a little better? My objection was that there are reasons to do things, and reasons why we do things, and while all reasons to do things are also reasons why, there are reasons why that are not reasons to do things. For example, having a micro-stroke might be the reason why I drive my car over an embankment, but it's not a reason to drive one's car over an embankment. No rational person could say to themselves "Huh, I just had a micro-stroke. I guess that means I should drive over this embankment."

I think this is immediately obvious for family members and friends...do you want your mother to be happy? Your children?

Sure, but I take myself to have moral reasons for this. I may feel this way because of my biology, but my biology is never itself a reason for me to do anything.

Comment author: Swimmer963 14 April 2012 02:12:33AM 1 point [-]

Beliefs don't feel like beliefs, they feel like the way the world is.

Perhaps on some level this is right, but the fact that I can assess the truth of my beliefs means that they don't feel like the way the world is in an important respect.

OK, let me give you a better example. When you look at something, a lot of very complex hardware packed into your retina, optic nerve, and visual cortex, a lot of hard-won complexity optimized over millions of years, is going all out analyzing the data and presenting you with comprehensible shapes, colour, and movement, as well as helpful recognizing objects for you. When you look at something, are you aware of all that happening? Or do you just see it?

(Disclaimer: if you've read a lot about neuroscience, it's quite possible that sometimes you do think about your visual processing centres while you're looking at something. But the average person wouldn't, and the average person probably doesn't think 'well, there go my empathy centres again' when they see an old lady having trouble with her grocery bag and feel a desire to help her.)

I think I really don't understand your question. Could you explain the idea behind this a little better? My objection was that there are reasons to do things, and reasons why we do things, and while all reasons to do things are also reasons why, there are reasons why that are not reasons to do things.

Okay, let's try to unpack this. In my example, we have a sociopath who wants to murder someone. The reason why he wants to murder someone, when most people don't, is because there's a centre in his brain that's broken and so hasn't learned to see the world from another's perspective, thus hasn't internalized any social morality because it doesn't make sense to him...basically, people are objects to him, so why not kill them. His reason to murder someone is because, let's say, they're dating a girl he wants to date. Most non-sociopaths wouldn't consider that a reason to murder anyone, but the reason why they wouldn't is because they have an innate understanding that other people feel pain, of the concept of fairness, etc, and were thus capable of learning more complex moral rules as well.

Sure, but I take myself to have moral reasons for this. I may feel this way because of my biology, but my biology is never itself a reason for me to do anything.

The way I see it, the biology aspect is both necessary and sufficient for this kind of behaviour. Someone without the requisite biology wouldn't be a good parent or friend because they'd see no reason to make an effort (unless they were deliberately "faking it" to benefit from that person). And an ordinary human being raised with no exposure to moral rules, who isn't taught anything about it explicitly, will still want to make their friends happy and do the best they can raising children. They may not be very good at it, but unless they're downright abused/severely neglected, they won't be evil.

Comment author: orthonormal 14 April 2012 03:43:04AM 0 points [-]

I may feel this way because of my biology, but my biology is never itself a reason for me to do anything.

Relevant LW post.

Comment author: Konrad2 09 February 2008 03:47:07AM 6 points [-]

Summary: Aristotelianism considered harmful; Hilbert Space is the new industry standard.

Comment author: Ari 09 February 2008 06:29:59AM 0 points [-]

The question "Is this object a blegg?" may stand in for different queries on different occasions. If it weren't standing in for some query, you'd have no reason to care.

Basically, this is pragmatism in a nutshell -- right?

Cheers, Ari

Comment author: tcpkac 09 February 2008 09:52:45AM 1 point [-]

Excellent post, however, "But people often don't realize that their argument about where to draw a definitional boundary, is really a dispute over whether to infer a characteristic shared by most things inside an empirical cluster..." Indeed so, but there are other aspects. Humans also have obsessions with (a) how far your cluster is from mine (kinship or the lack of it) (b) given one empirical cluster, how can I pick a characteristic, however minor, which will allow me to split it into 'us vs them' (Robber's Cave). So when you get to discussing whether an uploaded human brain is part of the cluster 'human', those are the considerations which will be foremost.

Comment author: Joseph_Knecht 10 February 2008 12:03:28AM 6 points [-]

Or more concisely: sharp distinctions regarding fuzzy concepts are meaningless.

Comment author: TGGP4 10 February 2008 03:19:19AM 0 points [-]

My favorite example is, Is a fetus a person? Yes, but it's still okay to murder them.

Micha Gertner has an interesting essay on pragmatism & economics here.

Comment author: Rolf_Nelson2 11 February 2008 12:51:27AM 7 points [-]

What's really at stake is an atheist's claim of substantial difference and superiority relative to religion

Often semantics matter because laws and contracts are written in words. When "Congress shall make no law respecting an establishment of religion", it's sometimes advantageous to claim that you're not a religion, or that your enemy is a religion. If churches get preferential tax treatment, it may be advantageous to claim that you're a church.

Comment author: Doug_S. 11 February 2008 02:59:23AM 1 point [-]

Often semantics matter because laws and contracts are written in words.

What he said.

Comment author: metahacker 11 February 2008 10:09:05PM 0 points [-]

I'm having problems with the word "is" in your description.

This is not intended as a snarky comment...

Comment author: TGGP4 11 February 2008 10:26:22PM 0 points [-]

Rolf, have you been reading Unqualified Reservations?

Comment author: Julia_Galef 12 March 2010 08:01:31PM *  12 points [-]

This was a really clarifying post for me. I had gotten to the point of noticing that "What is X?" debates were really just debates over the definition of X, but I hadn't yet taken the next step of asking why people care about how X is defined.

I think another great example of a disguised query is the recurring debate, "Is this art?" People have really widely varying definitions of "art" (e.g., some people's definition includes "aesthetically interesting," other people's definition merely requires "conceptually interesting") -- and in one sense, once both parties explain how they use the word "art," the debate should resolve pretty quickly.

But of course, since it's a disguised query, the question "Is this art?" should really be followed up with the question "Why does it matter?" As far as I can tell, the disguised query in this case is usually "does this deserve to be taken seriously?" which can be translated in practice into, "Is this the sort of thing that deserves to be exhibited in a gallery?" And that's certainly a real, non-semantic debate. But we can have that debate without ever needing to decide whether to apply the label "art" to something -- in fact, I think the debate would be much clearer if we left the word "art" out of it altogether.

I've elaborated on this topic on Rationally Speaking: http://rationallyspeaking.blogspot.com/2010/03/is-this-art-and-why-thats-wrong.html ...and I cite this LW post. Thanks, Eliezer.

Comment author: David_Gerard 10 January 2011 03:20:57PM -1 points [-]

I like this post because it shows the usefulness of one of my favourite questions to answer a question with: "What's it for?" What use do you have for the answer to your question?

Comment author: Dojan 02 November 2011 08:39:08AM 0 points [-]

When I have discussions of the philosophical kind, I have learned that it often pays of to start with defining the words being used: For example, I recall one discussion where I defined Evil as a shorthand for "all corporations and institutions that try to compete by opposing the existence and legitimacy of competitors and newcomers instead of by trying to offer a better product, like Microsoft", and one other discussion where I defined Evil as "Working for Sauron or Saruman or Morgoth", i.e very different. I would never (that is, I try hard not to) use a word such as evil without defining it first: People are all to likely to think of something other than what I meant.