Fhyve comments on If you can see the box, you can open the box - Less Wrong

49 Post author: ThePrussian 26 February 2015 10:36AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (108)

You are viewing a single comment's thread. Show more comments above.

Comment author: Viliam_Bur 26 February 2015 03:13:27PM *  53 points [-]

This is how I understand it:

  • At some sufficiently high meta level, all people believe "I am trying to do the right thing".
  • But people have widely different models of world; therefore their "trying to do the right thing" may result in almost anything, including many things that we consider wrong.
  • Therefore, we are likely to conclude that our enemy, at his high meta level, believes "I am trying to do the wrong thing." Because this makes our model of him simpler for us. But that is not a realistic model. And having a wrong model of reality is potentially dangerous to our goals.

Generally, I think I agree with this. The question is, how specifically high is the level where people are trying to do "the right thing". Here I can imagine that people who didn't have experience with (conventionally called) evil people, can underestimate the necessary level of abstraction.

For some people, in their model of the world, "the right thing" includes e.g. torturing the nonbelievers. Not only because of some reasons that we could consider palatable -- for example, when someone burns a witch, we could say "well, in their model of the world, the witch is able to bring a lot of human suffering, and probably already did, so... while I think their model of the world is stupid, I can on the abstract level empathise with the concept of 'doing the right thing by killing someone who causes a lot of suffering'." No, this is actually very shallow thinking; something that naive people imagine, and that expert Dark Arts religious apologists like Chesterton make them believe. This is not a credible model of a religious fanatic.

For a credible model one has to go yet a few levels deeper. One has to imagine someone who would torture the nonbeliever because he believes that torturing nonbelievers per se is a good thing. Not because of some consequentialist reasoning based on wrong models... this is trying to push our thinking into someone else's head. No, there are people who believe that torturing nonbelievers is intrinsically the right thing. To explain why, it would be like to explain why having an orgasm is pleasant. It obviously is, that's the whole answer. This is how some true believers think, whether they are Nazis, Communists, Muslims, Christians, etc. They may also have some rationalization for why the stuff they consider right will have good (from their point of view) consequences, for example burning the heretics may appease the angry God. But those are just rationalizations. Some people burn heretics because burning the heretics is the right thing to do. Tell them they are wrong, and they will consider you insane.

And yes, one level higher, both these people and the happy hippies of San Francisco are trying to do the right thing.

Maybe the values are not all in the mind. Some values are on level of feelings, associations, reflexes. Orgasm is good because because it feels good. Eating chocolate feels good. Hearing a lullaby feels good. With the right kind of upbringing, the idea of burning the heretic will feel good. Not because we have a specific "heretic burning" sense receptor, but because the parts of the brain containing the idea of burning the heretics were connected by neural pathways to the pleasure centers, just like all associations are created. Given some upbringing, values like this can be hardwired. (The human hardware is not read-only. Associations in human brain develop as we live.) People from different cultures and subcultures can be wired differently, so they may perceive as inherently pleasant the things we abhor, and vice versa. They may feel genuine discomfort from the idea of not burning the heretic.

So what does this mean? Did "doing the right thing" become completely meaningless? Can it predict anything; explain any observation? I don't think it is completely meaningless. Specifically, I believe that some cultures require more brainwashing, some less. (Although I have no specific methodology for measuring the amount of brainwashing. Even the environment can be a component of brainwashing; what exactly would a "neutral" environment look like?) Some cultures are more reflectively coherent than others. Some cultures promote better models of reality. Giving better information and more intelligence to humans would destroy many cultures. It's just... the predictions of this hypothesis are less straightforward than a naive reader would imagine.

Comment author: Fhyve 02 March 2015 09:06:18AM *  3 points [-]

Burning cats is another good example. Can you feel how much fun it is to burn cats? Some people used to have all sorts of fun by burning cats. And this one is harder to do the wrong sort of justification based on bad models than either burning witches or torturing heretics.

Edit: Well, just scrolled down to where you talk about torturing animals. Beat me to it I guess...