SusanBrennan comments on Thoughts on moral intuitions - LessWrong

39 Post author: Kaj_Sotala 30 June 2012 06:01AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (199)

You are viewing a single comment's thread. Show more comments above.

Comment author: Kaj_Sotala 18 July 2012 11:20:33AM 6 points [-]

You were casting unfairly casting your political opponents not just as wrong, but as morally reprehensible.

Right - and the interesting thing is, I had no idea that I was doing it, and in fact was trying to do the opposite. I did my best to take extreme viewpoints like "eating meat is like committing genocide" and "everyone should be converted or they'll go to hell" and attempted to portray them as psychologically no different from any other belief. But although I think I did okay with that, an uncharitable and exaggerated strawman still managed to slip in earlier on.

For the most part, I think it's just about the general ingroup-outgroup tendency in humans, and the desire to look down on any outgroups. But as for that bias slipping into my writing, even when I was explicitly trying to avoid it - that seems to have more to do with the way that most of our thought and behavior is built on subconscious systems, with conscious thought only playing a small role. Or to use Jonathan Haidt's analogy, the conscious mind is the rider of an elephant:

'm holding the reins in my hands, and by pulling one way or the other I can tell the elephant to turn, to stop, or to go. I can direct things, but only when the elephant doesn't have desires of his own. When the elephant really wants to do something, I'm no match for him.

...The controlled system [can be] seen as an advisor. It's a rider placed on the elephant's back to help the elephant make better choices. The rider can see farther into the future, and the rider can learn valuable information by talking to other riders or by reading maps, but the rider cannot order the elephant around against its will...

...The elephant, in contrast, is everything else. The elephant includes gut feelings, visceral reactions, emotions, and intuitions that comprise much of the automatic system. The elephant and the rider each have their own intelligence, and when they work together well they enable the unique brilliance of human beings. But they don't always work together well.

That elephant is very eager to pick up on all sorts of connotations and biases from its social environment, and if we spend a lot of time in an environment where a specific group (conservatives, say) frequently gets bashed, then we'll start to imitate that behavior ourselves - automatically and almost as a reflex, and sometimes even when we think that we're doing the exact opposite.

It is a pity that this kind of a bias hasn't really been discussed much on LW. Probably because the original sequences drew most heavily upon cognitive psychology and math, whereas this kind of bias has been mostly explored in social psychology and the humanities.

Comment author: SusanBrennan 18 July 2012 11:54:26AM 2 points [-]

I remember coming across this paper during my PhD, and it provides a somewhat game theoretic analysis of in-group out-group bias, which is still fairly easy to follow. The paper is mainly about the implications for conflict resolution, as the authors are lecturers in business an law, so it should be of interest to those seeking to improve their rationality (particularly where keeping ones cool in arguments is involved), which is why we are here after all.

I've been thinking about doing my first mainspace post for LessWrong soon. Perhaps I could use it to address this. Unfortunately I've forgotten a very famous social psychology experiment wherein one group (group A) was allowed to dictate their preferred wage difference between their group and and another group (group B). They chose the option which gave them the least in an absolute sense because the option gave them more than group B by comparison. They were divided according to profession. It's a very famous experiment, so I'm sure someone here will know it.

Comment author: VincentYu 18 July 2012 02:03:50PM *  6 points [-]

Unfortunately I've forgotten a very famous social psychology experiment wherein one group (group A) was allowed to dictate their preferred wage difference between their group and and another group (group B). They chose the option which gave them the least in an absolute sense because the option gave them more than group B by comparison. They were divided according to profession. It's a very famous experiment, so I'm sure someone here will know it.

In Irrationality, Sutherland cites Brown (1978, "Divided we fall: An analysis of relations between sections of a factory workforce") and states:

In real life, the rivalry between groups may be so irrational that each may try to do the other down even at its own expense. In an aircraft factory in Britain the toolroom workers received a weekly wage very slightly higher than that of the production workers. In wage negotiations the toolroom shop stewards tried to preserve this differential, even when by so doing they would receive a smaller wage themselves. They preferred a settlement that gave them £67.30 a week and the production workers a pound less, to one that gave them an extra two pounds (£69.30) but gave the production workers more (£70.30).

In a highly-cited review, Tajfel (1982) states:

An intriguing aspect of the early data on minimal categorization was the importance of the strategy maximizing the difference between the awards made to the ingroup and the outgroup even at the cost of giving thereby less to members of the ingroup. This finding was replicated in a field study (Brown 1978) in which shop stewards representing different trades unions in a large factory filled distri­bution matrices which specified their preferred structure of comparative wages for members of the unions involved. It was not, however, replicated in another field study in Britain (Bourhis & Hill 1982) in which similar matrices were completed by polytechnic and university teachers.

A brief look at recent studies seems to suggest a more nuanced relation, but I'm not familiar with the literature. See, e.g., Card et al. (2010).

Comment author: SusanBrennan 18 July 2012 02:09:24PM *  2 points [-]

Bang on! Brown ("Divided we fall") is exactly what I was looking for. Thank you. I regret having only one up-vote to give you.