This is an interesting post, but...
Hating Hitler doesn't mean you're biased against Hitler. Likewise, having a belief about a particular ethnic group doesn't mean you're biased for or against them.
Then how do you know what score you should get on the IAT? I don't know what an unbiased score would be, but an equal-for-both-groups score is most likely biased.
In the Israel vs. Palestine case, your answer would depend more on some meta-level decisions than on ironing out another decimal point of bias. For instance: Should a settlement give equal benefits to both sides; should it compensate for historic injustices; should it maximize expected value for the participants; should it maximize expected value for the world?
If you want to maximize expected value for the world, you would end up calculating something like this:
The answer you would then arrive at will be more different from the answer you would arrive at if "equitable outcome" is your goal, than the difference made by any bias. (Unless you really, really hate lawyers.)
So I don't think this approach gets at any of the hard problems.
Aargh. That's a good point and I clearly need to think about this more. I don't have a clear theory yet, but I'm going to brain-dump my thoughts on this topic.
Then how do you know what score you should get on the IAT? I don't know what an unbiased score would be, but an equal-for-both-groups score is most likely biased.
The score you should get on the IAT should be correlated to your conscious opinion. If you consciously think Palestinians are inferior, then you should be happy with an IAT score showing you think Palestinians are inferior. If you consciously think Palestinians are equal to Israelis, you should be trying to get an IAT score reflecting that equality. It's all about trying to get the unconscious mind to correspond to your rational beliefs.
I consciously assent to the proposition "Palestinians are more likely to be suicide bombers than some other groups, but it's still only a tiny fraction of their population" My unconscious probably believes something closer to "Palestinians = suicide bombers!" Further, my conscious mind stops way short of the proposition "All Palestinians are bad people." My unconscious mind probably believes this second proposition. I'd like my unconscious mind to get way closer to my conscious beliefs in both areas.
My conscious, rational brain believes that most Palestinians are probably decent people who have been driven to extremes by their situation. My conscious mind also believes that the best Middle East peace plan is one where everyone, Israeli or Palestinian, is considered equally deserving of happiness simply because they are human. That's my moral system, and yours may differ. The point is, that is my moral system, and I would like to be able to operate on it. If I'm going around subconsciously thinking that Palestinians are bad and don't deserve happiness, I can't enact my goals.
This ties into the halo effect and the horns effect, where people tend to classify others as either all good or all bad. My belief that Palestinians are sometimes suicide bombers probably makes me think that they're uglier, stupider, and meaner than they really are. A lot of that is mediated through that concept "bad", which is one reason I'm so interested in getting my link between "Palestinians" and "bad" down.
If you wanted to adjust moral value for the Israelis' greater economic value, you'd still need a way to make sure you're not over-adjusting, ie that your unconscious mind doesn't dislike the Palestinians even more than your conscious mind does. Alternately, you'd want to make sure you weren't going soft and that your unconscious mind liked the Palestinians more than your conscious mind thought they deserved. I can't think of an easy way to do that with the IAT, but I bet there's a complicated one if you imagine a hypothetical IAT with perfect reliability and let me get away with a mere proof-of-concept.
Hope that makes sense.
It is claimed that Golda Meir once requested Nixon not to appoint jews as his delegates in Israel. There was a strong feeling in Israel that to avoid accusations of bias, delegates who happen to be jewish tend to lean the other way instead.
In general, how can we be sure that by “correcting” subconscious associations, we really shift our subconscious towards rationality rather than suppress what those around us consider a “thoughtcrime”?
There's a story - it might be by Jorge Luis Borges, but the setting makes me think it isn't - in which two men are competing for an appointment at Oxford, and it's well-known that one moderate on the committee is going to have the deciding vote.
One of the candidates is a friend of this moderate. The second candidate goes out of his way to not befriend the moderate with the deciding vote, and even insults him publicly on numerous occasions. The moderate votes for this second candidate.
After the decision is final, the second candidate reveals that he deliberately got on the wrong side of the man who cast the deciding vote, because he knew him to be so scrupulously fair (and to like having a reputation for being fair), that he would overcompensate for his friendship with the one candidate and his animosity towards the other - and thus the second candidate cleverly gained an unfair advantage.
No, that's Borges alright. (He's one of my favorite authors, so I've read all his fictions many times. I don't have my books handy to say exactly what story it is, but I'm fairly sure it's one of the later ones.)
EDIT: I leafed through my Collected Fictions until I found it. It's "The Bribe", in The Book of Sand (Borges's 2nd-to-last collection). Turns out the appointment is not to Oxford, which explains why my googling failed, and is actually who is to chair a literature conference in Wisconsin.
Anchoring may still be a problem here. Your Isaac may decide to try and debiase himself through the methods you suggest, but when will he stop these efforts? When he's reached a point where he feels reasonably debiased - which will be an improvement, but still on the Israeli side of 'reasonable'.
If Sayed Muhammed had done the same thing from the opposite side, he would have ended up on the Palestianian side of 'reasonable'. And the span of reasonable may be quite large.
Actual debiasing will require efforts "beyond the call of duty", to avoid the anchoring effect. Another approach, if both Issac and Sayed Muhammed are honest in their quest, is to lock them together in a room and only let them out when they agree (not when they "have negotiated a reasonable compromise", but actually agree on the issues).
The problem with trying to type of testing in the real-world is that for it to work you need people who really are trying - 100% good faith - to fight their biases.
Otherwise, you could have very promising tests results showing that people are successfully fighting or routing around their biases, while in fact they are performing that trick just during the testing.
In fact, if such a test was used to determine who would be most deserving of having decision-making powers, the most biases people, favoring one side much more than the other, would have the biggest incentive to try to appear as unbiased as possible while the good faith people might be more honest, disqualifying them.
Or am I missing a way to correct for that?
Hating Hitler doesn't mean you're biased against Hitler.
Doesn't it? Does it make it less likely that you'll give Hitler a job? Even one that he might be very good at?
Okay, Hitler's not a very good example - he's dead, and that makes him pretty bad at most jobs. Am I biased against dead people?
I think so.
You are biased against a person only if you devalue her too much, compared to the extent that you should. You are biased against Hitler if you e.g. are expected to systematically don't give him a job that you should've given. You determine what you should've done by considering the real consequences of your decision and carefully valuing them, as opposed to taking the whole problem "at a glance".
Continuation of: The Implicit Association Test
Response to: 3 Levels of Rationality Verification
I've not yet seen it pointed out before that we use "bias" to mean two different things.
Sometimes we use "bias" to mean a hard-coded cognitive process that results in faulty beliefs. Take as examples the in-group bias, the recall bias, the bad guy bias, and various other things discovered by Tversky and Kahneman.
Other times, we use "bias" to mean a specific faulty belief generated by such a process, especially one that itself results in other faulty beliefs. For example, Jews are sometimes accused of having a pro-Israel bias. By this we mean that they have a higher opinion of Israel than the evidence justifies; this is a specific belief created by the in-group bias. This belief may itself generate other faulty beliefs; for example, they may have a more negative opinion of Palestinians than the evidence justifies. It is both the effect of a bias, and the cause of other biases.
Let's be clear about this "more than the evidence justifies" bit. Hating Hitler doesn't mean you're biased against Hitler. Likewise, having a belief about a particular ethnic group doesn't mean you're biased for or against them. My Asian friends hate it when people sheepishly admit in a guilty whisper that they've heard Asians are good at academics. Asians are good at academics. Just say "55% chance an average Asian has a GPA above the American population mean" and leave it at that. This is one of Tetlock's critiques of the Implicit Association Test, and it's a good one. I'd probably link Asians to high achievement on an IAT, but it wouldn't be a bias or anything to get upset about.
And let's also be clear about this faulty belief thing. You don't have to believe something for it to be a belief; consider again the skeptic who flees the haunted house. She claims she doesn't belief in ghosts, and she's telling the truth one hundred percent. She's still going to be influenced by her belief in ghosts. She's not secretly supernaturalist any more than someone who gets "strongly biased" on the IAT is secretly racist. But she needs to know she's still going to run screaming from haunted houses, and IAT-takers should be aware they're still probably going to discriminate against black people in some tiny imperceptible way.
Okay, back to the example. So the President appoints Isaac, a synagogue-going Jew, as the new Middle East peace envoy. Due to some amazing breakthrough in the region, both the Israelis and Palestinians agree to accept whatever plan Isaac develops. Isaac's only job is to decide what long-term plan is best for both sides. And he's a good man: he has an honest desire to choose the maximum-utility solution.
Isaac legitimately worries that he has a bias for the Israelis and against the Palestinians. How can he test the hypothesis? He can take a hypothetical souped-up version of the Implicit Association Test1. He finds that yes, he has a strong pro-Israel anti-Palestine bias. Now what does he do?
He can try to route around the bias. This is the approach implicitly endorsed by Overcoming Bias and by rationalism in general. He can take the Outside View and look at successful approaches in other world conflicts. He can use some objective metric to calculate the utility of everything in Israel, and check to make sure neither group is getting an amount disproportionate to their numbers. He can open a prediction market on metrics of success, and implement whatever policies trades at the highest value. All of these will probably improve Isaac's solution a fair bit. But none of them are perfect. In the end, Isaac's the one who has to make a decision that will be underdetermined by all these clever methods, and Isaac is still biased against the Palestinians.
Or he can try to fight the bias.
Diversity workshops try to fight biases directly . These don't work, and that's no surprise. Diversity workshops are telling you, on a conscious level, that minorities really are great people, aren't they? Well, yes. On a conscious level, you already believe that. Isaac already knows, on a conscious level, that the Palestinians deserve a fair solution that protects their interests just as much as the Israelis do. A diversity workshop would be a flashy video in which a condescending narrator explains that point again and again.
We don't have a lot of literature on what does work here, but I predict a few things would help. Make some Palestinian friends, to build mental connections between Palestinians and positive feelings. Learn to distinguish between Palestinian faces. Read works of fiction with sympathetic Palestinian characters. I would say "live in Palestine" but by all accounts Palestine is a pretty grim place; he might do better to live in a Palestinian community in America for a while.
Those techniques aren't especially good, but I don't care. We know how to improve them. By making a group take the Implicit Association Test, applying a technique to them, giving them the test again, and seeing how their score changed, we gain the ability to test bias-fighting techniques. I wouldn't want to do this on one person, because the test only has moderate reliability at the individual level. But a group of a few dozen, all practicing the same technique, would be quite sufficient. If another group learns a different technique, we can compare their IAT score improvement and see which technique is better, or if different techniques are better in different circumstances.
Again, there's no reason why this method should be limited to racial biases. No matter how hard I try to evaluate policies on their merits rather than their politics, I am biased towards the US Democratic Party and I know it. This ought to be visible on an IAT, and there ought to be techniques to cure it. I don't know what they are, but I'd like to find them and start testing them.
What about the second method of overcoming bias, routing around it? The IAT is less directly valuable here, but it's not without a role.
In one of the IAT experiments, subjects evaluated essays written by black or white students. This is a fiendishly difficult task upon which to avoid bias. A sneaky researcher can deliberately select essays graded as superior by a blind observer and designate them "white essays", so anyone trying to take the easy way out by giving all essays the same grade can be caught immediately. I like this essay task. It's utterly open to any technique you want to use to reduce bias.
So give someone IATs until you find a group they're especially biased against - black people, Palestinians, Korean-Americans, frequentists; any will do. Then make them grade essays by the control group and the disliked group. Collect statistics correlating IAT bias with essay grading bias. If a person using a special technique to route around mental bias can grade essays more accurately than other people with the same level of IAT bias, that person has routed around their bias successfully.
So: How do we tell if a technique for routing around bias works? Test whether people are better able to conduct a rating task than their IAT scores would predict. How do we test a technique for fighting bias directly? See if it lowers IAT scores. All terribly inconvenient because of the IAT's low effect size and reliability, but with a large enough sample size or enough test-retest cycles the thing could be done. And the psychologists who transformed the Bona Fide Pipeline into the IAT may yet transform the IAT into something even more powerful.
This, then, is one solution to schools proliferating without evidence. With enough research, it could be turned into one of the missing techniques of rationality verification.
Footnotes
1: Remember, the IAT is only moderately good at evaluating individuals, and has a bad habit of changing its mind each time someone takes it. Much of what is in this essay would work poorly (though probably still better than nothing) with a simple IAT. But having someone take the IAT ten times over ten days and averaging the results might give a more accurate picture (I don't know of any studies on this). And in any case the IAT is quite good at comparing groups of people with sample size >1. And I expect that souped-up versions of the IAT will be out within a few years; these tests have gotten better and better as time goes on.