Today, I finally took a racial/sexual Implicit Association Test.
I had always more or less accepted that it was, if not perfect, at least a fairly meaningful indicator of some sort of bias in the testing population. Now, I'm rather less confident in that conclusion.
According to the test, in terms of positive associations, I rank black women above black men above white women above white men. I do not think this is accurate.
Obviously, this is an atypical result, but I believe that I received it due to confounding factors which prevented the test from being an accurate reflection of my associations are likely to affect a large proportion of the testing population.
First, the most significant factor in how successful I was in correctly associating words and faces was simply practice. I made more mistakes in the first phase than the second phase, and more in the second than the third, etc. I believe that my test could have showed significantly different results simply by re-ordering the phases.
Second, I suspect that I was trying harder in the phases where I was matching black faces than white faces. I don't want to corrupt the test, but I also don't want it to tell me I'm a racist; would I have been so enthusiastic about making the final phase my most accurate one of all, if it had been matching white male faces rather than black male faces?
Third, I felt that many of the questions on the survey that followed the matching phase were too loaded to properly answer on their own terms. They presented a series of options from "strongly agree" to "strongly disagree," where I felt that my real answer would most accurately be framed as ADBOC.
If anyone here has access to university resources and would like to collaborate on an experiment which would attempt to discern subjects' associations while correcting for these faults, please let me know.
I haven't heard of any attempts at comparing implicit association tests to behavior.
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.