khafra comments on The New Yorker article on cognitive biases - Less Wrong

2 Post author: BlazeOrangeDeer 13 June 2012 06:14AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (26)

Comments (26)

You are viewing a single comment's thread. Show more comments above.

Comment author: VincentYu 13 June 2012 07:23:01AM *  29 points [-]

Full paper (West, Meserve, & Stanovich, 2012).

The New Yorker article ("Why smart people are stupid") by Lehrer misrepresents the main point of West et al.'s paper, which is that – as the original paper title asserts – "cognitive sophistication does not attenuate the bias blind spot", instead of the article's claim that smart people are more susceptible to cognitive biases.

Three negligent misrepresentations by Lehrer:

First, contrary to the general message of the article, West et al. state twice that most cognitive biases are negatively correlated with cognitive ability (emphasis mine):

The finding that the bias blind spot is more apparent among the more cognitively sophisticated individuals is contrary to much of the rest of the heuristics and biases literature where most biases are negatively correlated (or at least independent) with cognitive abilities (Bruine de Bruin et al., 2007; Chiesi et al., 2011; Del Missier et al., 2010, 2011; Finucane & Gullion, 2010; Kokis, Macpherson, Toplak, West, & Stanovich, 2002; Stanovich & West, 1998, 2000; Toplak et al., 2011; Weller, Levin, & Denburg, 2011; West et al., 2008).

Most cognitive biases in the heuristics and biases literature are negatively correlated with cognitive sophistication, whether the latter is indexed by development, by cognitive ability, or by thinking dispositions (Bruine de Bruin et al., 2007; Chiesi et al., 2011; Finucane & Gullion, 2010; Kokis et al., 2002; Toplak & Stanovich, 2002; Toplak et al., 2011; Weller et al., 2011; West et al., 2008).

Second, West et al. refer to the bias blind spot as a metabias because it is a pattern of inaccurate judgment in reasoning about cognitive biases. In this context, the distinction between bias and metabias (which the article neglects to maintain) is vital because the whole point of West et al.'s paper is that this particular metabias does not exhibit the same negative correlation with cognitive ability that many cognitive biases exhibit. They do not contest or assert to have a counterexample to the claim that most cognitive biases are negatively correlated with cognitive ability.

There is an important metabias that remains unexamined in all of the previous work on individual differences, however. It is the so-called bias blind spot—explored in an important article by Pronin, Lin, and Ross (2002). They found that people thought that various cognitive and motivational biases were much more prevalent in others than in themselves. Bias turns out to be relatively easy to recognize in the decisions of others, but often difficult to detect in one’s own judgments.

Third, West et al. are rather cautious (in contrast with the article) in stating the correlation strengths of bias blind spots with cognitive ability. For instance, they have as their paper title "Cognitive sophistication does not attenuate the bias blind spot", rather than a stronger "Cognitive sophistication augments the bias blind spot". However, they do find a slight positive correlation (two-tailed p < 0.05):

We found that none of these bias blind spot effects displayed a negative correlation with measures of cognitive ability (SAT total, CRT) or with measures of thinking dispositions (need for cognition, actively open-minded thinking). If anything, the correlations went in the other direction.

Whatever explains the slightly positive correlation, a conservative way to characterize the findings here is to say that cognitive ability provides no inoculation at all from the bias blind spot—the tendency to believe that biased thinking is more prevalent in others than in ourselves. In our data, cognitive ability did not attenuate the tendency toward a blind spot at all. Thus, the bias blind spot joins a small group of other effects such as myside bias and noncausal base-rate neglect (Stanovich & West, 2008b; Toplak & Stanovich, 2003) in being unmitigated by increases in intelligence.

Social news posts about the New Yorker article:

ETA: OB post

Comment author: khafra 13 June 2012 03:39:45PM *  2 points [-]

I'm missing something basic, here: If cognitive sophistication does attenuate other biases, how can they tell that the cognitively sophisticated believe themselves less biased than the average person to the point of being biased in that belief, rather than merely to the point of being correct?

Comment author: Jack 14 June 2012 12:47:10AM 2 points [-]

VincentYu has it right. They 1) used biases that had previously been shown to be uncorrelated with intelligence and 2) checked and confirmed that the biases actually were uncorrelated with intelligence in their sample.

Comment author: gwern 13 June 2012 03:52:47PM 2 points [-]

Having read the paper, I don't think they can, and they admit this to some extent. What they can prove is that in the small selected set of biases they chose, the smart are incorrect; but to show that the smart are miscalibrated in general about their resistance to biases, one would need to do a lot more work. (There are scores of biases, after all.)

Comment author: VincentYu 13 June 2012 03:50:26PM *  2 points [-]

They studied the bias blind spots of cognitive biases that show no correlation with cognitive ability.

The finding that the bias blind spot is more apparent among the more cognitively sophisticated individuals is contrary to much of the rest of the heuristics and biases literature where most biases are negatively correlated (or at least independent) with cognitive abilities [...] Given the rarity of the result, it is important to consider alternative explanations for it, and an obvious one suggests itself. Perhaps more cognitively sophisticated individuals actually are less likely to display these classic cognitive biases. Such a finding would in fact cause a positive correlation if more cognitively able individuals correctly perceived that they were less prone to bias. This alternative explanation has diminished likelihood, however, when we reflect on the fact that these cognitive biases have been found in another study to be independent of cognitive ability (see Stanovich & West, 2008b, Table 1).

We explored the obvious explanation for the indications of a positive correlation between cognitive ability and the magnitude of the bias blind spot in our data. That explanation is the not unreasonable one that more cognitively sophisticated people might indeed show lower cognitive biases—so that it would be correct for them to view themselves as less biased than their peers. However, as the analyses in Tables 3 and 5 indicate, we found very little evidence that these classic biases were attenuated by cognitive ability. More intelligent people were not actually less biased—a finding that would have justified their displaying a larger bias blind spot.

ETA: On further thought, the lack of correlation merely simplifies the study, and is not a necessary assumption – for each subject, they administered tests on the selected cognitive biases, obtained several measures of cognitive ability/sophistication (CRT, SAT, etc.), and asked for subjective assessments on the extent of the biases:

To what extent do you believe that you are likely to commit base-rate neglect?

To what extent do you believe that the average JMU student is likely to commit base-rate neglect?

They used the subjective assessments along with the results of the administered tests to derive the extent of bias blind spots, independent of the measures of cognitive ability. These are then combined in a standard correlation study, and your hypothesis can be tested.

Comment author: fubarobfusco 13 June 2012 04:57:44PM 0 points [-]

If that were true, and if the relation between measures of "cognitive sophistication" and of bias were continuous, then by the intermediate value theorem there would be some level of "cognitive sophistication" which minimized overall bias. Of course, that might be an unstable point; anyone smarter than that tends to fall into a bias-blind-spot attractor and anyone less smart than that tends to fall into an object-level bias attractor.

Powered by Reddit Powered by Reddit