As many of you may be aware, the UK general election took place yesterday, resulting in a surprising victory for the Conservative Party. The pre-election opinion polls predicted that the Conservatives and Labour would be roughly equal in terms of votes cast, with perhaps a small Conservative advantage leading to a hung parliament; instead the Conservatives got 36.9% of the vote to Labour's 30.4%, and won the election outright.

There has already been a lot of discussion about why the polls were wrong, from methodological problems to incorrect adjustments. But perhaps more interesting is the possibility that the polls were right! For example, Survation did a poll on the evening before the election, which predicted the correct result (Conservatives 37%, Labour 31%). However, that poll was never published because the results seemed "out of line." Survation didn't want to look silly by breaking with the herd, so they just kept quiet about their results. Naturally this makes me wonder about the existence of other unpublished polls with similar readings.

This seems to be a case of two well know problems colliding with devastating effect. Conformity bias caused Survation to ignore the data and go with what they "knew" to be the case (for which they have now paid dearly). And then the file drawer effect meant that the generally available data was skewed, misleading third parties. The scientific thing to do is to publish all data, including "outliers," both so that information can change over time rather than be anchored, and to avoid artificially compressing the variance. Interestingly, the exit poll, which had a methodology agreed beforehand and was previously committed to be published, was basically right.

This is now the third time in living memory that opinion polls have been embarrassingly wrong about the UK general election. Each time this has lead to big changes in the polling industry. I would suggest that one important scientific improvement is for polling companies to announce the methodology of a poll and any adjustments to be made before the poll takes place, and commit to publishing all polls they carry out. Once this became the norm, data from any polling company that didn't follow this practice would be rightly seen as unreliable by comparison.

The head of the IMF was supposed to be Smith College's commencement speaker, but withdrew because of faculty and student protests.  A few professors (although none in the economics department) wrote to the IMF chief asking her to cancel.  The Smith Alumnae Quarterly asked several people, including myself, to write a 400 word article on the surrounding issues of diversity of thought and protest.  Below is a draft of my article.  I hope it's of interest and I would be grateful for any suggestions for improvement:

On Monday I need to give a presentation to a group of 6-15 finance professionals on Confirmation Bias. I intend to use the 2-4-6 task to demonstrate it.

Do people have any advise on how to make this work well? Do people tend to fall for it? Does it help them understand afterwards?

(In ages gone by I would have made this post longer, or in the Open Thread, or not at all. But I gather LW has been seeing a drop-off in volume, so I decided I'd lower the bar I set myself)

I read this article today and thought LW might find it interesting. The key finding is that in a number of different experiments, simple "self-affirmations" (such as writing about relationships with your friends or something else that makes you feel good about yourself) make people more open to changing their mind in cases where changing their mind would be damaging to their self-image. The proposed explanation is that people need to maintain a certain level of self-worth, and one way they do that is by refusing to accept evidence that would damage their sense of self-worth. But if they have a high enough sense of self-worth, they are less likely to do this. I haven't reviewed any of these studies personally, but the idea makes some sense and sounds pretty easy to try. Hat tip to Dan Keys for putting me onto the idea. I searched LW for "Sherman self-affirmation" and didn't see this discussed anywhere on LW, but I didn't look very hard.

Title: Accepting Threatening Information: Self–Affirmation and the Reduction of Defensive Biases

Authors: David K. Sherman and Geoffrey L. Cohen

Citation details: Current Directions in Psychological Science August 2002 vol. 11 no. 4 119-123

Abstract: Why do people resist evidence that challenges the validity of long–held beliefs? And why do they persist in maladaptive behavior even when persuasive information or personal experience recommends change? We argue that such defensive tendencies are driven, in large part, by a fundamental motivation to protect the perceived worth and integrity of the self. Studies of social–political debate, health–risk assessment, and responses to team victory or defeat have shown that people respond to information in a less defensive and more open–minded manner when their self–worth is buttressed by an affirmation of an alternative source of identity. Self–affirmed individuals are more likely to accept information that they would otherwise view as threatening, and subsequently to change their beliefs and even their behavior in a desirable fashion. Defensive biases have an adaptive function for maintaining self–worth, but maladaptive consequences for promoting change and reducing social conflict.

Key quotes: "Pro-choice partisans and pro-life partisans were presented with a debate between two activists on opposite sides of the abortion dispute….However, this confirmation bias was sharply attenuated among participants who affirmed a valued source of self-worth (by writing about a personally important value, such as their relations with friends)....although all participants left the debate feeling more confident in their beliefs about abortion than they had before, this polarization in attitude was significantly reduced among self-affirmed participants (cf. Lord et al., 1979)."  p. 120

"In one study (Cohen et al., 2000), devout opponents and proponents of capital punishment were presented with a persuasive scientific report that contradicted their beliefs about the death penalty’s effectiveness as a deterrent for crime....the responses of participants who received an affirmation of a valued self-identity (by writing about a personally important value, or by being provided with positive feedback on an important skill) proved more favorable.Self affirmed participants were less critical of the reported research, they suspected less bias on the part of the authors, and they even changed their overall attitudes toward capital punishment in the direction of the report they read." p. 121

"In one study, athletes who had just completed an intramural volleyball game assessed the extent to which each of a series of factors contributed to their team’s victory or defeat. As in past research (Lau & Russell, 1980),winners made more internal attributions for their victories than losers did for their defeats. However, among athletes who had reflected on an important value irrelevant to athletics, this self-serving bias was attenuated." p. 122

Here is the link. The context is nutritional science and epidemiology, but confirmation bias is the primary theme pumping throughout the discussion. Gary Taubes has gained a reputation for contrarianism.* According to Taubes, the current nutritional paradigm (fat is bad, exercise is good, carbs are OK) does not deserve high credibility.

Roberts brings up the role of identity in perpetuating confirmation bias--a hypothesis has become part of you, so it has become that much harder to countenance contrary evidence. In this context they also talk about theism (Roberts is Jewish, while Taubes is an atheist). And, the program being EconTalk, Roberts draws analogies with economics.

*Sometime between 45 and 50 minutes in, Roberts points out that given this reputation, Taubes is susceptible to belief distortion as well:

What's your evidence that you are not just falling prey to the Ancel Keys and other folks who have made the same mistake?

I do not think Taubes gives a direct answer.