Related to: Consider Conspiracies, What causes people to believe in conspiracy theories?

Here I consider in some detail a failure mode that classical rationality often recognizes. Unfortunately nearly all heuristics normally used to detect it seem remarkably vulnerable to misfiring or being exploited by others. I advocate an approach where we try our best to account for the key bias, seeing agency where there is none, while trying to minimize the risk of being tricked into dismissing claims because of boo lights.  

What does calling something a "conspiracy theory" tell us?

What is a conspiracy theory? Explanations that invoke plots orchestrated by covert groups are easily called or thought of as such. In a more legal sense conspiracy is an agreement between persons to mislead or defraud others. This simple story gets complicated because people aren't very clear on what they consider a conspiracy.

To give an example, is explicit negotiation or agreement really necessary to call something a conspiracy? Does silent cooperation on Prisoner's Dilemma count? What if the players are deceiving themselves that they are really following a different goal and the resulting cooperation is just a side effect? How could we tell the difference and would it matter? The latter is especially interesting if one applies the anthropic principle to social attitudes and norms.

The phrase is also a convenient tool to mark an opponent's tale as low status and unworthy of further investigation. A boo light easily applied to anything that has people acting in something that can be framed as self-interest and happens to be few inferential jumps away from the audience. Not only is its use in this way well known, this is arguably the primary meaning of calling an argument a conspiracy theory.

We have plenty of historical examples of high-stakes conspiracies so we know they can be the right answer. Noting this and putting aside the misuse of the label, people do engage in crafting conspiracy theories when they just aren't needed. Entire communities can fixate on them or fail to call such bad thinking out. Why does this happen? Humans being the social animals that we are, the group dynamics at work probably need an article or sequence of their own. It should suffice for now to point to belief as attire, the bandwagon effect and Robin Hanson's take on status. Let's rather consider the question of why individuals may be biased towards such explanations. Why do they privilege the hypothesis?

When do they seem more likely than they are?

First off we have a hard time understanding that coordination is hard. Seeing a large pay off available and thinking it easily in reach if "we could just get along" seems like a classical failing. Our pro-social sentiments lead us to downplay such barriers in our future plans. Motivated cognition on behalf of assessing the threat potential of perceived enemies or strangers likely shares this problem. Even if we avoid this, we may still be lost since the second big relevant thing is our tendency for anthropomorphizing things that better not be. Ours is a paranoid brain seeing agency in every shadow or strange sound. The cost of false positives was once reasonably low, while the cost of a false negative very high.

Our minds are also just plain lazy. We are pretty good at modelling other human minds and considering just how hard the task really is, we do a pretty remarkable job of it. If you are stuck in relative ignorance on a subject, say the weather, dancing to appease the sky spirits makes sense. After all the weather is pretty capricious and angry sky spirits is a model that makes as much or more sense as any other model you know. Unlike some other models this one is at least cheap to run on your brain! The modern world is remarkably complex. Do we see ghosts in it?

Our Dunbarian minds probably just plain can't get how a society can be that complex and unpredictable without it being "planned" by a cabal of Satan or Heterosexual White Males or the Illuminati (but I repeat myself twice) scheming to make weird things happen in our oblivious small stone age tribe. Learning about useful models helps people escape anthropomorphizing human society or the economy or government. The latter is particularly salient. I think most people slip up occasionally in assuming that say something like the United States government can be successfully modelled as a single agent to explain most of its "actions". To make matters worse it is a common literary device used by pundits.

A mysterious malignant agency or someone keeping a secret playing the role of the villain makes a good story. Humans love stories. Its fun to think in stories. Any real conspiracy revealed will probably be widely publicized. Peter Knight in his 2003 book cites historians who have put forward the idea, that the United States is something of a home for popular conspiracy theories because so many high-level ones have been undertaken and uncovered since the 1960s. We are more likely to hear about real confirmed conspiracies today than ever before.

Wishful thinking also plays a role. A universe where bad things happen because bad people make them to is appealing. Getting rid of bad people, even very bad people, is easy compared to all the different things one has to do to make sure bad things don't happen in a universe that doesn't care about us and where really bad things are allowed to happen. Finding bad people whether there are or aren't is a problematic tendency. The sad thing is that this may also be how we often manage to coordinate. Do all theories of legitimacy also perhaps rest on the same cognitive failings that conspiracy theories do? The difference between a shadowy cabal we need to get rid of and an institution worthy of respect may be just some bad luck.

How this misleads us

Putting aside such wild speculation, what should we take away from this? When do conspiracy theories seem more likely than they are?

• The phenomena is unpredictable or can't be modelled very well
• Models used by others are hard to understand or are very counter-intuitive
• Thinking about the subject significantly strains cognitive resources
• The theory explains why bad things happen or why something went wrong
• The theory requires coordination

When you see these features you probably find the theory more plausible than it is. 

But how many here are likely to accept "conspiracy theories"? To do so with stuff that actually gets called a conspiracy theory doesn't fit our tribal attire. Reverse stupidity may be particularly problematic for us on this topic. Being open to thinking conspiracy is recommended. Just remember to compare how probable it is in relation to other explanations. It is important to call out people who misuse the tag for rhetorical gain.

This applies to debunking as well. Don't go wildly contrarian. But remember that even things that are tagged conspiracy theories are surprisingly popular. How popular might false theories that avoid that tag be? History shows us we don't have the luxury of hoping that kind of thing just doesn't happen in human societies. When assessing an explanation sharing the key features that make conspiracy theories seem more plausible than they are, compensate as you would with a conspiracy theory. 

But don't listen to me, I'm talking conspiracy theories. 

Note: This article started out as a public draft, feedback to other such drafts is always welcomed.  Special thanks to user Villiam_Bur for his commentary and user copt for proofreading and suggestions. Also thanks to the LessWrong IRC chatroom for last minute corrections and stylistic tips.