Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Your intuitions are not magic

58 Post author: Kaj_Sotala 10 June 2010 12:11AM

This article is an attempt to summarize basic material, and thus probably won't have anything new for the hard core posting crowd. If you're new and this article got you curious, we recommend the Sequences.

People who know a little bit of statistics - enough to use statistical techniques, not enough to understand why or how they work - often end up horribly misusing them. Statistical tests are complicated mathematical techniques, and to work, they tend to make numerous assumptions. The problem is that if those assumptions are not valid, most statistical tests do not cleanly fail and produce obviously false results. Neither do they require you to carry out impossible mathematical operations, like dividing by zero. Instead, they simply produce results that do not tell you what you think they tell you. As a formal system, pure math exists only inside our heads. We can try to apply it to the real world, but if we are misapplying it, nothing in the system itself will tell us that we're making a mistake.

Examples of misapplied statistics have been discussed here before. Cyan discussed a "test" that could only produce one outcome. PhilGoetz critiqued a statistical method which implicitly assumed that taking a healthy dose of vitamins had a comparable effect as taking a toxic dose.

Even a very simple statistical technique, like taking the correlation between two variables, might be misleading if you forget about the assumptions it's making. When someone says "correlation", they are most commonly talking about Pearson's correlation coefficient, which seeks to gauge whether there's a linear relationship between two variables. In other words, if X increases, does Y also tend to increase. (Or decrease.) However, like with vitamin dosages and their effects on health, two variables might have a non-linear relationship. Increasing X might increase Y up to a certain point, after which increasing X would decrease Y. Simply calculating Pearson's correlation on two such variables might cause someone to get a low correlation, and therefore conclude that there's no relationship or there's only a weak relationship between the two. (See also Anscombe's quartet.)

The lesson here, then, is that not understanding how your analytical tools work will get you incorrect results when you try to analyze something. A person who doesn't stop to consider the assumptions of the techniques she's using is, in effect, thinking that her techniques are magical. No matter how she might use them, they will always produce the right results. Of course, assuming that makes about as much sense as assuming that your hammer is magical and can be used to repair anything. Even if you had a broken window, you could fix that by hitting it with your magic hammer. But I'm not only talking about statistics here, for the same principle can be applied in a more general manner.

Every moment in our lives, we are trying to make estimates of the way the world works. Of what causal relationships there are, of what ways of describing the world make sense and which ones don't, which plans will work and which ones will fail. In order to make those estimates, we need to draw on a vast amount of information our brains have gathered throughout our lives. Our brains keep track of countless pieces of information that we will not usually even think about. Few people will explicitly keep track of the amount of different restaurants they've seen. Yet in general, if people are asked about the relative number of restaurants in various fast-food chains, their estimates generally bear a close relation to the truth.

But like explicit statistical techniques, the brain makes numerous assumptions when building its models of the world. Newspapers are selective in their reporting of disasters, focusing on rare shocking ones above common mundane ones. Yet our brains assume that we hear about all those disasters because we've personally witnessed them, and that the distribution of disasters in the newspapers therefore reflects the distribution of disasters in the real world. Thus, people asked to estimate the frequency of different causes of death underestimate the frequency of those that are underreported in the media, and overestimate the ones that are overreported.

On this site, we've also discussed a variety of other ways by which the brain's reasoning sometimes goes wrong: the absurdity heuristic, the affect heuristic, the affective death spiral, the availability heuristic, the conjunction fallacy... the list goes on and on.

So what happens when you've read too many newspaper articles and then naively wonder about how frequent different disasters are? You are querying your unconscious processes about a certain kind of statistical relationship, and you get an answer back. But like the person who was naively misapplying her statistical tools, the process which generates the answers is a black box to you. You do not know how or why it works. If you would, you could tell when its results were reliable, when they needed to be explicitly corrected for, and when they were flat-out wrong.

Sometimes we rely on our intuitions even when they are being directly contradicted by math and science. The science seems absurd and unintuitive; our intuitions seem firm and clear. And indeed, sometimes there's a flaw in the science, and we are right to trust our intuitions. But on other occasions, our intuitions are wrong. Yet we frequently persist in holding onto our intuitions. And what is ironic is that we persist on holding onto them exactly because we do not know how they work, because we cannot see their insides and all the things inside them that could go wrong. We only get the feeling of certainty, a knowledge of this being right, and that feeling cannot be broken into parts that could be subjected to criticism to see if they add up.

But like statistical techniques in general, our intuitions are not magic. Hitting a broken window with a hammer will not fix the window, no matter how reliable the hammer. It would certainly be easy and convenient if our intuitions always gave us the right results, just like it would be easy and convenient if our statistical techniques always gave us the right results. Yet carelessness can cost lives. Misapplying a statistical technique when evaluating the safety of a new drug might kill people or cause them to spend money on a useless treatment. Blindly following our intuitions can cause our careers, relationships or lives to crash and burn, because we did not think of the possibility that we might be wrong.

That is why we need to study the cognitive sciences, figure out the way our intuitions work and how we might correct for mistakes. Above all, we need to learn to always question the workings of our minds, for we need to understand that they are not magical.

Comments (28)

Comment author: GreenRoot 10 June 2010 04:46:20PM *  10 points [-]

Thanks for the well-written article. I enjoyed the analogy between statistical tools and intuition. I'm used to questioning the former, but more often than not I still trust my intuition, though now that you point it out, I'm not sure why.

Comment author: xv15 11 June 2010 10:42:15AM 12 points [-]

You shouldn't take this post as a dismissal of intuition, just a reminder that intution is not magically reliable. Generally, intuition is a way of saying, "I sense similarities between this problem and other ones I have worked on. Before I work on this problem, I have some expectation about the answer." And often your expectation will be right, so it's not something to throw away. You just need to have the right degree of confidence in it.

Often one has worked through the argument before and remembers the conclusion but not the actual steps taken. In this case it is valid to use the memory of the result even though your thought process is a sort of black box at the time you apply it. "Intuition" is sometimes used to describe the inferences we draw from these sorts of memories; for example, people will say, "These problems will really build up your intuition for how mathematical structure X behaves." Even if you cannot immediately verbalize the reason you think something, it doesn't mean you are stupid to place confidence in your intuitions. How much confidence depends on how frequently you tend to be right after actually trying to prove your claim in whatever area you are concerned with.

Comment author: Jayson_Virissimo 10 June 2010 09:02:25PM *  7 points [-]

I do know why I trust my intuitions as much as I do. My intuitions are partly the result of natural selection and so I can expect that they can be trusted for the purposes of surviving and reproducing. In domains that closely resemble the environment where this selection process took place I trust my intuition more, in domains that do not resemble that environment I trust my intuition less.

Black box or not, the fact that we are here is good evidence that they (our intuitions) work (on net).

Comment author: diegocaleiro 12 June 2010 08:18:54AM 3 points [-]

How sexy is that?

If you are evaluating intuitions, there are two variables you should account for. The similarity with evolutionary environment, indeed. AND your current posterior belief of the importance of this kind of act in the variance of offspring production.

We definitely evolved in an environment full of ants. Does that mean my understanding of ant-colony intelligence is intuitive?

Comment author: therufs 29 September 2012 05:22:40PM 1 point [-]

I'm very curious how you decide what constitutes a similar environment to that of natural selection, and what sorts of decisions your intuition helps make in such an environment.

Comment author: tommyjohn 18 November 2011 10:27:32PM 0 points [-]

So then anything that has evolved may be relied upon for survival? It is impossible to rationalize faith in an irrational cognitive process. In the book Blink, the author asserts that many instances of intuition are just extremely rapid rational thoughts, possibly at a sub-conscious level.

Comment author: Wei_Dai 11 June 2010 02:45:47PM 4 points [-]

Intuition seems to be one of the least studied areas of cognitive science, at least until very recently. The Wikipedia entry on cognitive sciences that the post links to has no mention of "intuition", and one paper I found said that the 1999 MIT Encyclopedia of Cognitive Sciences doesn't even have a single index entry for it (while "logic" has almost 100 references).

After a bit more searching, I found a 2007 book titled Intuition in Judgment and Decision Making, which apparently represents the current state of the art in understanding the nature of intuition.

Comment author: fool_hill 10 June 2010 06:20:11PM 4 points [-]

i don't know why we prefer to hold on to our intuitions. your claim, that " we persist on holding onto them exactly because we do not know how they work" has not been proven, as far as I can tell, and seems unlikely. I also don't know why our own results seem sharper than what we learn from the outside [although about this later point, i bet there's some story about lack of trust in homo hypocritus societies or something] .

As somebody who fits into the "new to the site" category, I enjoyed your article.

Comment author: RobinZ 10 June 2010 07:15:23PM 3 points [-]

Welcome to Less Wrong! Feel free to post an explicit introduction on that thread, if you're hanging around.

I think the critical point is in the next sentence:

We only get the feeling of certainty, a knowledge of this being right, and that feeling cannot be broken into parts that could be subjected to criticism to see if they add up.

Yes, we don't know what the interiors are - but the original source of our confidence is our (frequently justified) trust in our intuitions. I think another related point is made in How An Algorithm Feels From Inside, which talks about an experience which is illusory, merely reflecting an artifact of the way the brain processes data. The brain usually doesn't bother flagging a result as a result, it just marks it as true and charges forward. And as a consequence we don't observe that we are generalizing from the pattern of news stories we watched, and therefore don't realize our generalization may be wrong.

Comment author: JDM 05 November 2012 01:18:55PM 0 points [-]

I think it's a combination of not understanding the process with a lifetime of experience where's it's far more right than wrong (Even for younger people, if they have 10-15 years of instinctive behavior being rewarded on some level, it's hard to accept there are situations it doesn't work as well). Combine that with the tendency of positive outcomes to be more memorable than others, and it's not too difficult to understand why people trust their intuition as much as they do.

your claim, that " we persist on holding onto them exactly because we do not know how they work" has not been proven, as far as I can tell, and seems unlikely.

It may not be the only reason, but an accurate understanding of how intuitions work would make it easier to rely less on it in situations it's not as we'll equipped for, just as an understanding of different biases makes it easier to fight them in our own thought processes.

Comment author: Douglas_Knight 10 June 2010 10:40:08PM *  3 points [-]

People who know a little bit of statistics - enough to use statistical techniques, not enough to understand why or how they work - often end up horribly misusing them.

How often do people harm themselves with statistics, rather than further their goals through deception? Scientists data-mining get publications; financiers get commissions; reporters get readers.

ETA: the people who are fooled are harming themselves with statistics. But I think the people want to understand for themselves generally only use statistics that they understand.

Comment author: SilasBarta 10 June 2010 10:44:02PM 4 points [-]

True, but many of those scientists and reporters really do want to unravel the actual truth, even if it means less material wealth or social status. These people would enjoy being corrected.

Comment author: Dre 11 June 2010 01:25:21AM 1 point [-]

There is also an opportunity cost to the poor use of statistics instead of proper use. This may be only externalities (the person doing the test may actually benefit more from deception), but overall the world would be better if all statistics were used correctly.

Comment author: akshatrathi 13 August 2010 01:50:57AM 2 points [-]

I enjoyed your article and as a scientist, I've been interested to understand this: what seems an intuitive method to use to solve a scientific problem is not seen as an intuitive method while solving 'other' problems.

By 'other', I mean things like psychological problems or problems that arise from conflicts amongst people. It may be obvious why it is not 'intuitive' but what goes beyond my understanding is most will not even consider using the scientific method for the latter types of problem ever.

Comment author: Louie 11 June 2010 12:59:14AM 2 points [-]

Having just pressed "Send" on an email that estimates statistics based on my intuitions, this feels particularly salient to me.

Really well written. Great work Kaj.

Thanks for reminding me that my thoughts aren't magic.

Comment author: RobinZ 10 June 2010 12:43:20AM 2 points [-]

Elegantly done - clear and informative.

Comment author: MikeDobbs 25 March 2013 01:28:08PM 0 points [-]

This was an excellent read- I particularly enjoyed the comparison drawn between our intuition and other potentially "black box" operations such as statistical analysis. As a mathematics teacher (and recreational mathematician) I am constantly faced with, and amused by, the various ways in which my intuition can fail me when faced with a particular problem.

A wonderful example of the general failure of intuition can be seen in the classic "Monty Hall Problem." In the old TV game show Monty Hall would offer the contestant their choice of one of three doors. One door would have a large amount of cash, the other two a non-prize such as a goat. Here's where it got interesting. After the contestant makes their choice, Monty opens one of the "loosing" doors, leaving only two closed (one of which contains the prize), then offers the contestant he opportunity to switch from their original door to the other remaining door.

The question is, should they switch? Does it even matter? For most people (myself included) our intuition tells us it doesn't matter. There are two doors, so there's a 50/50 chance of winning whether you switch or not. However a quick analysis of the probabilities involved shows us that they are in fact TWICE as likely to win the prize if they switch than if they stay with their original choice.

That's a big difference- and a very counterintuitive result when first encountered (at least in my opinion)

Comment author: TheOtherDave 25 March 2013 03:39:49PM 0 points [-]

I was first introduced to this problem by a friend who had received as a classroom assignment "Find someone unfamiliar with the Monty Hall problem and convince them of the right answer."

The friend in question was absolutely the sort of person who would think it was fun to convince me of a false result by means of plausible-sounding flawed arguments, so I was a very hard sell... I ended up digging my heels in on a weird position roughly akin to "well, OK, maybe the probability of winning isn't the same if I switch, but that's just because we're doing something weird with how we calculate probabilities... in the real world I wouldn't actually win more often by switching, cuz that's absurd."

Ultimately, we pulled out a deck of cards and ran simulated trials for a while, but we got interrupted before N got large enough to convince me.

So, yeah: counterintuitive.

Comment author: PhilipL 25 March 2013 04:40:32PM 0 points [-]

I remember how my roommates and I drew a game tree for the Monty Hall problem, assigned probabilities to outcomes, and lo, it was convincing.

Comment author: TheOtherDave 25 March 2013 04:59:38PM 0 points [-]


It continues to embarrass me that ultimately I was only "convinced" that the calculated answer really was right, and not some kind of plausible-sounding sleight-of-hand, when I confirmed that it was commonly believed by the right people.

Comment author: MikeDobbs 25 March 2013 05:29:23PM *  0 points [-]

One of my favorites for exactly that reason- if you don't mind, let me take a stab at convincing you absent "the right people agreeing."

The trick is that once Monty removes one door from the contest you are left with a binary decision. Now to understand why the probability differs from our "gut" feeling of 50/50 you must notice that switching amounts to winning IF your original choice was wrong, and loosing IF your original choice was correct (of course staying with your original choice results in winning if you were right and loosing if you were wrong).

So, consider the probability that you original guess was correct. Clearly this is 1/3. That means the probability of your original choice being incorrect is 2/3. And there's the rub. If you will initially guess the wrong door 2/3 of the time, then that means that when you are faced with the option to switch doors you're original choice will be wrong 2/3 of the time, and switching would result in you switching to the correct door. Only 1/3 of the time will your original choice be correct, making switching a loosing strategy.

It becomes more clear if you begin with 10 doors. In this modified Monty Hall problem, you pick a door, then Monty opens 8 doors, leaving only your original choice and on other (one of which contains the prize money). In this case your original choice will be incorrect 9/10 times, which means when faced with the option to switch, switching will result in a win 9/10 times, as opposed to staying with your original choice, which will result in a win only 1/9 times.

Comment author: TheOtherDave 25 March 2013 06:07:25PM *  0 points [-]

(nods) Yah, I'm familiar with the argument. And like a lot of plausible-sounding-but-false arguments, it sounds reasonable enough each step of the way until the absurd conclusion, which I then want to reject. :-)

Not that I actually doubt the conclusion, you understand.

Of course, I've no doubt that with sufficient repeated exposure this particular problem will start to seem intuitive. I'm not sure how valuable that is.

Mostly, I think that the right response to this sort of counterintuitivity is to get seriously clear in my head the relationship between justified confidence and observed frequency. Which I've never taken the time to do.

Comment author: JamesCole 11 June 2010 07:07:17AM 0 points [-]

Yet our brains assume that we hear about all those disasters [we read about in the newspaper] because we've personally witnessed them, and that the distribution of disasters in the newspapers therefore reflects the distribution of disasters in the real world.

Even if we had personally witnessed them, that wouldn't, in itself, be any reason to assume that they are representative of things in general. The representativeness of any data is always something that can be critically assessed.

Comment author: Kutta 11 June 2010 09:25:53AM *  0 points [-]

For many people, representativeness is the primary governing factor in any data analysis, not just a mere facet of reasoning that should be critically assessed. Also, aside from the mentioned media bias that is indeed relatively easily correctable, there are many subtler instances of biasing via representativess, on the level of cognitive processes.

Comment author: nazgulnarsil 11 June 2010 02:23:38AM 0 points [-]

"However, like with vitamin dosages and their effects on health, two variables might have a non-linear relationship."

if we limit our interval we can make a linear approximation within that interval. this is often good enough if we don't much care about data outside that interval. the easy pitfall of course is people wanting to extend the linearization beyond the bounds of the interval.

Comment author: Nanani 16 June 2010 03:10:25AM -1 points [-]

Voted down because tangential replies that belong elsewhere really get on my nerves. Please comment on the post about the vitamin study, linked in the OP.

Comment author: nazgulnarsil 17 June 2010 04:04:15PM 0 points [-]

0_o I was responding directly to the OP.