Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

A Taxonomy of Bias: Mindware Problems

14 Post author: Kaj_Sotala 07 July 2010 09:53PM

This is the third part in a mini-sequence presenting content from Keith E. Stanovich's excellent book What Intelligence Tests Miss: The psychology of rational thought. It will culminate in a review of the book itself.

Noting that there are many different kinds of bias, Keith Stanovich proposes a classification scheme for bias that has two primary categories: the Cognitive Miser, and Mindware Gaps. Last time, I discussed the Cognitive Miser category. Today, I will discuss Mindware Problems, which has the subcategories of Mindware Gaps and Corrupted Mindware.

Mindware Problems

Stanovich defines "mindware" as "a generic label for the rules, knowledge, procedures, and strategies that a person can retrieve from memory in order to aid decision making and problem solving".

Mindware Gaps

Previously, I mentioned two tragic cases. In one, a pediatrician incorrectly testified the odds of a two children in the same family suffering infant death syndrome to be 73 million to 1. In the other, people bought into a story of "facilitated communication" helping previously non-verbal children to communicate, without looking at it in a critical manner. Stanovich uses these two as examples of a mindware gap. The people involved were lacking critical mindware: in one case, that of probabilistic reasoning, in the other, that of scientific thinking. One of the reasons why so many intelligent people can act in an irrational manner is that they're simply missing the mindware necessary for rational decision-making.

Much of the useful mindware is a matter of knowledge: knowledge of Bayes' theorem, taking into account alternative hypotheses and falsifiability, awareness of the conjunction fallacy, and so on. Stanovich also mentions something he calls strategic mindware, which refers to the disposition towards engaging the reflective mind in problem solving. These were previously mentioned as thinking dispositions, and some of them can be measured by performance-based tasks. For instance, in the Matching Familiar Figures Test (MFFT), participants are presented with a picture of an object, and told to find the correct match from an array of six other similar pictures. Reflective people have long response times and few errors, while impulsive people have short response times and numerous errors. These types of mindware are closer to strategies, tendencies, procedures, and dispositions than to knowledge structures.

Stanovich identifies mindware gaps to be involved in at least conjunction errors and ignoring base rates (missing probability knowledge), as well as the Wason selection task and confirmation bias (not considering alternate hypotheses). Incorrect lay psychological theories are identified as a combination of a mindware gap and contaminated mindware (see below). For instance, people are often blind to their own biases, because they incorrectly think that biased thinking on their part would be detectable by conscious introspection. In addition to bias blind spot, lay psychological theory is likely to be involved in errors of affective forecasting (the forecasting of one's future emotional state).

Contaminated Mindware

I previously also mentioned the case of Albania, where a full one half of the adult population fell victim to Ponzi schemes. As another example, in the 1980s psychotherapists thought they had found a way to uncover repressed memories of childhood sexual abuse. The ideas spread via professional association but without any evidence of them actually being correct. Even when the patients had no memories of abuse prior to entering therapy, and during therapy began to come up with elaborate memories of being abused in rituals with satanic overtones, nobody questioned their theories or sought any kind of independent evidence. The belief system of the therapists was basically "if the patient thinks she was abused then she was", and the mindware represented by this belief system required only the patient and the therapist to believe in the story. Several people were convicted of abuse charges because of this mindware.

Even though mindware gaps were definitely involved in both of these cases, they are also examples of contaminated mindware sweeping through a population. Stanovich defines contaminated mindware as mindware that leads to maladaptive actions and resists critical evaluation. Contaminated mindware is often somewhat complicated and may be just as enticing to high-IQ people than low-IQ people, if not more so. In a survey of paranormal beliefs conducted on members of a Mensa club in Canada, 44% believed in astrology, 51% believed in biorhytms, and 56% believed in extraterrestrial visitors.

To explain why we acquire mindware that is harmful to us, Stanovich draws upon memetic theory. In the same way that organisms are built to advance the interests of the genes rather than for the interest of the gene hosts themselves, beliefs may spread without being true or helping the human who holds the belief in any way. Chain letters such as "send me to five other people or experience misfortune" are passed on despite being both untrue and useless to the people passing them on, surviving because of their own "self-replicative" properties. Memetic theory shifts our perspective from "how do people acquire beliefs" to "how do beliefs acquire people". (Here, we're treating "memes" and "mindware" as rough synonyms, with the main difference being one of emphasis.)

Stanovich lists four reasons for why mindware might spread:

  1. Mindware survives and spreads because it is helpful to the people that store it.
  2. Certain mindware proliferates because it is a good fit to pre-existing genetic dispositions or domain-specific evolutionary modules.
  3. Certain mindware spreads because it facilitates the replication of genes that make vehicles that are good hosts for that particular mindware (e.g. religious beliefs that urge people to have more children).
  4. Mindware survives and spreads because of the self-perpetuating properties of the mindware itself.

Stanovich notes that reasons 1-3 are relatively uncontroversial, with 1 being a standard assumption in cultural anthropology, 2 being emphasized by evolutionary psychology, and 3 being meant to capture the types of effects emphasized by theorists stressing gene/culture co-evolution.

There are several subversive ways by which contaminated mindware might spread. It might mimick the properties of beneficial mindware and disguise itself as one, it might cause its bearers to want to kill anyone who speaks up against it, it might be unfalsifiable and prevent us from replacing it, it might contain beliefs that actively prohibit evaluation ("blind faith is a virtue") or it might impose a prohibitively high cost on testing it (some groups practicing female genital mutilation believe that if a baby's head touches the clitoris during delivery, the baby will die).

Stanovich identifies contaminated mindware to be involved in at least self-centered biases (self and egocentric processing) and confirmation bias (evaluation-disabling strategies), as well combining with mindware gaps to cause problems in the form of lay psychological theory.

Comments (18)

Comment author: Louie 07 July 2010 11:12:52PM 7 points [-]

people are often blind to their own biases, because they incorrectly think that biased thinking on their part would be detectable by conscious introspection

The illusion of a unified, transparent self seems to constantly undermine our thinking.

Is this a problem with our language or our culture? Has a culture / language ever been studied that didn't suffer from this as badly?

Comment author: Dirk 16 March 2013 07:14:25PM 0 points [-]

Is this a problem with our language or our culture? Has a culture / language ever been studied that didn't suffer from this as badly?

Good question. It seems to me, that it is mostly a problem of logistics. When talking about ourselves and objects near us, it is simply easier to reduce those objects to single words. such as 'me'.

Comment author: CarlShulman 08 July 2010 05:46:50AM 4 points [-]

(some groups practicing female genital mutilation believe that if a baby's head touches the clitoris during delivery, the baby will die).

If this belief isn't held to be specific to the group in question (the gods will kill only the babies of our tribe upon such an event), foreigners really ought to come visit and demonstrate a safe clitoris-touching delivery.

Comment author: Douglas_Knight 08 July 2010 06:19:34AM 3 points [-]

really ought to come visit and demonstrate

Have public demonstrations ever dispelled a superstition? The only story I've ever heard along these lines is Robert Johnson eating the tomato. One should ask, also, how many public demonstrations have been attempted; most superstitions are probably not precise enough. I suppose, say, Randi, does things along these lines.

Comment author: Sniffnoy 08 July 2010 10:22:58AM 1 point [-]

The only story I've ever heard along these lines is Robert Johnson eating the tomato.

And I don't think that actually happened, either. People in America seem to have been eating tomatoes well before that was supposed to have happened.

Comment author: JoshuaZ 07 July 2010 10:05:19PM 3 points [-]

I don't think the Mensa data is that useful necessarily. The sort of people who join Mensa are often people who think they are smart and think that intelligence matters a lot more than critical thinking or listening to criticism from people even if they aren't as bright. It shouldn't be that surprising if they are going to pick up unjustified fringe beliefs.

I'm not sure I completely see the difference between memes and mindware. Are they they the same thing but with different emphasis? If not, what is the difference? (I know that there are four of five different notions of memes floating around so this may not be a well-defined question).

Comment author: Kaj_Sotala 07 July 2010 10:20:29PM *  1 point [-]

I'm not sure I completely see the difference between memes and mindware.

For the purposes of this discussion, pretty much the same thing. I added a note to the post clarifying this.

Comment author: fiddlemath 07 July 2010 10:31:28PM *  5 points [-]

To be super-specific: memes are a kind of mindware. Though they're practically synonymous, a piece of mindware that you cannot or do not replicate is not a meme. A secret belief that you actually keep secret, say, would be mindware but not a meme.

That said, the whole point of memetics is that ideas survive to the extent that they replicate well; so we should expect non-memetic mindware to be rare.

Comment author: jmmcd 08 July 2010 12:33:53PM 1 point [-]

Doesn't that seem like a weakness in the definition of memes? After all, a gene that does not replicate (eg arising through mutation and causing the organism to be non-viable) is still a gene.

Comment author: RolfAndreassen 08 July 2010 01:22:59PM 3 points [-]

It is more a question of the emphasis of analysis. If you speak of a meme, you are thinking of its replicative power; if you speak of mindware, you are thinking about its effect on the possessor's thoughts; but either way it is the same concept. To draw the analogy with genes, it is as though the word 'protein-builder' had been coined for the purpose of speaking of a gene's effect on the body, and we retained the word 'gene' for use when discussing allele frequencies.

Comment author: xamdam 08 July 2010 04:22:44PM *  1 point [-]

online MFFT test: http://www.project-hortus.net/mfft-framework/groupfiles/webexperimentlist.html

keep score on paper, the app does not report the score... selfish researchers

Mine was

friragrra pbeerpg

Comment author: RichardKennaway 08 July 2010 04:53:34PM 2 points [-]

Interesting test to do, but it didn't give me a score at the end.

Comment author: xamdam 08 July 2010 06:25:38PM 0 points [-]

yep, I remembered how many I got wrong, added to the comment.

Comment author: RichardKennaway 08 July 2010 07:03:50PM *  1 point [-]

I got all of them correct, because I didn't choose an answer until I knew which one was right, but it's supposed to be a test of "reflectiveness vs. impulsiveness" (not mentioned on the web page, but I googled the name of the test).

Presumably "impulsive" people are supposed to guess before they're sure of the answer, but is reading a general trait into performance on a test like this any better than reading meaning into an inkblot? The first hit on Google Scholar is critical of the test.

ETA: Here and here are articles saying it's only for use on children aged 6 to 12.

Comment author: xamdam 08 July 2010 07:59:24PM 0 points [-]

I agree with you, it is not clear from the test what you're trying to optimize; sorting rotten fruit vs. detecting cancer in xrays are two examples of 'visual examination' where optimal behavior is very different.

So this only makes a good point when your confidence is not well calibrated (rather than consciously settling for lower confidence). I suspect the "impulsive" group Stenovich has in mind are people who are "sure" of the wrong answer, and do not update with feedback.

Comment author: Sniffnoy 08 July 2010 08:23:29PM 0 points [-]

Yes, nowhere does it say whether you're supposed to be going for time or for accuracy!

Comment author: RolfAndreassen 08 July 2010 01:19:14PM 1 point [-]

A truly minor nitpick: You mention 'biorhythms' as an example of irrational beliefs along with astrology and aliens. Quite possibly this is correct, but I suggest that it is nowhere near as well known, and therefore is a bad example; I for one do not have enough information about the belief to dismiss it as I do with astrology. For example, I believe that I need sleep at fairly regular intervals, which is presumably a biological rhythm of sorts. Without any information other than the bare label, I might identify myself as a biorhythm-believer. I'm sure you are referring to some rather wackier belief-structure, but the word 'biorhythm' does not have any such association for me at least, as astrology does. So it kind of jars the presentation; to borrow a trope, it's as though we're being told the Mensa members believe in arson, murder, and breach of estoppel.

Comment author: arundelo 08 July 2010 02:42:25PM 6 points [-]

Maybe a Wikipedia link would help.