Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Matt_Simpson comments on So You Think You're a Bayesian? The Natural Mode of Probabilistic Reasoning - Less Wrong

48 Post author: Matt_Simpson 14 July 2010 04:51PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (79)

You are viewing a single comment's thread. Show more comments above.

Comment author: Matt_Simpson 18 July 2010 06:50:49AM *  19 points [-]

Thanks for the links!

Reading both of these papers sent me on a trawl through the literature. Kahneman's paper sparked it. He reported an experiment to test Gigerenzer's hypothesis that recasting the problem in terms of frequencies reduces or eliminates the prevalence of the conjunction fallacy (pg 586-7). The results, according to Kahneman, confirm his hypothesis that frequencies just cue people to think in terms of set relations and reject Gigerenzer's hypothesis that people are natural frequentists.

In an unpublished manuscript (that I did not read), Hertwick (1997) challenged Kahneman's interpretation of his experiment, arguing that the language Kahneman used led subjects to misinterpret "and" to be disjunctive - i.e., when Kahneman asked how many out of 1000 women fitting the Linda description are "feminists and active bank tellers," the subjects interpreted this as "how many are feminists and how many bank tellers." Hertwick ran an experiment to test this, confirming his hypothesis.

Then Hertwick, Kahneman, and Mellers wrote an "adversarial collaboration" where Mellers arbitrated the disagreement between Hertwick and Kahneman (pdf). They ran 3 experiments to test the different interpretations that Hertwick and Kahneman were giving the data. The experiments didn't resolve the disagreement completely, but both parties moved closer to the other's position.

Finally, Tentori, Bonini, and Osherson devised a way to directly test whether subjects were misinterpreting the term "and" while they gave their subjects Linda-like problems (pdf). They ran two treatments - one with the probability language and the other with the frequency language. In both treatments, the majority of subjects interpreted "and" conjunctively, and those who interpreted it conjunctively still fell prey to the conjunction fallacy. There was no difference between treatments on the interpretation, and of those who interpreted "and" conjunctively, there was a minor but statistically insignificant difference in the prevalence of conjunction errors between the frequency and probability treatments.

Mea culpa. We aren't natural frequentists. When the frequency language helps, it seems to be due to cuing the set inclusion relation, as Kahneman argued. I don't think the issue is completely settled - in particular I think the last paper doesn't quite get at the crux of the issue - but the evidence is pointing in the direction of Kahneman's hypothesis.

Comment author: XiXiDu 24 July 2010 06:56:12PM 2 points [-]