http://lesswrong.com/lw/ji/conjunction_fallacy/

The moral? Adding more detail or extra assumptions can make an event seem more plausible, even though the event necessarily becomes less probable.

This moral is not what the researchers said; they were careful not to say much in the way of a conclusion, only hint at it by sticking it in the title and some general remarks. It is an interpretation added on by Less Wrong people (as the researchers intended it to be).

This moral contains equivocation: it says it "can" happen. The Less Wrong people obviously think it's more than "can": it's pretty common and worth worrying about, not a one in a million.

The moral, if taken literally, is pretty vacuous. Removing detail or assumptions can also make an event seem more plausible, if you do it in particular ways. Changing ideas can change people's judgment of them. Duh.

It's not meant to be taken that literally. It's meant ot say: this is a serious problem, it's meaningful, it really has something to do with adding conjunctions!

The research is compatible with this moral being false (if we interpret it to have any substance at all), and with it only being a one in a million event.

You may think one in a million is an exaggeration. But it's not. The size of set of possible questions to ask that the researchers selected from was ... I have no idea but surely far more than trillions. How many would have worked? The research does not and cannot say. The only things that could tell us that are philosophical arugments or perhaps further research.

The research says this stuff clearly enough:

http://www.econ.ucdavis.edu/faculty/nehring/teaching/econ106/readings/Extensional%20Versus%20Intuitive.pdf

Our problems, of course, were constructed to elicit conjunction errors, and they do not provide an unbiased estimate of the prevalence of these errors.

In this paper, the researchers clearly state that their research does not tell us anything about the prevalence of the conjunction fallacy. But Less Wrongers have a different view of the matter.

They also state that it's not arbitrary conjunctions which trigger the "conjunction" fallacy, but ones specifically designed to elicit it. But Less Wrong people are under the impression that people are in danger of doing a conjunction fallacy when there is no one designing situations to elicit it. That may or may not be true; the research certainly doesn't demonstrate it is true.

Note: flawed results can tell you something about prevalence, but only if you estimate the amount of error introduced by the flaws. They did not do that (and it is very hard to do in this case). Error estimates of that type are difficult and would need to be subjected to peer review, not made up by readers of the paper.

Here's various statements by Less Wrongers:

http://lesswrong.com/lw/56m/the_conjunction_fallacy_does_not_exist/

There is no plausible way that the students could have misinterpreted this question because of ambiguous understandings of phrases involving "probability".

The research does not claim the students could not have misinterpretted. It suggests they wouldn't have -- the less wronger has gotten the idea the researchers wanted him to -- but they won't go so far as to actually say miscommunication is ruled out because it's not. Given they were designing their interactions with their subjects on purpose in a way to get people to make errors, miscommunication is actually a very plausible interpretation, even in cases where the not very detailed writeup of what they actually did fails to explicitly record any blatant miscommunication.

The issue is that when a person intuitively tries to make a judgement on probabilities, their intuition gives them the wrong result, because it seems to use a heuristic based on representativeness rather than actual probability.

This also goes beyond what the research says.

Also note that he forgot to equivocate about how often this happen. He didn't even put a "sometimes" or a "more often than never". But the paper doesn't support that.

In short how is the experimental setting so different that we should completely ignore experimental results? If you have a detailed argument for that, then you'd actually be making a point.

He is apparently unaware that it differs from normal life in that in normal life people's careers don't depend on tricking you into making mistakes which they can call conjunction fallacies.

When this was pointed out to him, he did not rethink things but pressed forward:

So, your position is that it is completely unrealistic to try to trick people, because that never happens in real life?

But when you read about the fallacy in the Less Wrong articles about it, they do not state "only happens in the cases where people are trying to trick you". If it only applies in those situations, well, say so when telling it to people so they know when it is and isn't relevant. But this cosntraint on applicability, from the paper, is simply ignored most of the time to reach a conclusion the paper does not support.

I understand this criticism when applied to the Linda experiment, but not when it is applied to the color experiment. There was no "trickery" here.

Here's someone from Less Wrong denying there was any trickery in one of the experiments, even though the paper says there was.

It addresses the question of whether that kind of mistake is one that people do sometimes make; it does not address the question of how often people make that kind of mistake in practice.

Note the heavy equivocation. He retreats from "always" which is a straw man, to "sometimes" which is heavily ambiguous about how often. He has in mind: often enough to be important; pretty often. But the paper does not say that.

BTW the paper is very misleading in that it says things like, in the words of a Less Wronger:

65% still chose sequence 2 despite it being a conjunction of sequence 1 with another event

The paper is full of statements that sound like they have something to do with the prevalence of the conjunction fallacy. And then one sentence admiting all those numbers should be disregarded. If there is any way to rescue those numbers as legitimate at all, it was too hard for the researchers and they didn't attempt it (I don't mean to criticize their skill here. I couldn't do it either. Too hard.)