sfb comments on Simpson's Paradox - Less Wrong

68 Post author: bentarm 12 January 2011 11:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (58)

You are viewing a single comment's thread. Show more comments above.

Comment author: sfb 13 January 2011 08:22:32PM *  1 point [-]

How are we to decide which partitions are useful? If someone tells us that women born under Aries, Leo or Sagittarius do better with treatment A, as do those born under the Earth, Air and Water signs, would we really be willing to switch treatments?

Assume I have not heard of Simpson's Paradox, have no more time to research and must make a decision now.

Am I justified in not switching treatments using the reasoning that I don't want Astrology to have any substance to it, and it must not be allowed to have any, so I'm going to wishful think this data away and therefore ignore the evidence I have (as I understand it)?

Or am I more rational to say "I will accept the evidence I have as far as I understand it and switch treatments, even though I expect there is something else going on which is nothing to do with Astrology, but I have no time to find out what that is"?

The second has a better thought process but leads to a worse conclusion due to lack of understanding or lack of information, but the first one is based on a worse thought process which could lead to much worse outcomes in future if it is kept up.

Comment author: bentarm 14 January 2011 03:12:46AM 1 point [-]

Well, an important question to ask is how the data were generated. If the only thing we know about each patient is whether they were male or female and whether they were born under a Fire sign, and being born under a Fire sign seems to have some explanatory power, then by all means go for it. As Dave suggests below - it is perfectly possible that the astrological data is hiding some genuine phenomenon.

However, if someone collected every possible piece of astrological data, and tried splitting the patients along every one of the 2^11 possible partitions of the twelve starsigns, you would not be surprised to find that at least one of them displayed this sort of behaviour.

I think the key message is that you shouldn't be making causal inferences from correlational conclusions unless you have some good reason to do so.

Comment author: sfb 14 January 2011 03:49:36PM 0 points [-]

I think the key message is that you shouldn't be making causal inferences from correlational conclusions

I've sorted out what I was thinking a bit more. I was not saying "am I justified in believing that the alignment of stars and planets is the cause here", what I was saying is:

If someone tells us that women born under Aries, Leo or Sagittarius do better with treatment A, as do those born under the Earth, Air and Water signs, would we really be willing to switch treatments?

Yes we should be willing to act in a way that appears to support astrology - this paragraph is supporting wisdom as the opposite of stupidity, or decision making by fear of public embarrassment.