TheOtherDave comments on Simpson's Paradox - Less Wrong

68 Post author: bentarm 12 January 2011 11:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (58)

You are viewing a single comment's thread. Show more comments above.

Comment author: sfb 13 January 2011 08:22:32PM *  1 point [-]

How are we to decide which partitions are useful? If someone tells us that women born under Aries, Leo or Sagittarius do better with treatment A, as do those born under the Earth, Air and Water signs, would we really be willing to switch treatments?

Assume I have not heard of Simpson's Paradox, have no more time to research and must make a decision now.

Am I justified in not switching treatments using the reasoning that I don't want Astrology to have any substance to it, and it must not be allowed to have any, so I'm going to wishful think this data away and therefore ignore the evidence I have (as I understand it)?

Or am I more rational to say "I will accept the evidence I have as far as I understand it and switch treatments, even though I expect there is something else going on which is nothing to do with Astrology, but I have no time to find out what that is"?

The second has a better thought process but leads to a worse conclusion due to lack of understanding or lack of information, but the first one is based on a worse thought process which could lead to much worse outcomes in future if it is kept up.

Comment author: TheOtherDave 13 January 2011 08:27:56PM 1 point [-]

It might even lead to worse outcomes in the current case, if it turns out that the reason Water signs do better with treatment A in this data set is that the assignment of subjects to treatments in the study was in some way related to their date of birth.

If I have good reasons to believe that factor X doesn't cause events of class Y, and I have data that seems to demonstrate that factor X is causing an event of class Y in one particular case, and I don't have the time to explore that data further, I ought to take seriously the theory that the causation is not what it seems to be.