DSimon comments on Rationality Quotes May 2013 - Less Wrong

6 Post author: katydee 03 May 2013 08:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (387)

You are viewing a single comment's thread. Show more comments above.

Comment author: RolfAndreassen 07 May 2013 04:54:59PM 8 points [-]

If the effect is so small that a sample of several thousand is not sufficient to reliably observe it, then it doesn't even matter that it is positive. An analogy: Suppose I tell you that eating garlic daily increases your IQ, and point to a study with three million participants and P < 1e-7. Vastly significant, no? Now it turns out that the actual size of the effect is 0.01 points of IQ. Are you going to start eating garlic? What if it weren't garlic, but a several-billion-dollar government health program? Statistical significance is indeed not everything, but there's such a thing as considering the size of an effect, especially if there's a cost involved.

Moreover, please consider that "consistent with zero" means exactly that. If you throw a die ten times and it comes up heads six, do you "hesitantly update a very tiny bit" in the direction of the coin being biased? Would you do so, if you did not have a prior reason to hope that the coin was biased?

I respectfully suggest that you are letting your already-written bottom line interfere with your math.

Comment author: DSimon 08 May 2013 01:05:21AM 10 points [-]

If I throw a die and it comes up heads, I'd update in the direction of it being a very unusual die. :-)