Paul_Gowder comments on Beautiful Probability - Less Wrong

34 Post author: Eliezer_Yudkowsky 14 January 2008 07:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (109)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Paul_Gowder 15 January 2008 07:00:40AM 0 points [-]

Cyan, that source is slightly more convincing.

Although I'm a little concerned that it, too, is attacking another strawman. At the beginning of chapter 37, it seems that the author just doesn't understand what good researchers do. In the medical example given at the start of the chapter (458-462ish), many good researchers would use a one-sided hypothesis rather than a two-sided hypothesis (I would), which would better catch the weak relationship. One can also avoid false negatives by measuring the power of one's test. McKay also claims that "this answer does not say how much more effective A is than B." But that's just false: one can get an idea of the size of the effect with either sharper techniques (like doing a linear regression, getting beta values and calculating r-squared) or just by modifying one's null hypothesis (i.e. demanding that a datum improve on control by X amount before it counts in favor of the alternative hypothesis).

Given all that, I'm going to withhold judgment. McKay's argument on the coin flip example is convincing on the surface. But given his history from the prior pages of understating the counterarguments, I'm not going to give it credence until I find a better statistician than I to give me the response, if any, from a "sampling theory" perspective.