heredami comments on 2013 Survey Results - Less Wrong

74 Post author: Yvain 19 January 2014 02:51AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (558)

You are viewing a single comment's thread. Show more comments above.

Comment author: shminux 19 January 2014 04:15:24AM 14 points [-]

Yvain is not hugely on board with the idea of running correlations between everything and seeing what sticks, but will grudgingly publish the results because of the very high bar for significance (p < .001 on ~800 correlations suggests < 1 spurious result) and because he doesn't want to have to do it himself.

The standard way to fix this is to run them on half the data only and then test their predictive power on the other half. This eliminates almost all spurious correlations.

Comment author: Kawoomba 19 January 2014 08:48:10AM *  7 points [-]

Alternatively, Bonferroni correction.

Comment author: Pablo_Stafforini 19 January 2014 09:51:25AM *  8 points [-]

That's roughly what Yvain did, by taking into consideration the number of correlations tested when setting the significance level.