wedrifid comments on Case study: abuse of frequentist statistics - Less Wrong

25 Post author: Cyan 21 February 2010 06:35AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (96)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 26 February 2010 06:43:39AM 2 points [-]

Well, the blatant version would be to take 5 possible control variables and try all 32 possible omissions and inclusions to see if any of the combinations turns up "statistically significant". This might look a little suspicious if you collected the data and then threw some of it away. If you were running regressions on an existing database with lots of potential control variables, why, they'll just have to trust that you never secretly picked and chose.

Someone who did that might not be able to convince themselves they weren't cheating... but someone who, somehow or other, got an idea of which variables would be most convenient to control for, might well find themselves influenced just a bit in that direction.

Comment author: wedrifid 26 February 2010 07:04:10AM 0 points [-]

Someone who did that might not be able to convince themselves they were cheating...

Is that a 'were not'?