Douglas_Knight3 comments on An Especially Elegant Evpsych Experiment - Less Wrong

26 Post author: Eliezer_Yudkowsky 13 February 2009 02:58PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Douglas_Knight3 17 February 2009 01:07:23AM 0 points [-]

Bruce K. Britton, Let's start with simpler things, like having people make their data and calculations available. (Or to be really simple, journals with such rules should enforce them!) Without this, you can just hide the data-mining in poorly specified protocols, not to mention fraud.

Data-mining is not that bad because it has systematic effects that an outsider can predict and account for; at least you can hope that it will wash out in the meta-analyses. This reminds me of this Robin Hanson post on how to extract experiments from the medical literature you don't trust.

Perry E. Metzger makes similar recommendations to BKB and RH replies that it's not going to happen. Actually, the medical community is moving towards things like registering studies. I worry that actions taken with a definite sense of who is the bad guy (drug companies) may make us worse off than the status quo, though I don't see any downsides to anything that is actually going forward.

Yvain, read that post.