drc500free comments on Problems with Academia and the Rising Sea - Less Wrong

28 Post author: JonahSinick 23 May 2013 07:17PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (39)

You are viewing a single comment's thread. Show more comments above.

Comment author: drc500free 12 July 2013 05:59:11AM *  0 points [-]

That doesn't lower the pre-study prior for hypotheses, it (in combination with reporting bias) reduces the likelihood ratio a reported study gives you for the reported hypothesis.

Respectfully disagree. The ability to cheaply test hypotheses allows researchers to be less discriminating. They can check a correlation on a whim. Or just check every possible combination of parameters simply because they can. And they do.

That is very different from selecting a hypothesis out of the space of all possible hypotheses because it's an intuitive extension of some mental model. And I think it absolutely reduces the pre-study priors for hypotheses, which impacts the output signal even if no QRPs are used.