= 27d567bc2d48d9f40e9ccc20ea370b15)
drc500free comments on Problems with Academia and the Rising Sea - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (39)
A fundamental problem seems to be that there is a lower prior for any given hypothesis, driven by the increased number of researchers, use of automation, and incentive to go hypothesis-fishing.
Wouldn't a more direct solution be to simply increase the significance threshold required in the field?
That doesn't lower the pre-study prior for hypotheses, it (in combination with reporting bias) reduces the likelihood ratio a reported study gives you for the reported hypothesis.
Increasing the significance threshold would mean that adequately-powered honest studies would be much more expensive, but those willing to use questionable research practices could instead up the ante and use the QRPs more aggressively. That could actually make the published research literature worse.
Respectfully disagree. The ability to cheaply test hypotheses allows researchers to be less discriminating. They can check a correlation on a whim. Or just check every possible combination of parameters simply because they can. And they do.
That is very different from selecting a hypothesis out of the space of all possible hypotheses because it's an intuitive extension of some mental model. And I think it absolutely reduces the pre-study priors for hypotheses, which impacts the output signal even if no QRPs are used.