= 27d567bc2d48d9f40e9ccc20ea370b15)
drc500free comments on Problems with Academia and the Rising Sea - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (39)
Agreed that improved incentives for truth-seeking would improve details across the board, while local procedural patches would tend to be circumvented.
The first three metrics seem like they could even more strongly encourage sexy bogus findings by giving the general public more of a role: the science press seem to respond strongly to press releases and unsubstantiated findings, as do website hits (I say this based on the "most emailed" and "most read" categories at the NYTimes science section).
A fundamental problem seems to be that there is a lower prior for any given hypothesis, driven by the increased number of researchers, use of automation, and incentive to go hypothesis-fishing.
Wouldn't a more direct solution be to simply increase the significance threshold required in the field?
That doesn't lower the pre-study prior for hypotheses, it (in combination with reporting bias) reduces the likelihood ratio a reported study gives you for the reported hypothesis.
Increasing the significance threshold would mean that adequately-powered honest studies would be much more expensive, but those willing to use questionable research practices could instead up the ante and use the QRPs more aggressively. That could actually make the published research literature worse.
Respectfully disagree. The ability to cheaply test hypotheses allows researchers to be less discriminating. They can check a correlation on a whim. Or just check every possible combination of parameters simply because they can. And they do.
That is very different from selecting a hypothesis out of the space of all possible hypotheses because it's an intuitive extension of some mental model. And I think it absolutely reduces the pre-study priors for hypotheses, which impacts the output signal even if no QRPs are used.