= 27d567bc2d48d9f40e9ccc20ea370b15)
ciphergoth comments on The Decline Effect and the Scientific Method [link] - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (19)
The main problems here appear to be post hoc analysis and the file drawer effect. One reform that would make a huge difference would be a register of trials in advance of the trial taking place, including details of how they propose to analyze the data. Ideally, journals would accept papers for publication on the basis of the entry in the register, before the data arrives.
A register of proposals does seem like it would help to keep scientists honest, a step towards Feynman's "utter honesty." I would hesitate to say that journals should accept papers based solely on that register, though. Sometimes, the proposal might not end up as a wholly accurate description of the actual experiment, for a variety of reasons. I think that making both available would be a good way to judge how well the result actually applies to what is claimed by the scientists who published it. They could offer explanations for any differences between the proposal and the actual, and peer reviewers could give more thorough critiques with this extra information.
A recent psychology graduate tells me that what I propose here happens all the time already! I'd be curious to know where to go to find out more.