Contrarian viewpoint: I think this isn't as bad as the article makes it sound.
It makes sense to pay more attention to positive results. That's where the breakthroughs are. That's the reason humans evolved positive bias in the first place.
It makes sense to focus more on new research than on replication The end of science is precise mechanisms explaining how stuff works - if someone throws out a wrong result, we'll figure out something is wrong via conceptual replication, and maybe then an exact replication might be in order. Exact replication is expensive and benefits are likely lower than the equivalent resources into new research. (Of course, this does not apply when resources are not a limiting factor)
The real-world costs of being wrong in science are low - there are costs, but these costs only increase if you add in replication and stuff. The real world benefits of being right in science are tremendous. The signal-to-noise ratio is somewhat less important than the absolute amount of signal in this scenario.
Not saying that there isn't room any for improvement, but let's carefully think about what we want to reform. I don't think replication is necessarily the answer, and if we are going to pay more attention to null results we have to be smart about how we do it and not err in the opposite direction - as in, you shouldn't ignore nulls but it really is better to pay more attention to positive results.
(Unless you are doing extremely practical research, as in medicine. Then all of this is really really bad. I'm talking about strictly about research which is still several steps away from a practical implementation)
I'm vaguely annoyed with the way they explain statistical significance issues in genomics in this paper. It makes biologists sound worse than they are. They do control the family wise error rate when the 'family' in question is just the individual genes in an experiment. It's when the family in question consists of multiple experiments, or papers, that the trouble starts.
Wow I had no idea that would be taken so negatively. Anyone want to clear up the inferential silence here?
I'd like to retract it, but I can't retract it simply on the basis that it is very unpopular if I myself do not see the problem with it. :(
I didn't downvote initially, though I'm doing so now. If other people's reactions are anything like mine, it's a combination of finding the comment incoherent, reacting in isolation to boo-lights embedded in the comment, and having lost patience with the author to the point of being uninterested in asking for explanation or granting benefit of the doubt.
Thank you, that clarifies things considerably.
Reading around, I can't seem to decipher what you mean to say with, "boo-lights embedded in the comment." Can you clarify that for me?
"boo-lights" => phrases meant to evoke an emotional rejection-response. I would give examples, but the comment itself seems to have been completely rewritten.
I saved a copy in case you wanted to do that, if you're interested.
Thank you. I believe that clarifies everything that could be clarified here.
If you're not clear about which phrases I'm referring to , I'm willing to point them out. If you are, then there seems little to add.
Incidentally, editing a comment that's already downvoted below -3 is unlikely to achieve much (except for confusing the record), since basically nobody will read the edited comment.
In the same way (and for the same reason), simply retracting or deleting the comment serves little purpose. I see fit to attempt to clarify. The worst that can happen is that it will still yet attract downvotes. I have little to lose that I haven't already lost in failing to notice I had a comment receiving such negativity. At the very least, I am now aware that it is happening.
Yup, agreed that retracting/deleting the comment is similarly unlikely to achieve much.
Certainly, none of that activity is likely to clarify anything, since few people read comments after they've been downvoted below threshold. If anything, it's more likely confuse/obscure the record, as I mentioned.
Related: The Real End of Science
From the Economist.
I recommend reading the whole thing.