this could be a good project, any takers. We need to collect the information and put it all together on 1 site, we don't have to release our own studies.
In some fields most of the studies are refuted within 2 years of publishing, i wonder if it might be better to include any and all studies we can find and then rate them on a reliability scale like this one: http://www.cebm.net/index.aspx?o=1025 / http://chronopause.com/index.php/2011/08/12/interventive-gerontology-101-01-the-basics/ .
that scale might not work for all fields but we can probably think of a scale that does work. And then say any rating below X should be never be cited.
I do not however have a solution for how to become a respected agency.
I do not however have a solution for how to become a respected agency.
If journals start to reject publishing researched because it relies on 'poor' citations, that should have the effect of making this proposed archive-of-study-quality respected.
So maybe a more specific question: how could we get journals to use this archive as part of their review process?
From The Atlantic's Lies, Damned Lies and Medical Science:
Here's a suggested solution to the problem of refuted research still being cited. Have some respected agency maintain an archive of studies that have failed to replicate or that have otherwise been found lacking. Once such an archive existed, medical journals could adopt a policy of checking all citations in a proposed article against the archive, rejecting submissions that tried to cite refuted research as valid. This might also alleviate the problem of people not doing replications because replications don't get many cites. Once such an archive was established, getting your results there might become quite prestigious.
The one major problem that I can see with this proposal is that it's not always obvious when a study should be considered refuted. But even erring on the side of only including firmly refuted studies should be much better than nothing at all.
Such a fix seems obvious and simple to me, and while maintaining the archive and keeping it up to date would be expensive, it should be easily affordable for an organization such as a major university or the NIH. Similar archives could also be used for fields other than medicine. Is there some reason that I'm missing for why this isn't done?