If we see a significant number of instances where the conclusions of a widely-accepted paper are later debunked by a simple test, then we might begin to suspect that something like this is happening.
How so? Could you clarify your reasoning?
Scientists cite journals with conclusions that are convenient to cite (either because they corroborate their views or define a position to pivot from or argue with) whether or not they have been read. Journals with easily debunked conclusions might equivalently be not read (and thus unexamined) or read (and simply trusted).
I think that the real test for whether cited publications are read or not is the following: if a publication is consistently cited for a conclusion it does not actually present, then this is evidence of no one actually having read the publication.
I recall in my research that it was very convenient in the literature to cite one particular publication for a minor but foundational tenet in the field. However, when I finally got a hard-copy of the paper I couldn't find this idea explicitly written anywhere. The thing is -- contradicting what I say above, unfortunately -- I think the paper was well-read, but people don't double-check citations if the citation seems reasonable.
How so? Could you clarify your reasoning?
My thinking is: Given that a scientist has read (or looked at) a paper, they're more likely to cite it if it's correct and useful than if it's incorrect. (I'm assuming that affirmative citations are more common than "X & Y said Z but they're wrong because..." citations.) If that were all that happened, then the number of citations a paper gets would be strongly correlated with its correctness, and we would expect it to be rare for a bad paper to get a lot of citations. However, if we take into accou...
We're all familiar with false popular memes that spread faster than they can be stomped out: You only use 10% of your brain. Al Gore said he invented the internet. Perhaps it doesn't surprise you that some memes in popular culture can't be killed. But does the same thing happen in science?
Most of you have probably heard of Broca's aphasia and Wernicke's aphasia. Every textbook and every college course on language and the brain describes the connection between damage to these areas, and the speech deficits named after them.
Also, both are probably wrong. Both areas were mistakenly associated with their aphasias because they are near or surrounded by other areas which, when damaged, cause the aphasias. Yet our schools continue teaching the traditional, erroneous story; including a lecture in 9.14 at MIT given in 2005. Both the Wikipedia entry on Wernicke's aphasia and the Wikipedia entry on Broca's aphasia are still in error; the Wikipedia entry on Wernicke's area has got it straight.
Is it because this information is considered unimportant? Hardly; it's probably the only functional association you will find in every course and every book on the brain.
Is it because the information is too new to have penetrated the field? No; see the dates on the references below.
In spite of this failure in education, are the experts thoroughly familiar with this information? Possibly not; this 2006 paper on Broca's area by a renowned expert does not mention it. (In its defense, it references many other studies in which damage to Broca's area is associated with language deficits.)
So:
References
Bogen JE, Bogen GM (1976). Wernicke's region—Where is it? Ann. N. Y. Acad. Sci. 280: 834–43.
Dronkers, N. F., Shapiro, J. K., Redfern, B., & Knight, R. T. (1992). The role of Broca’s area in Broca’s aphasia.
Journal of Clinical and Experimental Neuropsychology, 14, 52–53.
Dronkers NF., Redfern B B., Knight R T. (2000). The neural architecture of language disorders. in Bizzi, Emilio; Gazzaniga, Michael S.. The New cognitive neurosciences (2nd ed.). Cambridge, Mass: MIT Press. pp. 949–58.
Dronkers et al. (2004). Lesion analysis of the brain areas involved in language comprehension. Cognition 92: 145-177.
Mohr, J. P. (1976). Broca’s area and Broca’s aphasia. In H. Whitaker, Studies in neurolinguistics, New York: Academic Press.