I am reminded of a paper by Simkin and Roychowdhury where they argued, on the basis of an analysis of misprints in scientific paper citations, that most scientists don't actually read the papers they cite, but instead just copy the citations from other papers. From this they show that the fact that some papers are widely cited in the literature can be explained by random chance alone.
Their evidence is not without flaws - the scientists might have just copied the citations for convenience, despite having actually read the papers. Still, we can easily imagine a similar effect arising if the scientists do read the papers they cite, but use the citation lists in other papers to direct their own reading. In that case, a paper that is read and cited once is more likely to be read and cited again, so a small number of papers acquire an unusual prominence independent of their inherent worth.
If we see a significant number of instances where the conclusions of a widely-accepted paper are later debunked by a simple test, then we might begin to suspect that something like this is happening.
a question
if referencing is not based on knowledge or perhaps even relevance what does this imply for Google algorithm?
does it not organize search responses according to page links?
We're all familiar with false popular memes that spread faster than they can be stomped out: You only use 10% of your brain. Al Gore said he invented the internet. Perhaps it doesn't surprise you that some memes in popular culture can't be killed. But does the same thing happen in science?
Most of you have probably heard of Broca's aphasia and Wernicke's aphasia. Every textbook and every college course on language and the brain describes the connection between damage to these areas, and the speech deficits named after them.
Also, both are probably wrong. Both areas were mistakenly associated with their aphasias because they are near or surrounded by other areas which, when damaged, cause the aphasias. Yet our schools continue teaching the traditional, erroneous story; including a lecture in 9.14 at MIT given in 2005. Both the Wikipedia entry on Wernicke's aphasia and the Wikipedia entry on Broca's aphasia are still in error; the Wikipedia entry on Wernicke's area has got it straight.
Is it because this information is considered unimportant? Hardly; it's probably the only functional association you will find in every course and every book on the brain.
Is it because the information is too new to have penetrated the field? No; see the dates on the references below.
In spite of this failure in education, are the experts thoroughly familiar with this information? Possibly not; this 2006 paper on Broca's area by a renowned expert does not mention it. (In its defense, it references many other studies in which damage to Broca's area is associated with language deficits.)
So:
References
Bogen JE, Bogen GM (1976). Wernicke's region—Where is it? Ann. N. Y. Acad. Sci. 280: 834–43.
Dronkers, N. F., Shapiro, J. K., Redfern, B., & Knight, R. T. (1992). The role of Broca’s area in Broca’s aphasia.
Journal of Clinical and Experimental Neuropsychology, 14, 52–53.
Dronkers NF., Redfern B B., Knight R T. (2000). The neural architecture of language disorders. in Bizzi, Emilio; Gazzaniga, Michael S.. The New cognitive neurosciences (2nd ed.). Cambridge, Mass: MIT Press. pp. 949–58.
Dronkers et al. (2004). Lesion analysis of the brain areas involved in language comprehension. Cognition 92: 145-177.
Mohr, J. P. (1976). Broca’s area and Broca’s aphasia. In H. Whitaker, Studies in neurolinguistics, New York: Academic Press.