Parapsychologists are constantly protesting that they are playing by all the standard scientific rules, and yet their results are being ignored - that they are unfairly being held to higher standards than everyone else. I'm willing to believe that. It just means that the standard statistical methods of science are so weak and flawed as to permit a field of study to sustain itself in the complete absence of any subject matter.
— Eliezer Yudkowsky, Frequentist Statistics are Frequently Subjective
Imagine if, way back at the start of the scientific enterprise, someone had said, "What we really need is a control group for science - people who will behave exactly like scientists, doing experiments, publishing journals, and so on, but whose field of study is completely empty: one in which the null hypothesis is always true.
"That way, we'll be able to gauge the effect of publication bias, experimental error, misuse of statistics, data fraud, and so on, which will help us understand how serious such problems are in the real scientific literature."
Isn't that a great idea?
By an accident of historical chance, we actually have exactly such a control group, namely parapsychologists: people who study extra-sensory perception, telepathy, precognition, and so on.
There's no particular reason to think parapsychologists are doing anything other than what scientists would do; their experiments are similar to those of scientists, they use statistics in similar ways, and there's no reason to think they falsify data any more than any other group. Yet despite the fact that their null hypotheses are always true, parapsychologists get positive results.
This is disturbing, and must lead us to wonder how many positive results in real science are actually wrong.
The point of all this is not to mock parapsychology for the sake of it, but rather to emphasise that parapsychology is useful as a control group for science. Scientists should aim to improve their procedures to the point where, if the control group used these same procedures, they would get an acceptably low level of positive results. That this is not yet the case indicates the need for more stringent scientific procedures.
Acknowledgements
The idea for this mini-essay and many of its actual points were suggested by (or stolen from) Eliezer Yudkowsky's Frequentist Statistics are Frequently Subjective, though the idea might have originated with Michael Vassar.
This was originally published at a different location on the web, but was moved here for bandwidth reasons at Eliezer's suggestion.
Comments / criticisms
A discussion on Hacker News contained one very astute criticism: that some things which may once have been considered part of parapsychology actually turned out to be real, though with perfectly sensible, physical causes. Still, I think this is unlikely for the more exotic subjects like telepathy, precognition, et cetera.
And of course, de La Mettrie was himself extending what Descartes argued a century before him about animals. So the serious doubt was there even in the 17th century, though still with a careful "But maybe humans are special (wink wink nudge nudge at the Catholic church)".
Both also handily take care of the notion that "In the 19th century, everyone knew that life was on a different order than mere matter"; yeah, it was the mainstream view of the British science establishment, Lord Kelvin being a prominent figure... but serious mechanistic view of life predates his offense by at least two centuries. That was still a time when it wasn't considered particularly controversial that science is different in different countries. French scientists were on board with the philosophical arguments; and heck, Leuwenhoek did most of his work in the 17th century too. We can learn from Kelvin's arrogance and indignity, but let's not forget that science took a lot of steps back thanks to politics, religion and someone simply declaring "people in the past were stupid". That doesn't mean we should follow the ideas of "the Ancients knew everything best" of course. Reverse stupidity and all that.
And as for Newton... we know now that he was wrong, don't we? There are no magical influences over distance. Everything affecting anything else in the universe must send something material to propagate that influence. Not to mention that the claim "everyone knew that real scientific physics only permitted a body to act upon another body through direct contact" doesn't hold water anyway - Thales described the magnetic properties of lodestones in 6th century BC; people were well aware that influences can somehow propagate through "empty" space. Another example would be when people first realized that lightning and thunder are two aspects of literally the same thing, each arriving to us at different time due to different propagation speed. Newton was rightly dismissed for his mystical beliefs - but the things that actually worked... worked. That's all the reason the learned people needed to believe them (while still being free to believe or dismiss alchemy, astrology and any other silly things Newton believed in).
Really, we should be wary of saying stuff like "In year X, everyone knew Y". It tends to turn out rather silly most of the time. Even when it's not outright propaganda, the view we have of the past is distorted at best.