Follow-up to:
Parapsychology: the control group for science
Some Heuristics for Evaluating the Soundness of the Academic Mainstream in Unfamiliar Fields
Recent renewed discussions of the parapsychology literature and Daryl Bem's recent precognition article brought to mind the "market test" of claims of precognition. Bem tells us that random undergraduate students were able to predict with 53% accuracy where an erotic image would appear in the future. If this effect was actually real, I would rerun the experiment before corporate earnings announcements, central bank interest rate changes, etc, and change the images based on the reaction of stocks and bonds to the announcements. In other words, I could easily convert "porn precognition" into "hedge fund trillionaire precognition."
If I was initially lacking in the capital to do trades, I could publish my predictions online using public key cryptography and amass an impressive track record before recruiting investors. If anti-psi prejudice was a problem, no one need know how I was making my predictions. Similar setups could exploit other effects claimed in the parapsychology literature (e.g. the remote viewing of the Scientologist-founded Stargate Project of the U.S. federal government). Those who assign a lot of credence to psi may want to actually try this, but for me this is an invitation to use parapsychology as control group for science, and to ponder a general heuristic for crudely estimating the soundness of academic fields for outsiders.
One reason we trust that physicists and chemists have some understanding of their subjects is that they produce valuable technological spinoffs with concrete and measurable economic benefit. In practice, I often make use of the spinoff heuristic: If an unfamiliar field has the sort of knowledge it claims, what commercial spinoffs and concrete results ought it to be producing? Do such spinoffs exist? What are the explanations for their absence?
For psychology, I might cite systematic desensitization of specific phobias such as fear of spiders, cognitive-behavioral therapy, and military use of IQ tests (with large measurable changes in accident rates, training costs, etc). In financial economics, I would raise the hundreds of billions of dollars invested in index funds, founded in response to academic research, and their outperformance relative to managed funds. Auction theory powers tens of billions of dollars of wireless spectrum auctions, not to mention evil dollar-auction sites.
This seems like a great task for crowdsourcing: the cloud of LessWrongers has broad knowledge, and sorting real science from cargo cult science is core to being Less Wrong. So I ask you, Less Wrongers, for your examples of practical spinoffs (or suspicious absences thereof) of sometimes-denigrated fields in the comments. Macroeconomics, personality psychology, physical anthropology, education research, gene-association studies, nutrition research, wherever you have knowledge to share.
ETA: This academic claims to be trying to use the Bem methods to predict roulette wheels, and to have passed statistical significance tests on his first runs. Such claims have been made for casinos in the past, but always trailed away in failures to replicate, repeat, or make actual money. I expect the same to happen here.
He was involved in calling out some fraud going on where he worked, he's honest about what motivated him to get involved in psi research (various personal experiences), he understands the statistics well enough to know the weaknesses of meta-analyses and the necessity of having powerful methods, he's pointed out various methodological problems with psi research as it's usually practiced, he doesn't try to hide weird results or pretend that weird results are the ones that the experiment was intended to find, he recognizes that most claimed psi experiences can be explained away by purely mundane factors, with a few exceptions he's very careful to pay attention to all reasonable hypotheses about possible mechanisms for psi given the limited and ambiguous evidence, et cetera.
Nor worse than chance, I presume? I'd figure you a goat after all.
I haven't done any rigorous self-experimentation as I'm superstitious and am mildly freaked out about the idea that reality actively corroborates whatever inductive biases you happen to have. Rationality is hard enough in a non-agentic world. Truth would seem to be about having terms in your utility function pertaining to cooperation with other agents, so if the information I get doesn't help me cooperate with others then I don't see any grounds for me to trust it or for me to go out and find it. Yay anti-epistemology. That's a rationalization; I'm not entirely sure why I'm afraid of rigorous self-tests.
You can have other Bay Area LessWrongers watch or help set up the experiments. That will at least help in cooperation with this community.