Sorry for butting in, but don't you find it strangely convenient that your psi effect is defined just so as to move it outside the domain of scientific inquiry? Do you anticipate ever finding a way to reliably distinguish it from random chance, or do you anticipate forming another excuse, ahem, reason why you should have expected from the start that the way you just tried would not reliably show it? I'd claim you're chasing invisible dragons, but I find it incredulous that you haven't thought of the comparison yourself, which leaves me confused. How does an effect look like that is real but cannot be distinguished from random chance by any reliable method? How would you extract utility from such an effect? And is it worth it to break your tools of inquiry that otherwise work very well, just so you can end up believing in an effect that is true but useless? Food for thought.
I am aware of this. I would have to be incredibly stupid not to be aware of it.
Do you anticipate ever finding a way to reliably distinguish it from random chance
I can reliably distinguish it from random chance, but by hypothesis I just can't tell you about it. I can get evidence, just not communicable evidence.
I think maybe every time I post about evasive psi I should include a standard disclaimer along the lines of "Yes, I realize how incredibly dodgy this sounds and I also find it rather frustrating, but bringing it up and harping on it never leads anywhere."
Follow-up to:
Parapsychology: the control group for science
Some Heuristics for Evaluating the Soundness of the Academic Mainstream in Unfamiliar Fields
Recent renewed discussions of the parapsychology literature and Daryl Bem's recent precognition article brought to mind the "market test" of claims of precognition. Bem tells us that random undergraduate students were able to predict with 53% accuracy where an erotic image would appear in the future. If this effect was actually real, I would rerun the experiment before corporate earnings announcements, central bank interest rate changes, etc, and change the images based on the reaction of stocks and bonds to the announcements. In other words, I could easily convert "porn precognition" into "hedge fund trillionaire precognition."
If I was initially lacking in the capital to do trades, I could publish my predictions online using public key cryptography and amass an impressive track record before recruiting investors. If anti-psi prejudice was a problem, no one need know how I was making my predictions. Similar setups could exploit other effects claimed in the parapsychology literature (e.g. the remote viewing of the Scientologist-founded Stargate Project of the U.S. federal government). Those who assign a lot of credence to psi may want to actually try this, but for me this is an invitation to use parapsychology as control group for science, and to ponder a general heuristic for crudely estimating the soundness of academic fields for outsiders.
One reason we trust that physicists and chemists have some understanding of their subjects is that they produce valuable technological spinoffs with concrete and measurable economic benefit. In practice, I often make use of the spinoff heuristic: If an unfamiliar field has the sort of knowledge it claims, what commercial spinoffs and concrete results ought it to be producing? Do such spinoffs exist? What are the explanations for their absence?
For psychology, I might cite systematic desensitization of specific phobias such as fear of spiders, cognitive-behavioral therapy, and military use of IQ tests (with large measurable changes in accident rates, training costs, etc). In financial economics, I would raise the hundreds of billions of dollars invested in index funds, founded in response to academic research, and their outperformance relative to managed funds. Auction theory powers tens of billions of dollars of wireless spectrum auctions, not to mention evil dollar-auction sites.
This seems like a great task for crowdsourcing: the cloud of LessWrongers has broad knowledge, and sorting real science from cargo cult science is core to being Less Wrong. So I ask you, Less Wrongers, for your examples of practical spinoffs (or suspicious absences thereof) of sometimes-denigrated fields in the comments. Macroeconomics, personality psychology, physical anthropology, education research, gene-association studies, nutrition research, wherever you have knowledge to share.
ETA: This academic claims to be trying to use the Bem methods to predict roulette wheels, and to have passed statistical significance tests on his first runs. Such claims have been made for casinos in the past, but always trailed away in failures to replicate, repeat, or make actual money. I expect the same to happen here.