LESSWRONG
LW

Comment Permalink
AndrewKemendo120

In no way do I think that the parapsychologists have good hypotheses or reasonable claims. I also am a firm adherent to the ethos: Extraordinary claims must have extraordinary proofs. However to state the following:

one in which the null hypothesis is always true.

is making a bold statement about your level of knowledge. You are going so far as to say that there is no possible way that there are hypotheses which have yet to be described which could be understood through the methodology of this particular subgroup. This exercise seems to me to be rejecting these studies intuitively,(without study) just from the ad hominem approach to rejection - well they are parapsychologists therefore they are wrong. If they are wrong, then proper analysis would indicate that, would it not?

I have never seen a parapsychology study, so I will go look for one. However does every single study have massive flaws in it?

Reply
Showing 3 of 6 replies (Click to show all)
Luaancz10

And of course, de La Mettrie was himself extending what Descartes argued a century before him about animals. So the serious doubt was there even in the 17th century, though still with a careful "But maybe humans are special (wink wink nudge nudge at the Catholic church)".

Both also handily take care of the notion that "In the 19th century, everyone knew that life was on a different order than mere matter"; yeah, it was the mainstream view of the British science establishment, Lord Kelvin being a prominent figure... but serious mechanistic view of life preda... (read more)

Reply
1AllanCrossman
OK. But the point about what we can conclude about regular science stands even if this is only mostly correct.
13Kaj_Sotala
Damien Broderick's Outside the Gates of Science summarizes a number of parapsychology studies, noting that several of the studies do indeed seem quite solid. It doesn't come to any definite conclusion over whether psi phenomena are actually real or if there's just something wrong with our statistical techniques, but it does seem like there might be enough to warrant more detailed study. See also e.g. Ben Goertzel's review of the book.
See in context
by AllanCrossman
2 min read
189

97

Parapsychologists are constantly protesting that they are playing by all the standard scientific rules, and yet their results are being ignored - that they are unfairly being held to higher standards than everyone else. I'm willing to believe that. It just means that the standard statistical methods of science are so weak and flawed as to permit a field of study to sustain itself in the complete absence of any subject matter.

— Eliezer Yudkowsky, Frequentist Statistics are Frequently Subjective

Imagine if, way back at the start of the scientific enterprise, someone had said, "What we really need is a control group for science - people who will behave exactly like scientists, doing experiments, publishing journals, and so on, but whose field of study is completely empty: one in which the null hypothesis is always true.

"That way, we'll be able to gauge the effect of publication bias, experimental error, misuse of statistics, data fraud, and so on, which will help us understand how serious such problems are in the real scientific literature."

Isn't that a great idea?

By an accident of historical chance, we actually have exactly such a control group, namely parapsychologists: people who study extra-sensory perception, telepathy, precognition, and so on.

There's no particular reason to think parapsychologists are doing anything other than what scientists would do; their experiments are similar to those of scientists, they use statistics in similar ways, and there's no reason to think they falsify data any more than any other group. Yet despite the fact that their null hypotheses are always true, parapsychologists get positive results.

This is disturbing, and must lead us to wonder how many positive results in real science are actually wrong.

The point of all this is not to mock parapsychology for the sake of it, but rather to emphasise that parapsychology is useful as a control group for science. Scientists should aim to improve their procedures to the point where, if the control group used these same procedures, they would get an acceptably low level of positive results. That this is not yet the case indicates the need for more stringent scientific procedures.

Acknowledgements

The idea for this mini-essay and many of its actual points were suggested by (or stolen from) Eliezer Yudkowsky's Frequentist Statistics are Frequently Subjective, though the idea might have originated with Michael Vassar.

This was originally published at a different location on the web, but was moved here for bandwidth reasons at Eliezer's suggestion.

Comments / criticisms

A discussion on Hacker News contained one very astute criticism: that some things which may once have been considered part of parapsychology actually turned out to be real, though with perfectly sensible, physical causes. Still, I think this is unlikely for the more exotic subjects like telepathy, precognition, et cetera.

Replication CrisisPractice & Philosophy of ScienceScience
Frontpage

97

Mentioned in
Load More (5/9)
Parapsychology: the control group for science
12AndrewKemendo
6Blueberry
New Comment
189 comments, sorted by
top scoring
Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]AlexMennen140

Parapsychologists make a poor control group of scientists because part of their job is collecting evidence that parapsychology works. In science, that step is already done. Biologists do not need to prove that life works, because life exists. Physicists do not need to prove that physics works, because physics, by definition, IS the way the universe works. Einstein did not dream up relativity and then start looking for evidence to support it. He looked at the evidence that was available, and came up with relativity as a way to explain it. Parapsychologists do it the other way around.

Reply
8brazil84
"Parapsychologists make a poor control group of scientists because part of their job is collecting evidence that parapsychology works." Why is that their job? In theory, they are just studying the question of psychic phenomena. If a parapsychologist found strong evidence against psychic phenonma, he would be doing his job. Of course, your real point is that such a parapsychologist would be working himself out of a job. But the same danger is there for biologists and physicists. And of course climatologists.
5Paul Crowley
What, they're going to discover there's no such thing as climate?
0Curiouskid
Why is research into climate funded? I imagine if the scare of global warming were eliminated, there would be much less funding.
3gjm
Some people have conjectured in the past that we might be in danger of a sort of global cooling. (Note: some things said about this by those who profess to disbelieve in anthropogenic climate change are false.) Understanding large-scale climate phenomena may help to predict natural disasters like hurricanes. The climate is extremely important for life on earth (human life included). I think there would be plenty for climate scientists to do "if the scare of global warming were eliminated". (In fact, there is near-total agreement among climate scientists about global warming. So probably most changes in climate-scientist opinion on this topic would make for greater uncertainty and therefore more funding...) [EDITED to add: At least one person has downvoted this; no one has replied to it. I would be interested to know what about it is downvote-worthy; it looks OK to me even on rereading. If I've made some idiotic mistake, I can't fix my brain unless someone points it out to me...]
0Luke_A_Somers
I suspect the issue is the vague nature of the 'some things said about this' comment. Yes, some things said about it are undoubtedly false, but some quite incisive things said about the conjecture are true!
1gjm
Oh yes, for sure. That comment was a lazy shorthand for something like this: "Some unscrupulous people have played up past conjectures about global cooling in order to discredit what is now said about global warming, and a great deal of what they have said on this subject is bullshit. By referring in this context to global cooling, I am not endorsing any of that bullshit." But that seemed like too much to cram into a comment that was actually about something else, and no way of making it much shorter sprang to mind.
-3brazil84
They might discover that global warming is not a serious threat after all.
[-]Paul Crowley120

Yes, I know that's what you meant, but the point of my comment is that unlike with parapsychology, the object of study for climatologists definitely exists, whether or not AGW is real. Any individual climatologist who started to think that AGW might not be real won't think "this is the end for climatology", because climate exists either way.

And of course if they worry that funding for climate research might drop as a result of what they've found, they can console themselves with the knowledge that their new belief allows them to earn twice as much money working for an oil-industry funded body such as the CEI.

Reply
-1brazil84
I understand your point, but the analogy still has a good deal of validity since the essential point is that the practical consequences of a negative result are to damage the careers of the scientists in question. "And of course if they worry that funding for climate research might drop as a result of what they've found, they can console themselves with the knowledge that their new belief allows them to earn twice as much money working for an oil-industry funded body such as the CEI." I disagree. If it turns out that global warming was wildly exaggerated, why would the oil-industry fund any climatology work at all?
2Paul Crowley
A researcher who thought it exaggerated would know that they would not personally be in a position to change the long-term trends whether or not they report their conclusions, but they could make a lot of money from the CEI in the short term. Denialists whose credentialed academic specialty is specifically climatology are few (I know of none), so they would be very much in demand.
-3brazil84
Ok I understand your point. It seems to me that you are distinguishing between long term and short term consequences. Without getting into an argument about the availability of funding for skeptical global warming research, I will concede the possibility that a climate researcher who is publishing skeptical results may do better in the short term than a parapsychologist who is publishing skeptical results.
5Paul Crowley
More than that: a fully credentialed climate researcher who is prepared to back the position of an organisation like the CEI can line their own pockets far more effectively than one who backs the scientific consensus. You'd be celebrated in many quarters as the world's leading authority on the subject, you'd get speaking engagements aplenty, lots of media attention, and more. It's an extraordinary testament to personal integrity that so few go for it - as I say, I know of none who have.
-2brazil84
I'm not sure I'd go that far. I don't know who CEI is, but Bob Carter, John Christy ,and Richard Lindzen don't seem to be drowning in oil money.
0gjm
CEI.
0taryneast
To reduce their currently extortionate insurance premiums.
0brazil84
Assuming the world discovers and accepts that global warming was wildly exaggerated, how would oil-industry-funded climatology research reduce insurance premiums?
0taryneast
I'm told by somebody in the industry, that premiums in cyclone-areas (notably the rigs in the gulf of mexico) are going through the roof right now as climate change predictions mean that cyclone activity is likely to continually increase. If climate change were wildly exaggerated, it could be used to argue for reduced insurance premiums in those regions.
-2brazil84
Assuming for the sake of argument that that that's true, so what? After that happens, what's the incentive to fund further research?
0taryneast
Um, I was responding to "why would the old industry fund climatology work at all" - my answer is "they might do it if it would reduce their insurance premiums". I do not postulate that they would continue to fund further research after that.
-4brazil84
"Um, I was responding to 'why would the old industry fund climatology work at all'" Ok, that's not the question I asked.
0taryneast
Looking back at your comment above, that is a word-for-word copy of what you asked. How have I misunderstood your question? Have I taken it out of context? If so - my apologies - and can you supply the correct context?
-4brazil84
Lol, yes you took it out of context. Here is the first part of my question: "If it turns out that global warming was wildly exaggerated" So the question is about what happens after anthropogenic CO2 triggered global warming is (hypothetically) debunked as a serious threat.
-1taryneast
Doh! yep I didn't realise you meant "after it'd already been debunked". I think we're in "violent agreement" :)
1prase
What they do in theory isn't much important. Parapsychologists have all their discipline at stake. They can't take one well established effect and gain experience and status by studying it, and then turn to investigation of a more controversial phenomenon. It's a serious difference from climatology.
-1brazil84
I don't think the difference is all that big. If climatologists discover that global warming is not a serious threat after all, it will seriously damage their ability to get funding and prestige.
0[anonymous]
.
5AllanCrossman
Only in general, but not for specific questions like: does compound XYZ affect tumour growth?
0AlexMennen
True, but the people studying whether compound XYZ affect tumour growth are not preselected to believe that it is.
3wedrifid
They are certainly not preselected to be unbiased on the subject.
[-]AndrewKemendo120

In no way do I think that the parapsychologists have good hypotheses or reasonable claims. I also am a firm adherent to the ethos: Extraordinary claims must have extraordinary proofs. However to state the following:

one in which the null hypothesis is always true.

is making a bold statement about your level of knowledge. You are going so far as to say that there is no possible way that there are hypotheses which have yet to be described which could be understood through the methodology of this particular subgroup. This exercise seems to me to be rejecting these studies intuitively,(without study) just from the ad hominem approach to rejection - well they are parapsychologists therefore they are wrong. If they are wrong, then proper analysis would indicate that, would it not?

I have never seen a parapsychology study, so I will go look for one. However does every single study have massive flaws in it?

Reply
[-]Kaj_Sotala130

I have never seen a parapsychology study, so I will go look for one. However does every single study have massive flaws in it?

Damien Broderick's Outside the Gates of Science summarizes a number of parapsychology studies, noting that several of the studies do indeed seem quite solid. It doesn't come to any definite conclusion over whether psi phenomena are actually real or if there's just something wrong with our statistical techniques, but it does seem like there might be enough to warrant more detailed study. See also e.g. Ben Goertzel's review of the book.

Reply
6Blueberry
This is exactly the point. Parapsychology is one of the very few things we can reject intuitively, because we understand the world well enough to know that psychic powers just can't exist. We can reject them even when proper analysis doesn't indicate that they're wrong, which tells us something about the limitations of analysis. ETA: Essentially, if the scientific method can't reject parapsychology, that means the scientific method isn't strong enough, not that parapsychology might be legitimate.
[-]Scott Alexander220

There are many other things that people have claimed can be rejected intuitively without study through the years.

In the 18th century, everyone knew that real scientific physics only permitted a body to act upon another body through direct contact. When Newton proposed his theory of gravity, many people rejected it as pseudoscientific or magical because it claimed the stars and planets could exert action at a distance, without saying how they did it.

In the 19th century, everyone knew that life was on a different order than mere matter, because obviously you couldn't produce the self-moving and self-regenerating qualities of life with just stuff like you get in rocks and sand.

In the 20th century, everyone knew that the mind was more than just the brain, since simple introspection could determine the existence of a consciousness inexplicable in simple material terms.

The absurdity heuristic is an okay heuristic, but I'd be really really careful before saying something is so absurd we can throw away any contradictory experimental evidence without a glance.

The possibility I give to some sort of psi effect existing (in a nice, scientific way that we can study once we figure out what form... (read more)

Reply
[-]MichaelVassar220

I used to think that way before I knew about Bayesianism. Once I learned about it I realized that the prior probability for psi was very VERY low, e.g. its complex and there's no reason to expect it so one in a bajillion, while the probability for the observed evidence for psi, given what we know about psychology, was well in excess of 50% in the absence of psi, so the update couldn't justify odds greater than two in a bajillion.

Reply
[-]Alex Flint180

One in a bajilion? Guys, the numbers matter. 10^-9 is very different from 10^-12, which is very different from 10^-15. If we start talking about some arbitrarily low number like "one-in-a-bajillion" against which no amount of evidence could change our mind, then we're really just saying "zero" but not admitting to ourselves that we're doing so.

Other than that, I agree with Yvain and have found this to be perhaps the most belief-changing so far on LW!

Reply
[-]Eliezer Yudkowsky220

It takes only 332 pieces of evidence with likelihood ratios of 2:1 to promote to 1:1 odds a hypothesis with prior odds of 1:googol, that is 10^-100, which would be the appropriate prior odds of something you could describe with around 70 symbols from a 26-letter equiprobable alphabet.

"A bajillion to one" are odds that Bayesian updating can overcome surprisingly quickly - it isn't anything remotely like "no amount of evidence can change my mind". Now odds of one to a googolplex - that might as well be zero, relative to the amount of evidence you could acquire over a human lifetime. But the prior probability of any possibility you can describe over a human lifetime should be much higher than that.

Reply
[-]roystgnr140

A nitpick: it takes 332 pieces of all mutually independent evidence to perform that level of update.

More confusing, for these purposes the independence level of the evidence depends on what hypotheses you're trying to distinguish with it. E.g. if you're trying to distinguish between "that subject has ESP powers" and "that experiment was random luck" then 332 repetitions of the same experiment will do. If you're trying to distinguish between "that subject has ESP powers" and "that experimenter's facial expressions differ based on what cards he was looking at", then you can't just repeat it; you've got to devise new and different experiments.

Reply
[-]Scott Alexander110

You're right that I completely missed the Bayesian boat, and I'm going to have to start thinking more before I speak and revise my estimates down to <1%.

But I'm still reluctant to put them as low as you seem to. The anthropic principle combined with large universe says that whatever complexity is necessary for the existence of conscious observers, we can expect to find at least that level of complexity. Questions like consciousness, qualia, and personal identity still haven't been resolved, and although past experience suggests there is probably a rational explanation to this question, it isn't nearly dissolved yet. If consciousness really is impossible without some exotic consciousness-related physics (Penrosean or otherwise), then our universe will have exotic consciousness-related physics no matter how complex they need to be. And since evolved beings have been so proficient at making use of normal physics to gain sensory information, it's a good bet they'd do the same with exotic consciousness-related physics too if they had them...

...is a somewhat hokey argument I just invented on the spot, and I'm sorry for it. But the ease with which I can put something like that together... (read more)

Reply
[-]MichaelVassar130

One problem with this argument is that if psi exists, we are very bad at using it, and we don't see other organisms using it well either. The world we see appears to be almost completely described by normal physics at worst.

I don't think that I'm double-counting evidence. I certainly know that there can be intelligent believers, after all, MANY intelligent people believe that one is compelled to accept the conclusions of the scientific method over those of the scientific community. Also, beliefs can be compelling for any variety of irrational reasons. The evidence I have seen though looks to me like exactly the evidence you would expect given known psychology and no psi. We can surely agree that there is a LOT of evidence that hyman psychology would create belief in psi in the absence of psi, can't we.

I would set my odds at "top twenty most astoundingly surprising things ever discovered but maybe not top ten". That seems to me like odds of many billions to one against, but not trillions. Unfortunately, the odds for almost any plausible winning conditions occurring without psi being real are much higher, making the bet difficult to judge. I have a standing 10,0... (read more)

Reply
[-]Scott Alexander150

Have donated $10 to SIAI (seemed less likely to lose you guys money in transaction fees than $1) with public comment about the bet . Will decide where you can donate your $33000 in the unlikely event it proves necessary.

Reply
3MixedNuts
I'd feel ridiculously overconfident stating a probability of less that 1e-6, yet I don't have the slightest hesitation to take that bet. (Brain sucks at small probabilities.) Condition is any two among {Randi, Dawkins, Eliezer, Vassar, me}, but if one is reported to have developed a new mental illness at least two months before they say psi is real, they don't count. Also, let's make it purchasing power as of 2011, not dollar amount. Assuming scarcity lasts long enough.
1CarlShulman
What if they're dead?
[-]Scott Alexander260

Well, then I lose the bet...unless someone contacts their ghosts...in which case I win the bet!

Reply
0FeepingCreature
Psi doesn't even explain consciousness or qualia. [edit] Oops, necro. Disregard me. [edit edit] okay! nevermind that then :D
3MC_Escherichia
I don't think there's a prejudice against replying to old posts around here...
0externalmonologue
One in a bajillion? You are saying you actually know how complex psi is without even saying what aspect of psi you are talking about. We know biology is very complex. So when testing a supplement like creatine, the pseudoskeptic could say "biology is extremely complex. We do not know the mechanism that makes creatine work so I assign a very low bayesian probability. Today I feel like a hundred trillion to one". Keep in mind this is after several studies have shown an effect in the predicted direction whose odds are not easily explained by chance. Indeed, there is nothing wrong with you assertion about complexity just the subjective part where you assign a number to a phenomena you are not very familiar with.
5wedrifid
Bajillion isn't exactly a precise measure. In this context it means 'lots'. That isn't hard to assign to the all aspects of psi.
[-]Sebastian_Hagen160

In the 20th century, everyone knew that the mind was more than just the brain, since simple introspection could determine the existence of a consciousness inexplicable in simple material terms.

No, they didn't. Superficial research indicates that serious materialism goes back to at least the Enlightenment in the 18th century. And the 20th century? That's not even plausible.

Reply
1Luaancz
And of course, de La Mettrie was himself extending what Descartes argued a century before him about animals. So the serious doubt was there even in the 17th century, though still with a careful "But maybe humans are special (wink wink nudge nudge at the Catholic church)". Both also handily take care of the notion that "In the 19th century, everyone knew that life was on a different order than mere matter"; yeah, it was the mainstream view of the British science establishment, Lord Kelvin being a prominent figure... but serious mechanistic view of life predates his offense by at least two centuries. That was still a time when it wasn't considered particularly controversial that science is different in different countries. French scientists were on board with the philosophical arguments; and heck, Leuwenhoek did most of his work in the 17th century too. We can learn from Kelvin's arrogance and indignity, but let's not forget that science took a lot of steps back thanks to politics, religion and someone simply declaring "people in the past were stupid". That doesn't mean we should follow the ideas of "the Ancients knew everything best" of course. Reverse stupidity and all that. And as for Newton... we know now that he was wrong, don't we? There are no magical influences over distance. Everything affecting anything else in the universe must send something material to propagate that influence. Not to mention that the claim "everyone knew that real scientific physics only permitted a body to act upon another body through direct contact" doesn't hold water anyway - Thales described the magnetic properties of lodestones in 6th century BC; people were well aware that influences can somehow propagate through "empty" space. Another example would be when people first realized that lightning and thunder are two aspects of literally the same thing, each arriving to us at different time due to different propagation speed. Newton was rightly dismissed for his mystical beliefs - b
0nwthomas
Good point, with the qualifier that many people (including professional philosophers) presently find themselves unable to wrap their heads around the idea that they have no non-material consciousness. The "argument from absurdity" against materialism is alive and kicking.
[-]Mitchell_Porter110

Parapsychology is one of the very few things we can reject intuitively, because we understand the world well enough to know that psychic powers just can't exist.

Do you think it is possible that we are "living in the Matrix"? If so, then you should consider something functionally indistinguishable from psychic powers to be possible.

Reply
4bigbad
If we are, in fact, living in the Matrix, then science has already characterized the rules of the simulation rather well. Barring further interference by the sysadmin/God/whatever, it should continue to operate by mechanistic, semipredictable rules. Science has little to say about one-time interventions from outside observable reality, whether you call them "Matrix hacks", "miracles", or what you will. Regarding such matters, the null hypothesis has yet to be convincingly falsified, but absence of proof is not proof of absence.
1Strange7
It's a fairly common thing, in videogame design, to include "cheat codes:" obscure, highly specific, and seemingly useless in-game behaviors which produce otherwise impossible results.
0Pablo
The hypothesis that we are living in the Matrix is best understood as a metaphysical hypothesis. The various claims made by parapsychologists, however, are not metaphysical claims about the nature of reality, but "scientific" claims about what goes on in reality. It is therefore unclear why such claims would be more probable on the assumption that the Matrix hypothesis is true.
0Psychohistorian
I am not surprised when a video game character consistently summons balls of fire out of nothingness. I would be absolutely astounded to see an actual person do this. This is because the system of rules governing a video game and the system governing a deterministic universe appear to be very, very different. If we were living in the matrix, this would not be the case. It would not mean that we are necessarily in the kind of video game where there are psychic powers, but it would provide a very clear mechanism through which psychic powers could act. Such a mechanism does not appear possible in a deterministic universe, or at least in the one we seem to occupy.
[-]Vladimir_Nesov100

Real world is uncaring, unsupervised. Magic is not just about the world being "complex", it's about the world containing mechanisms targeting specifically humans, and understanding the situation much like a human would. Being "deterministic" doesn't preclude anything, it's more of a way of seeing things than the way things are.

Reply
4wedrifid
An artificial dichotomy.
1Zack_M_Davis
I don't think so. Video games are specifically programmed to create a particular experience for the user. If something goes over the horizon and won't be needed again, it just doesn't get computed. Whereas the real universe seems to be---just the same physics. Everywhere. No complicated ad hoc programming describing levels or characters or points, or translating keypresses into useful actions---no user input at all, come to think of it.
3SilasBarta
Not quite. That's what we assume happens -- justifiably! -- because it would be a far more complicated hypothesis to disbelieve in the implied invisible. However, failing to see these implied invisibles is not itself independent evidence of universal law, just an inference from an Occamian prior. You would fail to see implied invisibles with equal probability whether or not the laws were fully universal. Interestingly, I explored the question of whether it's possible, if the universe is a simulation, to shut it down by forcing it to do more and more computational work in order to keep fooling us. But, I argue, it turns out that the 2nd law of thermodynamics implies that no matter what observations observers choose to make, it requires no more storage capacity to continue fooling them.
2gwern
I read this, but I'm a little confused. Conceptually, as a closed system, the demand of universe is constant, sure, when I imagine it as something like the game of Life. Are you assuming that any simulator will be a full and perfect emulator, with no optimizations like caches? Because if optimizations are applied, then it seems you can expand the necessary power by doing things that defeat the optimizations. Caches are ineffective if you keep generating intricately linked cryptographic junk, etc. One might think that no simulating agent would run a simulator whose worst-case requirements are beyond its abilities; but then, we humans routinely use QuickSort and don't mind our kernels over-committing memory... (Incidentally, I made an estimation of my own for how small our substrate could be: http://www.gwern.net/Simulation%20inferences.html . I concluded that the simulating computer could be as small as a Planck cube.)
1SilasBarta
It doesn't rely on that assumption. It's just based on the fact that any time you destroy entropy by forcing some system, from your perspective, to be in fewer possible states, you also allow another system, from your perspective, to be in proportionally more possible states. The more states something could be in, from your perspective, the less information the simulator has to store to consistently represent it for you.
2gwern
I vaguely see what you're getting at - every observation or interaction forces the simulator to calculate what you see, but also allows it to cheat in other areas. But I'm not sure how exactly this would work on the level of bits and programs?
0Vladimir_Nesov
This is a very conceptually interesting question.
-1SilasBarta
Bah! Implementation issue! :-P At the level you're asking about (if I understand you correctly), the program can just reallocate the memory for whatever gained entropy, to whatever lost entropy. Like in the comments section of my blog, if you learn the location of a ball, the program now has to store it as being in a definite location, but I also powered my brain to learn that, so the program doesn't have to be as precise in storing information about chemical bonds, which were moved to a higher entropy state.
0gwern
Spoken like a true theoretician. But it's hard to see an implementation that is optimal in exploiting this memory bound. I mean, imagine that we have a pocket universe where we can have many numbers (particles?) which all must add up to 1000, and we have your normal programming types like bit, byte/int, integer etc. If we start out with 1 1000, and then the 'laws of physics' begins dividing it by 10, (giving us 10 100s), how is the simulator going to be smart enough to take its fixed section of RAM and rewrite the single large 1000 integer into 10 smaller ints, and so on down to 1000 1s which could be single bits? Is there any representation of the universe's state which achieves these tricks automatically, or does the simulation really just have to include all sorts of conditionals like 'if (changed? x), then if x > 128, convert x Integer; x <= 128 && > 1, convert x int; else convert x bit' in order to preserve the constant-memory usage?
-1SilasBarta
I don't think this hypothetical universe is comparable in the relevant ways: it must be capable of representing the concept of an observer, and what that observer knows (has mutual information with), and adhere to the 2nd law of thermodynamics. Which I don't think is the case here.
0gwern
Wait, there has to be an observer? I thought you were really just talking about entangled wave-functions etc.
1SilasBarta
No, that's the point Jack brought up. I was only discussing the issues that arise in the hypothetical scenario in which the universe is simulated in an "overworld" and must successfully continue to fool us.
1Tyrrell_McAllister
You make an interesting observation. I'm still trying to think it through, so I might not yet be making sense. But, right now, I have the following difficulty with accepting your argument. Any simulation has "true" physical laws. These are just the rules that govern how in fact the simulation's algorithm unfolds, including all optimizations, etc. However, we expect, a priori, the ultimate laws of reality to satisfy certain invariances. For example, perhaps we expect the ultimate laws to work identically at different points in real physical space. The true laws of the simulation might not satisfy such invariances with respect to the simulation. For example, the simulation's laws might not work identically at different points in the simulated physical space. [ETA: Optimization makes this likely. The simulation could evolve in a "chunkier" way far from us than it does close to us.] So maybe this is how we can define what it means to hide the simulated nature of our universe from us: "Hiding the simulation" means "making our universe appear to us as though its laws satisfy all the expected invariances, even though they don't". Here's the issue that I hope you address: I'm convinced by your argument that "any time you destroy entropy by forcing some system, from your perspective, to be in fewer possible states, you also allow another system, from your perspective, to be in proportionally more possible states." Say that, when I start out, system A could be in any one of the states in some state-set X. Then I learn about system B, and so, as you point out, system A could now be in any one of the states in some larger state-set Y, as far as I know. But what if the larger state-set Y includes states that do not obey the expected invariances? And what if, as I learn more about the universe, the state-set that A's state must be in grows, all right, but eventually consists almost entirely of states that violate our expected invariances? Wouldn't that amount to discoveri
1SilasBarta
Good point -- I've struggled with the same problem, in different terms. Let me know if my statement of the problem matches the point you're making here: "It's possible to discover, not just particulars about individual systems, but universal laws. These universal laws put a constraint on all future observations, thus reducing the subjective entropy of the universe, without (apparently) needing any corresponding gain of entropy." It's something I was wondering about when going over the E. T. Jaynes papers and Yudkowsky's Engines of Cognition. I haven't gotten it resolved in terms of 2nd law and the "subjective entropy" idea, but I think I know how to resolve it in the context of the simulated universe question: basically, if the simulation starts out adhering to the invariances that have to be obeyed (even though they might be more than necessary to fool observers), then it is no additional burden for the observers to notice these invariances. Though the observers have (apparently) violated the 2nd law -- and this is an area for further research -- the simulator was already expending the computational resources necessary to make the invariances hold. It is an exception to the general principle I derived, in that it's a case where net destruction of entropy requires no additional RAM. I'm still working on how to resolve the remaining problems, but it shows how discovery of universal physical laws needn't be a problem for the simulator.
0pengvado
I'll try to bring your solution back to thermodynamics terms: The universe always has and always will obey certain invariances, and those are a redundancy in your observations, which (along with any other redundancy that could possibly be derived) is already taken into account when computing information-theoretic entropy. If you had plenty of data already to derive the invariance but just hadn't previously noticed it, that lack of logical omniscience is why the 2nd law is an inequality. Including the invariance into your future predictions isn't a net reduction in entropy. It just removes some of the slack between the exact phase-volume preserving transforms of physics and the upper bounds that a computationally bounded agent has to use.
0Tyrrell_McAllister
Your restatement looks exactly right, and your solution would resolve the issue I raised. One question is, how much optimization can the simulators do if the true laws are as invariant as they "ought to be"? For example, if the universe has to evolve according to the same rules everywhere, that would seem to keep it from evolving in a chunkier way far away from us, which closes off a potential way to save on computation.
0SilasBarta
The simulator can maintain conservation of e.g. mass, while not churning through the computations required for e.g. gravity until people see enough that they can check if gravity isn't holding. This would save on having to do the gravity calculations. Then, when people, armed with their knowledge of gravity, start looking in more places, the universe must pick a configuration and stick with it -- but at that point, all of their observations have the original problem of freeing up memory somewhere else in the form of higher entropy. On second thought, that doesn't work either, since discovery of gravitational laws will constrain their existing predictions of where the planets will be, and this destruction of entropy is unrelated to the entropy needed to create it, which was your objection to begin with. My best guess at this point is that any resolution will ultimately hinge on a finer-grained information-theoretic analysis of the discovery of universal laws. That is, as you gain evidence pointing to the validity of laws you notice, you assign a high-but-not-unity probability to the laws continuing to hold. Each time your probability goes up, that corresponds to a particular reduction in the entropy of your probability distribution. But, as they say, "to make inferences you have to make assumptions". There is some entropic cost to making the assumptions necessary for the model with invariants to work, and this must be properly accounted for. I'll continue to research this.
0Vladimir_Nesov
This is wrong (even assuming that previous coarse-grained observations don't matter). If you are changing the model by refining it, choosing one option of more detailed data arbitrarily, then this process on the world-model isn't reversible: you can't "un-choose" that arbitrary data and remain able to reconstruct it (unless the data is not arbitrary after all and only depends on the world model that is already there). As a result, no magical increase in entropy occurs, and no resources get saved: it's not an operation on the subsystems within the modeled world, it's an operation on the system of whole-world model within the world of modelers. Also, consider the fact that ultimate laws can never be discovered, strictly speaking: there will always be uncertainty, and maybe there won't even be asymptotically certain candidates, only turtles always deeper and deeper.
2Jack
When I was first introduced to quantum mechanics my professor taught us the Copenhagen Interpretation. I was immediately reminded of occasional moments in video games where features of a room aren't run until the player gets to the room. It seemed to me that only collapsing the wave function when it interacted with a particular kind of physical system (or a conscious system!) would be a really good way to conserve computing power and that it seemed like the kind of hack programmers in an fully Newtonian universe might use to approximate their universe without having to calculate the trajectories of a googolplex (ed) subatomic particles. Can anyone tell me if this actually would save computing power/memory?
2SilasBarta
The answer basically comes down to the issue of saving on RAM vs. saving on ROM. (RAM = amount of memory need to implement the algorithm, ROM = amount of memory needed to describe the algorithm) Video game programmers have to care about RAM, while the universe (in its capacity as a simulator) does not. That's why programmers generate only what they have to, while the universe can afford to just compute everything. However, I asked the same question, which is what led to the blog post linked above, where I concluded that you wouldn't save memory by only doing the computations for things observers look at: first, because they check for consistency and come back to verify that the laws of physics still work, forcing you to generate the object twice. But more importantly (as I mentioned) because the 2nd law of thermodynamics means that any time you gain information about something in the universe, you necessarily lose just as much in the process of making that observation (for a human, it takes the form of e.g. waste heat, higher-entropy decomposition of fuels). So by learning about the universe through observation, you simultaneously relieve it of having to store at least as much information (about e.g. subatomic particles). (This argument has not been peer-reviewed, but was based on Yudkowsky's Engines of Cognition post.)
1pengvado
Assuming they don't make any approximations other than collapse, yes a classical computer simulating Copenhagen takes fewer arithmetic ops than simulating MWI. At least until someone in the simulation builds a sufficiently large coherent system (quantum computer), at which point the simulator has to choose between forbidding it (i.e. breaking the approximation guarantee) or spending exponentially many arithmetic ops. Copenhagen (even in the absence of large coherent subsystems) does not take significantly less memory than MWI: both are in PSPACE. Otoh, if the simulator is running on quantum-like physics too, then there's no asymptotic difference in arithmetic either. And if you're not going to assume that the simulator's physics is similar to ours, who says it's less rather than more computationally capable?
1matt
googleplex = Google Inc's HQ googolplex = 10^(10^100)
-2Blueberry
It's truly sad now how people are less familiar with the original spelling and meaning of a googol. Now the first thing we think of is the search engine, instead of 10^100.
6gwern
Is that really so sad? googol was named in jest and I do not think I have ever seen it seriously needed for anything; Google on the other hand...
3Baughn
If you implemented the laws of physics on a computer, using lazy evaluation, then whatever is "over the horizon" from the observer process(es) would not be computed. However, this would not in the least be observable from inside the system. If the observer moved to serve you, your past would be "retroactively" computed. I'm not claiming this is very likely to be the case, since at the very least it requires an additional agent - the observer process - to cause anything to happen at all, but lazy evaluation isn't some weird ad-hoc concept; it's a basic concept in computer science that also happens to make programs shorter, a lot of the time. Hopefully not sufficiently shorter that a universe using lazy evaluation with one random point in space somewhere as the observer is less complex than one using strict evaluation. That.. would be impossible for us to detect, of course, but I believe it'd still have consequences.
2NancyLebovitz
If the universe we're living in is a work of art or a game, it's made for minds with much greater processing power than we've got. It isn't obvious that they'd be satisfied with something as crude as a video game.
4Baughn
How about a video game where you attempt to control a pre-singularity global civilization by directly playing a few thousand randomly selected humans simultaneously, while not letting this fact be noticed by the NPCs? It's interesting to wonder what sort of games post-humans might play, though I hope it won't be anything quite that ethically objectionable.
1wedrifid
Or, from the perspective of a pre-post-human, quite that dull. If I am going to play that kind of sim I'm going to pick the 'elves' faction.
1Baughn
Considering that there exist fork-lift simulation games, I hesitate to claim that anything is too dull to be made.
0wedrifid
You're serious? That scares me.
0Baughn
I think it was originally meant for training, but yes. People play it. As a game. http://www.youtube.com/watch?v=HIVFjtZzDr8
1Lightwave
It could be that it was the elves who picked the 'humans' faction.
0Psychohistorian
If you can understand how the two are truly the same, you are far wiser than anyone I've ever met, and I would very much like to subscribe to your newsletter. I hope thefirst issue explains how this dichotomy is invalid.
3wedrifid
A video game can be deterministic or not in the same way any other kind of universe can. "Video game" vs "deterministic" is just a silly comparison. I don't know what word to use in place of 'deterministic', I just don't think that one is the right one.
2Blueberry
I'm thinking "algorithmic". That is, the universe, or a video game, follows a certain algorithm to determine what happens next, whether the algorithm is the laws of physics or a computer program. Algorithms aren't necessarily deterministic: we could have a step for "generate a truly random (quantum) number".
0[anonymous]
Just plain, "no." There is, to my knowledge, exactly zero evidence indicating that the creation and execution of the laws governing the universe resembles that of video games in any way. There's some sense in which the term "system" applies to both, I admit, but that's about it, and "system" is a pretty broad word.
0Pavitra
You mean, besides the predictive power of the mathematical formalizations of Occam's Razor, as opposed to a linguistic or pathetic formulation? The universe looks very falsifiably like a computer program.
0Jack
Huh? "Metaphysics" refers to an incredibly wide variety of claims. But I'd say that metaphysics tries to answer questions about reality that aren't the kind of questions that can be answered by experimental science. Since we lack a good method for answering these questions our confidence in metaphysical claims is usually substantially lower than it is for empirical claims. But why should we think all metaphysical questions are radically different from scientific questions such that the answer to one can't influence our estimations of the other? Of hand I can't think of a number of metaphysical hypotheses that have been greatly effected by scientific knowledge and vice versa-- materialism, substance dualism, determinism and indeterminism, free will, eternalism and philosophies of time etc. In this case it seems rather obvious that if we are "living in the Matrix" the probability that the basic laws of physics are complicated rather than simple is dramatically higher.
0Pablo
I never denied that a our assessment of an empirical claim may be influenced by the metaphysical views we hold. I simply noted that, once the Matrix hypothesis is understood as a metaphysical hypothesis, it is unclear why believing that we live in the Matrix should increase our credence in the various claims of parapsychology.
0Jack
I have no idea what your argument actually is. Why does it matter whether or not the Matrix hypothesis is a metaphysical hypothesis?
2Pablo
My original comment was a reply to Mitchell Porter, who suggested that parapsychology would somehow receive support from the Matrix hypothesis. I replied by saying that this would not be true, or at least not clearly, if that hypothesis is understood as a claim about the ultimate nature of reality. To take another example, suppose someone argued Berkeleyan idealists should be more open to psychic phenomena, since we are all ideas in the mind of God. I would reply that this is not so, since the fact that the world is ultimately made of mind has in itself no implications about whether certain kinds of mental phenomena take place within that world.
1Jack
The ability of certain collections of atoms to communicate large amounts of information to other collections of atoms over vast distances without there being any detectable emissions is an incredibly complex power. Complex entities are a priori improbable compared to simple entities. You need some kind of creation mechanism to make them probable... with biological systems we have evolution, with pocket watches and jet planes you have human inventors. If you accept a metaphysical hypothesis that involves an intelligence creating the universe-- programmers or God-- you have a mechanism for making complex entities probable. That is why the Matrix hypothesis makes psychic phenomena more likely
2wedrifid
This remains the case no matter what the universe is made of. All evidence suggests that psychic mechanisms are not available to us in our current mode of existence, whatever that may be. That evidence doesn't change until you get more. At least, that seems to me to be the point ben is making.
1Jack
Yes, but not no matter how the universe was created. The Matrix hypothesis includes a claim that the universe was created by some intelligence and that makes psychic phenomena substantially more plausible. That doesn't mean all religious people have to believe in psychic phenomena or even that they should. If there is no evidence for psychic phenomena then there is not evidence for psychic phenomena. But if you think the universe was created claims of psychic phenomena should be less absurd on their face.
1wedrifid
Another position that could be taken is "the evidence suggests that if we are living in a matrix scenario then it is probably one of the ones without matrix psychic powers". That is, assuming rational reasoning without granting a counter-factual premise. The evidence can then be considered to have a fixed effect on the probability of psychic powers. Whether it causes you to also lower your probability for a Matrix or to alter your description of probable Matrix type would be considered immaterial. Again, this is just what my impression of Ben's position. He'll correct me if I'm wrong. For my part I don't care about Matrixes (especially No. 2. I walked out of that one in disgust! Actually, I do care about the 'dodge this!' line. It's infuriating.)
0Jack
This all started when Michell Porter responded to Blueberry's claim that we know, intuitively, that psychic phenomena are just not possible. I'm not quite sure I know just what Blueberry was talking about. But his estimate of the existence of psychic phenomena was zero and not just because of parapsychology's failure to provide convincing evidence but because of our understanding of the world. Mitchell provides Blueberry with a hypothesis that is consistent with what we know about the world but under which the existence of psychic phenomena is not prohibitively improbable. None of this changes the fact that finding evidence of psychic phenomenon should cause us to revise our probabilities of its existence up and that non finding evidence should cause us to revise our probabilities down. But if your probability is zero, and especially if your probability is zero for reasons other than the failure of parapsychology, a hypothesis with P>0 where P(psi) is >0 looks like information you needs to update on. Ben says it isn't clear why this is so. Well creation makes complex, unselected entities more probable. But maybe I should wait to have this argument with him. As far as the movie goes, it is all downhill right after Neo wakes up gooey pink tub and sees all the other people hooked into the Matrix. The whole movie should have taken place in the Matrix and kept us in the dark about what it really was until the very end. Would have been way cooler that way.
1wedrifid
Once your probability is zero is it even possible to update away? That'd more be 'completely discarding your entire understanding of the universe for reasons that cannot be modelled within that understanding'. If something is impossible then something else, no matter how unlikely, must be the truth. This includes the hypothesis "every thought I have that suggests the p(0) hypothesis must be true is the product of cosmic rays messing with my brain."
1Jack
I've always been really confused by this but it isn't clear that an event with P=0 is an impossible event unless we're talking about the probability of an event in a finite set of possible events. (Edit again: You can skip the rest of this paragraph and the next if you are smarter than me and already get continuous probability distributions. I'm obviously behind today.)This is how it was explained to me: Think of a dart board with a geometric line across it. That line represents probability space. An event with P=.5 is modeled by marking the middle of the line. If someone throws a dart at the line there is an equal chance that it lands at any point along the line. However, at any given point the probability that the dart lands there is zero. I think the probability of any particular complex entity, event or law existing can be said to have a probability of zero absent a creator or natural selection or some other mechanism for enabling complexity. Of course this is really counterintuitive since our evolved understanding of probability deals with finite sets of possibilities. Also it means that 'impossible' can't be assigned a probability. (Edit: Also, the converse is true. The probability that the dart lands anywhere other than the spot you pick is 1 so certainty can't be mapped as 1 either.) Also, imperfect Bayesians will sometimes assign less than ideal probabilities to things. A perfect Bayesian would presumably never wrongly declare something impossible because it could envision possible future evidence that would render the thing possible. But regular people are going to misinterpret evidence and fail to generate hypotheses so they might sometimes think something is impossible only to later have it's possibility thrown in their faces.
1Eliezer Yudkowsky
Interesting point. Since physics does appear on the surface to be continuous, I can't rule out continuous propositions. Perhaps the amended saying should read "0 and 1 are not probability masses, and 0 is not a probability density."
0Steve_Rayhawk
Oh. I was expecting your belief to be as with infinite-set atheism: that we never actually see an infinitely precise measurement.
0Eliezer Yudkowsky
We don't, but what if there are infinitely precise truths nonetheless? The math of Bayesianism would require assigning them probabilities.
0[anonymous]
And, having assigned p(A) = 0 to such an event A I will not be able to rationally update away from zero without completely discarding my former model. There is no evidence that can cause me to rationally update p(A) = 0 to something else ever. Discard it and overwrite with something completely unrelated perhaps, but never update.
0wedrifid
And, having assigned p(A) = 0 to such an event A I will not be able to rationally update away from zero without completely discarding my former model. There is no evidence that can cause me to rationally update p(A) = 0 to something else ever. Discard it and overwrite with something completely unrelated perhaps, but never update. p(A) is right there as the numerator! (But yes, I take your point and tentatively withdraw the use of 'impossible' to refer to p=0.) ETA: Well, maybe you're allowed to use some mathematical magic to cancel out the 0 if p(B) = 0 too. But then, the chance of that ever happening is, well, 0.
0Jack
Er, my bad. I missed your point. I see it now, duh. So my friend thinks something S has a probability of zero but I know otherwise and point out that it is possible give assumption which I know my friend believes has a .1 chance of being true. He says "Oh right. I guess S is possible after all." What has just happened? What do we say when we see the dart land at a specific point on the line?
2pengvado
Your friend had incorrectly computed the implications of his prior to the problem in question. On your prompting he re-ran the computation, and got the right answer (or at least a different answer) this time. Perfect Bayesians are normally assumed to be logically omniscient, so this just wouldn't happen to them in the first place. In order to specify a point on the line you need an infinite amount of evidence, which is sufficient to counteract the infinitesimal prior. (The dart won't hit a rational number or anything else that has a finite exact description.) Or if you only have a finite precision observation, then you have only narrowed the dart's position to some finite interval, and each point in that interval still has probability 0.
1wedrifid
You wasted a great gambling opportunity. Pengvado gives one good answer. I'll add that your friend saying something has a probability of zero most likely means a different thing than what a Bayesian agent means when it says the same thing. Often people give probability estimates that don't take their own fallibility into account without actually intending to imply that they do not need to. That is, if asked to actually bet on something they will essentially use a different probability figure that incorporates their confidence in their reasoning. In fact, I've engaged with philosophers who insist that you have to do it that way. "Did not! Look closer, you missed by 1/infinity miles!"
1Blueberry
Well, if you believe in Cartesian skepticism (that is, that we might be in the Matrix), then your probability for anything can't ever be zero. Say the probability that this world is an illusion within another world is epsilon: in that case anything could be true. So the lowest probability we can assign to anything is epsilon. EY has a post on how 1 and 0 aren't probabilities. If you observed psychic powers with no possibility of cheating, you should probably conclude that something metaphysically weird is going on: you're in a dream, you're insane, or there's a hole in the Matrix.
0wedrifid
Find the probability p(A | B) where B = "nothing weird like what the Cartesian sceptics talk about is going on". That gets rid of the Matrix issue. Then it is just a matter of whether or not you want to let dart players divide by infinity all willy nilly.
0Blueberry
Yes, but then we would have to give up on the scientific method anyway, because the laws of physics would be whatever the adminstrators of the Matrix felt like changing them to.
5SilasBarta
What Jack said -- you can still notice regularities in the Matrix even if it's sometimes capriciously tampered with. That is, these overlords do not automatically make our universe a high-entropy white noise bath. I discussed some of the implications of a simulated universe and the implications of the 2nd law of thermodynamics on it here.
3Jack
The universe being a simulation does not entail that the administrators of such a simulation are capricious. And that we have demonstrated the existence of a large number of natural regularities suggests they are not.
6LauraABJ
Exactly! I guess Allan needs to explain further why parapsychology is bunk. As an example, a person 'reading the mind' of another person a mile away without the emission of any kind of detectable electromagnetic wave or signal capable of traveling that far is in violation of the laws of physics as we know them (and if people did emit such signals, it would be intensively studied). For this to be true, physics itself would need to be complicated on a level that would specifically allow this phenomenon to occur, which seems very, very unlikely. To quote Michael Vassar, "Magic is the hypothesis that physics is complicated." We should expect positive results from the field of parapsychology, since so many people (in total over the years) are trying to prove it exists, and there is an extreme positive results bias. Thus by chance positive results will be obtained and published, while negative results are largely ignored or not even submitted (I assume a 'scientist' trying to prove parapsychology wants to do so, and so may only bother submitting a paper on the topic if the results are positive).
5Kaj_Sotala
-- Damien Broderick, Outside the Gates of Science
3Jack
Would it really surprise anyone here if, say, 10 percent of parapsychologists are either rigging experiments, hiding negative results or falsifying data? 20%? Thirty-seven percent.
4Kaj_Sotala
What would be the incentive? Forging results for highly public performances that allowed you to make money off people, sure. But for results published in obscure journals, when even academics in well-respected fields may need to fight tooth and claw for their next yearly funding? In a field that won't even get you the respect of most other academics, and might very well ruin your scientific reputation? Trying to prove a view that doesn't have powerful ideological backers pouring money into it the way creationists do? And with the number of fake researchers apparently staying roughly even for a period of fifty years, looking from the way the effect size hasn't changed?
7Jack
That right there is a really good point I didn't think of. As for motive, my impression is that a lot of parapsychologists are trying to demonstrate the truth of beliefs that are incredibly significant to them-- their new age spirituality is at stake. For that matter, if they've dedicated their lives to the subject. If there are no psychic phenomena they have literally spent their lives studying nothing. You might as well ask why theologians never come up with arguments disproving the existence of God. But your point about consistency makes this all moot. I'll check out the book.
6AllanCrossman
Well if they do they get called philosophers of religion instead...
2CarlShulman
To get more funding for their work, more fame within the parapsychology community, and to make it more likely that the world at large will realize the truth via "fake-but-accurate" experiments. Some parapsychologists pay for their own experiments, using resources garnered from a "day job" in some other field, but many rely on donations from wacky psi-enthusiasts (people who also get excited about ghosts, "subtle energies" and so forth), or selling psi-controlled meditation lamps. Many others think that it's critically important for mainstream funding sources to provide grants to parapsychologists (such as themselves) to do the work they find interesting and important. Under those circumstances, a psychic believer could come up with all sorts of justifications:
1CarlShulman
Note that this is a bogus calculation: it says that if there was no publication bias, so that unpublished studies were just as likely to show positive results as published ones, then adding the stated number of chance studies would "dilute" the results below a threshold significance level. But of course the whole point of publication bias is the enrichment of the file-drawer with negative results. See this paper by Scargle. You need far fewer studies in the file-drawer given the presence of bias. Further, various positive biases will be focused in the published literature, e.g. people doing outright fraud will normally do it for an audience. The number of studies needed also collapses if various questionable research practices (optional stopping, post hoc reporting of subgroups as separate experiments, etc) are used to concentrate 'hits' into some experiments while misses can be concentrated in a small file drawer. Parapsychologists counter that the few attempts to audit for unpublished studies (which would not catch everything) have not found large skew in the unpublished studies, but these inflated "fail-safe" statistics are misleadingly large regardless.
1LauraABJ
"Honorton's estimate required fourty-six unreported chance-level experiments for each of those in the meta-study, including those that themselves gave no significant support for the paranormal hypothesis." Why is this at all unlikely? This is a 52 year span of time, and who knows how many times each of these (only 62) 'scientists' ran the trials or tweaked the procedure before they decided they had a set of data worth submitting. Who knows how many people looked for these phenomena, didn't find them, and gave up without submission? Even without outright fraud (which I wouldn't doubt), people lie to themselves. I've worked with scientists who had evidence that their previously obtained results were bunk and submitted them anyway... 'maybe the retest was flawed...' The significant effect that was found may just be the threshold at which an investigator needs to see (or fake) results to submit a paper. There's the answer to the question Allan originally posed... Also, on another note, not all 'forced choice' tests are conducted in the same way. Some of them involve the person looking at the card being in the same room as the guesser, and well, it's not hard to imagine ways of getting a score above chance like that.
-1Kaj_Sotala
309 times 46 is 14 214, which divided by 52 equals approximately 273 unpublished studies per year. I haven't seen any figures on how many studies were conducted for e.g. a specific experimental paradigm in psychology during that time, so I can't say for certain how plausible this is or isn't. It does sound a bit high considering that parapsychology hasn't exactly been the best-funded field around, though it might have had more money in the 1930's. Does anyone have numbers?
3LauraABJ
I'm not saying there were 14k unpublished completed full studies, I'm suggesting that what got published was already biased. There is room for selection bias at every level of a study, including which trials and which methods are finally taken, written up, and submitted. If the 'scientists' are trying to prove that psi exists, they can find it, one or another. Fraud isn't even required, just wishful thinking. The consistency of the effect is interesting, but may only be measuring the psychological phenomenon of deliberate self-deception.-- ah we've discovered the threshold deviation from chance at which people will believe their own crap hasn't been tampered by their own meddling. Think about the alternative explanation: If the forced choice test is run properly- the subject guesses which order 5 symbols in a 25 card deck will appear before the deck has been shuffled. The deck is then shuffled by machine (or associate) in a different room, and the order of the cards are examined. Now, how do you propose the subject is entangled with the card shuffling machine and deck, without violating current physical law, such that he can predict the order? This is magical thinking, with no basis in reality as we know it. Unless there is a pattern in the card shuffling machine and some people are very aware of it due to practice with it... but that is hardly psi.
1Kaj_Sotala
I'm not saying that psi must be real, only that it seems to merit a closer look than most people in this thread have been implying. Yes, it does seem rather unlikely that psi would exist, which is why I'm still undecided myself. But the fact that we can't come up with any physical explanation for it doesn't mean that it couldn't be real. As Yvain pointed out, Newton's theories may at the time have seemed like magical thinking as well. There could be some physical mechanism we're just not aware of, but which the brain has nonetheless evolved to take advantage of. Or then it might just be a sign of our statistical methods being flawed, made worse by psi researchers being insufficiently rigorous in their methods.
8LauraABJ
"I'm not saying that psi must be real, only that it seems to merit a closer look than most people in this thread have been implying." I strongly disagree. Psi has been looked at very intensively for a very long time, and the best it can yield is that it's not completely statistically insignificant. No theories have been posed as to how it works, it hasn't been quantified (ie, how far away, in what time frame can the subject predict the future), and it cannot be demonstrated reliably and repeatedly from even a few individuals who could then be studied more elaborately. Even one person who could always predict the order of a deck of cards would be fascinating. At some point, you just have to say a line of research does not merit further study. In the mean time, giving these theories credence wastes time and resources and leads people to think they can believe anything they want about the world, including the outstanding religious dogma, since, hey, you never know.
0scav
Here's a closer look: to accept psi, you would have to reject evolutionary biology. It would be such a humungous advantage to communicate telepathically, see the future or remote locations, or manipulate the physical world by thinking, that there's no way evolution wouldn't have optimised the f* out of it by now. We don't wonder whether birds have wings, or whether dogs have a sense of smell. That we can wonder whether we might have psychic powers means we DON'T, to a very high probability indeed.
8timtyler
Cellphones are advantageous too - yet their evolution seems to be currently in progress. That should be enough to indicate why this line of argument fails.
6Kaj_Sotala
Just because an ability would be useful doesn't mean that evolution could (or would, if reaching that ability required several intermediate steps with very low fitness advantages) optimize it without limit. The ability to digest literally everything we put in our mouths would be useful as well, but the fact that we don't have that doesn't mean we need to reject evolution.
0scav
Voted up for making me think harder. I'm not talking about an ability (like digesting cellulose) which would be really advantageous but we don't have and would require a lot of unlikely steps. The non-null hypothesis of human psychic powers is that we do already have them and ancient humans did too. Yet we don't seem to have evolved psychic abilities that are even detectable by now. Compare: the abilies to cope with milk and beer in our diet have been evolving in humans since the invention of dairy farming and brewing (a few thousand years ago?) There is large population variation in these digestive abilities after that short time. Would the selection pressure in favour of telepathy be that much less than for drinking beer?
1Blueberry
The problem here is that you're assuming a) psychic abilities would have some degree of heritability, instead of being random accidents that aren't passed on genetically, and b) that psychic abilities can vary in degree, so that there could be selection pressure to make them larger, instead of being binary. Also consider that psychic abilities in small amounts could have detrimental effects on fitness: for instance, they could make you more sensitive to bad moods, more temperamental, or even insane.
0Strange7
Beer consumption has all sorts of implications for social interaction and waterborne disease, and in some environments, there are no close substitutes. Digestive efficiency is a major factor in survival, one way or another; not being able to cope with the food and drink you've got can kill you, and synthesizing a lot of tricky enzymes you don't need (or, equivalently, hosting intestinal flora which aren't pulling their weight) can also kill you. Telepathy, on the other hand, doesn't seem to involve enough of the body to have significant metabolic effects one way or another, and is unreliable and vague even for the best performers. What life-or-death/reproduce-or-don't outcomes would it be exerting selection pressure through?
0CCC
The ability to find someone willing to reproduce with you? Or a heightened ability to persuade someone to do so? Earlier warning of malicious intent from a potential murderer?
0Jiro
Small statistical effects accumulate over evolutionary timescales. Ifr telepathy is unreliable and vague, but is more reliable than chance and less vague than making stuff up, it will be selected for, even if the effects of telepathy are very difficult to detect on an individual scale.
0Vladimir_Nesov
This does mean estimating it to be much more probably real than seems reasonable at this point.
1Neil
If parapsychology is studying the patently non-existent, then the fact that parapsychologists don't typically spend their time debunking their own subject might suggest they are not up to par in some way, as a group, with "the rest of" science - unless you concede that other branches of science would also carry on in the face of total collapse in the credibility of their subject.
3Eliezer Yudkowsky
The nonexistence of psychic powers is less patently obvious than the truth of many-worlds in physics, so there is no proof that parapsychologists are less rational than average physicists. They are studying a widely despised subject, but that if anything should raise our estimate of their level. That said, it's entirely possible that, in reality, parapsychologists are lower-level. But we should not be so quick to assume this. And it remains that other sciences may also tend to contain some low-level people. Scientific protocols for saying when a theory has been verified are not supposed to rely on such things.
2Alex Flint
What is this "level" attribute you refer to? Does it mean intelligence or something more?
[-]Eliezer Yudkowsky210

Those little numbers that appear above people's heads. You can't see them?

Reply
0Strange7
Please record your "level number" observations so that we mere mortals can test whether they correlate with anything independently verifiable.
2billswift
This http://www.susanblackmore.co.uk/Articles/si87.html isn't a study, it's Susan Blackmore's article discussing 10 years of research attempting to demonstrate psi phenomena.
1AllanCrossman
OK. But the point about what we can conclude about regular science stands even if this is only mostly correct.
1CronoDAS
Not really. The ones that aren't flawed are either negative or not replicable, though.
[-]Morendil80

See also Cosma Shalizi's The Neutral Model of Inquiry.

Reply
0Douglas_Knight
moved here
[-]prase80

There's no particular reason to think parapsychologists are doing anything other than what scientists would do; their experiments are similar to those of scientists, they use statistics in similar ways, and there's no reason to think they falsify data any more than any other group. Yet despite the fact that their null hypotheses are always true, parapsychologists get positive results.

This seems self-contradictory. If they get positive results by methods we approve (as you write, nothing other that what scientists do), what entitles us to dismiss those r... (read more)

Reply
0timtyler
Parapsychology is obviously a load of old nonsense based on wishful thinking.
8prase
Not that I disagree, but when one wants to make a serious inquiry with control groups and formal analysis of evidence, "obviously" is word which should rather be omitted.
2HiddenTruth
I agree with prase and would like to point out that Academia and Science are not the same thing. Academia is the establishment based on commonly and popularly accepted scientific truth while Science is a method to achieve objective truth from empirical evidence. The use of the word "nonsense" and "absurdity" is common to Academia as well as propagators of all faith based establishments. The use of parapsychology as a control group would need more than a faith based assumption of null hypothesis being true in every case. Serious inquiry would have to be made in every case of parapsychology.
0erniebornheimer
I agree with HiddenTruth and prase. The original post is flawed, because it starts with a perfectly good idea: "if there were a group that 'did science' but was always wrong, it would be a good control group to compare to 'real science'", but then blows it by assuming parapsychologists are indeed always wrong. FWIW, I too believe parapsychologists are probably almost always wrong, but so what? Who cares what I believe? No one does, and no one should (without evidence), and that's the point.
[-]PhilGoetz70

The idea is good. I'm afraid that it may be interpreted as meaning that we need to increase our publication standards from 95% confidence intervals to 98% confidence intervals. I think scientists already have a dangerously strong bias to reject anything that fails to meet a 95% confidence interval. If someone has a good idea, with good theoretical reasoning behind it; and they run some experiments but don't hit 95%, it's still worth considering.

There are also all sorts of data-collection tasks which are routinely thrown out if they fall below 95% confid... (read more)

Reply
2Cyan
I agree that all data should be saved, and that there's much more information in 100,000 50%-confidence predictions than in a dozen 95%-confidence predictions. But ask a biologist which they'd prefer (ETA: I have actually done this, more or less) and they'll take the dozen 95%-confidence predictions, because they're just going to turn around and use bog-standard low-throughput experimental techniques to dig deeper. From the biologists' decision theory perspective, false positives are a lot more costly than false negatives.
2PhilGoetz
That's why we need to replace biologists with robots. Like this one.
2Cyan
That approach only works because yeast has been subjected to intense investigation by low-throughput techniques, providing a huge knowledge base that constrains and guides the automated investigation. (It also helps that yeast doesn't do alternative splicing.) So it's not so much "replacing" as "building upon".
[-]Alex Flint60

I would like to see more discussions on LW that, like this one, concern the rational conclusion that can be drawn from a particular body of research (as opposed to discussion of rationality itself). I will endeavour to start some such discussions.

Reply
[-]timtyler50

Re: "There's no particular reason to think parapsychologists are doing anything other than what scientists would do"

Sure there is. They are studying something that doesn't exist - so they are probably stupider than most scientists, and more likely to believe nonsense, have a relatively poor history of undestanding experimental results - and so on.

Reply
[-]arundelo50

A discussion on Hacker News contained one very astute criticism: that some things which may once have been considered part of parapsychology actually turned out to be real, though with perfectly sensible, physical causes.

What gwern said.](http://news.ycombinator.com/item?id=978927

Reply
9bigbad
As Feynman said, one of the characteristics of the truth is that, as you look more closely at it, it gets clearer. Most of the parapsych crowd tends to report results that are have a 1% probability of occurring randomly, after having done hundreds of experiments and failing to report the rest. The difference is that the level of confidence in the best experiment in a real effect doesn't scale simply with the number of experiments. A real effect should show millions-to-one odds in a few trials, once solid experimental procedures have been devised.
6Douglas_Knight
Susan Blackmore is an example of someone who did both research on psi and out of body experiences because she saw them as connected.
4arundelo
This was interesting, but it isn't quite what I'm looking for, which is cases of something being considered paranormal, and rejected by the scientific community, and then later being accepted by the scientific community and explained in a scientific way.
1Paul Crowley
Or even more, something that meets those criteria and was actually investigated by parapsychologists.
1Paul Crowley
FTR and for convenience, gwern said:
[-]LauraABJ40

If we assume that approximately p% of results are false positives, and that only positive results are published, then the question becomes how many scientists are trying to prove (and disprove) the same hypothesis. If 1000 scientists are trying to prove that Drug Y slows the progression of Alzheimer's disease, and a p of 0.01 is required for publication, then we need to see more than 10 independent publications supporting this result before we should believe it. Things would be so much easier if negative results were given as much weight as positive ones... Can anyone think of a good way of calibrating the publication bias towards positives?

Reply
4rps
This is what they do in the wretched hive of scum and villainly that is medical research: http://www.cochrane-net.org/openlearning/HTML/mod15-3.htm
[-]RobinHanson20

We could make more productive use of this control if we could compare some other methods as applied to both it and some more standard topic. So far we see that standard academic and "scientific" institutions find results even when there are none. Are there some other institutions that might do better?

Reply
1wedrifid
You would seem to be well qualified to make such a proposal. I am sure you have put thought into how a truth seeking institution could be constructed that relied significantly upon prediction markets or perhaps something similar but more specialised to the context. Ideas and or links to some suggestions you have already made?
[-]Curiouskid00

Just out of curiosity, has anybody read into the evidence for parapsychology? The best book I've found is "The Conscious Universe" by Dean Radin. There is a critical review of the book here: http://www.skepticreport.com/sr/?p=537

Reply
[-]onthefence-30

Since when is pre-judging the validity or importance of a subject--in the spirit of "it's obviously nonsense so why even bother studying it?"--considered a "scientific" stance to take? It's dogmatic comments like these that sadly lead many non-scientists to have a less than favorable view on the seeming "objectivity" of the field and its researchers.

Reply
[-]David_Gerard140

The big problem with parapsychology as a field is that science is all of a piece. Thus, physics is consistent with chemistry, biology and so on. So the question is not "what knowledge can we derive on the assumption that we know nothing?" - but "what knowledge can we derive given what we know already?" And we know really quite a lot about areas that directly impinge on this question.

Basic physics leaves it not looking good for parapsychology as a field in any way. Sean M. Carroll points out that both human brains and the spoons they try to bend are made, like all normal matter, of quarks and electrons; everything else they do is properties of the behaviour of quarks and electrons. And normal matter, made of quarks and electrons, interacts through the four forces: strong, weak, electromagnetic and gravitational. Thus either it's one of the four known forces or it's a new force, and any new force with range over 1 millimetre must be at most a billionth the strength of gravity or it will have been captured in experiments already done. So either it's electromagnetism, gravity or something weaker than gravity.

This leaves no force that could possibly account for telek... (read more)

Reply
6TheOtherDave
Do you actually believe that? That is, if a majority of scientists started instead saying "Actually, we've looked into this, here's a calculation of the expected frequency of non-fraudulent positive results from properly run parapsychological experiments given an assumption of no actual parapsychological phenomena, and here's a survey of results in the field. Notice that the actual positive results are not exceeding the expected positive results given that assumption?" (with the associated responsibility for maintaining such things instead of working on something else), you're suggesting that a majority of the folks who dismiss the objectivity of scientists to go "Oh! Well, all right, then." and decide the scientists really are objective after all? That would really surprise me, if it happened. I expect instead that the majority of those folks are far more likely to continue dismissing scientists, they'll just have some other reason for doing it.
2deeb
actually, this is precisely how I would like people to discuss parapsychology. What, are you going to defend science or rationalism using unscientific or irrational tactics just because you think that is going to work better? Even if that wasn't detrimental to your own agenda in the long run, you would need to ask yourself at that point what makes you different from any politician defending any ideology at all. Parapsychology isn't "wrong" because it is obvious to the bigwigs in your camp (the "rationalists") that it is wrong. It is "wrong" (or, unsubstantiated) because and only because positive results are not exceeding the positive results expected assuming the null hypothesis. If positive results DID exceed these, we WOULD need to recognize there is an effect. Actually, most people here would probably just see this as proof that we do indeed live in a simulation and would actually be pretty cool with that as they had half-hoped that we did all along.
1TheOtherDave
You're making a lot of assumptions about me on the basis of, as far as I can tell, no data. (Either that, or you're using "you" to refer to someone other than me.) For what it's worth, I agree that this is an excellent way to discuss unsubstantiated theories, although I would also say that after a certain point the onus is on those presenting the theory to show that their methodology and results are meaningfully different (and better) than previously disproved attempts to do the same. Otherwise, each new re-presentation of the same theory becomes, not part of the process of discovery, but rather just a tedious nuisance. What I was doubting (and still doubt) is that doing so would change the way science is thought about among those who dismiss it out of hand.
4jimrandomh
The consensus belief that parapsychology is nonsense is not a pre-judgment; it is an informed judgment based on overwhelming evidence.
1deeb
...this overwhelming evidence coming from paraphsychology studies, and parapsychology studies only. Before people did these, all we had was overwhelming anecdotal evidence in favour of parapsychology. Every culture, nay, every family is chock-full of reliable witnesses that give accounts of how they personally experienced paranormal phenomena. In the face of such persistent, recurring reports, you can hardly blame people for wanting to investigate. It is only after you do studies under laboratory conditions that you can begin to show that this anecdotal evidence is a product of selection bias. While I am personally quite convinced that selection bias is all that is needed to explain the phenomena, this doesn't take away the immense cultural significance of the phenomena that were selected in this way. In this sense, parapsychology is not "wrong", it's just cultural (as opposed to supernatural). At the end of the day, science doesn't attach value to anything. It is just capable of describing what arises from what. Meaning arises from subjective choice alone, and as humans we are much more interested in meaning and made-up patterns than in a full list of all hydrogen atoms in the biosphere, no matter how "objective".
2khafra
No, the evidence against precognition comes from overwhelming evidence in favor of a model of physics in which the arrow of time doesn't reverse. The evidence against telepathy comes from studies of communication channels between remote humans that don't show anything outside sound waves and visual-frequency electromagnetic radiation. It's the constraints imposed by an underlying model we're extremely certain of; not the direct experiments on the parapsychological theory in question.
[+]erica-50
Moderation Log
More from AllanCrossman
View more
Curated and popular this week
189Comments