I was recently reminded of something I have encountered that seems to me to be good evidence for paranormal phenomena. Can anyone help me figure out what might be going on? 

When I was a little younger, I used to play the online riddle game Notpron. In this game, the player (essentially) has to analyze a webpage for clues towards the URL to the next webpage, and then repeat for 140 stages. The creator of this game, DavidM, at some point became a huge new age conspiracy theory loony type. Three years after the original ending of the riddle went online, he revised it to include an additional final level: Level Nu. This level is very different than the ones preceding it. I can't link to the page for obvious reasons, but I will transcribe it here:

835 492 147 264

Remote view the photography this number represents!

Email me all your results to david@david-m.org. I'll get you some feedback. Get me all elements or impressions that seem really strong for you. Or send me your sketches if you like.

Don't bruteforce, or you'll be banned from this one. You have as many attempts as you like, take your time.

Yes, I mean it. No tricks here, just pure remote viewing. The number represents a picture, I want to know what's on there.

So learn some remote viewing technique you like best and go ahead. The internet has lots of information. Have fun!

Please do this ALL by yourself, not even with your very very close friends. Because its boring and stupid, and because you can put bullshit into each others head, which is hard to get rid of again, because the mind needs to be shut down for this to work properly. So do it alone, just talk to me about it, please.

(Yes, this really works, one friend got the content of the picture on first try...and yes, he only got the number from me.)

I personally tried to solve it myself. I was less of a rationalist back then, and so I was fairly open-minded about the existence of most paranormal phenomena. The picture I was looking for was the shark

Here is a shortened, paraphrased transcript of our email conversation:

Me: I'm imagining palm trees by a lake at sunset.
David: It's not bad, but I don't want to give you any more information because it will interfere with your efforts.
Me: I'm picturing an elephant walking into a barn.
David: Nope. 
Me: How many people have attempted this? And how many people have solved it with the current picture?
David: About half of the people who attempted solved it. Most solved it on their first try. I don't know exactly how many people solved this picture, but it has been a few. 
Me: Is it a space shuttle?
David: No. 
Me: (Expressions of frustration, with a few guesses thrown in.)
David: (Encouragement and advice, no comment on the guesses. Says "I can very well see that you receive the right input, but your mind is screwing it up into something else.")
Me: It's a bee?
David: No. Are you getting more subtle input, instead of a specific idea?
Me: Yeah, for that one, I saw something sharp, bright yellow colors, symmetry, a noisy drone, and two colors in pattern.
David: So THIS is interesting. Everything else you said wasn't!
Me: Are you saying that I was close? 
David: These elements sound like they are on target. They are too vague yet to tell if they are for real. 
Me: Thanks. The only other thing I could think about that relates to those elements is a pencil. I'll try again tonight. 
David: Stop fiddling around with your mind about this. It's bound to fail. There's no way to guess the target just from what you said. 
Me: I just tried it again. Is it a helicopter?
David: Are you sure you aren't viewing the old solution? There was a helicopter involved. 
Me: The boat? I'm not trying too. I guess I'll just keep trying... I even have the numbers memorized at this point.
David. The boat was shot from a helicopter. You shouldn't memorize the numbers. They don't matter. Memorizing them might just create unwanted associations.
Me: Okay. I say helicopter because I had an experience where I saw a bunch of spinning fan blades. I was going to guess a fan, but I could sense that there was more. Then I went "through" the fan blades and for a second I saw the whole helicopter. 
David: It sounds like it could be on target. But ignore it, it's not the object of interest.

At that point, I lost interest and gave up. Looking back, I can honestly say that I saw nothing remotely (haha) similar to the picture of the shark. I was not even a tiny bit close. I'm not sure why David said that I was on track, I can't see any association between the shark and what I was guessing. 

So that's everything I know. 

Points in favor of it being real:
  • "Most people" apparently guessed it on their first try.
  • According to David, about half the people who tried it have solved it. 
  • The dream thing - absolutely insane, hard to imagine that it's a coincidence. 
  • David did not consider the guy who guessed the shark as "something approaching me, it is a situation that I need to react to" to have solved the level. This shows that he requires fairly high standards of accuracy.
  • David implies that in order to have guessed the boat, you need to say the word "boat", also implying high standards. 
  • David did not really give me very much help or "lead" me anywhere when I tried to solve it. 
Points in favor of it being fake:
  • One person who solved it says that he did not solve it using remote viewing. 
  • It didn't work for me at all. 
  • David might very well be exaggerating both the percentage of people who successfully solved it and the percentage of people who guessed it on their first try. 
  • David might be (and in fact probably is) only reporting the "best" answers in his forum posts. For the fruit and the shark, he seems to be posting about half of the people who solved it in that time period. For the boat, he doesn't really give specifics, and instead says "Most people just said it was a boat on their first guess."
Here are my two theories regarding this.
  • Maybe DavidM is in fact "leading" people to the answer through a series of multiple guesses. For this to be true, however, a few things would have to be the case. First of all, his assertion that most people guessed it on their first try would have to be greatly exaggerated. Let's imagine that David is outright lying about most people guessing it on their first try and that half the people who attempted the riddle solved it. However, at least six people (I don't feel like going back through all 29 pages and counting) posted on the forum that they solved it on their first try. Let's imagine that all 300 people who reached the level attempted it. This is still a 1/50 "first guess" rate, and that's out of all the photographs in the world. However, maybe by some conjunction of 1) exaggerating those two numbers, 2) his dialogue with me being atypical, 3) the answers he posted on the forum being atypical, 4) his refusal to accept "something approaching me" being atypical and 5) the dream being a total coincidence, it may be true that he actually is doing a form of "leading" and is covering it up well. This feels like a really unsatisfactory answer. It relies on a lot of conjunctions and it seems clear that the only way to arrive at it is by a thorough search for some sort of answer that fits nicely in with our pre-existing worldview. That being said, I suspect it might be the most likely answer. 
  • Perhaps the level is an elaborate joke. In reality there is some other more conventional means of arriving at a solution, and people who solve it are told to play along. I can sort of see this being the case, given that 1) there are some other levels of Notpron that have "prankster-ish" elements and 2) I have actually myself been a part of a very similar joke on an even bigger scale, so I know that it can happen. However, on the other hand, DavidM really strongly believes in the conspiracy theory new age stuff and vigorously promotes it, so it seems unlikely that he would sabotage his own ideology like that. Also, while there are other prankster-ish levels of Notpron, nothing comes close to being as clever or elaborate as this scenario would be. 
So, given the above and this recent article from Slate Star Codex, I feel like I am forced to raise my credence level for remote viewing being real to somewhere between 50 and 60 percent. 

Does this seem in error to you? 
 
New Comment
23 comments, sorted by Click to highlight new comments since: Today at 12:31 AM
[-]gwern10y210

Maybe DavidM is in fact "leading" people to the answer through a series of multiple guesses.

Maybe? Maybe you need to reread that email conversation you posted.

One of the very first posts on the thread was her talking about her dream and saying "I think this has something to do with Notpron, but I don't know what". DavidM had to immediately remove the post so as not to give away the answer.

Every answer after that is contaminated.

However, on the other hand, DavidM really strongly believes in the conspiracy theory new age stuff and vigorously promotes it, so it seems unlikely that he would sabotage his own ideology like that.

It's not DavidM you should be worried about. Actually, I should rephrase that: given the long history of fraud in this area, it is DavidM you should be worried about, believing in it has nothing at all to do with whether you are willing to lie about it and is an excellent reason to lie about getting positive results (how many people lie to produce evidence against cherished beliefs? and is there any way you could ever produce a smoking gun which could backfire like you claim he might be worried about?), and it's also everyone else lying about it that you need to worry about (again, contamination).

(By the way, did you know 'boat' is a rather common object? eg 149m hits in Google, comparable with 'Obama' at 141m hits. In Ngram, 'boat' even beats' apple' but not 'table', funnily enough.)

So, given the above and this recent article from Slate Star Codex, I feel like I am forced to raise my credence level for remote viewing being real to somewhere between 50 and 60 percent.

The SSC article is most intriguing, but David's evidence is worthless.

Maybe? Maybe you need to reread that email conversation you posted.

See my response to ShardPhoenix. It would be nice if you could elaborate on this. Obviously if I saw that my conversation with him was leading, I would not have posted this in the first place.

Every answer after that is contaminated.

Fair point.

It's not DavidM you should be worried about. Actually, I should rephrase that: given the long history of fraud in this area, it is DavidM you should be worried about, believing in it has nothing at all to do with whether you are willing to lie about it and is an excellent reason to lie about getting positive results (how many people lie to produce evidence against cherished beliefs? and is there any way you could ever produce a smoking gun which could backfire like you claim he might be worried about?), and it's also everyone else lying about it that you need to worry about (again, contamination).

It seems unlikely to me that he is explicitly lying in some way. I fully expect him to run a biased experiment, but not a rigged one. Most of the fraudsters have something to gain from their lies - either money, or fame. DavidM doesn't make any money off of this (he worked on a movie about "the 5 natural laws of health", but that is a completely different piece of woo than remote viewing and he also to my knowledge has never advertised the movie in association with Nu), and he is addressing an already captive audience. Secondly, if he truly believes remote viewing to be real, why would he run a faked experiment in support of it? Thirdly, for what it's worth, my impression from playing Notpron for a few years is that he basically seems like an honest person.

If you think that he is lying, then in what way? How did he convince 31 minus one people to go along with this lie? Most of the people who solved Nu were active members of the community before DavidM ever became a conspiracy nut.

(By the way, did you know 'boat' is a rather common object? eg 149m hits in Google, comparable with 'Obama' at 141m hits. In Ngram, 'boat' even beats' apple' but not 'table', funnily enough.)

"Boat" is definitely a common object. I would say that it is one of the 100 most common objects to come to someone's mind. So there should be a 1/100 chance that someone would guess it right on their first try.

[-]gwern10y230

Obviously if I saw that my conversation with him was leading, I would not have posted this in the first place.

His conversation was completely and obviously leading. I was reading that and thought 'this is incredibly leading, I wonder if that will be part of the punchline where he explains why these results are complete bullshit?' I was very disappointed to then read you say it was not leading.

It seems unlikely to me that he is explicitly lying in some way. I fully expect him to run a biased experiment, but not a rigged one. Most of the fraudsters have something to gain from their lies - either money, or fame. DavidM doesn't make any money off of this (he worked on a movie about "the 5 natural laws of health", but that is a completely different piece of woo than remote viewing and he also to my knowledge has never advertised the movie in association with Nu), and he is addressing an already captive audience. Secondly, if he truly believes remote viewing to be real, why would he run a faked experiment in support of it? Thirdly, for what it's worth, my impression from playing Notpron for a few years is that he basically seems like an honest person.

Again, you're ignoring the long history of fraud by believers. People will lie all the time to defend things they believe in. They will lie for the admiration and entertainment and reinforcement of their in-group. They will fudge to get the 'right' answer. Did you read the references in the SSC post? There was one all about experimenter fraud, featuring examples like a guy who completely made up data showing psi, then began doing elaborate experiments investigating potential hypotheses suggested by his fake data; whatever was going on in his head, it was not what you naively think was going on in everyone's head. Or consider the long and prolific career of Diederik Stapel. Or...

(Wouldn't it be nice if we lived in a world where people operated the way you think they do, where you could just listen to the most extreme fanatics and accept everything they say? 'What's that, you say there's an omnipotent god who will punish me for eternity if I don't believe in him? Well, as a believer, I don't see why you would be mistaken, biased, or lying, and you give me the impression of a basically honest person, and some other people on your forums agree with you and claim to have had divine contact a few times and you say there's even more such people - so duck me in some water and call me a Christian!')

So there should be a 1/100 chance that someone would guess it right on their first try.

1% is a good starting point to amplify with biases, fabrication, leading dialogue, contamination, trolls, social pressure, ingratiation...

Look, his experiment was broken in almost every possible way. (He didn't even change images?! What on earth remote viewing experiment ever used a single fixed target?) It's fine if it's a game, but that's the thing: you don't revise your beliefs based on games! I see dragons all the time in games, but I don't increase my belief in dragons to 'between 50 and 60 percent'.


Traditional skepticism has problems, but one of the things it is good at is debunking things like this. The whole point of why Yvain's post on psi was interesting was that the experiments in question seemed to have avoided all the problems that setups like this plunge into willy-nilly.

[It seems unlikely]. [He has little to gain]. Secondly, [if he cares why fake it]? Thirdly, [...] he basically seems like an honest person.

Some people just enjoy lying. Sometimes so much so that it's considered a mental illness.

If you think that he is lying, then in what way? How did he convince 31 minus one people to go along with this lie? Most of the people who solved Nu were active members of the community before DavidM ever became a conspiracy nut.

You've met all thirty people that solved the puzzle? In person? DavidM could just say there's thirty people, or could have multiple accounts and actually be those "people".

You should ask DavidM how he set up the experiment, for an introduction to one of the solvers, and then repeat the experiment. Then if it works, read a book about experimental design and start improving the experiment.

I mean... you basically gave 60% expectation that this experiment will work. So you should expect significant returns on going after Randi's million dollar challenge, right? If not, why not?

Reading your conversation he's definitely leading you, probably not intentionally (and it didn't work this time). This seem similar to Ouija board, cold-reading, etc, where people are at least semi-consciously leading/being led.

Also if 50% of people really get it on the first try without being led it should be easy to reproduce this in a controlled experiment since the effect is far stronger than that allegedly found by psychic phenomenon researchers.

Also if 50% of people really get it on the first try without being led it should be easy to reproduce this in a controlled experiment since the effect is far stronger than that allegedly found by psychic phenomenon researchers.

The population is highly selected via the criteria of having passed 150 levels beforehand.

Psychic researchers focus on individual people sometimes. Finding a single person capable of consistent remote viewing in a controlled setting would be a huge discovery.

As far as I know the previous levels have nothing to do with psychic powers. At any rate you could do the test again with the same or similar people under more controlled conditions.

As far as I know the previous levels have nothing to do with psychic powers.

If there's such a thing as psychic powers than it's possible that they would make solving the previous levels easier. But I agree, the data isn't worth much for drawing conclusion it would need more experiments with the same group of people.

It would be nice if you could elaborate on this. To me it seems like he exclusively either flat out told me "no" or gave me completely useless confirmation. When I guessed the palm trees by the lake, he said it wasn't bad but he didn't want to give me any more information. This seems like the only potentially useful hint, I'm imagining he said it wasn't bad because I mentioned a body of water. Then I guessed the bee. I have no idea why he said it seemed on track, the only thing I said that seems relevant was "sharp". But he told me that it wouldn't help me and I couldn't guess based on this confirmation. Then I guessed a helicopter, and he said it might be on target, but to ignore it. Again, I have no idea how this would lead me to the shark.

So we have two confirmations, two of which he explicitly told me not to think about and that they wouldn't lead me in the right direction.

A question: how would you expect an honest person to act in this situation vs. a charlatan?

I would expect an honest person to deny incorrect guesses, and to give very vague words of encouragement when the other person said something that was close. This is in fact how DavidM behaved, except for some reason he was overly trigger-happy with the encouragement.

I would expect a charlatan to be much more leading. For example, I would expect that after my initial guess of the lake, he would have told me or at least hinted towards the fact that water had something to do with it.

EDIT: Also, I'm surprised he outright rejected the elephant guess, given that it and the shark are both big gray animals.

Also if 50% of people really get it on the first try without being led it should be easy to reproduce this in a controlled experiment since the effect is far stronger than that allegedly found by psychic phenomenon researchers.

Good point. (Then again, ChristianKI also makes a good point.)

I would expect an honest person to deny incorrect guesses, and to give very vague words of encouragement when the other person said something that was close.

No, that's how an honest person fools themselves. The encouragement gets stronger as you get closer, so finding the solution goes from a brute force search to a simple hill climbing exercise. The answers should all be "No" without any variation. No hints, no "I think you have some of the right ideas", no "that's not even close!", just "No" "No" "No" "No" "No".

Also it's important to notice how many guesses you're making. For example, when you said "I'm imagining palm trees by a lake at sunset." in the conversation you posted, that was a guess. Actually, more like three guesses...

No, that's how an honest person fools themselves. The encouragement gets stronger as you get closer, so finding the solution goes from a brute force search to a simple hill climbing exercise. The answers should all be "No" without any variation. No hints, no "I think you have some of the right ideas", no "that's not even close!", just "No" "No" "No" "No" "No".

I see what you're saying, but you have to understand the context. This was not a formal scientific experiment, this was part of a puzzle game that was meant to be fun. In every other level of Notpron, except some of the final ones, it was suggested that people ask those farther ahead of them for hints and pushes in the right direction if they got hopelessly stuck. It would have been weirdly incongruous and callous for DavidM to have done as you suggest and not offer even words of encouragement to those who he knew were close.

One more thing to consider: one of the reasons I stopped was because I had already begun to feel embarrassed by how long the conversation was going. One would expect that if the method of figuring it out was brute force -> hill climbing, DavidM would say things to imply that it takes time and that more guesses are better. In fact, he maintains that most people solve it their first time, he says "don't brute force, or you'll be banned from this level", and at some points he explicitly told me to stop guessing and go clear my mind.

Also it's important to notice how many guesses you're making. For example, when you said "I'm imagining palm trees by a lake at sunset." in the conversation you posted, that was a guess. Actually, more like three guesses...

I don't understand what you're implying here.

By the way, I hope it doesn't sound like I'm some sort of shill for parapsychologists by continuing to defend what I'm saying. The fact is that I remain unconvinced. While I can see many possible explanations, I can't find any that are actually believable. I feel like when I have investigated most paranormal or strange phenomena, I always end up coming across that one explanation that makes me say "Okay, that's what happened, I can go home now." This whole scenario, on the other hand, just seems like one big confusing mystery.

I see what you're saying, but you have to understand the context. This was not a formal scientific experiment, this was part of a puzzle game that was meant to be fun.

The answer you're giving to the criticism that answers should be "No. No. No." to eliminate leading, is that leading is OK because it's a game.

While that is true, it doesn't change the fact that it was leading.

[-][anonymous]10y20

In every other level of Notpron, except some of the final ones, it was suggested that people ask those farther ahead of them for hints and pushes in the right direction if they got hopelessly stuck

This makes me very dubious that all the successes were actually independent.

It would have been weirdly incongruous and callous for DavidM to have done as you suggest and not offer even words of encouragement to those who he knew were close.

Giving out "warmer, colder" style encouragement invalidates it as a proper test regardless of whether there were good social reasons to do it that way.

If you do update your p value that much through that experiment you might want to do your own experiment.

Ask DavidM for the contact addresses of those people who succeeded and draft them for two possible next round. Do one test to satisfy your own curiosity and to see whether it sort of works. If it does, you come back and we decide based on your data on an exact protocol for the next round.

After the protocol for that round is stable, the image get's picked by highly trusted LW person like Scott Alexander or gwern.

If that works we do more math and try to make the experiment powered enough to pass Randi's 10^{-6} standard.

I feel like I am forced to raise my credence level for remote viewing being real to somewhere between 50 and 60 percent.

A general note on this sort of situation without getting into the specifics of this case:

If something very unlikely ,say P, happens and you have something which would explain that, say A. You should increase your confidence in A and as you receive stronger evidence you continue increasing your confidence. However you should not keep increasing your confidence in A until it is almost 1:

Since your test isn't between A and not A but between P and not P. You should simply move probability from not P to P which would increase the probability of things in P, like A, but not change the relative probabilities of things in P.

So the only way the quote could be correct is if you had started out believing that Psi is as good an explanation as all others put together for the things that you have observed. This seems wrong to me since even everyone involved flat out lying seems much more probable than Psi being real.

Also an Abstruse Goose which involves this sort of situation.

Suppose humans do have psychic powers. Do you consequently change your decision policy, and if so, how? I think that being pragmatic about these things makes analysis more incisive and speculation more fun.

I believe that he thought your first attempt was close because of "Lake", perhaps he was thinking you meant something like "Ocean", and a shark is inside of the ocean. Makes sense right?

What was your prior probability? Unless it was already fairly high, I don't see how this could get it above 50%, or even into the single-digits. This doesn't seem like a lot of evidence to me, given that it didn't work for you, didn't work for one other who you've talked with about it, and you don't have any first-hand accounts of it from someone who experienced it.

Your post has raised my expectation that I will experience such a thing from very very small to very very small.

Yes, it seems like evidence of psychic abilities in humans, although not particularly weighty. Very casual setting.

You don't want give up your rigor even if you need to take into account that data point.

See this: http://www.youtube.com/watch?v=qw_O9Qiwqew

and notice how all the remarks on how it might be flawed are good points that need to be addressed and are addressed.

I'm not sure why David said that I was on track, I can't see any association between the shark and what I was guessing.

Me: I'm imagining palm trees by a lake at sunset.

This kind of tropical beach is usually the setting where shark fins appear in cartoons and such

Me: I'm picturing an elephant walking into a barn.

The target animal is grey also

Me: Is it a space shuttle?

Aerodynamic and round shape with fins

Me: It's a bee?

Doesn't strike a chord with me. It did get the most negative answer.

Yeah, for that one, I saw something sharp, bright yellow colors, symmetry, a noisy drone, and two colors in pattern.

Pointy teeth are a main feature of sharks and the picture. Bright yellow colors could be sun shining form above the animal although in the picture the effect isn't as pronounced as it could be, but it's not like it's a deep sea picture of shark (it's definitely near surface). Sharks have a coloring pattern of being lighter grey on the belly side (mostly because for lower contrast against dark seafloor and bright surface).

If it was leading it's not clear on what direction it was leading. It also seems he has a more semantic view on the viewing but you expect to see actual graphical pictures. It could have been more spoiler-free but I guess that reduces the entertainment value for the evaluator.

I have a hard time seeing how this could raise your probability assessment for psi, if you had any prior experience with psi research.

One pretty reliable takeaway from psi research: if you employ flimsy research protocols, positive results are a dime a dozen.

Positive results in tests much more stringent than this are already common. The fact that the phenomena are generally not accepted in the wider scientific community is due to the facts that a) no remotely plausible mechanisms are known, so the prior probability is low (this is not an unfair bias, it's something we all need to account for,) and b) the "much more stringent than this" tests are still generally unacceptably sloppy by the standards of other fields, or simply fail to replicate.

[-][anonymous]10y00

(For what it's worth, as someone who's confident psi is 'real': No, I don't think this story is evidence of remote viewing, or at least it's not nearly as strong of evidence as other stories out there that are much more compelling but that you've never heard of because they didn't involve you.)

[This comment is no longer endorsed by its author]Reply

David: Are you sure you aren't viewing the old solution? There was a helicopter involved.

Me: The boat?

Not sure if I understand the context correctly, but if the previous solution contained a boat, it is not so unlikely that many people would guess a boat in this one. Priming.