98.5% "non crackpot" cases is very different from 98.5% non-false positive.
I'm using a hotel lobby computer right now, and can't rewatch the video for confirmation, but I believe that this video contains a highly relevant anecdote (as well as being generally relevant.) The author of the video was witness to a phenomenon which his host believed could only be accounted for by ghost activity. The author of the video realized that the phenomenon was, in fact, caused by a fan being on which his host didn't notice. But if he hadn't been there, his host could have conveyed the story as a phenomenon that could only be explained by ghost activity, and the relevant detail, the fan, that was the real explanation of the phenomenon, would have been omitted from the account. Nobody hearing the secondhand account could have known the real explanation, only lumped it in with the expanse of possible answer space. And a normal, non-crazy person who believes they witnessed ghost activity, would most likely deny that there had been a fan on that they had failed to notice, and indeed take offense at the very suggestion, because they would interpret it as an attack on their credibility.
If a phenomenon has a real incidence rate of zero, but any false positive rate at all, then all accounts will be false positives. Suppose that the real incidence of alien visitations of earth is zero, but 0.1% of the population has experiences they interpret as signs of alien visitation, for which they cannot come up with alternative explanations. That would account for hundreds of thousands of reports of alien visitation in America, all of which would be false positives.
"98.5% "non crackpot" cases is very different from 98.5% non-false positive."
I fully agree, and indeed most observations turns out to be easily explainable. The interesting question is not "can eye witness reports be fallible?" Of course they can. The interesting question is "is every single ufo observation completely unreliable?" The science says that 22% of thousands of observations of ufos are truly explainable by any phenomenom we now. Thus the answer to the latter question is an unevoquivally "NO".
sour...
Recently I've been struck with a belief in Aliens being present on this Earth. It happened after I watched this documenary (and subsequently several others). My feeling of belief is not particular interesting in itself - I could be lunatic or otherwise psychological dysfunctional. What I'm interested in knowing is to what extend other people, who consider themselves rationalists, feel belief in the existence of aliens on this earth, after watching this documentary. Is anyone willing to try and watch it and then report back?
Another question arising in this matter is how to treat evidence of extraordinary things. Should one require 'extraordinary evidence for extraordinary claims'? I somehow feel that this notion is misguided - it discriminates evidence prior to observation. That is not the right time to start discriminating. At most we should ascribe a prior probability of zero and then do some Bayesian updating to get a posterior. Hmm, if no one has seen a black swan and some bayesian thinking person then sees a black swan a) in the distance or b) up front, what will his a posterior probability of the existence of black swans then be?