"98.5% "non crackpot" cases is very different from 98.5% non-false positive."
I fully agree, and indeed most observations turns out to be easily explainable. The interesting question is not "can eye witness reports be fallible?" Of course they can. The interesting question is "is every single ufo observation completely unreliable?" The science says that 22% of thousands of observations of ufos are truly explainable by any phenomenom we now. Thus the answer to the latter question is an unevoquivally "NO".
source: http://en.wikipedia.org/wiki/Project_Blue_Book#Project_Blue_Book_Special_Report_No._14
"If a phenomenon has a real incidence rate of zero, but any false positive rate at all, then all accounts will be false positives. Suppose that the real incidence of alien visitations of earth is zero, but 0.1% of the population has experiences they interpret as signs of alien visitation, for which they cannot come up with alternative explanations. That would account for hundreds of thousands of reports of alien visitation in America, all of which would be false positives."
There's a trick here. Were not interested in eyewitnesses own ideas about whether they saw an alien spaceship or not. Were interested in the subsequent analysis performed by expert scientists and the like. All we want from the eyewitnesses are their accounts of size, shape, flight path, lights etc. Then we corroborate this with radar data and see if we can find a plausible earthly explanation. Also your scenario simply does not explain all the cases that involve radar trackings, videos and photos. The latter can be faked but it is rare that we hear about military personnel faking radar data before handing it over to their superiors, just for fun.
All we want from the eyewitnesses are their accounts of size, shape, flight path, lights etc. Then we corroborate this with radar data and see if we can find a plausible earthly explanation. Also your scenario simply does not explain all the cases that involve radar trackings, videos and photos. The latter can be faked but it is rare that we hear about military personnel faking radar data before handing it over to their superiors, just for fun.
There's a huge difference between "we can't think of a plausible earthly explanation" and "alien...
Recently I've been struck with a belief in Aliens being present on this Earth. It happened after I watched this documenary (and subsequently several others). My feeling of belief is not particular interesting in itself - I could be lunatic or otherwise psychological dysfunctional. What I'm interested in knowing is to what extend other people, who consider themselves rationalists, feel belief in the existence of aliens on this earth, after watching this documentary. Is anyone willing to try and watch it and then report back?
Another question arising in this matter is how to treat evidence of extraordinary things. Should one require 'extraordinary evidence for extraordinary claims'? I somehow feel that this notion is misguided - it discriminates evidence prior to observation. That is not the right time to start discriminating. At most we should ascribe a prior probability of zero and then do some Bayesian updating to get a posterior. Hmm, if no one has seen a black swan and some bayesian thinking person then sees a black swan a) in the distance or b) up front, what will his a posterior probability of the existence of black swans then be?