If you think there's an inconsistency in people's dismissal of UFO eyewitness accounts, I think you may not have grasped the principles of Bayesian reasoning yet.
Eliezer's introduction is here, but to add some subject relevant commentary:
Eyewitness reports of unfamiliar things tend to be much less reliable than eyewitness reports of familiar ones. If a person witnesses someone they already recognize committing a crime, for instance, their description of the perpetrator is likely to be highly accurate, whereas a person who witnesses someone they don't recognize committing a crime is likely to give a description that's extremely unhelpful, and often worse than useless.
So as a general principle, people reporting on unfamiliar phenomena tends to be weaker evidence than people reporting on familiar ones. If your girlfriend says she saw an eagle by the creek, and she often sees eagles and is well equipped to recognize them on sight, then her report is stronger evidence than that of a tourist who thinks they know what eagles look like, and saw a bird which fit their idea of what an eagle is supposed to look like, so they're pretty sure that's what it was, which is in turn stronger evidence than a report from someone who doesn't know what eagles look like at all, but saw a bird which they think fit the description of an eagle which they heard after seeing it.
This is in addition to the fact that the single standard of Bayesian evidential reasoning demands larger amounts of evidence to raise less probable events to the point of likelihood. If seeing an eagle over the creek has a prior of .02, your girlfriend's say-so is more than enough evidence to accept the proposition, unless she's unusually dishonest. If she claims to have seen a gremlin, on the other hand, the prior is going to be much, much lower; if gremlins exist at all, they must be awfully rare and elusive to have avoided reliable observation thus far. So the likelihood that the report is due to misidentification, dishonesty, confusion, or some other reason for a false report, swamps the likelihood that she really saw a gremlin, under the same standards of evidence as we would apply in the case of the eagle.
Recently I've been struck with a belief in Aliens being present on this Earth. It happened after I watched this documenary (and subsequently several others). My feeling of belief is not particular interesting in itself - I could be lunatic or otherwise psychological dysfunctional. What I'm interested in knowing is to what extend other people, who consider themselves rationalists, feel belief in the existence of aliens on this earth, after watching this documentary. Is anyone willing to try and watch it and then report back?
Another question arising in this matter is how to treat evidence of extraordinary things. Should one require 'extraordinary evidence for extraordinary claims'? I somehow feel that this notion is misguided - it discriminates evidence prior to observation. That is not the right time to start discriminating. At most we should ascribe a prior probability of zero and then do some Bayesian updating to get a posterior. Hmm, if no one has seen a black swan and some bayesian thinking person then sees a black swan a) in the distance or b) up front, what will his a posterior probability of the existence of black swans then be?