Humans aren't just not perfect Bayesians. Very very few of us are even Bayesian wannabes. In essence, everyone who thinks that it is more moral/ethical to hold some proposition than to hold it's converse is taking some criterion other than appearent truth as normative with respect to the evaluation of beliefs.

Hmm... thanks for writing this. I just realized that I may resemble your argumentative friend in some ways. I should bookmark this.

I didn't know whether to post this reply to "Black swans from the future" or here, so I'll just reference it:

http://www.overcomingbias.com/2007/04/black_swans_fro.html#comment-65404590

Good post, Eliezer.

I've pointed before to this very good review of Philip Tetlock's book, Expert Political Judgment. The review describes the results of Tetlock's experiments evaluating expert predictions in the field of international politics, where they did very poorly. On average the experts did about as well as random predictions and were badly outperformed by simple statistical extrapolations.

Even after going over the many ways the experts failed in detail, and even though the review is titled "Everybody’s An Expert", the reviewer concludes, "But the best lesson of Tetlock’s book may be the one that he seems most reluctant to draw: Think for yourself."

Does that make sense, though? Think for yourself? If you've just read an entire book describing how poorly people did who thought for themselves and had a lot more knowledge than you do, is it really likely that you will do better to think for yourself? This advice looks like the same kind of flaw Eliezer describes here, the failure to generalize from knowledge of others' failures to appreciation of your own.

Hal, to be precise, the bias is generalizing from knowledge of others' failures to skepticism about disliked conclusions, but failing to generalize to skepticism about preferred conclusions or one's own conclusions. That is, the error is not absence of generalization, but imbalance of generalization, which is far deadlier. I do agree with you that the reviewer's conclusion is not supported (to put it mildly) by the evidence under review.

So why, then, is this blog not incorporating more statistical and collective de-biasing mechanisms? There are some out-of-the-box web widgets and mildly manual methods to incorporate that would at the very least provide new grist for the discussion mill.

The error here is similar to one I see all the time in beginning philosophy students: when confronted with reasons to be skeptics, they instead become relativists. That is, where the rational conclusion is to suspend judgment about an issue, all too many people instead conclude that any judgment is as plausible as any other.

I would love to hear more about such methods, Rafe. This blog tends to be a somewhat abstract and "meta" but I would like to do more case studies on specific issues and look at how we could come to a less biased view of the truth. I did a couple of postings on the "Peak Oil" controversy a few months ago along these lines.

Rafe, name three.

Rooney, I don't disagree that this would be a mistake, but in my experience the balance of evidence is very rarely exactly even - because hypotheses have inherent penalties for complexity. Where there is no evidence in favor of a complicated proposed belief, it is almost always correct to reject it, not suspend judgment. The only cases I can think of where I suspend judgment are binary or small discrete hypothesis spaces, like "Was it murder or suicide?", or matters like the anthropic principle, where there is no null hypothesis to take refuge in, and any position is attackable.

I have also had repeated encounters with individuals who take the bias literature to provide 'equal and opposite biases' for every situation, and take this as reason to continue to hold their initial beliefs. The situation is reminiscent of many economic discussions, where bright minds question whether the effect of a change on some quantity will be positive, negative or ambiguous. The discussants eagerly search for at least one theoretical effect that could move the quantity in a positive direction, one that could move it in the negative, and then declare the effect ambiguous after demonstrating their cleverness, without evaluating the actual size of the opposed effects.

I would recommend that when we talk about opposed biases, at least those for which there is an experimental literature, we should give rough indications of their magnitudes to discourage our audiences from utilizing the 'it's all a wash' excuse to avoid analysis.

As someone who seems to have "thrown the kitchen sink" of cognitive biases at the free will problem, I wonder if I've suffered from this meta-bias myself. I find only modest reassurance in the facts that: (i) others have agreed with me and (ii) my challenge for others to find biases that would favor disbelief in free will has gone almost entirely unanswered.

But this is a good reminder that one can get carried away...

Eliezer, I agree that exactly even balances of evidence are rare. However, I would think suspending judgment to be rational in many situations where the balance of evidence is not exactly even. For example, if I roll a die, it would hardly be rational to believe "it will not come up 5 or 6", despite the balance of evidence being in favor of such a belief. If you are willing to make >50% the threshold of rational belief, you will hold numerous false and contradictory beliefs.

Also, I have some doubt about your claim that when "there is no evidence in favor of a complicated proposed belief, it is almost always correct to reject it". If you proposed a complicated belief of 20th century physics (say, Bell's theorem) to Archimedes, he would be right to say he has no evidence in its favor. Nonetheless, it would not be correct for Archimedes to conclude that Bell's theorem is therefore false.

Perhaps I am misunderstanding you.

"Nonetheless, it would not be correct for Archimedes to conclude that Bell's theorem is therefore false."

I think this is a terrible hypothetical to use to illuminate your point, since most of Archimedes' decision would be based on how much evidence is proper to give to the source of information he gets the theorem from. I would say that, for any historically plausible mechanism, he'd certainly be correct in rejecting it.

Rooney, where there isn't any evidence, then indeed it may be appropriate to suspend judgment over a large hypothesis space, which indeed is not the same as being able to justifiably adopt a random such judgment - anyone who wants to assign more than default probability mass is being irrational.

I concur that Bell's theorem is a terrible hypothetical, because the whole point is that, in real life, without evidence, there's absolutely no way for Archimedes to just accidentally hit on Bell's theorem - in his lifetime he will not reach that part of the search space; anything he tries without evidence will be wrong. It's exactly like saying, "But what if you did buy the winning lottery ticket? Then it would have high expected utility."

I don't think that 50% is a distinguished threshold for probability. Heck, I don't think 1 in 20 is a distinguished threshold for probability. The point of a binary decision space is that it is small and discrete, not that it is binary.

Eliezer, I think we are misunderstanding each other, possibly merely about terminology.

When you (and pdf) say "reject", I am taking you to mean "regard as false". I may be mistaken about that.

I would hope that you don't mean that, for if so, your claim that "no evidence in favor -> almost always false" seems bound to lead to massive errors. For example, you have no evidence in favor of the claim "Rooney has string in his pockets". But you wouldn't on such grounds aver that such a claim is almost certainly false. The appropriate response would be to suspend judgment, i.e., to neither reject nor accept. Perhaps I am not understanding what counts as a suitably "complicated" belief.

As for Archimedes meeting Bell's theorem, perhaps it was too counter-factual an example. However, I wouldn't say it's comparable to the "high utility" of the winning lottery ticket: it the case of the lottery, the relevant probabilities are known. By contrast, Archimedes (supposing he were able to understand the theorem) would be ignorant of any evidence to confirm or disconfirm it. Thus I would hope that he would refrain from rejecting it, merely regarding it as a puzzling vision from Zeus, perhaps.

The probability that an arbitrary person has string in their pockets (given that they're wearing pockets at the time) is knowable, and given no other information we could say that it's X%. The proper attitude towards the claim "Rooney has string in his pockets" is that it has about an X% chance of being true. (Unless we get other evidence to the contrary--and the fact that someone made the claim might be evidence here.)

Say X is 3%. Then I should say that Rooney very likely has no string in his pockets. Say X were 50%. *Then* I should say that there's an even chance Rooney has string in his pockets. In neither case am I withholding judgment. Given what you've said, Rooney, I think you might say that the latter *would* be withholding judgment? Or would you say that neither assertion is justified, and in that case, what does it mean to withhold judgment?

I think there's a post somewhere last year where Eliezer went over these points.

Pdf, maybe you're referring to "I Don't Know"?

Rooney, I think you're interpreting "reject" as "state with certainty that it is not true" or "behave as if there is definite evidence against it". Whereas what I mean is that one should bet at odds that are tiny or even infinitesimal when dealing with an evidentially unsupported belief in a very large search space. You have no choice but to deal this way with the vast majority of such beliefs if you want your total probabilities to sum to 1.

By "suspending judgment" I mean neither accepting a claim as true, nor rejecting it as false. Claims about the probability of a given claim being true, helpful as they may be in many cases, are distinct from the claim itself. So, pdf, when you say "The proper attitude towards the claim "Rooney has string in his pockets" is that it has about an X% chance of being true", where X is unknown, I don't see how this is materially different from saying "I don't know if Rooney has string in his pockets", which is to say that you are (for the moment at least) suspending judgment about whether the claim (call it 'string') is true or false. And where X is estimated (on the basis of some hypothetical evidence) to be (say) .4, what is the proper attitude toward 'string'? Saying "'string' has a 40% chance of being true" doesn't answer the question, it makes a different claim, assigning probability. In such situations, the rational course of action is to suspend judgment about 'string'. You may of course hold beliefs about the probability of 'string' being true and act on those beliefs accordingly (by placing real or hypothetical bets, etc.), but in such cases you're neither accepting nor rejecting 'string'.

You have no choice but to bet at some odds. Life is about action, action is about expected utility, and expected utility demands that you assign some subjective weighting to outcomes based on how likely they are. Walking down the street, I offer to bet you a million dollars against one dollar that a stranger has string in their pockets. Do you take the bet? Whether you say yes or no, you've just made a statement of probability. The null action is also an action. Refusing to bet is like refusing to allow time to pass.

Nor do I permit probabilities of zero and one. All belief is belief of probability.

I have to bet on every possible claim I (or any sentient entity capable of propositional attitudes in the universe) might entertain as a belief? That is highly implausible as a descriptive claim. Consider the claim "Xinwei has string in his pockets" (where Xinwei is a Chinese male I've never met). I have no choice but to assign probability to that claim? And all other claims, from "language is the house of being" to "a proof for Goldbach's conjecture will be found by an unaided human mind"? If Eliezer offers me a million dollars to bet on someone's pocket-contents, then, yes, if the utility is right, I will calculate probabilities, meager though my access to evidence may be. But that is not life. The null action may be an action, but lack of belief is not a belief. "I've never thought about it" is not equivalent to "it's false" or "it's very improbable".

(Did Neanderthals assign probabilities, or was it a module that emerged at about the same time as the FOXP gene? Or did it have to wait until the invention of games of chance in western Europe? Is someone who refuses to bet on anything for religious reasons ipso facto irrational?)

And you don't take the belief "2 + 2 = 4" as having probability of 1? Nor "2 + 2 = 5" as 0?

I'm off, out of ISP range for a day, so I won't reply for a bit. Cheers.

Michael Rooney: I don't think Eliezer is saying that it's invalid to say "I don't know." He's saying it's invalid to have as your position "I should not have a position."

The analogy of betting only means that every action you take will have consequences. For example, the decision not to try to assign a probability to the statement that Xinwei has a string in his pocket will have some butterfly effect. You have recognized this, and have also recognized that you don't care, and have taken the position that it doesn't matter. The key here is that, as you admit, you have taken a position.