Right, I'm not denying that overconfidence bias exists and is a coherent concept. I was trying to point out that when we reinterpret the predictions to be more easily verified/falsified (like I have been doing before adding some of the predictions to PredictionBook) the prediction is transformed in a way that doesn't necessarily preserve the original framing (whether positive or negative), so it would be unclear from the proposition we would be scoring whether or not the original predictor was under or overconfident.
Right; in fact, we can see pretty easily that just transferring the predictor's probability to our better more precise predictions will intrinsically increase their apparent confidence. The point of our versions is to be narrower and better defined, and so we will judge our prediction correct in fewer states of the world than they would have judged their own prediction (independent of any biases); fewer states of the world means less confidence is justified (P(A&B) <= P(A)).
Of course, in practice, due to the many biases and lack of experience afflic...
Going through expiring predictions reminded me. Just as we did for 2010 and 2011, it's time for LessWrong to make its beliefs pay rent and give hostages to fortune in making predictions for events in 2012 and beyond.
Suggested topics include: Methods of Rationality updates (eg. "will there be any?"), economic benchmarks (price of gold has been an educational one for me this past year), medical advances (but be careful not to be too optimistic!), personal precommitments (signing up for cryonics?), being curmudgeonly about self-improvement, making daring predictions about the future of AGI, and so on.
As before, please be fairly specific. I intend to put most predictions on PredictionBook.com and it'd be nice if they weren't too hard to judge in the future.
(If you want advice on making good predictions, I've tried to write up a few useful heuristics I've learned. So far in the judging process, I've done pretty well this year, although I'm a little annoyed I got a Yemen prediction right but for the wrong reasons.)