JoshuaZ comments on 1001 PredictionBook Nights - Less Wrong

51 Post author: gwern 08 October 2011 04:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (49)

You are viewing a single comment's thread. Show more comments above.

Comment author: JoshuaZ 12 October 2011 01:25:19PM 1 point [-]

A related issue I seem to be having on Prediction Book. Judging from my past predictions I'm generally underconfident by quite a bit. According to the outcome graph no matter what percentage I estimate for something to happen, as long as I am giving it more than 50%, it seems to happen around 88% of the time. Slightly more for the >90% section, and similarly for the rounding to 100%, so my probability are at least weakly correlated in the right direction. Any thoughts on what I can do to better calibrate myself?

Comment author: gwern 12 October 2011 03:03:19PM *  3 points [-]

You don't have very many predictions judged, so I'm not sure how reliable your worry is - only 8-11 predictions for each decile seems quite possible to just get lucky. Assuming that's not the case, you could try mechanically bumping up every prediction by 10% and see what happens.

Comment author: JoshuaZ 12 October 2011 08:59:45PM 1 point [-]

Yes, that seems like a good point. I'm especially worried that my long-term predictions will turn out to be much too overconfident. And presumably I won't see that until at least a while has gone by.