steven0461 comments on 2012 Survey Results - Less Wrong

80 Post author: Yvain 07 December 2012 09:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (640)

You are viewing a single comment's thread. Show more comments above.

Comment author: steven0461 29 November 2012 11:28:36PM *  1 point [-]

I would agree that this explains the apparent atrocious calibration. It's worth an edit to the main post. No reason to beat ourselves up needlessly.

People were answering different questions in the sense that they each had an interval of their own choosing to assign a probability to, but obviously different people's performance here was going to be strongly correlated. Bayes just happens to be the kind of guy who was born surprisingly early. If everyone had literally been asked to assign a probability to the exact same proposition, like "Bayes was born before 1750" or "this coin will come up heads", that would have been a more extreme case. We'd have found that events that people predicted with probability x% actually happened either 0% or 100% of the time, and it wouldn't mean people were infinitely badly calibrated.

Comment author: [deleted] 30 November 2012 12:12:00AM -1 points [-]

All of that also applies to the year calibration questions in previous surveys and yet people did much better in those.

Comment author: steven0461 30 November 2012 12:48:15AM 4 points [-]

Because they weren't about events that occurred surprisingly early.