Kindly comments on Raising the forecasting waterline (part 1) - Less Wrong

32 Post author: Morendil 09 October 2012 03:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (108)

You are viewing a single comment's thread. Show more comments above.

Comment author: Kindly 11 October 2012 02:11:54PM 1 point [-]

How does GJP score predictions that change over time?

Comment author: Morendil 11 October 2012 03:04:15PM *  4 points [-]

They compute your Brier score for each day that the question is open, according to what your forecast is on that day, and average over all days.

Suppose you start at 80%, six days pass, you switch to 40% three days before the deadline, and the event doesn't happen, your score is (6*(0.8)^2+3*(0.4)^2)/9 = .48, which is a so-so score - but an improvement over the .64 that you'd get if you didn't change your mind.