= 27d567bc2d48d9f40e9ccc20ea370b15)
Morendil comments on [LINK] Get paid to train your rationality - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (55)
The Good Judgment project has started publishing a leaderboard. FWIW, as of this writing I am in pole position with a "Brier score" of 0.18, with numbers 2 and 3 at 0.2 and 0.23 respectively. (I'm not sure whether other participants are also from LW.)
(ETA: dethroned! I'm #2 now, #1 has a score of .16.)
Team scores seem a bit below the best individual scores: 0.32, 0.33 and 0.36 for the best three teams.
From the emails I've been getting from the organizers, they have trouble sustaining participation from all who signed up; poor participation is leading to poor forecasting scores.
FYI the leaderboard rankings are fake, or at least generated strategically to provide users with specific information. I am near the top of my own leaderboard, while my friend sees his own name but not mine. Also, my Brier is listed at 0.19, strikingly close to yours. I wonder if they are generated with some apparent distribution.
My take is that the leader stats are some kind of specific experimental treatment they're toying with.
This is almost more interesting than the study itself. :)
Are your friend and you able to see each other's comments on predictions?
Hmm, correlation v. causation maybe? It is possible that some people were doing poorly and so started participating less?
Yes, it's possible too. I used "causing" referring to a direct link: some predictions are of the form "event X will happen before date D", and you lose points if you fail to revise your estimates as D draws nearer.
Apparently many people weren't aware of this aspect - they took a "fire and forget" approach to prediction. (That is in itself an interesting lesson.) That was before the leaderboard was set up.