Thank you for your detailed contribution!
Ah, but if you didn't make a prediction on a question, you still got a Brier score for it — GJP gave you the median score of the group members who did make a prediction.
Hm, yes, that makes sense as these scores are listed in grey in my coulmn too. I just didn't make that connection and can't seem to remember that it was explicitly explained that way - but maybe I misunderstood which averaging applies to which. Esp. before actually seeing the UI.
So the trick is to choose questions where you expect to do better than the median predictor, even if those questions look difficult...
Yes. That seems like another sensible strategy to game it,
I recommend that you send this as a reply to one of the last mails. They seem to really read them.
Follow-Up to Good Judgment Project, Season Three.
During the last forecasting season I took part in the Good Judgment Project (GJP; see also the blog) and this is a short summary of my participation (actually triggered by hamnox comment).
The GJP estimates world events like
To participate in that study one has to register (can't remember where exactly I stumbled over the link, possibly the one at the top). And one has to do an preparatory online course and one has to pass an online test. At least I had to complete it. Whether the result affected my assignment to any group I can't say. The course explains the scoring and gives recommendations for making good forecasts (choose forecasts one has an edge in, estimate early, update often, do post-mortems). The test seems to test for calibration and accuracy by asking for known (mostly political) events and whether one is sure about them.
The current forecasting season started in November 2014 and has just ended. I invested significantly less then half an hour a week on 8 questions of about 100 (and thus less than I projected in an early questionaire). I did 2 to 15 updates for these questions and I earned a score in the middle range (mostly due to getting hit by an unexpected terrorist attack). As I just learned I was assigned to the study condition were I could neither see the total group estimate nor the estimates of the other group members - only their comments. I was somewhat disappointed by this as I had hoped to learn something from how the scores developed. Too bad I wasn't in a prediction marked group. But I hope to get the study results later.
I will not take part in further rounds as I shy the effort for the types of forecasts which are mostly political. They are political because the sponsor (guess who) is interested mostly in political events - less in economical, environmental, scientific or other types. But I enjoyed forecasting artic ice cap melting and ebola - and netted a better than average score on that.
The scoring - at least in this group - is interesting and uses an averaged Brier Score - averaged over a) all forecast questiontion and b) within a question over all the days for which a forecast is provided. I intended to game that by betting on questions that a) I could forecast well and b) that had an expected reliable outcome. Sadly there were few of type a.
From this experience I learned that