If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.
What was the result of the IARPA prediction contest (2010-2015)?
Below I present what seem to me very basic questions about the results. I have read vague statements about the results that sound like people are willing to answer these questions, but the details seem oddly elusive. Is there is some write-up I am missing?
How many teams were there? 5 academic teams? What were their names, schools, or PIs? What was the “control group”? Were there two, an official control group and another group consisting of intelligence analysts with access to classified information?
Added: perhaps a third control group "a prediction market operating within the IARPA tournament."
What were the Brier scores of the various teams in various years?
When Tetlock says that A did 10% better than B, does he mean that the Brier score of A was 90% of the Brier score of B?
I can identify 4 schools involved, composing 3-4 teams:
GJP (Berkeley: Tetlock, Mellers, Moore)
DAGGRE/SciCast (GMU: Twardy, Hanson)
Michigan, MIT - 2 teams or a joint team?
In Superforcasting Tetlock writes that the main documents of the comparison between the GJP forecasters against the intelligence analysts with access to classified information is classified. Tetlock doesn't directly say something about that comparison but reports in his book that a newspaper article says that the GJP forecasters were 30% better (if I remember right).