RandomThinker comments on Raising the forecasting waterline (part 2) - Less Wrong

15 Post author: Morendil 12 October 2012 03:56PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (22)

You are viewing a single comment's thread. Show more comments above.

Comment author: Morendil 12 October 2012 06:40:52PM *  3 points [-]

The DAGGRE project is based on just that, decomposition of forecasts. This PDF explains how it works. It's an interesting approach, and the reason I mentioned in an aside, in part 1, that I might have liked to join that team.

The GJP, on the other hand, uses different tools - as I understand it some teams have "survey" type interfaces, where we enter just a probability and our reasoning, other teams have "prediction market" interfaces.

I don't personally find it very useful (yet?) to explicitly decompose my forecasts.

For instance a recent question was "Will the sentence of any of the three members of the band Pussy Riot who were convicted of hooliganism be reduced, nullified, or suspended before 1 December 2012?" It's not clear how you'd decompose that:

  • chance that each individual girl member of PR would have her sentence reduced
  • chance for each possible grounds for a sentence reduction
  • chance for each possible political influence on sentencing (public opinion, Putin, Medvedev)

ISTM that making a fine-grained forecast on any of the above is to presume way too much of my detailed knowledge of the situation. Maybe someone close to the case might have predicted that Yekaterina would walk while the other two would serve a full sentence. The reason given was "because she was thrown out of the cathedral by guards before she could remove her guitar from its case and take part in the performance." I only learned about that just now, looking at news reports on the appeal result; this was never mentioned previously.

So, I don't know how I feel about decomposition. What I'm reminded of is the distinction between "fox" and "hedgehog" approaches that originated with Tetlock and which Silver discusses in his book: "Hedgehogs know one big thing, while foxes know many little things."

Silver says that a "fox" usually does better because they approach different predictions in different ways and bring a variety of perspectives to each, whereas the "hedgehog" tends to be more ideological, to insist that there is One True Way to tackle every forecasst. The decomposition approach strikes me as less fox-y and more hedgehog-gy.

The results of the questionnaire I filled when I joined GJP identified me as more of a hedgehog: 4.5 on a 1-7 scale, compared to a mean of 3.81, SD .52. I'm pretty sure that my actual forecasting behavior, at least this year, is foxier.

Comment author: RandomThinker 14 October 2012 07:37:17AM 0 points [-]

I also scored slightly on the hedgehog scale. I think people who like to "think about thinking" are already slightly hedgehog. True foxes don't believe in such grand theories.