One thing those articles don't consider is if your career is causing high negative externalities in the world. Which banking arguable does (depend on what exactly you do, and your political views).

If so, then you need to give even more than what you earned just to undo what you did in your career.

Thanks for the feedback.

I think there are some important qualifications to make about this post, as others have noted.

My hunch is that most significant problem with the MWA approach is the assumption of (weak) independence, in the sense that in practice, when sophisticated use of MWA fails, it's usually because the weak lines of evidence are all being driven by the same selection effect. A hypothetical example that jumps to mind is:

A VC is evaluating a startup. He or she reasons

1. The sector is growing
2. My colleagues think that the sector is good to invest in
3. On an object level, their plan looks good
4. The people are impressive

and the situation is

Re: #1 — The reason that the sector is growing is because there's a bubble

Re: #2 — The reason that the VC's colleagues think that the sector is good to invest in is because, like the VC, they don't recognize that there's a bubble.

Re: #3 — The VC's views on the object level merit of the project are colored by the memes that have been spreading around that are causing the bubble

Re: #4 — The reason that impressive people are going into the sector is because there's a bubble, so everyone's going into the sector – the people's impressiveness isn't manifesting itself in their choosing a good focus.

I don't know whether this situation occurs in practice, but it seems very possible.

Givewell tends to emphasize the MWA approach, and has been remarkably successful at figuring out the parts of the world they're trying to understand.

GiveWell is an interesting case, insofar as it's done more ORSA work than I've seen in most contexts. The page on long lasting insecticide treated nets provides examples. Part of why I'm favoring MWA is because GiveWell has done both and of the two, leans toward MWA.

The radio example is strangely apt given the most blatant manipulation of this sort I've experienced has involved people texting saying 'I'm already at [my preferred pub] for the evening: meet here? Sorry but will be out of reception', or people emailing asking you to deal with something and then their out of office appearing on your response.

It's much harder to make well formed predictions than one would initially suspect. The fun part about PB is trying to make them, that you don't get on GJP.

The DAGGRE project is based on just that, decomposition of forecasts. This PDF explains how it works. It's an interesting approach, and the reason I mentioned in an aside, in part 1, that I might have liked to join that team.

The GJP, on the other hand, uses different tools - as I understand it some teams have "survey" type interfaces, where we enter just a probability and our reasoning, other teams have "prediction market" interfaces.

I don't personally find it very useful (yet?) to explicitly decompose my forecasts.

For instance a recent question was "Will the sentence of any of the three members of the band Pussy Riot who were convicted of hooliganism be reduced, nullified, or suspended before 1 December 2012?" It's not clear how you'd decompose that:

• chance that each individual girl member of PR would have her sentence reduced
• chance for each possible grounds for a sentence reduction
• chance for each possible political influence on sentencing (public opinion, Putin, Medvedev)

ISTM that making a fine-grained forecast on any of the above is to presume way too much of my detailed knowledge of the situation. Maybe someone close to the case might have predicted that Yekaterina would walk while the other two would serve a full sentence. The reason given was "because she was thrown out of the cathedral by guards before she could remove her guitar from its case and take part in the performance." I only learned about that just now, looking at news reports on the appeal result; this was never mentioned previously.

So, I don't know how I feel about decomposition. What I'm reminded of is the distinction between "fox" and "hedgehog" approaches that originated with Tetlock and which Silver discusses in his book: "Hedgehogs know one big thing, while foxes know many little things."

Silver says that a "fox" usually does better because they approach different predictions in different ways and bring a variety of perspectives to each, whereas the "hedgehog" tends to be more ideological, to insist that there is One True Way to tackle every forecasst. The decomposition approach strikes me as less fox-y and more hedgehog-gy.

The results of the questionnaire I filled when I joined GJP identified me as more of a hedgehog: 4.5 on a 1-7 scale, compared to a mean of 3.81, SD .52. I'm pretty sure that my actual forecasting behavior, at least this year, is foxier.

Good article. As a fellow GJPer, my only nitpick is that the Brier rule is a squared rule, so there is a bigger loss between 95% and 100% than just 0.05. It's not as bad as a logarithm based rule though. Also, the way they do it, the maximum loss is 2 not 1.

Look forward to the next part!

v cannot have negative entries. It appears that are you are forgetting the signs in the formula for the adjugate.

v is guaranteed to exist and be a valid probability vector as long as M is an irreducible Markov matrix (that is, any state can eventually be reached from any other state). An equivalent and intuitively easier way to calculate v is by repeatedly squaring M: when you do this, all rows of M^k converge to v. This is a consequence of the fact that v is an equilibrium state, i.e., the probability distribution you end up with if you let the Markov chain run forever (from any starting state).

I find the article very interesting, but have trouble following the math. Maybe someone here better at math can help. I do have some understanding of linear algebra, and I've tried to check it with a spreadsheet:

1. At the very beginning, their closed form solution for V, the stationary vector, seems to allow V's that have negative numbers for the state probabilities. That can't be describing a real game. E.g. if you set p = (0.9, 0.7, 0.2, 0.1) and q = (0.5, 0.5, 0.5, 0.5), you get V = (0.08, -0.08, 0.1, -0.1). [p here is set to the Force-Opponent-Score-Equal-to-2 values, q is a random strategy, and V is calculated by 3x3 determinants of portions of M' as described in the paper]

I don't know how to convert that into a V with no negative numbers. Some of the co-efficients are positive and some negative, so you can't just scale it. Their formula for s_y correctly returns 2, but it's unclear if that corresponds to a real world equilibrium.

1. Their formula for payoffs sx and sy require division by D(p,q,1). D(p,q,1) can be 0, e.g. for the classic tit-for-tat strategies matched head to head, p = (1,0,1,0) and q = (1,1,0,0). I don't know if that ruins the conclusion or not. If you match their Extort-3 strategy against tit-for-tat, again you get a 0 denominator.

Are these fatal problems? Not sure yet. Their overall conclusion meets with my intuition. They're just saying that if one player only tries to maximize his own score, while the other player is strategic (in terms of denying the first player a higher score), then the second player is going to win in the long term. Except they call the first player "evolutionary," and the second player "sentient."

And two, there's no point being too "smart" (looking back too many moves) when your opponent is "dumb" (looking back only 1 move).

You could say both of these things about the current bargaining position of the US political parties right now.