I don't follow. As a project progresses it seems common to acquire new information and continuously update your valuation of the project.

I taught game theory at Princeton and wish I'd seen this explanation beforehand, excellent framing.

In the territory, bad event happens [husband hits wife, missile hits child, car hits pedestrian]. There is no confusion about the territory: everyone understands the trajectories of particles that led to the catastrophe. But somehow there is a long and tortuous debate about who is responsible/to blame ["She was wearing a dark hoodie that night," "He should have come to a complete stop at the stop sign", "Why did she jaywalk when the crosswalk was just 10 feet away!"].

The problem is that we mean a bunch of different things simultaneously by blame/responsibility:

1. Causality. The actual causal structure of the event. ["If she'd worn a reflective vest this wouldn't have happened," "If your left headlight wasn't broken you'd have seen her."]
2. Blame. Who should be punished/shamed in this situation. This question already branches into a bunch of cruxes about the purpose and effectiveness of punishment.
3. Responsibility. What is the most effective way of preventing such events in the future? ["If we passed a law that all pedestrians wear reflective vests it would halve incidents like this", "How about we institute mandatory pedestrian-sighting courses for drivers, and not blame the victim?"]

People argue about the same event with different causal models, different definitions of blame, and different notions of responsibility, and the conversation collapses. Fill in your own politically-charged example. 

Setting the zero point seems to be one "move" in this blame game [if the default is that all drivers take pedestrian-sighting courses, then you're to blame if you skipped it. if the default is that all pedestrians must wear reflective vests, then you're to blame if you didn't wear one.]

I don't have a complete reply to this yet, but wanted to clarify if it was not clear that the position in this dialogue was written with the audience (a particularly circumspect broad-map-building audience) in mind. I certainly think that the vast majority of young people outside this community would benefit from spending more time building broad maps of reality before committing to career/identity/community choices. So I certainly don't prescribe giving up entirely.

ETA: Maybe a useful analogy is that for Amazon shopping I have found doing serious research into products (past looking at purchase volume and average ratings) largely unhelpful. Usually if I read reviews carefully, I end up more confused than anything else as a large list of tail risks and second-order considerations are brought to my attention. Career choice I suspect is similar with much higher stakes.

Seeing patterns where there are none is also part of my writing process.

This paper of mine answers exactly this question (nonconstructively, using the minimax theorem).

I feel there is an important thing here but [setting the zero point] is either not the right frame, or a special case of the real thing, [blame and responsibility are often part of the map and not part of the territory] closely related to asymmetric justice and the copenhagen interpretation of ethics. 

Afaict, the first simple game is not the prisoner's dilemma, nor is it zero-sum, nor is the prisoner's dilemma zero-sum.

This is not intended as a criticism in any way, but this post seems to overlap largely with https://www.lesswrong.com/posts/k9dsbn8LZ6tTesDS3/sazen. 

[Edit: After looking at the timestamps it looks like that post actually came out after, anyway it might be an helpful alternative perspective on the same phenomenon.]

Is it just me or are alignment-related post titles getting longer and longer?