There is no global "good and fair" pure external Archimedean viewpoint. Just a flock of agents only trying to maximize their own utilities, who are all using the same base decision algorithm, and who all know it, and who understand the correlation this implies.

My understanding is that cousin_it is suggesting just such a base decision algorithm, which works like this:

• compute a global outcome that is "good and fair" for everyone who is using this decision algorithm
• choose the option that implements the above outcome
• (anyone who doesn't follow this algorithm will be left out of the "good and fair" computation and presumably will fail to maximize its utility as a result)

We can all probably see a whole bunch of difficulties here, both technical and philosophical. But Eliezer, it's not clear from reading your comment what your objection is exactly.

ETA: I just noticed that cousin it's proposed "good and fair" formula doesn't actually ensure my point above in parenthesis (that anyone who doesn't follow the decision algorithm will fail to maximize its utility). To see this, suppose in PD one of the players can choose a third option, which is not Pareto-optimal but unilaterally gives it a higher payoff than assigned by cousin_it's formula.

cousin_it, if you're reading this, please see http://lesswrong.com/lw/102/indexical_uncertainty_and_the_axiom_of/sk8 , where Vladimir Nesov proposed a notion that turns out to coincide with the concept of the core in cooperative game theory. This is necessary to ensure that the "good and fair" solution will be self-enforcing.