Eugine_Nier comments on Robust Cooperation in the Prisoner's Dilemma - Less Wrong

69 Post author: orthonormal 07 June 2013 08:30AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (145)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eugine_Nier 12 June 2013 07:59:04AM -1 points [-]

This does have the problem that in practice it cashes out as defect against anyone who is sufficiently smarter than you.

Comment author: orthonormal 13 June 2013 04:15:49PM 4 points [-]

It depends on how they use their intelligence. For instance, consider the variant of FairBot that appears in the last Masquerade post, which oscillates between seeking proofs of its opponent's cooperation and defection, up to a high threshold.

The original FairBot, or PrudentBot, or Masquerade, all reach mutual cooperation with this variant (call it ToughButFairBot), despite its undecidability in general up to PA+N. That's because if it finds a proof at a lower level, it acts on that immediately, so you can figure out what it does in those particular cases without climbing too high up the tower of formal systems.

The upshot is that you can have higher levels of strategy when necessary, without sacrificing your ability to provably act on lower levels in other cases. So this principle doesn't cash out as "defect against anyone smarter than you", but rather as "defect against anyone who refuses to let you figure out how they're going to respond to you in particular".