orthonormal comments on Decision Theories, Part 3.75: Hang On, I Think This Works After All - Less Wrong

23 Post author: orthonormal 06 September 2012 04:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: orthonormal 15 September 2012 09:53:45PM *  1 point [-]

Sorry for the delay.

As I was saying to cousin_it, it's instructive to think about what happens when you throw a delay like this at Masquerade: it basically has to wait it out and start from scratch.

Qualitatively, Masquerade needs to go up a few steps before reaching mutual cooperation with FairBot. That kind of delay will make it "start out" at PA+4. Notably, if your SlowFairBot delays to PA+3 but tries to prove cooperation in PA+7, it'll still work.

I'm willing to accept that sometimes you run out of time to find mutual cooperation and end up mutually defecting. The property that OpportunisticBot has that I don't like is that it sometimes "regrets it has too much power"- it would have done better on my example if it had stopped searching sooner. That's not the case for Masquerade.

EDIT: Actually, as Masquerade is written, it would end up with mutual defection here. But if, instead of having all the proof searches capped at the same N, the MaskSearch capped out at a lower (but still sufficiently high) level than SlowFairBot and Masquerade's sanity check, it would mutually cooperate.