= 27d567bc2d48d9f40e9ccc20ea370b15)
orthonormal comments on Robust Cooperation in the Prisoner's Dilemma - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (145)
This agent is unexploitable, and mutually cooperates with PrudentBot and FairBot, but it cooperates with CooperateBot (since the fixed point reduces to "P iff Box(P)", which makes the Löbian sentence valid).
Notably, this agent is exactly the same as FairBot. Clearly it satisfies P => Provable (Q). We can also check Provable(Q) =>P, since if provable(Q), then P iff Provable(P), so P. Thus P <==> Provable(Q), so it's FairBot.
Notions of sameness need to be handled with care here in general. There are lots: literally identical code is the most obvious one, then provably identical behavior in PA (against some class of reasonable agents), then provably identical behavior in PA+1 (against some class of reasonable agents)... it's sometimes necessary to use the first notion of sameness because opponents may not behave fairly, e.g. they may punish exactly one program, so in general the set of opponents you want to prove sameness with respect to also matters.
I believe this argument proves identical behavior against all modal agents, and identical behavior of modal agents against.