= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on Thomas C. Schelling's "Strategy of Conflict" - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (148)
Actually the book tells about a psychological experiment conducted similarly to the situation I described, but there the subjects were told outright by the trustworthy experimenter that their partner couldn't learn their whereabouts. But I still think that average-rational humans would often fall for the radio trick. Expected utility suggests you don't have to believe your partner 100%; a small initial doubt reinforced over a day could suffice.
And yep, the problem of making your precommitments trustworthy is also discussed in much detail in the book.
There may be an interesting connection between this example and AIs knowing each other's source code. The idea is, if one AI can unilaterally prove its source code to another without the receiver being able to credibly deny receipt of the proof, then it should change its source code to commit to an unfair agreement that favors itself, then prove this. If it succeeds in being the first to do so, the other side then has no choice but to accept. So, Freaky Fairness seems to depend on the details of the proof process in some way.
Unless, of course, you already know that most AIs will go ahead and "suicidally" deny the unfair agreement.