DeterminateJacobian comments on Blackmail, continued: communal blackmail, uncoordinated responses - Less Wrong

11 Post author: Stuart_Armstrong 22 October 2014 05:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (33)

You are viewing a single comment's thread.

Comment author: DeterminateJacobian 24 October 2014 12:50:15AM 0 points [-]

I was thinking about this question in regards to whether CDT agents might have a simply bayesian reason to mimic UDT agents, not only in any pre-decision signaling, but also in the actual decision. And I realized an important feature of these problems is that the game ends precisely when the agent submits a decision, which highlights the feature of UDT that distinguishes its cooperation from simple bayesian reasoning: A distinction that becomes important when you start adding qualifiers that include unknowns about other agents' source codes. The game may have as many confounders and additional decision steps before the final step, but UDT is exclusively the feature that allows cooperation on that final step.