Nisan comments on Open Thread: June 2010 - Less Wrong

5 Post author: Morendil 01 June 2010 06:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (651)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nisan 02 June 2010 03:16:39PM 0 points [-]

"Duality" has become more complex because we're now talking about a more complex problem — a version of Counterfactual Mugging where you believe that all superintelligent agents are trustworthy. The old version of duality suffices for the ordinary Counterfactual Mugging problem.

My thesis is that there's always a symmetry in the space of black swans like No-mega.

In the case currently under consideration, I'm assuming Omega's spiel goes something like "I just flipped a coin. If it had been heads, I would have predicted what you would do if I had approached you and given my spiel...." Notice the use of first-person pronouns. Omega* would have almost the same spiel verbatim, also using first-person pronouns, and make no reference to Omega. And, being non-deceptive, it would behave the way it says it does. So it wouldn't condition on your being muggable by Omega.

You could object to this by claiming that Omega actually says "I am Omega. If Omega had come up to you and said....", in which case I can come up with a third notion of duality.

Comment author: cousin_it 02 June 2010 05:21:54PM *  1 point [-]

If Omega* makes no reference to the original Omega, I don't understand why they have "opposite behavior with respect to my status as being counterfactually-muggable" (by the original Omega), which was your reason for inventing "duality" in the first place. I apologize, but at this point it's unclear to me that you actually have a proof of anything. Maybe we can take this discussion to email?