Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Psychohistorian comments on The AI in a box boxes you - Less Wrong

81 Post author: Stuart_Armstrong 02 February 2010 10:10AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (262)

Sort By: Best

You are viewing a single comment's thread. Show more comments above.

Comment author: Psychohistorian 02 February 2010 06:10:15PM 1 point [-]

After all, if he's inside the box, he can't let the AI out. His decision wouldn't mean anything - it's outside-Dave's choice.

I think it's pretty fair to assume that there's a button or a lever or some kind of mechanism for letting the AI out, and that mechanism could be duplicated for a virtual Dave. That is, while virtual Dave pulling the lever would not release the AI, the exact same action by real Dave would release the AI. So while your decision might not mean something, it certainly could.

This, of course, is granting the assumption that the AI can credibly make such a threat, both with respect to its programmed morality and its actual capacity to simulate you, neither of which I'm sure I accept as meaningfully possible.