Sewing-Machine comments on Cryptographic Boxes for Unfriendly AI - Less Wrong

24 Post author: paulfchristiano 18 December 2010 08:28AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (155)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 18 December 2010 03:25:06PM *  4 points [-]

Moral value can manipulate your concerns, even as you prevent causal influence. Maybe the AI will create extraordinary people in its mind, and use that as leverage to work on weak points of your defense. It's just too difficult, you are bound to miss something. The winning move is not to play.

Comment author: [deleted] 18 December 2010 06:21:21PM 4 points [-]

Separate "let's develop a theory of quarantines" from "let's implement some quarantines."

It's just too difficult, you are bound to miss something.

Christiano should take it as a compliment that his idea is formal enough that one could imagine proving that it doesn't work. Other than that, I don't see why your remark should go for "quarantining an AI using cryptography" and not "creating a friendly AI."

The winning move is not to play.

Prove it. Prove it by developing a theory of quarantines.

Comment author: Vladimir_Nesov 18 December 2010 07:22:39PM 1 point [-]

Separate "let's develop a theory of quarantines" from "let's implement some quarantines."

I agree.