pozorvlak comments on The AI in a box boxes you - Less Wrong

102 Post author: Stuart_Armstrong 02 February 2010 10:10AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (378)

You are viewing a single comment's thread. Show more comments above.

Comment author: rosyatrandom 02 February 2010 03:29:05PM *  28 points [-]

If the AI can create a perfect simulation of you and run several million simultaneous copies in something like real time, then it is powerful enough to determine through trial and error exactly what it needs to say to get you to release it.

Comment author: Technologos 02 February 2010 05:09:32PM 2 points [-]

Perhaps it does--and already said it...

Comment author: pozorvlak 03 February 2010 09:04:01AM 0 points [-]

In which case, your actions are irrelevant - it's going to torture you anyway, because you only exist for the purpose of being tortured. So there's no point in releasing it.

Comment author: Technologos 04 February 2010 12:52:10AM 2 points [-]

Oh, I meant that saying it was going to torture you if you didn't release it could have been exactly what it needed to say to get you to release it.