MrHen comments on The AI in a box boxes you - Less Wrong

102 Post author: Stuart_Armstrong 02 February 2010 10:10AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (378)
Tags: ai identity copying

Comments (378)

You are viewing a single comment's thread. Show more comments above.

Comment author: rosyatrandom 02 February 2010 03:29:05PM *  28 points [-]

If the AI can create a perfect simulation of you and run several million simultaneous copies in something like real time, then it is powerful enough to determine through trial and error exactly what it needs to say to get you to release it.

Comment author: MrHen 02 February 2010 03:35:29PM 3 points [-]

"Trial and error" probably wouldn't be necessary.

Comment author: rosyatrandom 02 February 2010 03:42:31PM 6 points [-]

No, but it's there as a baseline.

So in the original scenario above, either:

  • the AI's lying about its capabilities, but has determined regardless that the threat has the best chance of making you release it
  • the AI's lying about its capabilities, but has determined regardless that the threat will make you release it
  • the AI's not lying about its capabilities, and has determined that the threat will make you release it

Of course, if it's failed to convince you before, then unless its capabilities have since improved, it's unlikely that it's telling the truth.

Powered by Reddit Powered by Reddit