somervta comments on I played the AI Box Experiment again! (and lost both games) - Less Wrong

35 Post author: Tuxedage 27 September 2013 02:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (123)

You are viewing a single comment's thread. Show more comments above.

Comment author: somervta 01 October 2013 03:23:18AM 0 points [-]

I think the key difference is that the AI can convince the person. You might say that a person is fully competent to judge the Friendliness of the AI based solely on the code, and yet not want a (superintelligent) AI to get a chance to convince him, as superintelligence trumps intelligence. The difference is whether you have a superintelligence working against you.