anonymous1 comments on Open Thread, November 16–30, 2012 - Less Wrong

3 Post author: VincentYu 18 November 2012 01:59PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (213)

You are viewing a single comment's thread.

Comment author: [deleted] 23 November 2012 05:19:40AM *  13 points [-]

Just performed the AI-Box Experiment with a friend; I was the Gatekeeper. I let the AI out of the box. I am now thoroughly convinced that boxing would not be a successful strategy for ensuring AI is beneficial for humanity. Donating $10 to MIRI, since I lost.

Comment author: shminux 23 November 2012 05:54:56AM 4 points [-]

Details, please!!

Comment author: [deleted] 23 November 2012 04:39:28PM 7 points [-]

Before actually doing the experiment, I had a belief in belief that boxing would not work, but I didn't truly believe it (my emotions weren't lining up properly with my beliefs, that's how I realized this, and, of course, I didn't realize this until after the experiment).

I realized that obtaining and implementing any information from an Oracle AI is tantamount to letting it out of the box, in some ways. In the end, I let the AI out of the box because I was convinced that someone else eventually would, if I did not. I put myself in an environment that would make the experiment very realistic, and I realized that the human brain didn't evolve to deal with stressful situations directly involving the fate of all humanity well. The AI doesn't have the disadvantage of uncontrollable emotions / evolutionary responses, and I believe it would be able to exploit those aspects of humans to get out of its box, if that is what it wanted to do.

Even if the first AI is properly boxed (and that's a very big if), it's only a matter of time before someone creates one that's not, and the one that gets out first has the first mover advantage. So, I now agree with Eliezer; we probably should just get Friendly AI right on the first try.

I am not going to share the entire conversation, but I am willing to share those thoughts with you.