You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Mark_Friedenbach comments on What are your contrarian views? - Less Wrong Discussion

10 Post author: Metus 15 September 2014 09:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (806)

You are viewing a single comment's thread.

Comment author: [deleted] 15 September 2014 04:15:09PM *  18 points [-]

AI boxing will work.

EDIT: Used to be "AI boxing can work." My intent was to contradict the common LW positions that AI boxing is either (1) a logical impossibility, or (2) more difficult or more likely to fail than FAI.

Comment author: Jayson_Virissimo 15 September 2014 06:26:28PM 6 points [-]

"Can" is a very weak claim. With what probability will it work?

Comment author: D_Malik 15 September 2014 06:44:32PM 1 point [-]

It seems unlikely that the first people to build fooming AGI will box it sufficiently thoughtfully.

I think it's likely to work if implemented very carefully by the first people to build AGI. For instance, if they were careful, a team of 100 people could manually watch everything the AI thinks, stopping its execution after every step and spending a year poring over its thoughts. With lots of fail-safes, with people assigned to watch researchers in case they try anything, with several nested layers of people watching so that if the AI infects an inner layer of people, an outer layer can just pull a lever and kill them all, etc. And with the AI inside several layers of simulated realities, so that if it does bad things in an inner layer we just kill it, and so on. Plus a thousand other precautions that we can think up if we have a couple centuries. Basically, there are asymmetries such that a little bit of human effort can make it astronomically more difficult for an AI to escape. But it seems likely that we won't take advantage of all these asymmetries, especially if e.g. there's something like an arms race.

(See also this, which details several ways to box AIs.)

Comment author: [deleted] 15 September 2014 09:43:51PM 0 points [-]

Seems like an ad hominum attack. Why wouldn't the people working on this be aware of the issues? My contrarian point is that people concerned about FAI should be working on AI boxing instead.