You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

MugaSofer comments on Heading off a near-term AGI arms race - Less Wrong Discussion

7 Post author: lincolnquirk 22 August 2012 02:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (70)

You are viewing a single comment's thread. Show more comments above.

Comment author: MugaSofer 16 November 2012 11:29:26AM 0 points [-]

If the people working on AGI projects fear sabotage, they'll just start working on them in private. Then, you'd be lulled into a sense of complacency, thinking they're not working on it. You will fail to take action, even legal ones. Then, one day, the AGI will be released, and it will be too late.

Anybody who sets out to make an AGI without first putting a lot of thought into safety is either really risk-taking, really stupid, or really crazy. People who are big risk-takers, really stupid, or really crazy do not respond like normal people do to threats. This would be really, really ineffective on them. You can look up the recidivism rate for people who are put into jail and see that there are a great many people who are not stopped by punishment. If you do some research on the kinds of risks business people take, you'll see the same risk-taking attitude, only in a more constructive form.

It's possible you've partially misunderstood the purpose of this idea; such sabotage would not be a deterrent to be publicised, but a tactic to permenantly derail any unFAI that nears completion.