You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Armok_GoB comments on Non-orthogonality implies uncontrollable superintelligence - Less Wrong Discussion

14 Post author: Stuart_Armstrong 30 April 2012 01:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread.

Comment author: Armok_GoB 02 May 2012 09:28:11PM 0 points [-]

Even if this was the case, by murder-pill logic a papercliper would stop self improving just below the relevant "superintelegence" threshold.

Comment author: Eugine_Nier 02 May 2012 11:47:47PM 2 points [-]

Assuming it knew where that threshold was ahead of time.

Comment author: Armok_GoB 03 May 2012 09:57:28PM 0 points [-]

Well, yea. Probably a bunch of more complicated techniques it could still use in that case thou. Have some specific ideas, but debating over technicalities of irrelevant counterfactuals is not my idea of fun so I wont mention them.