Giles comments on Criticisms of intelligence explosion - Less Wrong

15 Post author: lukeprog 22 November 2011 05:42PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (123)

You are viewing a single comment's thread. Show more comments above.

Comment author: Giles 24 November 2011 04:25:45AM 0 points [-]

This is a good point. I think there's one reason to give special attention to the intelligence explosion concept though... it's part of the proposed solution as well as one of the possible problems.

The two main ideas here are:

  • Recursive self-improvement is possible and powerful
  • Human values are fragile; "most" recursive self-improvers will very much not do what we want

These ideas seem to be central to the utliity-maximizing FAI concept.