You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

ChristianKl comments on For FAI: Is "Molecular Nanotechnology" putting our best foot forward? - Less Wrong Discussion

48 Post author: leplen 22 June 2013 04:44AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (117)

You are viewing a single comment's thread. Show more comments above.

Comment author: ChristianKl 22 June 2013 07:48:16PM 7 points [-]

In the space of possible mind-designs, very few AIs care about humans as anything but raw resources.

An AGI cares about not being killed by humans.

In the space of possible mind-designs, your mind (and every human mind) is an extreme specialist in manipulating humans.

Corn manipulates humans to kill parasites that might damage the corn by a variety of ways. An entity doesn't need to be smart to be engaged in manipulating humans.

As long as humans have the kind of power over our world that they have at the moment and AGI will either be skilled in dealing with humans or humans will shut it down if there seems to be a danger of the AGI amazing power and not caring about humans.