You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Dreaded_Anomaly comments on Open thread, May 17-31 2013 - Less Wrong Discussion

2 [deleted] 17 May 2013 01:47PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (311)

You are viewing a single comment's thread.

Comment author: Dreaded_Anomaly 22 May 2013 10:59:56PM *  6 points [-]

Norbert Weiner, a mathematician from MIT, postulated unfriendly AI in 1949.

The possibility of learning may be built in by allowing the taping to be re-established in a new way by the performance of the machine and the external impulses coming into it, rather than having it determined by a closed and rigid setup, to be imposed on the apparatus from the beginning.

...

Moreover, if we move in the direction of making machines which learn and whose behavior is modified by experience, we must face the fact that every degree of independence we give the machine is a degree of possible defiance of our wishes. The genie in the bottle will not willingly go back in the bottle, nor have we any reason to expect them to be well disposed to us.