wedrifid comments on Welcome to Less Wrong! (2012) - Less Wrong

25 Post author: orthonormal 26 December 2011 10:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (1430)

You are viewing a single comment's thread. Show more comments above.

Comment author: wedrifid 25 April 2012 12:37:09AM 2 points [-]

i.e., to rewrite its own code, and possibly rebuild its own hardware, in order to become smarter and smarter -- then its intelligence will grow exponentially, until it becomes smart enough to easily outsmart everyone on the planet.

Recursively, not necessarily exponentially. It may exploit the low hanging fruit early and improve somewhat slower once those are gone. Same conclusion applies - the threat is that it improves rapidly, not that it improves exponentially.

Comment author: Bugmaster 25 April 2012 12:42:48AM 0 points [-]

Good point, though if the AI's intelligence grew linearly or as O(log T) or something, I doubt that it would be able to achieve the kind of speed that we'd need to worry about. But you're right, the speed is what ultimately matters, not the growth curve as such.