PeterS comments on The Curve of Capability - Less Wrong

18 Post author: rwallace 04 November 2010 08:22PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (264)

You are viewing a single comment's thread. Show more comments above.

Comment author: PeterS 05 November 2010 10:39:43AM *  0 points [-]

Strong recursion: Software designs new software to design newer software; money begets money begets more money. Think of the foom as compound interest on intelligence.

Suppose A designs B, which then designs C. Why does it follow that C is more capable than B (logically, disregarding any hardware advances made between B and C)? Alternatively, why couldn't A have designed C initially?

Comment author: khafra 05 November 2010 01:53:13PM 2 points [-]

It does not necessarily follow; but the FOOM contention is that once A can design a B more capable than itself, B's increased capability will include the capability to design C, which would have been impossible for A. C can then design D, which would have been impossible for B and even more impossible for A.

Currently, each round of technology aids in developing the next, but the feedback isn't quite this strong.

Comment author: shokwave 05 November 2010 06:39:45PM *  0 points [-]

As per khafra's post, though I would add that it looks likely: after all, that we as humans are capable of any kind of AI at all is proof that designing intelligent agents is the work of intelligent agents. It would be surprising if there was some hard cap on how intelligent an agent you can make - like if it topped out at exactly your level or below.