Kingreaper comments on Ben Goertzel: The Singularity Institute's Scary Idea (and Why I Don't Buy It) - Less Wrong

32 Post author: ciphergoth 30 October 2010 09:31AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

Show:

You are viewing a single comment's thread. Show more comments above.

Comment author: Kingreaper 31 October 2010 11:20:47AM 6 points [-]

If it has a bug in its utility function, it won't want to fix it.

If it has a bug in its bug-detection-and-fixing techniques, you can guess what happens.

So, no, you can't rely on the AGI to fix itself, unless you're certain that the bugs are localised in regions that will be fixed.

Comment author: timtyler 31 October 2010 12:08:21PM -1 points [-]

So: bug-free is not needed - and a controlled ascent is possible.

The unreferenced "hubris verging on sheer insanity" asumption seems like a straw man - nobody assumed that in the first place.

Comments (432)