BethMo comments on Rebelling Within Nature - Less Wrong

23 Post author: Eliezer_Yudkowsky 13 July 2008 12:32PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: BethMo 17 May 2011 06:43:47AM 0 points [-]

I'm still going through the Sequences too. I've seen plenty of stuff resembling the top part of your post, but nothing like the bottom part, which I really enjoyed. The best "how to get to paperclips" story I've seen yet!

I suspect the problem with the final paragraph is that any AI architecture is unlikely to be decomposable in such a well-defined fashion that would allow drawing those boundary lines between "the main process" and "the paperclip subroutine". Well, besides the whole "genie" problem of defining what is a Friendly goal in the first place, as discussed through many, many posts here.