BethMo comments on Rebelling Within Nature - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (37)
I'm still going through the Sequences too. I've seen plenty of stuff resembling the top part of your post, but nothing like the bottom part, which I really enjoyed. The best "how to get to paperclips" story I've seen yet!
I suspect the problem with the final paragraph is that any AI architecture is unlikely to be decomposable in such a well-defined fashion that would allow drawing those boundary lines between "the main process" and "the paperclip subroutine". Well, besides the whole "genie" problem of defining what is a Friendly goal in the first place, as discussed through many, many posts here.