Dmytry comments on Complexity based moral values. - Less Wrong

-6 Post author: Dmytry 06 April 2012 05:09PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (100)

You are viewing a single comment's thread. Show more comments above.

Comment author: Dmytry 06 April 2012 07:17:17PM *  0 points [-]

I think if you get paperclips slightly wrong, you get a crash of some kind. If I get a ray-tracer slightly wrong, it doesn't trace electrons instead of photons.

edit: To clarify. It's about definition of person vs definition of paperclip. You need a very broad definition of person for FAI, so that it won't misidentify a person as non-person (misidentifying dolphins as persons won't be a big problem), and you need a very narrow definition of paperclip for uFAI, so that a person holding two papers together is not a paperclip. It's not always intuitive how broad definitions compare to narrow in difficulty, but it is worth noting that it is ridiculously hard to define paperclip making so that a Soviet factory anxious to maximize the paperclips would make anything at all, while it wasn't particularly difficult to define what a person is (or to define what 'money' are so that capitalist paperclip factory would make paperclips to maximize profit).