TheOtherDave comments on Holden's Objection 1: Friendliness is dangerous - Less Wrong

11 Post author: PhilGoetz 18 May 2012 12:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (428)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheOtherDave 18 May 2012 02:21:08PM 0 points [-]

I don't mean to imply that, given that we've performed a bottom-up extrapolation of human values into a coherent whole, that implementing that whole in a system vastly different from a human is necessarily difficult.

Indeed, by comparison to the first part, it's almost undoubtedly trivial, as you suggest.

Rather, I mean that what is at issue is extrapolating the currently instantiated value systems into "a coherent whole that can be implemented by a system vastly different from a human".

That said, I do think it's worthwhile to distinguish between "Catholicism" and "the result of extrapolating Catholicism into a coherent whole." The latter, supposing it existed, might not qualify as an example of the former. The same is true of "human value".