= 27d567bc2d48d9f40e9ccc20ea370b15)
TimS comments on Holden's Objection 1: Friendliness is dangerous - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (428)
Why do you think this part is difficult? If there are any coherent human value systems, then it seems very plausible (if difficult to build) for any agent to implement the value system, even if the agent isn't human.
Put slightly differently, my objection to the possibility of a friendly-to-Catholicism AI is that Catholicism (like basically all human value systems) is not coherent. If it were proved coherent, I would agree that it was possible to build an AI that followed it (obviously, I'd personally oppose building such an AI - it would be an act of violence against me)
I don't mean to imply that, given that we've performed a bottom-up extrapolation of human values into a coherent whole, that implementing that whole in a system vastly different from a human is necessarily difficult.
Indeed, by comparison to the first part, it's almost undoubtedly trivial, as you suggest.
Rather, I mean that what is at issue is extrapolating the currently instantiated value systems into "a coherent whole that can be implemented by a system vastly different from a human".
That said, I do think it's worthwhile to distinguish between "Catholicism" and "the result of extrapolating Catholicism into a coherent whole." The latter, supposing it existed, might not qualify as an example of the former. The same is true of "human value".