= 27d567bc2d48d9f40e9ccc20ea370b15)
jimrandomh comments on What is Eliezer Yudkowsky's meta-ethical theory? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (368)
As I understand it, Eliezer has taken the position that human values are too complex for humans to reliably formalize, and that all formalizations presented so far are or probably are incorrect. This may explain some of your difficulty in trying to find Eliezer's preferred formalization.
One project is the descriptive one of moral psychology and moral anthropology. Because Coherent Extrapolated Volition begins with data from moral psychology and moral anthropology, that descriptive project is important for Eliezer's design of Friendly AI. Certainly, I agree with Eliezer that human values are too complex to easily formalize, because our terminal values are the product of millions of years of messy biological and cultural evolution.
"Morality" is a term usually used in speech acts to refer to a set of normative questions about what we ought to do, or what we ought to value. Even if you're an ethical reductionist as I am, and reduce 'ought' such that it is a particular species of 'is', there are lots of ways to do that, and I'm not clear on how Eliezer does it.
Moral psychology and anthropology are pretty useless, because morality is too complex for humans to manually capture with accuracy, and too fragile to allow capturing without accuracy. We need better tools.
Your first claim doesn't follow from the (correct) supporting evidence.
In actually implementing a CEV or other such object, it's true that one daren't program in specific object-level moral truths derived from human study. The implementation should be much more meta-level, in order to not get locked into bad assumptions.
However, you and I can think of classes of possible implementation failures that might be missed if we had too naive a theory of moral psychology. Maybe a researcher who didn't know about the conscious/unconscious divide at all would come to the same implementation algorithm as one who did, but it's not out of the question that our limited knowledge could be relevant.