Eliezer_Yudkowsky comments on Moral Error and Moral Disagreement - Less Wrong

14 Post author: Eliezer_Yudkowsky 10 August 2008 11:32PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (125)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 12 August 2008 12:37:35PM 5 points [-]

I think that views are being attributed to me that I do not possess - perhaps on the Gricean notion that if someone attacks me for holding these views, I ought to hold them; but on a blog like this one, that leaves you prey to everyone else's misunderstanding.

I do not assert that all humans end up in the same moral frame of reference (with regard to any particular extrapolation method). I do think that psychological unity is typically underestimated, and I have a hard time taking modern culture at face value (we're the ancient Greeks, guys, not a finished product) - read Tooby and Cosmides's "The Psychological Foundations of Culture" to get a picture of where I'm coming from.

But if you read "Coherent Extrapolated Volition" you'll see that it's specifically designed to handle, among other problems, the problem of, "What if we don't all want the same thing?" What then can an AI programmer do that does not constitute being a jerk? That was my attempt to answer.

Michael Vassar has tried to convince me that I should be more worried about a majority of humanity being "selfish bastards" as defined in CEV. I have a couple of thoughts on this problem, may post on them later.