thomblake comments on Holden's Objection 1: Friendliness is dangerous - Less Wrong

11 Post author: PhilGoetz 18 May 2012 12:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (428)

You are viewing a single comment's thread. Show more comments above.

Comment author: thomblake 23 May 2012 02:15:58PM 0 points [-]

I am saying that a CEV that extrapolated human morality would generally be utilitarian, but that it would grant a utility value of zero to satisfying what I call "malicious preferences."

This is because I think that a CEV of human morality would find the concept of malicious preferences to be immoral and discard or suppress it.

Zero is a strange number to have specified there, but then I don't know the shape of the function you're describing. I would have expected a non-specific "negative utility" in its place.

Comment author: Ghatanathoah 23 May 2012 04:08:47PM -1 points [-]

Zero is a strange number to have specified there, but then I don't know the shape of the function you're describing. I would have expected a non-specific "negative utility" in its place.

You're probably right, I was typing fairly quickly last night.