Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Baughn comments on Amputation of Destiny - Less Wrong

26 Post author: Eliezer_Yudkowsky 29 December 2008 06:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (66)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Baughn 24 September 2014 11:51:02AM *  1 point [-]

He is not the only one who'd be horrified. Median humanity scares me.

Would it be fair to ignore them and make a geek/LW-specific CEV?

No, but I'm not sure how much I care.

Comment author: [deleted] 24 September 2014 02:34:16PM 0 points [-]

It's not as if average-utilitarianism is the only possible answer. Real life today already allows for subcultures who enjoy diverging from most of humanity. Any goal system halfway worth implementing would also allow for such.

Comment author: Baughn 24 September 2014 02:49:11PM *  1 point [-]

According to us. How certain are you that the CEV of all of humanity agrees?

The fact that they exist today isn't an answer; it could be (and to some degree is) because eradicating them would be too costly, morally or economically.

Comment author: [deleted] 24 September 2014 03:16:50PM 0 points [-]

How certain are you that the CEV of all of humanity agrees?

Since CEV is, AFAICT, defined by the mean of all humans' utility functions (derived from reflective equilibrium), it disagrees by definition. But CEV is not divine revelation: it's just the best Eliezer could do at the time. As we learn more about the evaluative and social cognition underlying our "moral" judgements, I expect to be able to dissolve a priori philosophizing and build a better Friendly goal-system than CEV.

Of course, this is because I don't believe in normative ethics in the normal sense (post-hoc consequentialist based on Peter Railton's style of moral realism, blah blah blah), so I'm sure this will wind up a massive debate at some point. I strongly doubt the armchair philosophers will go down without a fight.