wuwei comments on Hacking the CEV for Fun and Profit - Less Wrong

52 Post author: Wei_Dai 03 June 2010 08:30PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (194)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 03 June 2010 09:43:44PM 12 points [-]

I can see why you might feel that way, if this was just a technical flaw in CEV that can be fixed with a simple patch. But I've been having a growing suspicion that the main philosophical underpinning of CEV, namely preference utilitarianism, is seriously wrong, and this story was meant to offer more evidence in that vein.

Comment author: wuwei 03 June 2010 11:18:33PM *  2 points [-]

CEV is not preference utilitarianism, or any other first-order ethical theory. Rather, preference utilitarianism is the sort of thing that might be CEV's output.

Comment author: Wei_Dai 04 June 2010 10:17:10AM 6 points [-]

Obviously CEV isn't identical to preference utilitarianism, but CEV and preference utilitarianism have the following principles in common, which the hack exploits:

  • Give people what they want, instead of what you think is good for them.
  • If different people want different things, give each individual equal weight.

It seems clear that Eliezer got these ideas from preference utilitarianism, and they share some of the same flaws as a result.