MugaSofer comments on Hacking the CEV for Fun and Profit - Less Wrong

52 Post author: Wei_Dai 03 June 2010 08:30PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (194)

You are viewing a single comment's thread. Show more comments above.

Comment author: AlexMennen 08 June 2010 02:04:46AM 7 points [-]

Simple solution: Build an FAI to optimize the universe to your own utility function instead of humanity's average utility function. They will be nearly the same thing anyway (remember, you were tempted to have the FAI use the average human utility function instead, so clearly, you sincerely care about other people's wishes). And in weird situations in which the two are radically different (like this one), your own utility function more closely tracks the intended purpose of an FAI.

Comment author: MugaSofer 24 January 2013 11:00:19AM -2 points [-]

What if you're a preference utilitarian?

Comment author: AlexMennen 24 January 2013 07:08:41PM 2 points [-]

If you are a true preference utilitarian, then the FAI will implement preference utilitarianism when it maximizes your utility function.

Comment author: MugaSofer 25 January 2013 09:17:11AM -1 points [-]

My point was that a preference utilitarian would let Dr Evil rule the world, in that scenario.

Although, obviously, if you're a preference utilitarian then that's what you actually want.