MugaSofer comments on Hacking the CEV for Fun and Profit - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (194)
Simple solution: Build an FAI to optimize the universe to your own utility function instead of humanity's average utility function. They will be nearly the same thing anyway (remember, you were tempted to have the FAI use the average human utility function instead, so clearly, you sincerely care about other people's wishes). And in weird situations in which the two are radically different (like this one), your own utility function more closely tracks the intended purpose of an FAI.
What if you're a preference utilitarian?
If you are a true preference utilitarian, then the FAI will implement preference utilitarianism when it maximizes your utility function.
My point was that a preference utilitarian would let Dr Evil rule the world, in that scenario.
Although, obviously, if you're a preference utilitarian then that's what you actually want.