Why even stop at psychopaths? There are perfectly neurotypical people with strong desires for revenge-based justice, purity norms that I strongly dislike, etc. I'm not extremely confident that extrapolation will dissolve these values into deeper-order values, although my perception that intelligence in humans does at least seem to be correlated to values similar to mine is comforting in this respect.
Although really, I think this is reaching the point where we have to stop talking in terms of idealized agents with values and start thinking about how these models can be mapped to actual meat brains.
What exactly happens to all the psychopaths and people with psychopathic traits when you start extrapolating human values?
Well, under the shaky assumption that we have the ability to extrapolate in the first place, in practice what happens is that whoever controls the extrapolation sets which values are to be extrapolated, and they have a very strong incentive to put in only their own values.
By definition, no one wants to implement the CEV of humanity more than they want to implement their own CEV. But I would hope that most of the worlds impacted by the various human's CEVs would be a pretty nice places to live.
By definition, no one wants to implement the CEV of humanity more than they want to implement their own CEV.
That depends. The more interconnected our lives become, the harder it gets to enhance the life of myself or my loved ones through highly localized improvements. Once you get up to a sufficiently high level (vaccination programs are an obvious example), helping yourself and your loved ones is easiest to accomplish by helping everyone all together, because of the ripple effects down to my loved ones' loved ones thus having an effect on my loved on...
There seems to be a widespread impression that the metaethics sequence was not very successful as an explanation of Eliezer Yudkowsky's views. It even says so on the wiki. And frankly, I'm puzzled by this... hence the "apparently" in this post's title. When I read the metaethics sequence, it seemed to make perfect sense to me. I can think of a couple things that may have made me different from the average OB/LW reader in this regard: