Alternatively, we could bite the bullet and just say that some humans simply end up with alien values that are not "good",
Seeing as about 1% of the population are estimated to be psychopaths, not to mention pathological narcissists megalomaniacs etc, it seems hard to argue that there isn't a large (if statistically insignificant) portion of the population who are natural ethical egoists rather than altruists. You could try to weasel around it like Mr Yudkowski does, saying that they are not "neurologically intact," except that there is evidence that psychopathy at least is a stable evolutionary strategy rather than a malfunction of normal systems.
I'm usually not one to play the "evil psychopaths" card online, mainly because it's crass and diminishes the meaning of a useful medical term, but it's pretty applicable here. What exactly happens to all the psychopaths and people with psychopathic traits when you start extrapolating human values?
Seems to me that to a significant degree the psychopaths are successful because people around them have problems communicating. Information about what the specific psychopath did to whom are usually not shared. If they were easily accessible to people before interacting with the psychopath, a lot of their power would be lost.
Despite being introverted by nature, these days my heuristics for dealing with problematic people is to establish good communication lines among the non-problematic people. Then people often realize that what seemed like their specific...
There seems to be a widespread impression that the metaethics sequence was not very successful as an explanation of Eliezer Yudkowsky's views. It even says so on the wiki. And frankly, I'm puzzled by this... hence the "apparently" in this post's title. When I read the metaethics sequence, it seemed to make perfect sense to me. I can think of a couple things that may have made me different from the average OB/LW reader in this regard: