I'm really having trouble understanding how this isn't tantamount to moral relativism -- or indeed moral nihilism. The whole point of "morality" is that it's supposed to provide a way of arbitrating between beings, or groups, with different interests -- such as ourselves and Pebblesorters. Once you give up on that idea, you're reduced, as in this post, to the tribalist position of arguing that we humans should pursue our own interests, and the Pebblesorters be damned. When a conflict arises (as it inevitably will), the winner will then be whoever has the bigger guns, or builds AI first.

Mind you, I don't disagree that this is the situation in which we in fact find ourselves. But we should be honest about the implications. The concept of "morality" is entirely population-specific: when groups of individuals with common interests come into contact, "morality" is the label they give to their common interests. So for us humans, "morality" is art, music, science, compassion, etc. in short, all the things that we humans (as opposed to Pebblesorters) like. This is what I understand Eliezer to be arguing. But if this is your position, you may as well come out and admit that you're a moral relativist, because this is the position that the people who are scared of moral relativism are in fact scared of. What they dread is a world in which Dennis could go on saying that Dennis-morality is what really matters, the rest of us disagree, war breaks out, Dennis kills us all, eats the whole pie, and is not spanked by any cosmic force. But this is indeed the world we live in.