Anyone who predicts that some decision may result in the world being optimized according to something other than their own values, and is okay with that, is probably not thinking about terminal values. More likely, they're thinking that humanity (or its successor) will clarify its terminal values and/or get better at reasoning from them to instrumental values to concrete decisions, and that their understanding of their own values will follow that. Of course, when people are considering whether it's a good idea to create a certain kind of mind, that kind of thinking probably means they're presuming that Friendliness comes mostly automatically. It's hard for the idea of an agent with different terminal values to really sink in; I've had a little bit of experience with trying to explain to people the idea of minds with really fundamentally different values, and they still often try to understand it in terms of justifications that are compelling (or at least comprehensible) to them personally. Like, imagining that a paperclip maximizer is just like a quirky highly-intelligent human who happens to love paperclips, or is under the mistaken impression that maximizing paperclips is the right thing could do and could be talked out of it by the right arguments. I think Ben and Robin ("Human value ... will continue to [evolve]", "I think it is ok (if not ideal) if our descendants' values deviate from ours") are thinking as though AI-aided value loss would be similar to the gradual refinement of instrumental values that takes place within societies consisting of largely-similar human brains (the kind of refinement that we can anticipate in advance and expect we'll be okay with), rather than something that could result in powerful minds that actually don't care about morality.

And I feel like anyone who really has internalized the idea that minds are allowed to fundamentally care about completely different things that we do, and still thinks they're okay with that actually happening, probably just haven't taken five minutes to think creatively about what kinds of terrible worlds or non-worlds could be created as a result of powerfully optimizing for a value system based on our present muddled values plus just a little bit of drift.

I suppose what remains are the people who don't buy the generalized idea of optimization processes as a superset of person-like minds in the first place, with really powerful optimization processes being another subset. Would Ben be in that group? Some of his statements (e.g. "It's possible that with sufficient real-world intelligence tends to come a sense of connectedness with the universe that militates against squashing other sentiences. But I'm not terribly certain of this, any more than I'm terribly certain of its opposite." (still implying a privileged 50-50 credence in this unsupported idea)) do suggest that he is expecting AGIs to automatically be people in some sense.

(I do think that "Value Deathism" gives the wrong impression of what this post is about. Something like "Value Loss Escapism" might be better; the analogy to deathism seems too much like a surface analogy of minor relevance. I'm not convinced that the tendency to believe that value loss is illusory or desirable is caused by the same thought processes that cause those beliefs about death. More likely, most people who try to think about AI ethics are going to be genuinely really confused about it for a while or forever, whereas "is death okay/good?" is not a confusing question.)