Tyrrell_McAllister3 comments on The Super Happy People (3/8) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (50)
Akon forgot to mention the possibility of trying to genetically modify the Babyeaters.
So the Lady 3rd says, basically, "modifying ourselves to exclude this pain would be hard, and we don't want to do it until all other options are proved harder."
Akon, then should be able to say exactly the same thing, with equal truth, regarding the human pain that the Lady 3rd wants eliminated. She is too quick in dismissing this symmetry.
Or was Akon really trying to argue that the Lady 3rd's kind keep their sympathy-pain as a terminal value? He is described as thinking hastily on his feet, so I suppose it's plausible for him to make such a silly argument. And in that case it's plausible to the Lady 3rd to dismiss this argument as she did. But it's not plausible that Akon really thinks that humans want to hold on to their pain as a terminal value. Star Trek V notwithstanding, it's hard to imagine a human society of the kind depicted here feeling that kind of attachment to pain.
So, for this scene to be believable, Akon should very quickly realize that humans are reluctant to eliminate their pain because they don't know how to do so without interfering with other values. And that should be a reason that is immediately understandable to the Lady 3rd, because she has offered essentially the same justification for not proceeding immediately to eliminate her own sympathy-pain.
Of course, she should then offer immediately to figure out how to eliminate our pain for us if we can't do it ourselves. But it shouldn't be hard for her to see why we would be reluctant to trust her ability to do that without interfering with our other values, given what we've seen of their abilities to understand us so far. Her evident bafflement at our reluctance to modify ourselves is prima facie evidence that they do not understand us well enough that we would be willing to let them muck around with our source code.
Of course, she might still conclude that they do understand us well enough to modify us, even against our wishes. But it shouldn't be surprising to her that it would be against our wishes, at least at this stage of the encounter.