torekp comments on Pluralistic Moral Reductionism - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (316)
No, but it sets a high standard - If you value, say, the company of your family, then modifying to not want that (and therefore not spend much time with your family) costs as much as if you were kept away from your family by force for the rest of your life. So any threats have to be pretty damn serious, and maybe not even death would work if you know important secrets or do not highly value living without some key values.
I wouldn't call all of those cases of modifying terminal values. From some quick googling (I didn't know about the Vicar of Bray), what the Vicar of Bray cared about was being the vicar of Bray. What Pierre Laval cared about was being the head of the government and not being killed, maybe. So they're maybe not good examples of changing terminal values, as opposed to instrumental ones.
Also "improving their moral beliefs as they mature" is a very odd concept once you think about it. How do you judge whether a moral belief is right to hold correctly without having a correct ultimate belief from the start, to do the judging? It's really an example of how humans are emphatically not rational agents - we follow a bunch of evolved and cultural rules, which can appear to produce consistent behavior, but really have all these holes and internal conflicts. And things can change suddenly, without the sort of rational deliberation described above.
You could say the same about "improving our standards of scientific inference." Circular? Perhaps, but it needn't be a vicious circle. It's pretty clear that we've accomplished it, so it must be possible.