timtyler comments on A Less Wrong singularity article? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (210)
My use is not counterintuitive. The fact that it is the intuitive use - that only humans ever think of what they should do in the ordinary sense, while aliens do what is babyeating; that looking at a paperclipper's actions conveys no more information about what we should do than looking at evolution or a rockslide - is counterintuitive.
If you tell me that "should" has a usage which is unrelated to "right", "good", and "ought", then that usage could be adapted for aliens.
It's counterintuitive to me - and I'm not the only one - if you look at the other comments here.
Aliens could have the "right", "good", "ought" and "should" concept cluster - just as some other social animals can, or other tribes, or humans at other times.
Basically, there are a whole bunch of possible and actual moral frameworks - and these words normally operate relative to the framework under consideration.
There are some people who think that "right" and "wrong" have some kind of universal moral meaning. However most of those people are religious, and think morality comes straight from god - or some such nonsense.