Viliam_Bur comments on Confusion about Normative Morality - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (103)
I am probably using the words incorrectly, because I don't know how philosophers define them, or even whether they can agree on a definition. I essentially used "morality" to mean "any system which says what you should", and added an observation that if you take literally any such system, most of them will not fit your intuition of morality. Why? Because they recommend things you find repulsive or just stupid. But this is a fact about you or about humans in general, so in order to find "a system which says what you should, and it makes sense and is not repulsive", you must study humans. Specifically, human desires.
In other words, I define "morality" as "a system of 'shoulds' that humans can agree with".
Paperclip maximizers, capable of reflexivity and knowing game theory, could derive their own "system of 'shoulds'" they could agree with. It could include rules like "don't destroy your neighbor's two paperclips just to build one yourself", which would be similar to our morality, but that's because the game theory is the same.
But it would be game theory plus paperclip-maximizer desires, so even if it would contain some concepts of friendship and non-violence (cooperating with each other in the iterated Prisonner's Dilemma's) which would make all human hippies happy, when given a choice "sending all sentient beings into eternal hell of maximum suffering in exchange for a machine that tiles the universe with the paperclips" would seem to them like a great idea. Don't ever forget it when dealing with paperclip maximizers.
If I am a psychopath now, I don't give a **** about morality, do I? So I decide according to whatever psychopaths consider important. (I guess it would be according to my whim at the moment.)
If you want a name for your position on this (which, as far as I can tell, is very well put,) a suitable philosophical equivalent is Moral Contractualism, a la Thomas Scanlon in "What We Owe To Each Other." He defines certain kinds of acts as morally wrong thus: