Stirling_Westrup comments on The Meaning of Right - Less Wrong

30 Post author: Eliezer_Yudkowsky 29 July 2008 01:28AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (147)

Sort By: Old

You are viewing a single comment's thread.

Comment author: JulianMorrison 29 July 2008 06:04:00PM -1 points [-]

So, what do you do about inter-morality? Suppose a paperclip maximizer with the intellect and self-improvement limits of a typical human thirteen-year-old. They are slightly less powerful than you and can be expected to stay that way, so they aren't an existential threat. They are communicative and rational but they have one terminal value, and it's paperclips.

How ought a moral human to react? Can you expect to negotiate an inter-morality? Or must the stronger party win by force and make the weaker party suffer non-fulfillment? Is this mis-treatment and immoral? Ought you to allow them a bale of wire so they can at least make *some* paperclips?