timtyler comments on A Less Wrong singularity article? - Less Wrong

28 Post author: Kaj_Sotala 17 November 2009 02:15PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (210)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 18 November 2009 12:32:41PM *  -1 points [-]

It's a good thing - from their point of view. They probably think that there should be more paperclips. The term "should" makes sense in the context of a set of preferences.

Comment author: Eliezer_Yudkowsky 18 November 2009 07:05:35PM *  1 point [-]

No, it's a paperclip-maximizing thing. From their point of view, and ours. No disagreement. They just care about what's paperclip-maximizing, not what's good.

Comment author: timtyler 18 November 2009 07:43:36PM *  0 points [-]

This is not a real point of disagreement.

IMO, in this context, "good" just means "favoured by this moral system". An action that "should" be performed is just one that would be morally obligatory - according to the specified moral system. Both terms are relative to a set of moral standards.

I was talking as though a paperclip maximiser would have morals that reflected their values. You were apparently assuming the opposite. Which perspective is better would depend on which particular paperclip maximiser was being examined.

Personally, I think there are often good reasons for morals and values being in tune with one another.