TimS comments on On the fragility of values - Less Wrong

4 Post author: Stuart_Armstrong 04 November 2011 06:15PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread.

Comment author: TimS 05 November 2011 05:06:03AM 0 points [-]

V seems quite close to U when compared to the utility function programmed into a paperclip maximizer.

Secondly, I see how V could allow the coefficient for autonomy to be zero or even negative, in principle. But I don't think our values are actually that inconsistent. More precisely, if Value A and Value B require different choices with high frequency and have relatively similar priority, then U cannot implement both A and B unless it outputs very little of either one. Or so it seems to me.