pdf23ds comments on Open Thread: January 2010 - Less Wrong

5 Post author: Kaj_Sotala 01 January 2010 05:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (725)

You are viewing a single comment's thread. Show more comments above.

Comment author: pdf23ds 23 January 2010 12:51:10PM *  1 point [-]

So you know how to divide the pie? There is no interpersonal "best way" to resolve directly conflicting values. (This is further than Eliezer went.) Sure, "divide equally" makes a big dent in the problem, but I find it much more likely any given AI will be a Zaire than a Yancy. As a simple case, say AI1 values X at 1, and AI2 values Y at 1, and X+Y must, empirically, equal 1. I mean, there are plenty of cases where there's more overlap and orthogonal values, but this kind of conflict is unavoidable between any reasonably complex utility functions.

Comment author: Vladimir_Nesov 23 January 2010 01:11:15PM *  1 point [-]

here is no interpersonal "best way" to resolve directly conflicting values.

I'm not suggesting an "interpersonal" way (as in, by a philosopher of perfect emptiness). The possibilities open for the search of "off-line" resolution of conflict (with abstract transformation of preference) are wider than those for the "on-line" method (with AIs fighting/arguing it over) and so the "best" option, for any given criterion of "best", is going to be better in "off-line" case.