Vladimir_Nesov comments on Human values differ as much as values can differ - Less Wrong

13 Post author: PhilGoetz 03 May 2010 07:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (205)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 03 May 2010 11:59:56PM 1 point [-]

I read it; now it seems you're protesting against presenting an AI with a single value to optimize, rather than my source code.

I suggest that it's very hard to form a coherent concept of an AI that only cares about one particular wish/aspect/value.

If something poses a problem in the very simple case of an AI with one single value to optimize, I don't see how giving it a whole bunch of values to optimize, along with their algorithmic definitions and context, is going to make things easier.

FAI is only supposed to improve on status quo. In the worst impossible case, this improvement is small. Unless AI actually makes things worse (in which case, it's by definition not Friendly), I don't see what your argument could possibly be about.