You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

torekp comments on Against utility functions - Less Wrong Discussion

40 Post author: Qiaochu_Yuan 19 June 2014 05:56AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (87)

You are viewing a single comment's thread. Show more comments above.

Comment author: Viliam_Bur 19 June 2014 07:26:47AM 11 points [-]

Like the word "rational" is sometimes used instead of "optimal" or "good", words "utility function" are probably used to mean "good" or "our values" or something like that.

Therefore, analogically to the suggestion of only using the word 'rational' when talking about cognitive algorithms and thinking techniques, we should only use the words 'utility function' when talking about computer programs. When speaking about humans, "good / better / the best" probably expresses what we need well enough.

Comment author: torekp 22 June 2014 09:26:36PM 1 point [-]

Except ... maybe "utility function" need not apply to AI either. Why suppose that the best way to get what we want from an AI is to give it a utility function? Maybe AIs that do have utility functions are generally more dangerous than certain classes of AIs that don't.