Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

timtyler comments on Bayesian Utility: Representing Preference by Probability Measures - Less Wrong

10 Post author: Vladimir_Nesov 27 July 2009 02:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (35)

You are viewing a single comment's thread.

Comment author: timtyler 27 July 2009 05:09:01PM *  -1 points [-]

I've critiqued this "value is complex" [http://lesswrong.com/lw/y3/value_is_fragile/] material before. To summarise from my objections there:

The utility function of Deep Blue has 8,000 parts - and contained a lot of information. Throw all that information away, and all you really need to reconstruct Deep Blue is the knowledge that it's aim is to win games of chess. The exact details of the information in the original utility function are not recovered - but the eventual functional outcome would be much the same - a powerful chess computer.

The supposed complexity is actually a bunch of implementation details that can be effectively recreated from the goal - if that should prove to be necessary.

It is not precious information that must be preserved. If anything, attempts to preserve the 8,000 parts of Deep Blue's utility function while improving it would actually have a crippling negative effect on its future development. For example, the "look 9 moves ahead" heuristic is a feature when the program is weak, but a serious bug when it grows stronger.

Similarly with complexity of human values: those are a bunch of implementation details to deal with the problem of limited resources - not some kind of representation of the real target.

Comment author: Jonathan_Graehl 27 July 2009 08:55:19PM *  2 points [-]

It looks like this is a response to the passing link to http://wiki.lesswrong.com/wiki/Complexity_of_value in the article. At first I didn't understand what in the article you were responding to.

Comment author: timtyler 27 July 2009 09:08:21PM *  -2 points [-]

The article it was posted in response to was this one - from the conclusion of the post:


That's a wiki article - which can't be responded to directly. The point I raise is an old controversy now. This message seems rather redundant now - since the question it responded to has subsequently been dramatically edited.

Comment author: Jonathan_Graehl 28 July 2009 05:49:46AM 0 points [-]

Yes, I edited, but before your response. Sorry for the confusion.

Comment author: Wei_Dai 27 July 2009 08:07:37PM 1 point [-]

Why was this comment voted down so much (to -4 as of now)? It seems to be a reasonable point, clearly written, not an obvious troll or off-topic. Why does it deserve to be ignored?