Tyrrell_McAllister comments on A Master-Slave Model of Human Preferences - Less Wrong

58 Post author: Wei_Dai 29 December 2009 01:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (80)

Comments (80)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 29 December 2009 05:50:43PM *  1 point [-]

Thus, the criterion for ascribing preferences to a physical system is that the actual physics has to be well-approximated by a function that optimizes for a preferred state, for some value of "preferred state".

I don't think this simple characterisation resembles the truth: the whole point of this enterprise is to make sure things go differently, in a way they just couldn't proceed by themselves. Thus, observing existing "tendencies" doesn't quite capture the idea of preference.

Comment deleted 29 December 2009 08:03:17PM [-]
Comment author: Tyrrell_McAllister 29 December 2009 08:23:36PM *  0 points [-]

[Y]ou have to draw a boundary around the "optimizing agent", and look at the difference between the tendencies of the environment without the optimizer, and the tendencies of the environment with the optimizer.

And there's your "opinion or interpretation" --- not just in how you draw the boundary (which didn't exist in the original ontology), but in your choice of the theory that you use to evaluate your counterfactuals.

Of course, such theories can be better or worse, but only with respect to some prior system of evaluation.

Comment author: Vladimir_Nesov 29 December 2009 08:49:47PM 2 points [-]

Still, probably a question of Aristotelian vs. Newtonian mechanics, i.e. not hard to see who wins.

Comment author: Tyrrell_McAllister 29 December 2009 08:55:55PM *  0 points [-]

Still, probably a question of Aristotelian vs. Newtonian mechanics, i.e. not hard to see who wins.

Agreed, but not responsive to Mitchell Porter's original point. (ETA: . . . unless I'm missing your point.)

Powered by Reddit Powered by Reddit