Kaj_Sotala comments on Applying utility functions to humans considered harmful - Less Wrong

26 Post author: Kaj_Sotala 03 February 2010 07:22PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (114)
Tags: decision_theory utility decision_field_theory

Comments (114)

You are viewing a single comment's thread. Show more comments above.

Comment author: Kaj_Sotala 05 February 2010 07:48:23PM 0 points [-]

I don't think I understand what you're trying to describe here. Could you give an example of a scenario where you usefully transform a model into a utility-based one the way you describe?

I'm not bashing utility-based models, I'm quite aware of their good sides. I'm just saying they shouldn't be used universally and without criticism. That's not bashing any more than it's bashing to say that integrals aren't the most natural way to do matrix multiplication with.

Comment author: timtyler 05 February 2010 08:08:25PM *  0 points [-]

Could you give an example of a scenario where you usefully transform a model into a utility-based one the way you describe?

Call the original model M.

"Wrap" the model M - by preprocessing its sensory inputs and post-processing its motor outputs.

Then, post-process M's motor outputs - by enumerating its possible actions at each moment, assign utility 1 to the action corresponding to the action M output, and assign utility 0 to all other actions.

Then output the action with the highest utility.

I'm not bashing utility-based models, I'm quite aware of their good sides.

Check with your subject line. There are plenty of good reasons for applying utility functions to humans. A rather obvious one is figuring out your own utility function - in order to clarify your goals to yourself.

Comment author: Kaj_Sotala 05 February 2010 08:37:03PM 0 points [-]

Okay, I'm with you so far. But what I was actually asking for was an example of a scenario where this wrapping gives us some benefit that we wouldn't have otherwise.

I don't think utility functions are a very good tool to use when seeking to clarify one's goals to yourself. Things like PJ Eby's writings have given me rather powerful insights to my goals, content which would be pointless to try to convert to the utility function framework.

Comment author: timtyler 05 February 2010 09:14:25PM *  0 points [-]

I don't think utility functions are a very good tool to use when seeking to clarify one's goals to yourself.

Personally, I found thinking of myself as a utility maximiser enlightening. However YMMV.

Comment author: timtyler 05 February 2010 09:04:16PM *  0 points [-]

But what I was actually asking for was an example of a scenario where this wrapping gives us some benefit that we wouldn't have otherwise.

My original comment on that topic was:

Utility based models are most useful when applying general theorems - or comparing across architectures. For example when comparing the utility function of a human with that of a machine intelligence - or considering the "robustness" of the utility function to environmental perturbations.

Utility-based models are a general framework that can represent any computable intelligent agent. That is the benefit that you don't otherwise have. Utility-based models let you compare and contrast different agents - and different types of agent.

Powered by Reddit Powered by Reddit