RichardKennaway comments on Applying utility functions to humans considered harmful - Less Wrong

26 Post author: Kaj_Sotala 03 February 2010 07:22PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (114)
Tags: decision_theory utility decision_field_theory

Comments (114)

You are viewing a single comment's thread. Show more comments above.

Comment author: RichardKennaway 05 February 2010 06:33:02PM -1 points [-]

I am well aware of the perception-action feedback - but what does it have to do with this discussion?

It renders wrong the passage that I quoted above. You have described agents as choosing an outcome (from utility calculations, which I'd dispute, but that's not the point at issue here) deciding on an action which will produce that outcome, and emitting that action, whereupon the world then produces the chosen outcome. Agents, that is, in the grip of the planning fallacy.

Planning plays a fairly limited role in human activity. An artificial agent designed to plan everything will do nothing useful. "No plan of battle survives contact with the enemy." "What you do changes who you are." "Life is what happens when you're making other plans." Etc.

Comment author: timtyler 05 February 2010 06:47:40PM *  0 points [-]

I don't know what you are thinking - but it seems fairly probable that you are still misinterpreting me - since your first paragraph contains:

You have described agents as choosing an outcome [...] deciding on an action which will produce that outcome, and emitting that action

...which appears to me to have rather little to do with what I originally wrote.

Rather, agents pick an action to execute, enumerate their possible actions, have a utility (1 or 0) assigned to each action by the I/O wrapper I described, select the highest utility action and then pass that on to the associated actuators.

Notice the lack of mention of outcomes here - in contrast to your description.

I stand by the passage that you quoted above, which you claim is wrong.

Comment author: RichardKennaway 05 February 2010 09:42:59PM 0 points [-]

In that case, I disagree even more. The perceived outcome is what matters to an agent. The actions it takes to get there have no utility attached to them; if utility is involved, it attaches to the perceived outcomes.

I continue to be perplexed that you take seriously the epiphenomal utility function you described in these words:

Simply wrap the I/O of the non-utility model, and then assign the (possibly compound) action the agent will actually take in each timestep utility 1 and assign all other actions a utility 0 - and then take the highest utility action in each timestep.

and previously here. These functions require you to know what action the agent will take in order to assign it a utility. The agent is not using the utility to choose its action. The utility function plays no role in the agent's decision process.

Comment author: timtyler 05 February 2010 09:59:58PM *  0 points [-]

The utility function plays no role in the agent's decision process.

The utility function determines what the agent does. It is the agent's utility function.

Utilities are numbers. They are associated with actions - that association is what allows utility-based agents to choose between their possible actions.

The actions produces outcomes - so, the utilities are also associated with the relevant outcomes.

Powered by Reddit Powered by Reddit