= 27d567bc2d48d9f40e9ccc20ea370b15)
timtyler comments on Applying utility functions to humans considered harmful - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (114)
It seems simple to convert any computable agent-based input-transform-output model into a utility-based model - provided you are allowed utility functions with Turing complete languages.
Simply wrap the I/O of the non-utility model, and then assign the (possibly compound) action the agent will actually take in each timestep utility 1 and assign all other actions a utility 0 - and then take the highest utility action in each timestep.
That neatly converts almost any practical agent model into a utility-based model.
So: there is nothing "wrong" with utility-based models. A good job too - they are economics 101.
I don't think that's the right wrapping.
Utilities are over outcomes, not decisions.
Decisions change the distribution of outcomes but rarely force a single absolutely predictable outcome. At the very least, your outcome is contingent on other actors' unpredictable effects.
Maybe you have some way of handling this in your wrapping; it's not clear to me.
This reminds me: often it seems like people think they can negotiate outcomes by combining personal utility functions in some way. Your quirky utility function is just one example of how it's actually in general impossible to do so without normalizing and weighting in some fair way the components of each person's claimed utility.
Utilities are typically scalars calculated from sensory inputs and memories - which are the sum total of everything the agent knows at the time.
Each utility is associated with one of the agent's possible actions at each moment.
The outcome is that the agent performs the "best" action (according to the utility function) - and then the rest of the world responds to it according to physical law. The agent can only control its actions. Outcomes are determined from them by physics and the rest of the world.
...but an agent only takes one action at any moment (if you enumerate its possible actions appropriately). So this is a non-issue from the perspective of constructing a utility-based "wrapper".
I personally feel happy or sad about the present state of affairs, including expectation of future events ("Oh no, my parachute won't deploy! I sure am going to hit the ground fast."). I can call how satisfied I am with the current state of things as I perceive it "utility". Of course, by using that word, it's usually assumed that my preferences obey some axioms, e.g. von Neumann-Morgenstern, which I doubt your wrapping satisfies in any meaningful way.
Perhaps there's some retrospective sense in which I'd talk about the true utility of the actual situation at the time (in hindsight I have a more accurate understanding of how things really were and what the consequences for me would be), but as for my current assessment it is indeed entirely a function of my present mental state (including perceptions and beliefs about the state of the universe salient to me). I think we agree on that.
I'm still not entirely sure I understand the wrapping you described. It feels like it's too simple to be used for anything.
Perhaps it's this: given the life story of some individual (call her Ray), you can vacuously (in hindsight) model her decisions with the following story:
1) Ray always acts so that the immediately resulting state of things has the highest expected utility. Ray can be thought of as moving through time and having a utility at each time, which must include some factor for her expectation of her future e.g. health, wealth, etc.
2) Ray is very stupid and forms some arbitrary belief about the result of her actions, expecting with 100% confidence that the predicted future of her life will come to pass. Her expectation in the next moment will usually turn out to revise many things she previously wrongly expected with certainty, i.e. she's not actually predicting the future exactly.
3) Whatever Ray believed the outcome would be at each choice, she assigned utility 1. To all other possibilities she assigned utility 0.
That's the sort of fully-described scenario that your proposal evoked in me. If you want to explain how she's forecasting more than singleton expectation set, and yet the expected utility for each decision she takes magically works out to be 1, I'd enjoy that.
In other words, I don't see any point modeling intelligent yet not omniscient+deterministic decision making unless the utility at a given state includes an anticipation of expectation of future states.
There's no point in discussing "utility maximisers" - rather than "expected utility maximisers"?
I don't really agree - "utility maximisers" is a simple generalisation of the concept of "expected utility maximiser". Since there are very many ways of predicting the future, this seems like a useful abstraction to me.
...anyway, if you were wrapping a model a human, the actions would clearly be based on predictions of future events. If you mean you want the prediction process to be abstracted out in the wrapper, obviously there is no easy way to do that.
You could claim that a human - while a "utility maximiser" was not clearly an "expected utility maximiser". My wrapper doesn't disprove such a claim. I generally think that the "expected utility maximiser" claim is highly appropriate for a human as well - but there is not such a neat demonstration of this.
I certanly did not intend any such implication. Which set of axioms is using the word "utility" supposed to imply?
Perhaps check with the definition of "utility". It means something like "goodness" or "value". There isn't an obvious implication of any specific set of axioms.
This is backwards. Agents control their perceptions, not their actions. They vary their actions in such a manner as to produce the perceptions they desire. There is a causal path from action to perception outside the agent, and another from perception (and desired perception) to action inside the agent.
It is only by mistakenly looking at those paths separately and ignoring their connection that one can maintain the stimulus-response model of an organism (whether of the behaviourist or cognitive type), whereby perceptions control actions. But the two are bound together in a loop, whose properties are completely different: actions control perceptions. The loop as a whole operates in such a way that the perception takes on whatever value the agent intends it to. The action varies all over the place, while the perception hardly changes. The agent controls its perceptions by means of its actions; the environment does not control the agent's actions by means of the perceptions it supplies.
Um. Agents do control their actions.
I am well aware of the perception-action feedback - but what does it have to do with this discussion?
It renders wrong the passage that I quoted above. You have described agents as choosing an outcome (from utility calculations, which I'd dispute, but that's not the point at issue here) deciding on an action which will produce that outcome, and emitting that action, whereupon the world then produces the chosen outcome. Agents, that is, in the grip of the planning fallacy.
Planning plays a fairly limited role in human activity. An artificial agent designed to plan everything will do nothing useful. "No plan of battle survives contact with the enemy." "What you do changes who you are." "Life is what happens when you're making other plans." Etc.
I don't know what you are thinking - but it seems fairly probable that you are still misinterpreting me - since your first paragraph contains:
...which appears to me to have rather little to do with what I originally wrote.
Rather, agents pick an action to execute, enumerate their possible actions, have a utility (1 or 0) assigned to each action by the I/O wrapper I described, select the highest utility action and then pass that on to the associated actuators.
Notice the lack of mention of outcomes here - in contrast to your description.
I stand by the passage that you quoted above, which you claim is wrong.
In that case, I disagree even more. The perceived outcome is what matters to an agent. The actions it takes to get there have no utility attached to them; if utility is involved, it attaches to the perceived outcomes.
I continue to be perplexed that you take seriously the epiphenomal utility function you described in these words:
and previously here. These functions require you to know what action the agent will take in order to assign it a utility. The agent is not using the utility to choose its action. The utility function plays no role in the agent's decision process.
The utility function determines what the agent does. It is the agent's utility function.
Utilities are numbers. They are associated with actions - that association is what allows utility-based agents to choose between their possible actions.
The actions produces outcomes - so, the utilities are also associated with the relevant outcomes.
"Control" is being used in two different senses in the above two quotes. In control theory parlance, timtyler is saying that actions are the manipulated variable, and you're saying that perceptions are the process variable.
Is this an argument in favor of using utility functions to model agents, or against?
It is just saying that you can do it - without much in the way of fuss or mess - contrary to the thesis of this post.
Did you miss the second paragraph of the post?
No, I didn't. My construction shows that the utility function need not be "insanely complex". Instead, a utility based model can be constructed that is only slightly more complex than the simplest possible model.
It is partly this simplicity that makes the utility-based framework such an excellent general purpose model of goal-directed agents - including, of course, humans.
Wait, do you mean that your construction is simply acting as a wrapper on some underlying model, and converting the outputs of that model into a different format?
If that's what you mean, then well, sure. You could do that without noticeably increasing the complexity. But in that case the utility wrapping doesn't really give us any useful additional information, and it'd still be the underlying model we'd be mainly interested in.
The outputs from the utility based model would be the same as from the model it was derived from - a bunch of actuator/motor outputs. The difference would be the utility-maximizing action "under the hood".
Utility based models are most useful when applying general theorems - or comparing across architectures. For example when comparing the utility function of a human with that of a machine intelligence - or considering the "robustness" of the utility function to environmental perturbations.
If you don't need a general-purpose model, then sure - use a specific one, if it suits your purposes.
Please don't "bash" utility-based models, though. They are great! Bashers simply don't appreciate their virtues. There are a lot of utility bashers out there. They make a lot of noise - and AFAICS, it is all pointless and vacuous hot air.
My hypothesis is that they think that their brain being a mechanism-like expected utility maximiser somehow diminishes their awe and majesty. It's the same thing that makes people believe in souls - just one step removed.
I don't think I understand what you're trying to describe here. Could you give an example of a scenario where you usefully transform a model into a utility-based one the way you describe?
I'm not bashing utility-based models, I'm quite aware of their good sides. I'm just saying they shouldn't be used universally and without criticism. That's not bashing any more than it's bashing to say that integrals aren't the most natural way to do matrix multiplication with.
Call the original model M.
"Wrap" the model M - by preprocessing its sensory inputs and post-processing its motor outputs.
Then, post-process M's motor outputs - by enumerating its possible actions at each moment, assign utility 1 to the action corresponding to the action M output, and assign utility 0 to all other actions.
Then output the action with the highest utility.
Check with your subject line. There are plenty of good reasons for applying utility functions to humans. A rather obvious one is figuring out your own utility function - in order to clarify your goals to yourself.
Okay, I'm with you so far. But what I was actually asking for was an example of a scenario where this wrapping gives us some benefit that we wouldn't have otherwise.
I don't think utility functions are a very good tool to use when seeking to clarify one's goals to yourself. Things like PJ Eby's writings have given me rather powerful insights to my goals, content which would be pointless to try to convert to the utility function framework.
Incidentally, I do not like writing "utility-based model" over and over again. These models should be called "utilitarian". We should hijack that term away from the ridiculous and useless definition used by the ethicists. They don't have the rights to this term.
You get plenty of absurdities following this route. Like atoms are utility maximising agents that want to follow brownian motion and are optimal!
You mean like the principle of least action...? ...or like the maximum entropy principle...?
Slapping the label "utility" on any quantity optimized in any situation adds zero content.
It is not supposed to. "Utility" in such contexts just means "that which is optimized". It is terminology.
"That which is optimized" is a mouthful - "utility" is shorter.
There's already a word for that: "optimand". The latter is the better terminology because (i) science-y types familiar with the "-and" suffix will instantly understand it and (ii) it's not in a name collision with another concept.
If "utility" is just terminology for "that which is optimized", then
is vacuous: goal-directed agents attempt to optimize something by definition.
Right - but you can't say "expected optimand maximiser". There is a loooong history of using the term "utility" in this context in economics. Think you have better terminology? Go for it - but so far, I don't see much of a case.
That would be the "other concept" (link edited to point to specific subsection of linked article) referred to in the grandparent.
Not "vacuous" - true. We have people saying that utility-based frameworks are "harmful". That needs correcting, is all.
I suspect that by "utility-based frameworks" they mean something more specific than you do.
Maybe - but if suspicions are all you have, then someone is not being clear - and I don't think it is me.
I find it hilarious that you think you're being perfectly clear and yet cannot be bothered to employ standard terminology.
Or they want to move in straight lines forever but are suboptimal.
This does not work. The trivial assignment of 1 to what happens and 0 to what does not happen is not a model of anything. A real utility model would enable you to evaluate the utility of various actions in order to predict which one will be performed. Your fake utility model requires you to know the action that was taken in order to evaluate its utility. It enables no predictions. It is not a model at all.