= 27d567bc2d48d9f40e9ccc20ea370b15)
Nisan comments on A fungibility theorem - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (66)
I think I found a clearer way to state an argument that I and a few others have been trying to make. Sorry for the repetition if you already understood! The claim is that Pareto-optimal is not equivalent to utility weighting, in the following important sense:
Consider a decision procedure as a function from a set of feasible outcomes (lotteries over world-histories) to a particular outcome. Let's say we have a decision procedure that is guaranteed to always output Pareto optimal outcomes, against some set of utility functions. Is this decision procedure necessarily equivalent to maximizing EU using a linear aggregation of those utility functions? No, because for different feasible sets, you may need different weights on the individual utility functions to reach the decisions that the original decision procedure would make, in which case we cannot specify an equivalent function using EU maximization.
(Feedback requested as to whether this made the argument clearer to anyone.)
Ah yes, this is clear to me.
Now, if we have a prior over the possible sets of lotteries you'll be presented with, then for each decision procedure and each utility function, we have the expected utility given that you follow that decision procedure. These expected utilities give us a new sense of Pareto optimality: A non-optimizing decision procedure that is Pareto-optimal in your sense will not be Pareto-optimal with respect to these expected utilities.
So, Pareto-optimal decision procedure is not equivalent to utility weighting, but Pareto-optimal (decision-procedure + prior) is equivalent to utility weighting.
The complete class theorem gives a reason for having a prior, but it assumes that you're an optimizer :)
Benja answered a similar point recently in this comment, in his third paragraph which starts with "I disagree". If you apply the Pareto-optimal decision procedure to the prior instead of after updating, then it will be Pareto-optimal with respect to these expected utilities. And in general, given different priors the decision will be equivalent to maximizing different linear aggregations of individual utility functions, so you still have the same issue that the decision procedure as a function cannot be reproduced by EU maximization of a single linear aggregation.
You might ask, why does this matter if in real life we just have one prior to deal with? I guess the answer is that it's a way of making clear that a Pareto-optimal decision procedure need not be algorithmically equivalent to EU maximization, so we can't conclude that we should become EU maximizers instead of implementing some other algorithm, at least not based just on considerations of Pareto optimality.
ETA: Your response to this comment seems to indicate a misunderstanding. It's probably easier to clear up this via online chat. I sent you a PM with my contact info.
Suppose you have a prior over all possible priors, and your first action after determining your utility function is to figure out which prior you should use. Before choosing a particular prior, you can define the expected utility of policies in terms of the "expected prior" of your distribution over priors. No matter how you arrived at your utility function, you will want to remember it as a linear combination of values while updating on the prior you chose.
So if I understand you correctly, if I wanted to switch from a non-optimizing policy to an optimizing policy, I'd have to choose whether to switch to a policy that's Pareto-optimal with respect to my current beliefs, or to a policy that's Pareto-optimal with respect to old beliefs. And if we don't know which beliefs to use, we can hardly say that we "should" choose one or the other.
Is that statement close to your point of view?