tom_cr comments on To what extent does improved rationality lead to effective altruism? - Less Wrong

10 Post author: JonahSinick 20 March 2014 07:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (156)

You are viewing a single comment's thread. Show more comments above.

Comment author: SaidAchmiz 23 March 2014 09:57:11PM *  0 points [-]

Dawes' argument, as promised.

The context is: Dawes is explaining von Neumann and Morgenstern's axioms.


Aside: I don't know how familiar you are with the VNM utility theorem, but just in case, here's a brief primer.

The VNM utility theorem presents a set of axioms, and then says that if an agent's preferences satisfy these axioms, then we can assign any outcome a number, called its utility, written as U(x); and it will then be the case that given any two alternatives X and Y, the agent will prefer X to Y if and only if E(U(X)) > E(U(Y)). (The notation E(x) is read as "the expected value of x".) That is to say, the agent's preferences can be understood as assigning utility values to outcomes, and then preferring to have more (expected) utility rather than less (that is, preferring those alternatives which are expected to result in greater utility).

In other words, if you are an agent whose preferences adhere to the VNM axioms, then maximizing your utility will always, without exception, result in satisfying your preferences. And in yet other words, if you are such an agent, then your preferences can be understood to boil down to wanting more utility; you assign various utility values to various outcomes, and your goal is to have as much utility as possible. (Of course this need not be anything like a conscious goal; the theorem only says that a VNM-satisfying agent's preferences are equivalent to, or able to be represented as, such a utility formulation, not that the agent consciously thinks of things in terms of utility.)

(Dawes presents the axioms in terms alternatives or gambles; a formulation of the axioms directly in terms of the consequences is exactly equivalent, but not quite as elegant.)

N.B.: "Alternatives" in this usage are gambles, of the form ApB: you receive outcome A with probability p, and otherwise (i.e. with probability 1–p) you receive outcome B. (For example, your choice might be between two alternatives X and Y, where in X, with p = 0.3 you get consequence A and with p = 0.7 you get consequence B, and in Y, with p = 0.4 you get consequence A and with p = 0.6 you get consequence B.) Alternatives, by the way, can also be thought of as actions; if you take action X, the probability distribution over the outcomes is so-and-so; but if you take action Y, the probability distribution over the outcomes is different.

(If all of this is old hat to you, apologies; I didn't want to assume.)


The question is: do our preferences satisfy VNM? And: should our preferences satisfy VNM?

It is commonly said (although this is in no way entailed by the theorem!) that if your preferences don't adhere to the axioms, then they are irrational. Dawes examines each axiom, with an eye toward determining whether it's mandatory for a rational agent to satisfy that axiom.

Dawes presents seven axioms (which, as I understand it, are equivalent to the set of four listed in the wikipedia article, just with a difference in emphasis), of which the fifth is Independence.

The independence axiom says that AB (i.e., A is preferred to B) if and only if ApCBpC. In other words, if you prefer receiving cake to receiving pie, you also prefer receiving (cake with probability p and death with probability 1–p) to receiving (pie with probability p and death with probability 1–p).

Dawes examines one possible justification for violating this axiom — framing effects, or pseudocertainty — and concludes that it is irrational. (Framing is the usual explanation given for why the expressed or revealed preferences of actual humans often violate the independence axiom.) Dawes then suggests another possibility:

Is such irrationality the only reason for violating the independence axiom? I believe there is another reason. Axiom 5 [Independence] implies that the decision maker cannot be affected by the skewness of the consequences, which can be conceptualized as a probability distribution over personal values. Figure 8.1 shows (Note: This is my reproduction of the figure. I've tried to make it as exact as possible.) the skewed distributions of two different alternatives. Both distributions have the same average, hence the same expected personal value, which is a criterion of choice implied by the axioms. These distributions also have the same variance.

If the distributions in Figure 8.1 were those of wealth in a society, I have a definite preference for distribution a; its positive skewness means that income can be increased from any point — an incentive for productive work. Moreover, those people lowest in the distribution are not as distant from the average as in distribution b. In contrast, in distribution b, a large number of people are already earning a maximal amount of money, and there is a "tail" of people in the negatively skewed part of this distribution who are quite distant from the average income.[5] If I have such concerns about the distribution of outcomes in society, why not of the consequences for choosing alternatives in my own life? In fact, I believe that I do. Counter to the implications of prospect theory, I do not like alternatives with large negative skews, especially when the consequences in the negatively skewed part of the distribution have negative personal value.

[5] This is Dawes' footnote; it talks about an objection to "Reaganomics" on similar grounds.

Essentially, Dawes is asking us to imagine two possible actions. Both have the same expected utility; that is, the "degree of goal satisfaction" which will result from each action, averaged appropriately across all possible outcomes of that action (weighted by probability of each outcome), is exactly equal.

But the actual probability distribution over outcomes (the form of the distrbution) is different. If you do action A, then you're quite likely to do alright, there's a reasonable chance of doing pretty well, and a small chance of doing really great. If you do action B, then you're quite likely to do pretty well, there's a reasonable chance to do ok, and a small chance of doing disastrously, ruinously badly. On average, you'll do equally well either way.

The Independence axiom dictates that we have no preference between those two actions. To prefer action A, with its attendant distribution of outcomes, to action B with its distribution, is to violate the axiom. Is this irrational? Dawes says no. I agree with him. Why shouldn't I prefer to avoid the chance of disaster and ruin? Consider what happens when the choice is repeated, over the course of a lifetime. Should I really not care whether I occasionally suffer horrible tragedy or not, as long as it all averages out?

But if it's really a preference — if I'm not totally indifferent — then I should also prefer less "risky" (i.e. less negatively skewed) distributions even when the expectation is lower than that of distributions with more risk (i.e. more negative skew) — so long as the difference in expectation is not too large, of course. And indeed we see such a preference not only expressed and revealed in actual humans, but enshrined in our society: it's called insurance. Purchasing insurance is an expression of exactly the preference to reduce the negative skew in the probability distribution over outcomes (and thus in the distributions of outcomes over your lifetime), at the cost of a lower expectation.

Comment author: tom_cr 24 March 2014 03:49:46PM 0 points [-]

Thanks very much for the taking the time to explain this.

It seems like the argument (very crudely) is that, "if I lose this game, that's it, I won't get a chance to play again, which makes this game a bad option." If so, again, I wonder if our measure of utility has been properly calibrated.

It seems to me like the expected utility of option B, where I might get kicked out of the game, is lower than the expected utility of option A, where this is impossible. Your example of insurance may not be a good one, as one insures against financial loss, but money is not identical to utility.

Nonetheless, those exponential distributions make a very interesting argument.

I'm not entirely sure, I need to mull it over a bit more.

Thanks again, I appreciate it.

Comment author: SaidAchmiz 24 March 2014 05:47:21PM 0 points [-]

Just a brief comment: the argument is not predicated on being "kicked out" of the game. We're not assuming that even the lowest-utility outcomes cause you to no longer be able to continue "playing". We're merely saying that they are significantly worse than average.

Comment author: tom_cr 24 March 2014 06:18:23PM 0 points [-]

Sure, I used that as what I take to be the case where the argument would be most easily recognized as valid.

One generalization might be something like, "losing makes it harder to continue playing competitively." But if it becomes harder to play, then I have lost something useful, i.e. my stock of utility has gone down, perhaps by an amount not reflected in the inferred utility functions. My feeling is that this must be the case, by definition (if the assumed functions have the same expectation), but I'll continue to ponder.

The problem feels related to Pascal's wager - how to deal with the low-probability disaster.

Comment author: SaidAchmiz 24 March 2014 06:36:02PM *  1 point [-]

I really do want to emphasize that if you assume that "losing" (i.e. encountering an outcome with a utility value on the low end of the scale) has some additional effects, whether that be "losing takes you out of the game", or "losing makes it harder to keep playing", or whatever, then you are modifying the scenario, in a critical way. You are, in effect, stipulating that that outcome actually has a lower utility than it's stated to have.

I want to urge you to take those graphs literally, with the x-axis being Utility, not money, or "utility but without taking into account secondary effects", or anything like that. Whatever the actual utility of an outcome is, after everything is accounted for — that's what determines that outcome's position on the graph's x-axis. (Edit: And it's crucial that the expectation of the two distributions is the same. If you find yourself concluding that the expectations are actually different, then you are misinterpreting the graphs, and should re-examine your assumptions; or else suitably modify the graphs to match your assumptions, such that the expectations are the same, and then re-evaluate.)

This is not a Pascal's Wager argument. The low-utility outcomes aren't assumed to be "infinitely" bad, or somehow massively, disproportionately, unrealistically bad; they're just... bad. (I don't want to get into the realm of offering up examples of bad things, because people's lives are different and personal value scales are not absolute, but I hope that I've been able to clarify things at least a bit.)

Comment author: tom_cr 24 March 2014 08:11:09PM -1 points [-]

If you assume.... [y]ou are, in effect, stipulating that that outcome actually has a lower utility than it's stated to have.

Thanks, that focuses the argument for me a bit.

So if we assume those curves represent actual utility functions, he seems to be saying that the shape of curve B, relative to A makes A better (because A is bounded in how bad it could be, but unbounded in how good it could be). But since the curves are supposed to quantify betterness, I am attracted to the conclusion that curve B hasn't been correctly drawn. If B is worse than A, how can their average payoffs be the same?

To put it the other way around, maybe the curves are correct, but in that case, where does the conclusion that B is worse come from? Is there an algebraic formula to choose between two such cases? What if A had a slightly larger decay constant, at what point would A cease to be better?

I'm not saying I'm sure Dawes' argument is wrong, I just have no intuition at the moment for how it could be right.

Comment author: SaidAchmiz 24 March 2014 08:45:49PM 1 point [-]

A point of terminology: "utility function" usually refers to a function that maps things (in our case, outcomes) to utilities. (Some dimension, or else some set, of things on the x-axis; utility on the y-axis.) Here, we instead are mapping utility to frequency, or more precisely, outcomes (arranged — ranked and grouped — along the x-axis by their utility) to the frequency (or, equivalently, probability) of the outcomes' occurrence. (Utility on the x-axis, frequency on the y-axis.) The term for this sort of graph is "distribution" (or more fully, "frequency [or probability] distribution over utility of outcomes").

To the rest of your comment, I'm afraid I will have to postpone my full reply; but off the top of my head, I suspect the conceptual mismatch here stems from saying that the curves are meant to "quantify betterness". It seems to me (again, from only brief consideration) that this is a confused notion. I think your best bet would be to try taking the curves as literally as possible, attempting no reformulation on any basis of what you think they are "supposed" to say, and proceed from there.

I will reply more fully when I have time.