= 27d567bc2d48d9f40e9ccc20ea370b15)
Manfred comments on Papers framing anthropic questions as decision problems? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (36)
I don't think so. But I can give you the critique here :)
SUMMARY: If you don't actually calculate expected utility, don't expect to automatically make choices that correspond to a relevant utility function. Also, don't name a specific function for a general feeling - you might accidentally start calling the function when you just mean the feeling, and then you get really wrong answers.
-
I've already made the obvious criticism - since Stuart's anthropic decision theory is basically a way to avoid using subjective probabilities (also known as "probabilities") when they're confusing, it makes sense that it usually can't do probability-associated things like maximizing the expectation of a specific utility function.
The replacement for actually figuring out the probabilities is an "anthropic preference" that takes a list of utilities of different people you could be inside a "world," then outputs some effective utility for that world. That is, it's some function A(U(1), U(2), U(3), ... ), where U is the utility of being a certain person. The "worlds" are the different outcomes that you would know the probabilities for if no confusing anthropic things happened (for example, in a coin flip, the probabilities of the heads and tails worlds are 0.5). So the expected utility-ish-stuff is the sum over the worlds of P(world, if there was no anthropic stuff) * A(U(1), U(2), U(3), ... ).
Expected utility, on the other hand, can be written as the sum over people you could be of P(being this person) * U(being this person). So the only time when this anthropic decision theory actually maximizes utility is when its expected utility-ish-stuff is proportional to the actual expected utility - that is, you have to pick the correct anthropic preference out of all possible functions, and it can be a different one for every different problem. The easiest way to do this is simply to know what the expected utility is beforehand, and in that case you might as well just maximize that :P
On the other hand, it's not impossible to find a few useful "A"s, in the occasion that things are really simple. For example, if the different probabilities of being people are all a constant, multiplied by P(world, if there was no anthropic stuff), then the probability of being in a "world" is proportional to the number of people in that world, and the expected utility of being in a world is just the unweighted average of the utility. So the anthropic preference "A" just has to add everything together.
The Sleeping Beauty problem is an example that's simple enough to meet these conditions. There are a few other simple situations that can also yield simple "A" functions. However, if you drift away from simplicity, for example by giving people weak evidence about which world they're in, you can't even always find an "A" that gives you back the expected utility.
Another way to get into trouble is if you use an "A" that you think corresponds to a specific expected utilt, but you have not ever proved this. This was my real beef with Stuart's paper - he names his anthropic preferences with colorful descriptors like "altruistic" or "selfish." This is fine as long as you keep in mind that these names correspond to the anthropic preferences, not to any sort of utility function; but it leads to folly when he goes on to say things like "if we have an altruistic agent," as if he could tell how the agent acted in general. Because the "A" that corresponds to a utility function can change from problem to problem, calling an "A" function "selfish" can't somehow force it to correspond to a utility function we'd call selfish.
So in retrospect calling these anthropic preferences things like "altruistic" was a pretty bad idea, because it generated confusion. This confusion results in him whiffing the Presumptuous Philosopher problem, calling on what "selfish" versus "altruistic" methods of adding utilities together would say, with no guarantee that they have anything to do with any relevant expected utility. In fact, at no point does he ever compare anything to expected utility, because, if you remember back 20 paragraphs ago ( :D ), the whole point of this decision process was to avoid confusing probabilities.
Many thanks for your comments; it's nice to have someone engaging with it.
That said, I have to disagree with you! You're right, the whole point was to avoid using "anthropic probabilities" (though not "subjective probabilities" in general; I may have misused the word "objective" in that context). But the terms "altruistic", "selfish" and so on, do correspond to actual utility functions.
"Selfless" means your utility function has no hedonistic content, just an arbitrary utility function over world states that doesn't care about your own identity. "Altruistic" means your utility function is composed of some amalgam of the hedonistic utilities of you and others. And "selfish" means your utility function is equal to your own hedonistic utility.
The full picture is more complex - as always - and in some contexts it would be best to say "non-indexical" for selfless and "indexical" for selfish. But be as it may, these are honest, actual utility functions that you are trying to maximise, not "A"'s over utility functions. Some might be "A"'s over hedonistic utility functions, but they still are genuine utilities: I am only altruistic if I actually want other people to be happy (or achieve their goals or similar); their happiness is a term in my utility function.
Then ADT can be described (for the non-selfish agents) coloquially as "before the universe was created, if i wanted to maximise U, what decision theory would I want any U-maximiser to follow? (ie what decision theory maximises the expected U in this context)". So ADT is doubly a utility maximising theory: first pick the utility maximising decision theory, then the agents with it will try and maximise utility in accordance with the theory they have.
(for selfish/indexical agents it's a bit more tricky and I have to use symmetry or "veil of ignorance" arguments; we can get back to that).
Furthermore, ADT can perfectly deal with any other type of uncertainty - such as whether you are or aren't in a Sleeping Beauty problem, or when you have partial evidence that you're Monday, or whatever. There's no need for it to restrict to the simple cases. Admittedly for the presumptuous philosopher, I restricted to a simple, with simple binary altruistic/selfish utilites, but that was for illustrative purposes. Come up with a more complex problem, with more complicated utilties, and ADT will give a correspondingly more complex answer.
Okay, let's look at the "selfish" anthropic preference laid out in your paper, in two different problems.
In both of these problems there are two worlds, "H" and "T," which have equal "no anthropics" probabilities of 0.5. There are two people you could be in T and one person you could be in H. Standard Sleeping Beauty so far.
However, because I like comparing things to utility, I'm going to specify two sets of probabilities. In Problem 1, the probability of being each person is 1/3. In Problem 2, the probability of being the person in H is 1/2, while the probability of being a person in T is 1/4 each.
( These probabilities can be manipulated by giving people evidence of which world they are in - for example, you could spontaneously stop some H or T sessions of the experiment, so the people in the experiment can condition on the experiment not being stopped. The point is that both problems are entirely possible. )
And let's have winning each bet give 1 hedonic utilon (you get to eat a candybar).
ADT makes identical choices in both problems, because it just interacts with the "if there were no anthropics" probabilities. The selfish preference just says to give each world the average utility of all its inhabitants. To calculate the expected-utility-ish thing for betting on Tails, we take the average bet won in the winning side (1 candybar) and multiply it by the probability of the winning side if there were no anthropics (0.5). So our "selfish" ADT agent pays up to 0.5 candybars for a bet on Tails in both problems.
Now, what are the expected hedonic utilities? (in units of candybars, of course)
In problem 1, the probability of being a winner is 2/3, so a utility maximizer pays up to 2/3 of a candybar to bet on Tails.
In problem 2, the probability of being a winner is 1/2, so a utility maximizer pays up to 1/2 of a candybar to bet on Tails.
So in problem 2, the "selfish" ADT agent and the utility maximizer do the same thing. This looks like a good example of selfishness. But what's going on in problem 1? Even though the utilon is entirely candybar-based, the "selfish" anthropic preference seems to undervalue it. What anthropic preference would maximize expected hedonic utility?
Well, if you added up all the utilities in each world, rather than averaging, then an ADT agent would do the same thing as a utility maximizer in problem 1. But now in problem 2, this "total utility" ADT agent would overestimate the value of a candybar, relative to maximum expected utility.
There are in fact no ADT preferences that maximize candybars in both problems. There is no analogue of utility maximization, which makes sense because ADT doesn't deal with the subjective probabilities and expected utility does.
To translate this into ADT terms: in problem 2, the coin is fair, in problem 1, the coin is (1/3, 2/3) on (H, T) (or maybe the coin was fair, but we got extra info that pushed the postiori odds to (1/3, 2/3)).
Then ADT (and SSA) says that selfish agents should bet up to 2/3 of candybar on Tails in problem 1, and 1/2 in problem 2. Exactly the same as what you were saying. I don't understand why you think that ADT would make identical choices in both problems.
The reason that's "exactly as I was saying" is because you adjusted a free parameter to fit the problem, after you learned the subjective probabilities. The free parameter was which world to regard as "normal" and which one to apply a correction to. If you already know that the (1/2, 1/4, 1/4) problem is the "normal" one, then you already solved the probability problem and should just maximize expected utility.
Er no - you gave me an underspecified problem. You told me the agents were selfish (good), but then just gave me anthropic probabilities, without giving me the non-anthropic probabilities. I assumed you were meaning to use SSA, and worked back from there. This may have been incorrect - were you assuming SIA? In that case the coin odds are (1/2,1/2) and (2/3,1/3), and ADT would reach different conclusions. But only because the problem was underspecified (giving anthropic probabilities without explaining the theory that goes with them is not specifying the problem).
As long as you give a full specification of the problem, ADT doesn't have an issue. You don't need to adjust free parameters or anything.
I feel like I'm missing something here. Can you explain the hole in ADT you seem to find so glaring?
I intended "In both of these problems there are two worlds, "H" and "T," which have equal "no anthropics" probabilities of 0.5. "
In retrospect, my example of evidence (stopping some of the experiments) wasn't actually what I wanted, since an outside observer would notice it. In order to mess with anthropic probabilities in isolation you'd need to change the structure of coinflips and people-creation.
But you can't mess with the probabilities in isolation. Suppose I were an SIA agent, for instance; then you can't change my anthropic probabilities without changing non-anthropic facts about the world.
I'm uncertain whether what you're saying is relevant. The question at hand is, is there some change to a problem that changes anthropic probabilities, but is guaranteed not to change ADT decisions? Such a change would have to conserve the number of worlds, the number of people in each world, the possible utilities, and the "no anthropics" probabilities
For example, if my anthropic knowledge says that I'm an agent at a specific point in time, a change in how long Sleeping Beauty stays awake in different "worlds" will change how likely I am to find myself there overall.
Is there? It would require some sort of evidence that would change your own anthropic probabilities, but that would not change the opinion of any outside observer if they saw it.
Doesn't feel like that would work... if you remember how long you've been awake, that makes you into slightly different agents, and if the duration of the awakening gives you any extra info, it would show up in ADT too. And if you forget how long you're awake, that's just sleeping beauty with more awakenings...
Define "individual impact" as the belief that your own actions have no correlations with those of your copies (the belief your decisions control all your copies is "total impact"). Then ADT basically has the following equivalences:
If those equivalences are true, it seems that we cannot vary the anthropic probabilities without varying the ADT decision.
Thanks!