= 27d567bc2d48d9f40e9ccc20ea370b15)
Manfred comments on "Solving" selfishness for UDT - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (50)
I agree with most of this post, but not with most parts mentioning SSA/SIA or the sleeping beauty problem. In general, aside from those two areas I find your written works to be valuable resources. Now that I've said something nice, here's a long comment predictably focusing on the bad bits.
SSA and SIA, as interpreted by you, seem uninformative (treating them as two different black boxes rather than two settings on a transparent box), so I'm not surprised that you decided SSA vs SIA was meaningless. But this does not mean that anthropic probability is meaningless. Certainly you didn't prove that - you tried something else, that's all. It's analogous to how just because UDT solves Psy-Kosh's non-anthropic problem without mentioning classical probability updates, that doesn't mean classical probability updates are "meaningless."
This is the gnome's reasoning with different labels. But that doesn't mean that it has the right labels to be the human's reasoning.
It sounds like the sort of thing that a person who believed that anthropic probabilities were meaningless would write as the person's reasoning.
Let me try and give an analogy for how this sounds to me. It will be grossly unfair to you, and I apologize - pretend the content is a lot better even as the sound remains similar.
Suppose you're sitting in your room, and also in your room is a clock. Now imagine there was a gnome flying by with time dilation of 0.5. The gnome reasons as follows "I see a human and a clock moving past me together. The clock ticks at half a tick per second, and the person thinks at half normal speed, so the human sees the clock tick once per second"
My grossly unfair parody of you would then say: "Physics would be the same if I was moving past with time dilation 0.5. I'd see myself and my clock moving past me together. The clock would tick at half a tick per second, and I'd think at half normal speed, so I see the clock tick once per second."
This is the right conclusion, but it's just copying what the gnome said even when that's not appropriate.
What do I think the right way would look like? Well, it would have anthropic probabilities in it.
My full response can be found at:
http://www.fhi.ox.ac.uk/anthropics-why-probability-isnt-enough.pdf
But the gist of it is this: different people can assign different anthropic probabilities to certain problems, yet, due to having different decision theories, will make the same decision in every single case. That caused me to wonder what the meaning of "anthropic probability" was if you could shout "SIA" versus "SSA" but never actually do anything different because of this.
Probabilities are a way of encoding your knowledge about events. For example in the original Sleeping Beauty problem, probability of it being monday actually does correspond to what the agent with that probability would see if they could get on the internet, or walk outside.
Specifically, probabilities are a function of your information about events.
It seems like you disagree with this. Or maybe you just got fed up with arguments over the Sleeping Beauty problem and decided to declare the whole thing meaningless? Could you expand on that a little?
Consider what this sentence looks like if, like me, you think that probabilities are a function of the agent's information:
Here's one highly charitable rephrasing. "If I give two people different information, then there exists some set of values such that they'll make the same decision anyhow."
But this has zilch to do with anthropics. If I tell two people two different things about a coin, and ask them how much they'd pay for a candy bar that I only gave them if the coin landed heads, there exists some set of values such that these people will make the same decision.
Another reason that anthropic probabilities are different: agents can share all their information, but this will not convince a SIA agent to move to SSA or vice versa.
I don't think you understand what I'm saying about SSA and SIA. Hm. Maybe I should rewrite that post where I tried to explain this, since it had a lot of confused bits in it. Clearly you don't remember it, so I'm sure the ideas would look fresh and original.
Sorry, getting off track. I will attempt to recap:
SSA and SIA are identical to certain states of information. From these states of information, probabilities can be gotten using the details of the problem and the maximum entropy principle.
The information state identical to SSA says only that being in different possible worlds are mutually exclusive events, and that you are in some world. In the Sleeping Beauty problem, there are just two worlds, labeled Heads and Tails, and the max ent distribution is just 1/2, 1/2.
The information state identical to SIA says only that the information identical to SSA is true, and also that being in different states relative to the world (including different places or different times) are mutually exclusive and exhaustive events for you. There are then three mutually exclusive and exhaustive events, which all get 1/3 probability.
The reason why SIA doesn't make a distinction between different people and different worlds is because all these things are mutually exclusive and exhaustive - there is no such thing as a "special degree of mutual exclusivity" that might be awarded to the distinction between Heads and Tails, but not between Monday and Tuesday. All mutual exclusivity is the same.
--
Okay, so now imagine an SSA agent and an SIA agent get to talk to each other.
SSA agent says "me existing in diferent worlds is mutually exclusive and exhaustive."
"Wow, me too!" replies SIA. "Also, when I look at the world I see exactly one place and time, so those are mutually exclusive and exhaustive for me."
"Nope, that's not true for me at all." says SSA.
"Have you tried looking at the world?"
"Nope, not yet."
"Well, when you do, try and check out if you see exactly one place and time - since I'm talking to you my causal model predicts that you will."
"This conversation is just a rhetorical device, I'm really a pretty abstract entity."
"Okay then. Bye."
"See you later, and possibly also at other times as well."
Nothing particularly strange appears to be happening.
I think it's worth sorting the issue out (if you agree), so let's go slowly. Both SSA and SIA depend on priors, so you can't argue for them based on maximal entropy grounds. If the coin is biased, they will have different probabilities (so SSA+biased coin can have the same probabilities as SIA+unbiased coin and vice versa). That's probably obvious to you, but I'm mentioning it in case there's a disagreement.
Your model works, with a few tweaks. SSA starts with a probability distribution over worlds, throws away the ones where "you" don't exist (why? shush, don't ask questions!), and then locates themselves within the worlds by subdividing a somewhat arbitrary reference class. SIA starts with the same, uses the original probabilities to weigh every possible copy of themselves, sees these as separate events, and then renormalises (which is sometimes impossible, see http://lesswrong.com/lw/fg7/sia_fears_expected_infinity/).
I have to disagree with your conversation, however. Both SIA and SSA consider all statements of type "I exist in universe X and am the person in location Y" to be mutually exclusive and exhaustive. It's just that SIA stratifies by location only (and then deduces the probability of a universe by combining different locations in the same universe), while SSA first stratifies by universe and then by location.
But I still think this leads us astray. My point is different. Normally, given someone's utility, it's possible to disentangle whether someone is using a particular decision theory or a particular probability approach by observing their decisions. However, in anthropic (and Psy-Koch-like) situations, this becomes impossible. In the notation that I used in the paper I referred to, SIA+"divided responsibility" will always give the same decision as SSA+"total responsibility" (to a somewhat more arguable extent, for any fixed responsibility criteria, EDT+SSA gives the same decisions as CDT+SIA).
Since the decision is the same, this means that all the powerful arguments for using probability (which boil down to "if you don't act as if you have consistent probabilities, you'll lose utility pointlessly") don't apply in distinguishing between SIA and SSA. Thus we are not forced to have a theory of anthropic probability - it's a matter of taste whether to do so or not. Nothing hinges on whether the probability of heads is "really" 1/3 or 1/2. The full decision theory is what counts, not just the anthropic probability component.
I definitely agree that SSA + belief in a biased coin can have the same probabilities as SIA + belief in an unbiased coin. (I'm just calling them beliefs to reinforce that the thing that affects the probability directly is the belief, not that coin itself). But I think you're making an implied argument here - check if I'm right.
The implied argument would go like "because the biasedness of the coin is a prior, you can't say what the probabilities will be just from the information, because you can always change the prior."
The short answer is that the probabilities I calculated are simply for agents who "assume SSA" and "assume SIA" and have no other information.
The long answer is to explain how this interacts with priors. By the way, have you re-read the first three chapters of Jaynes recently? I have done so several times, and found it helpful.
Prior probabilities still reflect a state of information. Specifically, they reflect one's aptly named prior information. Then you learn something new, and you update, and now your probabilities are posterior probabilities and reflect your posterior information. Agents with different priors have different states of prior information.
Perhaps there was an implied argument that there's some problem with the fact that two states with different information (SSA+unbiased and SIA+biased) are giving the same probabilities for events relevant to the problem? Well, there's no problem. If we conserve information there must be differences somewhere, but they don't have to be in the probabilities used in decision-making.
Predictably, I'd prefer descriptions in terms of probability theory to mechanistic descriptions of how to get the results.
Whoops. Good point, I got SSA quite wrong. Hm. That's troubling. I think I made this mistake way back in the ambitious yet confused post I mentioned, and have been lugging it around ever since.
Consider an analogous game where a coin is flipped. If heads I get a white marble. If tails, somehow (so that this 'somehow' has a label, let's call it 'luck') I get either a white marble or a black marble. This is SSA with different labels. How does one get the probabilities from a specification like the one I gave for SIA in the sleeping beauty problem?
I think it's a causal condition, possibly because of something equivalent to "the coin flip does not affect what day it is." And I'm bad at doing this translation.
But I need to think a lot, so I'll get back to you later.
Just not a fan of Cox's theorem, eh?
And I'm still not seeing what that either assumption gives you, if your decision is already determined (by UDT, for instance) in a way that makes the assumption irrelevant.
Very much a fan. Anything that's probability-like needs to be an actual probability. I'm disputing whether anthropic probabilities are meaningful at all.
I'll delay talking about the point of all of this until later.
Probabilities are a function that represents what we know about events (where "events" is a technical term meaning things we don't control, in the context of Cox's theorem - for different formulations of probability this can take on somewhat different meanings). This is "what they mean."
As I said to lackofcheese:
If you accept that the events you're trying to predict are meaningful (e.g. "whether it's Monday or Tuesday when you look outside"), and you know Cox's theorem, then P(Monday) is meaningful, because it encodes your information about a meaningful event.
In the Sleeping Beauty problem, the answer still happens to be straightforward in terms of logical probabilities, but step one is definitely agreeing that this is not a meaningless statement.
(side note: If all your information is meaningless, that's no problem - then it's just like not knowing anything and it gets P=0.5)
As I said to lackofcheese:
In general, the question "how many copies are there" may not be answerable in certain weird situations (or can be answered only arbitrarily).
EDIT: with copying and merging and similar, you get odd scenarios like "the probability of seeing something is x, the probability of remembering seeing it is y, the probability of remembering remembering it is z, and x y and z are all different." Objectively it's clear what's going on, but in terms of "subjective anticipation", it's not clear at all.
Or put more simply: there are two identical copies of you. They will be merged soon. Do you currently have a 50% chance of dying soon?
I think that argument is highly suspect, primarily because I see no reason why a notion of "responsibility" should have any bearing on your decision theory. Decision theory is about achieving your goals, not avoiding blame for failing.
However, even if we assume that we do include some notion of responsibility, I think that your argument is still incorrect. Consider this version of the incubator Sleeping Beauty problem, where two coins are flipped.
HH => Sleeping Beauties created in Room 1, 2, and 3
HT => Sleeping Beauty created in Room 1
TH => Sleeping Beauty created in Room 2
TT => Sleeping Beauty created in Room 3
Moreover, in each room there is a sign. In Room 1 it is equally likely to say either "This is not Room 2" or "This is not Room 3", and so on for each of the three rooms.
Now, each Sleeping Beauty is offered a choice between two coupons; each coupon gives the specified amount to their preferred charity (by assumption, utility is proportional to $ given to charity), but only if each of them chose the same coupon. The payoff looks like this:
A => $12 if HH, $0 otherwise.
B => $6 if HH, $2.40 otherwise.
I'm sure you see where this is going, but I'll do the math anyway.
With SIA+divided responsibility, we have
p(HH) = p(not HH) = 1/2
The responsibility is divided among 3 people in HH-world, and among 1 person otherwise, therefore
EU(A) = (1/2)(1/3)$12 = $2.00
EU(B) = (1/2)(1/3)$6 + (1/2)$2.40 = $2.20
With SSA+total responsibility, we have
p(HH) = 1/3
p(not HH) = 2/3
EU(A) = (1/3)$12 = $4.00
EU(B) = (1/3)$6 + (2/3)$2.40 = $3.60
So SIA+divided responsibility suggests choosing B, but SSA+total responsibility suggests choosing A.
The SSA probability of HH is 1/4, not 1/3.
Proof: before opening their eyes, the SSA agents divide probability as: 1/12 HH1 (HH and they are in room 1), 1/12 HH2, 1/12 HH3, 1/4 HT, 1/4 TH, 1/4 TT.
Upon seeing a sign saying "this is not room X", they remove one possible agent from the HH world, and one possible world from the remaining three. So this gives odds of HH:¬HH of (1/12+1/12):(1/4+1/4) = 1/6:1/2, or 1:3, which is a probability of 1/4.
This means that SSA+divided responsibility says EU(A) is $3, and EU(B) is $3.3. - exactly the same ratios as the first setup, with B as the best choice.
That's not true. The SSA agents are only told about the conditions of the experiment after they're created and have already opened their eyes.
Consequently, isn't it equally valid for me to begin the SSA probability calculation with those two agents already excluded from my reference class?
Doesn't this mean that SSA probabilities are not uniquely defined given the same information, because they depend upon the order in which that information is incorporated?
Yep. The old reference class problem. Which is why, back when I thought anthropic probabilities were meaningful, I was an SIAer.
But SIA also has some issues with order of information, though it's connected with decisions ( http://lesswrong.com/lw/4fl/dead_men_tell_tales_falling_out_of_love_with_sia/ ).
Anyway, if your reference class consists of people who have seen "this is not room X", then "divided responsibility" is no longer 1/3, and you probably have to go whole UTD.
No, they have the same values. And same information. Just different decision theories (approximately CDT vs EDT).
As I have previously argued against this "different probabilities but same information" line and you just want to repeat it, I doubt there's much value in going further down this path.
The strongest argument against anthropic probabilities in decision-making comes from problems like the Absent-Minded Driver, in which the probabilities depend upon your decisions.
If anthropic probabilities don't form part of a general-purpose decision theory, and you can get the right answers by simply taking the UDT approach and going straight to optimising outcomes given the strategies you could have, what use are the probabilities?
I won't go so far as to say they're meaningless, but without a general theory of when and how they should be used I definitely think the idea is suspect.
Probabilities have a foundation independent of decision theory, as encoding beliefs about events. They're what you really do expect to see when you look outside.
This is an important note about the absent-minded driver problem et al, that gets lost if one gets comfortable in the effectiveness of UDT. The agent's probabilities are still accurate, and still correspond to the frequency with which they see things (truly!) - but they're no longer related to decision-making in quite the same way.
"The use" is then to predict, as accurately as ever, what you'll see when you look outside yourself.
And yes, probabilities can sometimes depend on decisions, not only in some anthropic problems but more generally in Newcomb-like ones. Yes, the idea of having a single unqualified belief, before making a decision, doesn't make much sense in these cases. But Sleeping Beauty is not one of these cases.
That's a reasonable point, although I still have two major criticisms of it.
1 - I don't have a general solution, there are plenty of things I'm confused about - and certain cases where anthropic probability depends on your action are at the top of the list. There is a sense in which a certain extension of UDT can handle these cases if you "pre-chew" indexical utility functions into world-state utility functions for it (like a more sophisticated version of what's described in this post, actually), but I'm not convinced that this is the last word.
Absurdity and confusion have a long (if slightly spotty) track record of indicating a lack in our understanding, rather than a lack of anything to understand.
2 - Same way that CDT gets the right answer on how much to pay for 50% chance of winning $1, even though CDT isn't correct. The Sleeping Beauty problem is literally so simple that it's within the zone of validity of CDT.
On 1), I agree that "pre-chewing" anthropic utility functions appears to be something of a hack. My current intuition in that regard is to reject the notion of anthropic utility (although not anthropic probability), but a solid formulation of anthropics could easily convince me otherwise.
On 2), if it's within the zone of validity then I guess that's sufficient to call something "a correct way" of solving the problem, but if there is an equally simple or simpler approach that has a strictly broader domain of validity I don't think you can be justified in calling it "the right way".