Treating anthropic selfish preferences as an extension of TDT

9 Manfred 01 January 2015 12:43AM

I

When preferences are selfless, anthropic problems are easily solved by a change of perspective. For example, if we do a Sleeping Beauty experiment for charity, all Sleeping Beauty has to do is follow the strategy that, from the charity's perspective, gets them the most money. This turns out to be an easy problem to solve, because the answer doesn't depend on Sleeping Beauty's subjective perception.

But selfish preferences - like being at a comfortable temperature, eating a candy bar, or going skydiving - are trickier, because they do rely on the agent's subjective experience. This trickiness really shines through when there are actions that can change the number of copies. For recent posts about these sorts of situations, see Pallas' sim game and Jan_Ryzmkowski's tropical paradise. I'm going to propose a model that makes answering these sorts of questions almost as easy as playing for charity.

To quote Jan's problem:

It's a cold cold winter. Radiators are hardly working, but it's not why you're sitting so anxiously in your chair. The real reason is that tomorrow is your assigned upload, and you just can't wait to leave your corporality behind. "Oh, I'm so sick of having a body, especially now. I'm freezing!" you think to yourself, "I wish I were already uploaded and could just pop myself off to a tropical island."

And now it strikes you. It's a weird solution, but it feels so appealing. You make a solemn oath (you'd say one in million chance you'd break it), that soon after upload you will simulate this exact scene a thousand times simultaneously and when the clock strikes 11 AM, you're gonna be transposed to a Hawaiian beach, with a fancy drink in your hand.

It's 10:59 on the clock. What's the probability that you'd be in a tropical paradise in one minute?

So question one is the probability question: what's your probability that you go to the tropical paradise? And question two is the decision problem: is this actually a good idea?

The probability question is straightforward, and is indeed about a 1000/1001 chance of tropical paradise. If this does not make sense, feel free to ask about it, or go check out these two rambling complementary posts: Deriving probabilities from causal diagrams, More marbles and Sleeping Beauty.

One might then make an argument about the decision question that goes like this: Before I swore this oath, my probability of going to a tropical island was very low. After, it was very high. Since I really like tropical islands, this is a great idea. In a nutshell, I have increased my expected utility by making this oath.

The counterargument is also simple, though: Making copies of myself has no causal effect on me. Swearing this oath does not move my body to a tropical paradise. What really happens is that I just sit there in the cold just the same, but then later I make some simulations where I lie to myself. This is not a higher-utility universe than the one where I don't swear the oath.

Hopefully you can see how this is confusing.

II

So, my proposal, in short form: You are a person. I mean this not in the abstract, non-causal, sense, where if I make a copy of you and then shoot you, "you live on." I mean that the isolated causal agent reading this is a person capable of selfish desires, where if you are one of two copies and I give the other copy a candy bar, your selfish desires for eating candy are unfulfilled1. Choose as if you were controlling the output of your decision algorithm, so that you maximize your expected utility, including selfish desires (if you have them), conditioned on the fact that you exist (I'll come back to what this last bit means in part III).

This is at its heart porting TDT to anthropic problems. When there is a decision your original body can make that creates a bunch of copies, and the copies are also faced with this decision, your decision lets you control whether you are the original or a copy. If you don't want to be a copy, as in Pallas' sim game, you have to take that into account. If you do want to be a copy, you take that into account too.

This leads to biting the bullet in Jan's tropical paradise. It is actually a good idea to take an action that, if you're the original body, creates a bunch of high-selfish-expected-utility copies that also undergo the decision you're making right now, because this decision controls whether you're one of those copies.

There is an important caveat: this only holds if you truly would like to be one of those copies. To repeat the short form, this decision algorithm assumes that you are a person trying to increase their own expected utility. These copies can't just be made and disposed of to manipulate your subjective experience - something which is possible, but is a bad idea. The copies have to be people who you would actually like to be, who go on to live long, fulfilling lives. This is not about gaming the system. It's just an extension of TDT to anthropic situations.

Interestingly, this usually gives the same results as playing for charity. Thus there's a sort of locality of money, where you make similar tradeoffs between selfish spending and charitable spending no matter how many copies of you there are.

To deliberately construct an intuitive case, imagine that you are already uploaded, and you're led into a room (a simulated room, of course) where Omega is waiting for you. Omega says hello, and asks you whether you think you're the original or a copy. "Huh? Have I been copied?" you say. Excellent, Omega says. It then presents two boxes to you, box A and box B. Box A always has some okay candy in it for you to eat (eating candy is my go-to selfish reward). If you pick box B and are the original, it is empty, but you will be copied a million times from a snapshot when you entered the room, and offered the same choice - and if you are a copy, box B contains very delicious candy to eat (and then the copies go on to live similar lives to the original). Again there's the odd property that the output of your decision algorithm controls whether you are likely a copy or not. If you would prefer to be a copy, then you should pick box B.

There's a precommitment problem here. Suppose I value my future selves by a sum of their utilities (given some zero point). Then even if being a copy was not so great (like in Pallas' sim game), I'd precommit to making as many copies as possible. But once the game starts, by my definition of selfish preferences I don't care much about whether the other copies get a selfish reward, and so I might try to fight that precommitment to raise my expected utility.

In fact, these precommitment problems crop up whenever I calculate expected value in any other way than by averaging utility among future copies. This is a statement about a small piece of population ethics, and as such, should be highly suspect - the fact that my preferred model of selfish preferences says anything about even this small subset of population ethics makes me significantly less confident that I'm right. Even though the thing it's saying seems sensible.

Footnote 1: The reader who has been following my posts may note how this identification of who has the preferences via causality makes selfish preferences well-defined no matter how many times I define the pattern "I" to map to my brain, which is good because it makes the process well-defined, but also somewhat difficult because it eliminates the last dependence on a lower level where we can think of anthropic probabilities as determined a priori, rather than depending on a definition of self grounded in decision-making as well as experiencing. On the other hand, with that level conflict gone, maybe there's nothing stopping us from thinking of anthropic probabilities on this more contingent level as "obvious" or "a priori."

III

It's worth bringing up Eliezer's anthropic trilemma (further thought by Katja Grace here). The idea is to subjectively experience winning the lottery by entering a lottery and then replicating yourself a trillion times, wake up to have the experience, and then merge back together. Thus, the argument goes, as long as probability flows along causal channels, by waking up a trillion times I have captured the subjective experience, and will go on to subjectively experience winning the lottery.

Again we can ask the two questions: What are the probabilities? And is this actually a good idea?

This is the part where I come back to explain that earlier terminology - why is it important that I specified that you condition on your own existence? When you condition on the fact that you exist, you get an anthropic probability. In the story about Omega I told above, your probability that you're the original before you enter the room is 1. But after you enter the room, if your decision algorithm chooses box B, your probability that you're the original should go down to one in a million. This update is possible because you're updating on new information about where you are in the game - you're conditioning on your own existence.

Note that I did not just say "use anthropic probabilities." When calculating expected utility, you condition on your own existence, but you most certainly do not condition on future selves' existence. After all, you might get hit by a meteor and die, so you don't actually know that you'll be around tomorrow, and you shouldn't condition on things you don't know. Thus the player at russian roulette who says "It's okay, I'll subjectively experience winning!" is making a decision by conditioning on information they do not have.

Katja Grace talks about two principles acting in the Anthropic Trilemma: Follow The Crowd, which sends your subjective experience into the branch with more people, and Blatantly Obvious Principle, which says that your subjective experience should follow causal paths. Katja points out that they do not just cause problems when merging, they also conflict when splitting - so Eliezer is being selective in applying these principles, and there's a deeper problem here. If you recall me mentioning my two-fluid model of anthropics, I partially resolved this by tracking two measures, one that obeyed FTC (subjective probability), and one that obeyed BOP (magic reality fluid).

But the model I'm presenting here dissolves those fluids (or would it be 'dilutes'?) - the thing that follows the crowd is who you think you are, and the blatantly obvious thing is your expectation for the future. There's no subjective experience fluid that it's possible to push around without changing the physical state of the universe. There's just people.

To give the probabilities in the Anthropic Trilemma, it is important to track what information you're conditioning on. If I condition on my existence just after I buy my ticket, my probability that I picked the winning numbers is small, no matter what anthropic hijinks might happen if I win, I still expect to see those hijinks happen with low probability2. If I condition on the fact that I wake up after possibly being copied, my probability that I picked the winning numbers is large, as is my probability that I will have picked the winning numbers in the future, even if I get copied or merged or what have you. Then I learn the result, and no longer have a single state of information which would give me a probability distribution. Compare this to the second horn of the trilemma; it's easy to get mixed up when giving probabilities if there's more than one set of probabilities to give.

Okay, so that's the probabilities - but is this actually a good idea? Suppose I'm just in it for the money. So I'm standing there considering whether to buy a ticket, and I condition on my own existence, and the chances of winning still look small, and so I don't buy the ticket. That's it. This is especially clear if I donate my winnings to charity - the only winning move is not to play the lottery.

Suppose then instead that I have a selfish desire to experience winning the lottery, independent of the money - does copying myself if I win help fulfill this desire? Or to put this another way, in calculating expected utility we weight the selfish utility of the many winning copies less because winning is unlikely, but do we weight it more because there are more of them?

This question is resolved by (possible warning sign) the almost-population-ethics result above, which says that as an attractor of self-modification we should average copies' utilities rather than summing them, and so copying does not increase expected utility. Again, I find this incompletely convincing, but it does seem to be the extension of TDT here. So this procedure does not bite the bullet in the anthropic trilemma. But remember the behavior in Jan's tropical paradise game? It is in fact possible to design a procedure that lets you satisfy your desire to win the lottery - just have the copies created when you win start from a snapshot of yourself before you bought the lottery ticket.

This is a weird bullet to bite. It's like, how come it's a good idea to create copies that go through the decision to create copies, but only a neutral idea to create copies that don't? After all, winning and then creating simulations has the same low chance no matter what. The difference is entirely anthropic - only when the copies also make the decision does the decision control whether you're a copy.

Footnote 2: One might complain that if you know what you'll expect in the future, you should update to believing that in the present. But if I'm going to be copied tomorrow, I don't expect to be a copy today.

IV

The problem of the Anthropic Trilemma is not actually gone, because if I'm indifferent to merging with my copies, there is some procedure that better fulfills my selfish desire to experience winning the lottery just by shuffling copies of me around: if I win, make a bunch of copies that start from a snapshot in the past, then merge a the copies together.

So let's talk about the merging. This is going to be the section with the unsolved problem.

Here's what Eliezer's post says about merging:

Just as computer programs or brains can split, they ought to be able to merge.  If we imagine a version of the Ebborian species that computes digitally, so that the brains remain synchronized so long as they go on getting the same sensory inputs, then we ought to be able to put two brains back together along the thickness, after dividing them.  In the case of computer programs, we should be able to perform an operation where we compare each two bits in the program, and if they are the same, copy them, and if they are different, delete the whole program.  (This seems to establish an equal causal dependency of the final program on the two original programs that went into it.  E.g., if you test the causal dependency via counterfactuals, then disturbing any bit of the two originals, results in the final program being completely different (namely deleted).)

In general, merging copies is some process where many identical copies go in, and only one comes out. If you know they're almost certainly identical, why bother checking them, then? Why not just delete all but one? It's the same pattern, after all.

Well, imagine that we performed a causal intervention on one of these identical copies - gave them candy or something. Now if we deleted all but one, the effect of our intervention is erased with high probability. In short, if you delete all but one, the person who comes out is not actually the causal descendant of the copies who go in - it's just one of the copies.

Just like how "selfish preferences" means that if I give another of your copies candy, that doesn't fulfill your selfish desire for candy, if another of your copies is the one who gets out of the murder-chamber, that doesn't fulfill your selfish desire to not get murdered. This is why Eliezer talks about going through the process of comparing each copy bit by bit and only merging them if they're identical, so that the person who comes out is the causal descendant of all the people who go in.

On the other hand, Eliezer's process is radically different from how things normally go. If I'm one of several copies, and a causal intervention gives me candy, and no merging shenanigans occur, then my causal descendant is me who's had some candy. If I'm one of several copies, and a causal intervention gives me candy, and then we're merged by Eliezer's method, then my causal descendant is utterly annihilated.

If we allow the character of causal arrows to matter, and not merely their existence, then it's possible that merging is not so neutral after all. But this seems like a preference issue independent of the definition of selfish preferences - although I would have said that about how to weight preferences of multiple copies, too, and I would likely have been wrong.

Does the strange behavior permitted by the neutrality of merging serve as a reductio of that neutrality, or of this extension of selfish preferences to anthropic information, or neither? In the immortal words of Socrates, "... I drank what?" 


EDIT:

A Problem:

This decision theory has precommitment issues. In the case of Jan's tropical paradise, I want to precommit to creating satisfied copies from a snapshot of my recent self. But once I'm my future self, I don't want to do it because I know I'm not a copy.

Metaproblems:

This decision theory doesn't have very many knobs to turn - it boils down to "choose the decision-algorithm output that causes maximum expected utility for you, conditioning on both the action and the information you possess." This is somewhat good news, because we don't much want free variables in a decision theory. But it's a metaproblem because it means that there's no obvious knob to turn to eliminate the problem above - creativity is required.

One approach that has worked in the past is to figure out what global variable we want to maximize, and just do UDT to this problem. But this doesn't work for this decision theory - as we expected, because it doesn't seem to work for selfish preferences in general. The selves at two different times in the tropical paradise problem just want to act selfishly - so are they allowed to be in conflict?

Solution Brainstorming (if one is needed at all):

One specific argument might run that when you precommit to creating copies, you decrease your amount of indexical information, and that this is just a form of lying to yourself and is therefore bad. I don't think this works at all, but it may be worth keeping in mind.

A more promising line might be to examine the analogy to evidential decision theory. Evidential decision theory fails when there's a difference between conditioning on the action and conditioning on a causal do(Action). What does the analogue look like for anthropic situations?

EDIT 2:

For somewhat of a resolution, see Selfish preferences and self-modification.

"Solving" selfishness for UDT

18 Stuart_Armstrong 27 October 2014 05:51PM

With many thanks to Beluga and lackofcheese.

When trying to decide between SIA and SSA, two anthropic probability theories, I concluded that the question of anthropic probability is badly posed and that it depends entirely on the values of the agents. When debating the issue of personal identity, I concluded that the question of personal identity is badly posed and depends entirely on the values of the agents. When the issue of selfishness in UDT came up recently, I concluded that the question of selfishness is...

But let's not get ahead of ourselves.

continue reading »

The Octopus, the Dolphin and Us: a Great Filter tale

48 Stuart_Armstrong 03 September 2014 09:37PM

Is intelligence hard to evolve? Well, we're intelligent, so it must be easy... except that only an intelligent species would be able to ask that question, so we run straight into the problem of anthropics. Any being that asked that question would have to be intelligent, so this can't tell us anything about its difficulty (a similar mistake would be to ask "is most of the universe hospitable to life?", and then looking around and noting that everything seems pretty hospitable at first glance...).

Instead, one could point at the great apes, note their high intelligence, see that intelligence arises separately, and hence that it can't be too hard to evolve.

One could do that... but one would be wrong. The key test is not whether intelligence can arise separately, but whether it can arise independently. Chimpanzees, Bonobos and Gorillas and such are all "on our line": they are close to common ancestors of ours, which we would expect to be intelligent because we are intelligent. Intelligent species tend to have intelligent relatives. So they don't provide any extra information about the ease or difficulty of evolving intelligence.

To get independent intelligence, we need to go far from our line. Enter the smart and cute icon on many student posters: the dolphin.

continue reading »

Anthropic Atheism

21 [deleted] 13 January 2014 05:00AM

(Crossposted from my blog)

I've been developing an approach to anthropic questions that I find less confusing than others, which I call Anthropic Atheism (AA). The name is a snarky reference to the ontologically basic status of observers (souls) in other anthropic theories. I'll have to explain myself.

We'll start with what I call the “Sherlock Holmes Axiom” (SHA), which will form the epistemic background for my approach:

How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?

Which I reinterpret as “Reason by eliminating those possibilities inconsistent with your observations. Period.” I use this as a basis of epistemology. Basically, think of all possible world-histories, assign probability to each of them according to whatever principles (eg occams razor), eliminate inconsistencies, and renormalize your probabilities. I won’t go into the details, but it turns out that probability theory (eg Bayes theorem) falls out of this just fine when you translate P(E|H) as “portion of possible worlds consistent with H that predict E”. So it’s not really any different, but using SHA as our basis, I find certain confusing questions less confusing, and certain unholy temptations less tempting.

With that out of the way, let’s have a look at some confusing questions. First up is the Doomsday Argument. From La Wik:

Simply put, it says that supposing the humans alive today are in a random place in the whole human history timeline, chances are we are about halfway through it.

The article goes on to claim that “There is a 95% chance of extinction within 9120 years.” Hard to refute, but nevertheless it makes one rather uncomfortable that the mere fact of one’s existence should have predictive consequences.

In response, Nick Bostrom formulated the “Self Indication Assumption”, which states that “All other things equal, an observer should reason as if they are randomly selected from the set of all possible observers.” Applied to the doomsday argument, it says that you are just as likely to exist in 2014 in a world where humanity grows up to create a glorious everlasting civilization, as one where we wipe ourselves out in the next hundred years, so you can’t update on that mere fact of your existence. This is comforting, as it defuses the doomsday argument.

By contrast, the Doomsday argument is the consequence of the “Self Sampling Assumption”, which states that “All other things equal, an observer should reason as if they are randomly selected from the set of all actually existent observers (past, present and future) in their reference class.”

Unfortunately for SIA, it implies that “Given the fact that you exist, you should (other things equal) favor hypotheses according to which many observers exist over hypotheses on which few observers exist.” Surely that should not follow, but clearly it does. So we can formulate another anthropic problem:

It is the year 2100 and physicists have narrowed down the search for a theory of everything to only two remaining plausible candidate theories, T1 and T2 (using considerations from super-duper symmetry). According to T1 the world is very, very big but finite, and there are a total of a trillion trillion observers in the cosmos. According to T2, the world is very, very, very big but finite, and there are a trillion trillion trillion observers. The super-duper symmetry considerations seem to be roughly indifferent between these two theories. The physicists are planning on carrying out a simple experiment that will falsify one of the theories. Enter the presumptuous philosopher: “Hey guys, it is completely unnecessary for you to do the experiment, because I can already show to you that T2 is about a trillion times more likely to be true than T1

This one is called the “presumptuous philosopher”. Clearly the presumptuous philosopher should not get a Nobel prize.

These questions have caused much psychological distress, and been beaten to death in certain corners of the internet, but as far as I know, few people have satisfactory answers. Wei Dai’s UDT might be satisfactory for this, and might be equivalent to my answer, when the dust settles.

So what’s my objection to these schemes, and what’s my scheme?

My objection is aesthetic; I don’t like that SIA and SSA seem to place some kind of ontological specialness on “observers”. This reminds me way too much of souls, which are nonsense. The whole “reference-class” thing rubs me the wrong way as well. Reference classes are useful tools for statistical approximation, not fundamental features of epistemology. So I'm hesitant to accept these theories.

Instead, I take the position that you can never conclude anything from your own existence except that you exist. That is, I eliminate all hypotheses that don’t predict my existence, and leave it at that, in accordance with SHA. No update happens in the Doomsday Argument; both glorious futures and impending doom are consistent with my existence, their relative probability comes from other reasoning. And the presumptuous philosopher is an idiot because both theories are consistent with us existing, so again we get no relative update.

By reasoning purely from consistency of possible worlds with observations, SHA gives us a reasonably principled way to just punt on these questions. Let’s see how it does on another anthropic question, the Sleeping Beauty Problem:

Sleeping Beauty volunteers to undergo the following experiment and is told all of the following details: On Sunday she will be put to sleep. Once or twice, during the experiment, Beauty will be wakened, interviewed, and put back to sleep with an amnesia-inducing drug that makes her forget that awakening. A fair coin will be tossed to determine which experimental procedure to undertake: if the coin comes up heads, Beauty will be wakened and interviewed on Monday only. If the coin comes up tails, she will be wakened and interviewed on Monday and Tuesday. In either case, she will be wakened on Wednesday without interview and the experiment ends.

Any time Sleeping Beauty is wakened and interviewed, she is asked, “What is your belief now for the proposition that the coin landed heads?”

SHA says that the coin came up heads in half of the worlds, and no further update happens based on existence. I'm slightly uncomfortable with this, because SHA is cheerfully biting a bullet that has confused many philosophers. However, I see no reason not to bite this bullet; it doesn’t seem to have any particularly controversial implications for actual decision making. If she is paid for each correct guess, for example, she'll say that she thinks the coin came up tails (this way she gets $2 half the time instead of $1 half the time for heads). If she’s paid only on Monday, she’s indifferent between the options, as she should be.

What if we modify the problem slightly, and ask sleeping beauty for her credence that it’s Monday? That is, her credence that “it” “is” Monday. If the coin came up heads, there is only Monday, but if it came up tails, there is a Monday observer and a Tuesday observer. AA/SHA reasons purely from the perspective of possible worlds, and says that Monday is consistent with observations, as is Tuesday, and refuses to speculate further on which “observer” among possible observers she “is”. Again, given an actual decision problem with an actual payoff structure, AA/SHA will quickly reach the correct decision, even while refusing to assign probabilities “between observers”.

I'd like to say that we've casually thrown out probability theory when it became inconvenient, but we haven’t; we've just refused to answer a meaningless question. The meaninglessness of indexical uncertainty becomes apparent when you stop believing in the specialness of observers. It’s like asking “What’s the probability that the Sun rather than the Earth?”. That the Sun what? The Sun and the Earth both exist, for example, but maybe you meant something else. Want to know which one this here comet is going to hit? Sure I'll answer that, but these generic “which one” questions are meaningless.

Not that I'm familiar with UDT, but this really is starting to remind me of UDT. Perhaps it even is part of UDT. In any case, Anthropic Atheism seems to easily give intuitive answers to anthropic questions. Maybe it breaks down on some edge case, though. If so, I'd like to see it. In the mean time, I don’t believe in observers.

ADDENDUM: As Wei Dai, DanielLC, and Tyrrell_McAllister point out below, it turns out this doesn't actually work. The objection is that by refusing to include the indexical hypothesis, we end up favoring universes with more variety of experiences (because they have a high chance of containing *our* experiences) and sacrificing the ability to predict much of anything. Oops. It was fun while it lasted ;)

Fundamentals of kicking anthropic butt

18 Manfred 26 March 2012 06:43AM


Galactus

Introduction

An anthropic problem is one where the very fact of your existence tells you something. "I woke up this morning, therefore the earth did not get eaten by Galactus while I slumbered." Applying your existence to certainties like that is simple - if an event would have stopped you from existing, your existence tells you that that it hasn't happened. If something would only kill you 99% of the time, though, you have to use probability instead of deductive logic. Usually, it's pretty clear what to do. You simply apply Bayes' rule: the probability of the world getting eaten by Galactus last night is equal to the prior probability of Galactus-consumption, times the probability of me waking up given that the world got eaten by Galactus, divided by the probability that I wake up at all. More exotic situations also show up under the umbrella of "anthropics," such as getting duplicated or forgetting which person you are. Even if you've been duplicated, you can still assign probabilities. If there are a hundred copies of you in a hundred-room hotel and you don't know which one you are, don't bet too much that you're in room number 68.

But this last sort of problem is harder, since it's not just a straightforward application of Bayes' rule. You have to determine the probability just from the information in the problem. Thinking in terms of information and symmetries is a useful problem-solving tool for getting probabilities in anthropic problems, which are simple enough to use it and confusing enough to need it. So first we'll cover what I mean by thinking in terms of information, and then we'll use this to solve a confusing-type anthropic problem.

continue reading »

Preference For (Many) Future Worlds

18 wedrifid 15 July 2011 11:31PM

Followup to: Quantum Russian Roulette; The Domain of Your Utility Function

The only way to win is cheat
And lay it down before I'm beat
and to another give my seat
for that's the only painless feat.

Suicide is painless
It brings on many changes
and I can take or leave it if I please.

-- M.A.S.H.

Let us pretend, for the moment, that we are rational Expected Utility Maximisers. We make our decisions with the intention of achieving outcomes that we judge to have high utility. Outcomes that satisfy our preferences. Since developments in physics have led us to abandon the notion of a simple single future world our decision making process must now grapple with the notion that some of our decisions will result in more than one future outcome. Not simply the possibility of more than one future outcome but multiple worlds, each of which with different events occurring. In extreme examples we can consider the possibility of staking our very lives on the toss of a quantum die, figuring that we are going to live in one world anyway!

How do preferences apply when making decisions with Many Worlds? The description I’m giving here will be obvious to the extent of being trivial to some, confusing to others and, I expect, considered outright wrong by others. But it is the post that I want to be able to link to whenever the question “Do you believe in quantum immortality?” comes up. Because it is a wrong question!

continue reading »

On the Anthropic Trilemma

33 KatjaGrace 19 May 2011 07:30PM

Cross-posted from Meteuphoric.

Eliezer's Anthropic Trilemma:

So here's a simple algorithm for winning the lottery: Buy a ticket.  Suspend your computer program just before the lottery drawing - which should of course be a quantum lottery, so that every ticket wins somewhere.  Program your computational environment to, if you win, make a trillion copies of yourself, and wake them up for ten seconds, long enough to experience winning the lottery.  Then suspend the programs, merge them again, and start the result.  If you don't win the lottery, then just wake up automatically. The odds of winning the lottery are ordinarily a billion to one.  But now the branch in which you win has your "measure", your "amount of experience", temporarily multiplied by a trillion.  So with the brief expenditure of a little extra computing power, you can subjectively win the lottery - be reasonably sure that when next you open your eyes, you will see a computer screen flashing "You won!"  As for what happens ten seconds after that, you have no way of knowing how many processors you run on, so you shouldn't feel a thing.

See the original post for assumptions, what merging minds entails etc. He proposes three alternative bullets to bite: accepting that this would work, denying that there is "any meaningful sense in which I can anticipate waking up as myself tomorrow, rather than Britney Spears" so undermining any question about what you should anticipate, and Nick Bostrom's response, paraphrased by Eliezer:

...you should anticipate winning the lottery after five seconds, but anticipate losing the lottery after fifteen seconds. To bite this bullet, you have to throw away the idea that your joint subjective probabilities are the product of your conditional subjective probabilities.  If you win the lottery, the subjective probability of having still won the lottery, ten seconds later, is ~1.  And if you lose the lottery, the subjective probability of having lost the lottery, ten seconds later, is ~1.  But we don't have p("experience win after 15s") = p("experience win after 15s"|"experience win after 5s")*p("experience win after 5s") + p("experience win after 15s"|"experience not-win after 5s")*p("experience not-win after 5s").

I think I already bit the bullet about there not being a meaningful sense in which I won't wake up as Britney Spears. However I would like to offer a better, relatively bullet biting free solution.

continue reading »

Anthropics makes sense with shorter people

24 KatjaGrace 11 April 2011 10:58PM

Reproduced from Meteuphoric by request.

Often people think that various forms of anthropic reasoning require you to change your beliefs in ways other than conditionalizing on evidence. This is false, at least in the cases I know of. I shall talk about Frank Arntzenius' paper Some Problems for Conditionalization and Reflection [gated] because it explains the issue well, though I believe his current views agree with mine.

He presents five thought experiments: Two Roads to Shangri La, The Prisoner, John Collins's Prisoner, Sleeping Beauty and Duplication. In each of them, it seems the (arguably) correct answer violates van Fraassen's reflection principle, which basically says that if you expect to believe something in the future without having been e.g. hit over the head between now and then, you should believe it now. For instance the thirder position in Sleeping Beauty seems to violate this principle because before the experiment Beauty believes there is a fifty percent chance of heads, and that when she wakes up she will think there is a thirty three percent chance. Arntzenius argued that these seemingly correct answers really are the correct ones, and claimed that they violate the reflection principle because credences can evolve in two ways other than by conditionalization.

First he said credences can shift, for instance through time. I know that tomorrow I will have a higher credence in it being Monday than I do today, and yet it would not be rational for me to increase my credence in it being Monday now on this basis. They can also 'spread out'. For instance if you know you are in Fairfax today, and that tomorrow a perfect replica of your brain experiencing Fairfax will be made and placed in a vat in Canberra, tomorrow your credence will go from being concentrated in Fairfax to being spread between there and Canberra. This is despite no damage having been done to your own brain. As Arntzenius pointed out, such an evolution of credence looks like quite the opposite of conditionalization, since conditionalization consists of striking out possibilities that your information excludes - it never opens up new possibilities.

I agree that beliefs should evolve in these two ways. However they are both really conditionalization, just obscured. They make sense as conditionalization when you think of them as carried out by different momentary agents, based on the information they infer from their connections to other momentary agents with certain beliefs (e.g. an immediately past self).

Normal cases can be considered this way quite easily. Knowing that you are the momentary agent that followed a few seconds after an agent who knew a certain set of facts about the objective world, and who is (you assume) completely trustworthy, means you can simply update the same prior with those same facts and come to the same conclusion. That is, you don't really have to do anything. You can treat a stream of moments as a single agent. This is what we usually do.

However sometimes being connected in a certain way to another agent does not make everything that is true for them true for you. Most obviously, if they are a past self and know it is 12 o clock, your connection via being their one second later self means you should exclude worlds where you are not at time 12:00:01. You have still learned from your known relationship to that agent and conditionalized, but you have not learned that what is true of them is true of you, because it isn't. This is the first way Arntzenius mentioned that credences seem to evolve through time not by by conditionalization.

The second way occurs when one person-moment is at location X, and another person moment has a certain connection to the person at X, but there is more than one possible connection of that sort. For instance when two later people both remember being an earlier person because the earlier person was replicated in some futuristic fashion. Then while the earlier person moment could condition on their exact location, the later one must condition on being in one of several locations connected that way to the earlier person's location, so their credence spreads over more possibilities than that of the earlier self. If you call one of these later momentary agents the same person as the earlier one, and say they are conditionalizing, it seems they are doing it wrong. But considered as three different momentary people learning from their connections they are just conditionalizing as usual.

What exactly the later momentary people should believe is a matter of debate, but I think that can be framed entirely as a question of what their state spaces and priors look like.

Momentary humans almost always pass lots of information from one to the next, chronologically along chains of memory through non-duplicated people, knowing their approximate distance from one another. So most of the time they can treat themselves as single units who just have to update on any information coming from outside, as I explained. But conditionalization is not specific to these particular biological constructions; and when it is applied to information gained through other connections between agents, the resulting time series of beliefs within one human will end up looking different to that in a chain with no unusual extra connections.

This view also suggests that having cognitive defects, such as memory loss, should not excuse anyone from having credences, as for instance Arntzenius argued it should in his paper Reflections on Sleeping Beauty: "in the face of forced irrational changes in one's degrees of belief one might do best simply to jettison them altogether". There is nothing special about credences derived from beliefs of a past agent you identify with. They are just another source of information. If the connection to other momentary agents is different to usual, for instance through forced memory loss, update on it as usual.

 

Conditioning on Observers

6 Jonathan_Lee 11 May 2010 05:15AM

Response to Beauty quips, "I'd shut up and multiply!"

Related to The Presumptuous Philosopher's Presumptuous FriendThe Absent-Minded Driver, Sleeping Beauty gets counterfactually mugged

This is somewhat introductory. Observers play a vital role in the classic anthropic thought experiments, most notably the Sleeping Beauty and Presumptuous Philosopher gedankens. Specifically, it is remarkably common to condition simply on the existence of an observer, in spite of the continuity problems this raises. The source of confusion appears to be based on the distinction between the probability of an observer and the expectation number of observers, with the former not being a linear function of problem definitions.

There is a related difference between the expected gain of a problem and the expected gain per decision, which has been exploited in more complex counterfactual mugging scenarios. As in the case of the 1/2 or 1/3 confusion, the issue is the number of decisions that are expected to be made, and recasting problems so that there is at most one decision provides a clear intuition pump.

continue reading »

Beauty quips, "I'd shut up and multiply!"

6 neq1 07 May 2010 02:34PM

When it comes to probability, you should trust probability laws over your intuition.  Many people got the Monty Hall problem wrong because their intuition was bad.  You can get the solution to that problem using probability laws that you learned in Stats 101 -- it's not a hard problem.  Similarly, there has been a lot of debate about the Sleeping Beauty problem.  Again, though, that's because people are starting with their intuition instead of letting probability laws lead them to understanding.

The Sleeping Beauty Problem

On Sunday she is given a drug that sends her to sleep. A fair coin is then tossed just once in the course of the experiment to determine which experimental procedure is undertaken. If the coin comes up heads, Beauty is awakened and interviewed on Monday, and then the experiment ends. If the coin comes up tails, she is awakened and interviewed on Monday, given a second dose of the sleeping drug, and awakened and interviewed again on Tuesday. The experiment then ends on Tuesday, without flipping the coin again. The sleeping drug induces a mild amnesia, so that she cannot remember any previous awakenings during the course of the experiment (if any). During the experiment, she has no access to anything that would give a clue as to the day of the week. However, she knows all the details of the experiment.

Each interview consists of one question, "What is your credence now for the proposition that our coin landed heads?"

Two popular solutions have been proposed: 1/3 and 1/2

The 1/3 solution

From wikipedia:

Suppose this experiment were repeated 1,000 times. We would expect to get 500 heads and 500 tails. So Beauty would be awoken 500 times after heads on Monday, 500 times after tails on Monday, and 500 times after tails on Tuesday. In other words, only in a third of the cases would heads precede her awakening. So the right answer for her to give is 1/3.

Yes, it's true that only in a third of cases would heads precede her awakening.

Radford Neal (a statistician!) argues that 1/3 is the correct solution.

This [the 1/3] view can be reinforced by supposing that on each awakening Beauty is offered a bet in which she wins 2 dollars if the coin lands Tails and loses 3 dollars if it lands Heads. (We suppose that Beauty knows such a bet will always be offered.) Beauty would not accept this bet if she assigns probability 1/2 to Heads. If she assigns a probability of 1/3 to Heads, however, her expected gain is 2 × (2/3) − 3 × (1/3) = 1/3, so she will accept, and if the experiment is repeated many times, she will come out ahead.

Neal is correct (about the gambling problem).

These two arguments for the 1/3 solution appeal to intuition and make no obvious mathematical errors.   So why are they wrong?

Let's first start with probability laws and show why the 1/2 solution is correct. Just like with the Monty Hall problem, once you understand the solution, the wrong answer will no longer appeal to your intuition.

The 1/2 solution

P(Beauty woken up at least once| heads)=P(Beauty woken up at least once | tails)=1.  Because of the amnesia, all Beauty knows when she is woken up is that she has woken up at least once.  That event had the same probability of occurring under either coin outcome.  Thus, P(heads | Beauty woken up at least once)=1/2.  You can use Bayes' rule to see this if it's unclear.

Here's another way to look at it:

If it landed heads then Beauty is woken up on Monday with probability 1.

If it landed tails then Beauty is woken up on Monday and Tuesday.  From her perspective, these days are indistinguishable.  She doesn't know if she was woken up the day before, and she doesn't know if she'll be woken up the next day.  Thus, we can view Monday and Tuesday as exchangeable here.

A probability tree can help with the intuition (this is a probability tree corresponding to an arbitrary wake up day):

If Beauty was told the coin came up heads, then she'd know it was Monday.  If she was told the coin came up tails, then she'd think there is a 50% chance it's Monday and a 50% chance it's Tuesday.  Of course, when Beauty is woken up she is not told the result of the flip, but she can calculate the probability of each.

When she is woken up, she's somewhere on the second set of branches.  We have the following joint probabilities: P(heads, Monday)=1/2; P(heads, not Monday)=0; P(tails, Monday)=1/4; P(tails, Tuesday)=1/4; P(tails, not Monday or Tuesday)=0.  Thus, P(heads)=1/2.

Where the 1/3 arguments fail

The 1/3 argument says with heads there is 1 interview, with tails there are 2 interviews, and therefore the probability of heads is 1/3.  However, the argument would only hold if all 3 interview days were equally likely.  That's not the case here. (on a wake up day, heads&Monday is more likely than tails&Monday, for example).

Neal's argument fails because he changed the problem. "on each awakening Beauty is offered a bet in which she wins 2 dollars if the coin lands Tails and loses 3 dollars if it lands Heads."  In this scenario, she would make the bet twice if tails came up and once if heads came up.  That has nothing to do with probability about the event at a particular awakening.  The fact that she should take the bet doesn't imply that heads is less likely.  Beauty just knows that she'll win the bet twice if tails landed.  We double count for tails.

Imagine I said "if you guess heads and you're wrong nothing will happen, but if you guess tails and you're wrong I'll punch you in the stomach."  In that case, you will probably guess heads.  That doesn't mean your credence for heads is 1 -- it just means I added a greater penalty to the other option.

Consider changing the problem to something more extreme.  Here, we start with heads having probability 0.99 and tails having probability 0.01.  If heads comes up we wake Beauty up once.  If tails, we wake her up 100 times.  Thirder logic would go like this:  if we repeated the experiment 1000 times, we'd expect her woken up 990 after heads on Monday, 10 times after tails on Monday (day 1), 10 times after tails on Tues (day 2),...., 10 times after tails on day 100.  In other words, ~50% of the cases would heads precede her awakening. So the right answer for her to give is 1/2.

Of course, this would be absurd reasoning.  Beauty knows heads has a 99% chance initially.  But when she wakes up (which she was guaranteed to do regardless of whether heads or tails came up), she suddenly thinks they're equally likely?  What if we made it even more extreme and woke her up even more times on tails?

Implausible consequence of 1/2 solution?

Nick Bostrom presents the Extreme Sleeping Beauty problem:

This is like the original problem, except that here, if the coin falls tails, Beauty will be awakened on a million subsequent days. As before, she will be given an amnesia drug each time she is put to sleep that makes her forget any previous awakenings. When she awakes on Monday, what should be her credence in HEADS?

He argues:

The adherent of the 1/2 view will maintain that Beauty, upon awakening, should retain her credence of 1/2 in HEADS, but also that, upon being informed that it is Monday, she should become extremely confident in HEADS:
P+(HEADS) = 1,000,001/1,000,002

This consequence is itself quite implausible. It is, after all, rather gutsy to have credence 0.999999% in the proposition that an unobserved fair coin will fall heads.

It's correct that, upon awakening on Monday (and not knowing it's Monday), she should retain her credence of 1/2 in heads.

However, if she is informed it's Monday, it's unclear what she conclude.  Why was she informed it was Monday?  Consider two alternatives.

Disclosure process 1:  regardless of the result of the coin toss she will be informed it's Monday on Monday with probability 1

Under disclosure process 1, her credence of heads on Monday is still 1/2.

Disclosure process 2: if heads she'll be woken up and informed that it's Monday.  If tails, she'll be woken up on Monday and one million subsequent days, and only be told the specific day on one randomly selected day.

Under disclosure process 2, if she's informed it's Monday, her credence of heads is 1,000,001/1,000,002.  However, this is not implausible at all.  It's correct.  This statement is misleading: "It is, after all, rather gutsy to have credence 0.999999% in the proposition that an unobserved fair coin will fall heads."  Beauty isn't predicting what will happen on the flip of a coin, she's predicting what did happen after receiving strong evidence that it's heads.

ETA (5/9/2010 5:38AM)

If we want to replicate the situation 1000 times, we shouldn't end up with 1500 observations.  The correct way to replicate the awakening decision is to use the probability tree I included above. You'd end up with expected cell counts of 500, 250, 250, instead of 500, 500, 500.

Suppose at each awakening, we offer Beauty the following wager:  she'd lose $1.50 if heads but win $1 if tails.  She is asked for a decision on that wager at every awakening, but we only accept her last decision. Thus, if tails we'll accept her Tuesday decision (but won't tell her it's Tuesday). If her credence of heads is 1/3 at each awakening, then she should take the bet. If her credence of heads is 1/2 at each awakening, she shouldn't take the bet.  If we repeat the experiment many times, she'd be expected to lose money if she accepts the bet every time.

The problem with the logic that leads to the 1/3 solution is it counts twice under tails, but the question was about her credence at an awakening (interview).

ETA (5/10/2010 10:18PM ET)


Suppose this experiment were repeated 1,000 times. We would expect to get 500 heads and 500 tails. So Beauty would be awoken 500 times after heads on Monday, 500 times after tails on Monday, and 500 times after tails on Tuesday. In other words, only in a third of the cases would heads precede her awakening. So the right answer for her to give is 1/3.

Another way to look at it:  the denominator is not a sum of mutually exclusive events.  Typically we use counts to estimate probabilities as follows:  the numerator is the number of times the event of interest occurred, and the denominator is the number of times that event could have occurred. 

For example, suppose Y can take values 1, 2 or 3 and follows a multinomial distribution with probabilities p1, p2 and p3=1-p1-p2, respectively.   If we generate n values of Y, we could estimate p1 by taking the ratio of #{Y=1}/(#{Y=1}+#{Y=2}+#{Y=3}). As n goes to infinity, the ratio will converge to p1.   Notice the events in the denominator are mutually exclusive and exhaustive.  The denominator is determined by n.

The thirder solution to the Sleeping Beauty problem has as its denominator sums of events that are not mutually exclusive.  The denominator is not determined by n.  For example, if we repeat it 1000 times, and we get 400 heads, our denominator would be 400+600+600=1600 (even though it was not possible to get 1600 heads!).  If we instead got 550 heads, our denominator would be 550+450+450=1450.  Our denominator is outcome dependent, where here the outcome is the occurrence of heads.  What does this ratio converge to as n goes to infinity?  I surely don't know.  But I do know it's not the posterior probability of heads.

View more: Next