## SIA fears (expected) infinity

6 12 November 2012 05:23PM

It's well known that the Self-Indication Assumption (SIA) has problems with infinite populations (one of the reasons I strongly recommend not using the probability as the fundamental object of interest, but instead the decision, as in anthropic decision theory).

SIA also has problems with arbitrarily large finite populations, at least in some cases. What cases are these? Imagine that we had these (non-anthropic) probabilities for various populations:

p0, p1, p2, p3, p4...

Now let us apply the anthropic correction from SIA; before renormalising, we have these weights for different population levels:

0, p1, 2p2, 3p3, 4p4...

To renormalise, we need to divide by the sum 0 + p1 + 2p2 + 3p3 + 4p4... This is actually the expected population! (note: we are using the population as a proxy for the size of the reference class of agents who are subjectively indistinguishable from us; see this post for more details)

So using SIA is possible if and only if the (non-anthropic) expected population is finite (and non-zero).

Note that it is possible for the anthropic expected population to be infinite! For instance if pj is C/j3, for some constant C, then the non-anthropic expected population is finite (being the infinite sum of C/j2). However once we have done the SIA correction, we can see that the SIA-corrected expected population is infinite (being the infinite sum of some constant times 1/j).

## No Anthropic Evidence

8 23 September 2012 10:33AM

Closely related to: How Many LHC Failures Is Too Many?

Consider the following thought experiment. At the start, an "original" coin is tossed, but not shown. If it was "tails", a gun is loaded, otherwise it's not. After that, you are offered a big number of rounds of decision, where in each one you can either quit the game, or toss a coin of your own. If your coin falls "tails", the gun gets triggered, and depending on how the original coin fell (whether the gun was loaded), you either get shot or not (if the gun doesn't fire, i.e. if the original coin was "heads", you are free to go). If your coin is "heads", you are all right for the round. If you quit the game, you will get shot at the exit with probability 75% independently of what was happening during the game (and of the original coin). The question is, should you keep playing or quit if you observe, say, 1000 "heads" in a row?

Intuitively, it seems as if 1000 "heads" is "anthropic evidence" for the original coin being "tails", that the long sequence of "heads" can only be explained by the fact that "tails" would have killed you. If you know that the original coin was "tails", then to keep playing is to face the certainty of eventually tossing "tails" and getting shot, which is worse than quitting, with only 75% chance of death. Thus, it seems preferable to quit.

On the other hand, each "heads" you observe doesn't distinguish the hypothetical where the original coin was "heads" from one where it was "tails". The first round can be modeled by a 4-element finite probability space consisting of options {HH, HT, TH, TT}, where HH and HT correspond to the original coin being "heads" and HH and TH to the coin-for-the-round being "heads". Observing "heads" is the event {HH, TH} that has the same 50% posterior probabilities for "heads" and "tails" of the original coin. Thus, each round that ends in "heads" doesn't change the knowledge about the original coin, even if there were 1000 rounds of this type. And since you only get shot if the original coin was "tails", you only get to 50% probability of dying as the game continues, which is better than the 75% from quitting the game.

(See also the comments by simon2 and Benja Fallenstein on the LHC post, and this thought experiment by Benja Fallenstein.)

The result of this exercise could be generalized by saying that counterfactual possibility of dying doesn't in itself influence the conclusions that can be drawn from observations that happened within the hypotheticals where one didn't die. Only if the possibility of dying influences the probability of observations that did take place, would it be possible to detect that possibility. For example, if in the above exercise, a loaded gun would cause the coin to become biased in a known way, only then would it be possible to detect the state of the gun (1000 "heads" would imply either that the gun is likely loaded, or that it's likely not).

## Request for feedback: paper on fine-tuning and the multiverse hypothesis

6 29 April 2012 09:19PM

A while back, I posted in the "What are you working on?" thread about a paper I was working on. A few people wanted to see it once I have a complete draft, and I'm of course independently interested in obtaining feedback before I move on with it.

The paper doesn't presuppose much philosophical jargon that isn't easily googleable, I think. Math-wise, you need to be somewhat comfortable with basic conditional probabilities. I'm interested in finding out about any math errors, other non sequiturs, and other flaws in my discussion. I'd also like to find out about general impressions, such as what I should have spilled more or less ink on. Some notation is unfinished (subscripts, singular/plural first person, etc.), but it's thoroughly readable.

ABSTRACT: According to a standard form of the fine-tuning argument, the apparent anthropic fine-tuning of the physical constants and boundary conditions of our universe confirms the multiverse hypothesis. According to the inverse gambler’s fallacy objection, this view is mistaken: although the multiverse hypothesis makes the existence of a life-permitting universe more probable than it would be on a single-universe theory, it does not make it any more probable that our universe should be life-permitting, and thus is not confirmed by our total evidence. We examine recent replies to this objection and conclude that they all fall short, usually due to a shared weakness. We then show how a synthetic reply, obtained by combining independent insights from the literature, can overcome the weakness afflicting its predecessors.

If you'd like a slightly more detailed description before deciding whether or not to read the whole thing, see my post.

Here is the actual paper: DOCX PDF (on some computers, italicized Times New Roman looks weird in the PDF)

EDIT 5/9/12: Current draft (edited, shortened to 13.5K words) is here:

DOCX: http://bit.ly/Jc4pXr

NOTE: The paper occasionally makes use of the notion of a person as a metaphysical individual. Roughly and likely inaccurately, this is the concept of an individual essence that can only be instantiated once in a possible world and is partly independent of the physical pattern it inhabits (i.e. you can have different possible worlds that are physically identical but contain different individuals -- I think this is what Eliezer refers to as "the philosophical notion of indexical identity apart from pattern identity"). I personally find this concept unmotivated to say the least; it figures in the paper only because some of the arguments discussed rely on it; and it is inessential for my proposed reply. If you're going to weight in on this, I'd rather you make suggestions as to how I could gracefully express that I find the concept unhelpful while still engaging with the arguments.

## The hundred-room problem

0 21 January 2012 06:12PM

This thought-experiment has been on my mind for a couple of days, and no doubt it's a special case of a more general problem identified somewhere by some philosopher that I haven't heard of yet. It goes like this:

You are blindfolded, and then scanned, and ninety-nine atom-for-atom copies of you are made, each blindfolded, meaning a hundred in all. To each one is explained (and for the sake of the thought experiment, you can take this explanation as true (p is approx. 1)) that earlier, a fair-coin was flipped. If it came down heads, ninety-nine out of a hundred small rooms were painted red, and the remaining one was painted blue. If it came down tails, ninety-nine out of a hundred small rooms were painted blue, and the remaining one was painted red. Now, put yourself in the shoes of just one of these copies. When asked what the probability is that the coin came down tails, you of course answer “.5”. It is now explained to you that each of the hundred copies is to be inserted into one of the hundred rooms, and will then be allowed to remove their blindfolds. You feel yourself being moved, and then hear a voice telling you you can take your blindfold off. The room you are in is blue. The voice then asks you for your revised probability estimate that the coin came down tails.

It seems at first (or maybe at second, depending on how your mind works) that the answer ought to be .99 – ninety-nine out of the hundred copies will, if they follow the rule “if red, then heads, if blue then tails”, get the answer right.

However, it also seems like the answer ought to be .5, because you have no new information to update on. You already knew that at least one copy of you would, at this time, remove their blindfold and find themselves in a blue room. What have you discovered that should allow you to revise your probability of .5 to .99?

And the answer, of course, cannot be both .5 and .99. Something has to give.

Is there something basically quite obvious that I'm missing that will resolve this problem, or is it really the mean sonofabitch it appears to be? As it goes, I'm inclined to say the probability is .5 – I'm just not quite sure why. Thoughts?

## Yet another Sleeping Beauty

-1 22 November 2011 05:00AM

leeping Beauty is put to sleep on Sunday. If the coin lands on heads, she is awakened only on Monday. If it lands on tails, she is awaken on Monday and Tuesday, and has her memory erased between them. Each time she is awoken, she is asked how likely it is the coin landed on tails.

According to the one theory, she would figure it's twice as likely to be her if the coin landed on tails, so it's now twice as likely to be tales. According to another, she would figure that the world she's in isn't eliminated by heads or tails, so it's equally likely. I'd like to use the second possibility, and add a simple modification:

The coin is tossed a second time. She's shown the result of this toss on Monday, and the opposite on Tuesday (if she's awake for it). She wakes up, and believes that there are four equally probable results: HH, HT, TH, and TT. She then is shown heads. This will happen at some point unless the coin has the result HT. In that case, she is only woken once, and is shown tails. She now spreads the probability between the remaining three outcomes: HH, TH, and TT. She is asked how likely it is that the coin landed on heads. She gives 1/3. Thanks to this modification, she got the same answer as if she had used SIA.

Now suppose that, instead of being told the result of second coin toss, she had some other observation. Perhaps she observed how tired she was when she woke up, or how long it took to open her eyes, or something else. In any case, if it's an unlikely observation, it probably won't happen twice, so she's about twice as likely to make it if she wakes up twice.

Edit: SIA and SSA don't seem to be what I thought they were. In both cases, you get approximately 1/3. As far as I can figure, the reason Wikipedia states that you get 1/2 with SIA is that it uses sleeping beauty during the course of this experiment as the entire reference class (rather than all existent observers). I've seen someone use this logic before (they only updated on the existence of such an observer). Does anyone know what it's called?

## Anthropics Does Not Work LIke That

-1 12 August 2011 06:52PM

People around here seem to think that a recent series of near-misses, such as not destroying the world in the Cold War, is evidence in favor of quantum immortality.

This fails to appreciate that the anthropic selection bias has no limit on how far back it can make things retroactively seem to happen. If, as has been suggested, a majority of the Everett branches from our 1950 destroyed the world, then it is equally true that a majority of the Everett branches from our 1750 in which there is someone still alive in 2010 failed to contain probably-world-destroying technology.

The existence of x-risk near-miss events should be taken as evidence against quantum immortality.

## Revisiting the anthropic trilemma III: solutions and interpretations

2 17 February 2011 03:14PM

In previous posts, I revisited Eliezer's anthropic trilemma, approaching it with ata's perspective that the decisions made are the objects of fundamental interest, not the probabilities or processes that gave rise to them. I initially applied my naive intuitions to the problem, and got nonsense. I then constructed a small collection of reasonable-seeming assumptions, and showed they defined a single method of spreading utility functions across copies.

This post will apply that method to the anthropic trilemma, and thus give us the "right" decisions to make. I'll then try and interpret these decisions, and see what they tell us about subjective anticipation, probabilities and the impact of decisions. As in the original post, I will be using the chocolate bar as the unit of indexical utility, as it is a well known fact that everyone's utility is linear in chocolate.

The details of the lottery winning setup can be found either here or here. The decisions I must make are:

Would I give up a chocolate bar now for two to be given to one of the copies if I win the lottery? No, this loses me one utility and gains me only 2/million.

Would I give up a chocolate bar now for two to given to every copy if I win the lottery? Yes, this loses me one utility and gains me 2*trillion/million = 2 million.

Would I give up one chocolate bar now, for two chocolate bars to the future merged me if I win the lottery? No, this gives me an expected utility of -1+2/million.

Now let it be after the lottery draw, after the possible duplication, but before I know whether I've won the lottery or not. Would I give up one chocolate bar now in exchange for two for me, if I had won the lottery (assume this deal is offered to everyone)? The SIA odds say that I should; I have an expected gain of 1999/1001 ≈ 2.

Now assume that I have been told I've won the lottery, so I'm one of the trillion duplicates. Would I give up a chocolate bar for the future merged copy having two? Yes, I would, the utility gain is 2-1=1.

So those are the decisions; how to interpret them? There are several ways of doing this. There are four things to keep in mind: probability, decision impact, utility function, and subjective anticipation.

## Sleeping anti-beauty and the presumptuous philosopher

1 17 February 2011 02:59PM

My approach for dividing utility between copies gives the usual and expected solutions to the sleeping beauty problem: if all copies are offered bets, take 1/3 odds, if only one copy is offered bets, take 1/2 odds.

This makes sense, because my approach is analogous to "some future version of Sleeping Beauty gets to keep all the profits".

The presumptuous philosopher problem is subtly different from the sleeping beauty problem. It can best be phrased as sleeping beauty problem where each copy doesn't care for any other copy. Solving this is a bit more subtle, but an useful half-way point is the "Sleeping Anti-Beauty" problem.

Here, as before, one or two copies are created depending on the result of a coin flip. However, if two copies are created, they are the reverse of mutually altruistic: they derive disutility from the other copy achieving its utility. So if both copies receive \$1, neither of their utilities increase: they are happy to have the cash, but angry the other copy also has cash.

Apart from this difference in indexical utility, the two copies are identical, and will reach the same decision. Now, as before, every copy is approached with bets on whether they are in the large universe (with two copies) or the small one (with a single copy). Using standard UDT/TDT Newcomb-problem type reasoning, they will always take the small universe side in any bet (as any gain/loss in the large universe is compensated for by the same gain/loss for the other copy they dislike).

Now, you could model the presumptuous philosopher by saying they have 50% chance of being in a Sleeping-Beauty (SB) situation and 50% of being in a Sleeping Anti-Beauty (SAB) situation (indifference modelled as half way between altruism and hate).

There are 4 equally likely possibilities here: small universe in SB, large universe in SB, small universe in SAB, large universe in SAB. A contract that gives \$1 in a small universe is worth 0.25 + 0 + 0.25 + 0 = \$0.5. While a contract that gives \$1 in a large universe is worth 0 + 0.25*2 + 0 + 0 = \$0.5 (as long as its offered to everyone). So it seems that a presumptuous philosopher should take even odds on the size of the universe if he doesn't care about the other presumptuous philosophers.

It's no coincidence this result can be reached by UDT-like arguments such as "take the objective probabilities of the universes, and consider the total impact of your decision being X, including all other decision that must be the same as yours". I'm hoping to find more fundamental reasons to justify this approach soon.

## Revisiting the anthropic trilemma I: intuitions and contradictions

1 15 February 2011 11:18AM

tl;dr: in which I apply intuition to the anthropic trilemma, and it all goes horribly, horribly wrong

Some time ago, Eliezer constructed an anthropic trilemma, where standard theories of anthropic reasoning seemed to come into conflict with subjective anticipation. rwallace subsequently argued that subjective anticipation was not ontologically fundamental, so we should not expect it to work out of the narrow confines of everyday experience, and Wei illustrated some of the difficulties inherent in "copy-delete-merge" types of reasoning.

Wei also made the point that UDT shifts the difficulty in anthropic reasoning away from probability and onto the utility function, and ata argued that neither the probabilities nor the utility function are fundamental, that it was the decisions that resulted from them that were important - after all, if two theories give the same behaviour in all cases, what grounds do we have for distinguishing them? I then noted that this argument could be extended to subjective anticipation: instead of talking about feelings of subjective anticipation, we could replace it by questions such as "would I give up a chocolate bar now for one of my copies to have two in these circumstances?"

In this post, I'll start by applying my intuitive utility/probability theory to the trilemma, to see what I would decide in these circumstance, and the problems that can result. I'll be sticking with classical situations rather than quantum, for simplicity.

So assume a (classical) lottery where I have ticket with million to one odds. The trilemma presented a lottery winning trick: set up the environment so that if ever I did win the lottery, a trillion copies of me would be created, they would experience winning the lottery, and then they will be merged/deleted down to one copy again.

So that's the problem; what's my intuition got to say about it? Now, my intuition claims there is a clear difference between my personal and my altruistic utility. Whether this is true doesn't matter, I'm just seeing whether my intuitions can be captured. I'll call the first my indexical utility ("I want chocolate bars") and the second my non-indexical utility ("I want everyone hungry to have a good meal"). I'll be neglecting the non-indexical utility, as it is not relevant to subjective anticipation.

Now, my intuitions tell me that SIA is the correct anthropic probability theory. It also tells me that having a hundred copies in the future all doing exactly the same thing is equivalent with having just one: therefore my current utility means I want to maximise the average utility of my future copies.

If I am a copy, then my intuitions tell me I want to selfishly maximise my own personal utility, even at the expense of my copies. However, if I were to be deleted, I would transfer my "interest" to my remaining copies. Hence my utility as a copy is my own personal utility, if I'm still alive in this universe, and the average of the remaining copies, if I'm not. This also means that if everyone is about to be deleted/merged, then I care about the single remaining copy that will come out of it, equally with myself.

Now I've setup my utility and probability; so what happens to my subjective anticipation in the anthropic trilemma? I'll use the chocolate bar as a unit of utility - because, as everyone knows, everybody's utility is linear in chocolate, this is just a fundamental fact about the universe.

First of all, would I give up a chocolate bar now for two to be given to one of the copies if I win the lottery? Certainly not, this loses me 1 utility and only gives me 2/million trillion in return. Would I give up a bar now for two to be given to every copy if I lose the lottery? No, this loses me 1 utility and only give me 2/million in return.

So I certainly do not anticipate winning the lottery through this trick.

Would I give up one chocolate bar now, for two chocolate bars to the future merged me if I win the lottery? No, this gives me an expected utility of -1+2/million, same as above.

So I do not anticipate having won the lottery through this trick, after merging.

Now let it be after the lottery draw, after the possible duplication, but before I know whether I've won the lottery or not. Would I give up one chocolate bar now in exchange for two for me, if I had won the lottery (assume this deal is offered to everyone)? The SIA odds say that I should; I have an expected gain of 1999/1001 ≈ 2.

So once the duplication has happened, I anticipate having won the lottery. This causes a preference reversal, as my previous version would pay to have my copies denied that choice.

Now assume that I have been told I've won the lottery, so I'm one of the trillion duplicates. Would I give up a chocolate bar for the future merged copy having two? Yes, I would, the utility gain is 2-1=1.

So once I've won the lottery, I anticipate continuing having won the lottery.

So, to put all these together:

• I do not anticipate winning the lottery through this trick.
• I do not anticipate having won the lottery once the trick is over.
• However, in the middle of the trick, I anticipate having won the lottery.
• This causes a money-pumpable preference reversal.
• And once I've won the lottery, I anticipate continuing to have won the lottery once the trick is over.

Now, some might argue that there are subtle considerations that make my behaviour the right one, despite the seeming contradictions. I'd rather say - especially seeing the money-pump - that my intuitions are wrong, very wrong, terminally wrong, just as non-utilitarian decision theories are.

However, what I started with was a perfectly respectable utility function. So we will need to add other consideration if we want to get an improved consistent system. Tomorrow, I'll be looking at some of the axioms and assumptions one could use to get one.

## Counterintuitive World - Good intro to some topics

2 02 October 2010 03:32AM

View more: Next