(or: "Please Do Anthropics with Actual Math")

The anthropic shadow argument states something like:

Anthropic principle! If the LHC had worked, it would have produced a black hole or strangelet or vacuum failure, and we wouldn't be here!

or:

You can't use "we survived the cold war without nuclear war" as evidence of anything. Because of the anthropic principle, we could have blown up the human race in the 1960's in 99% of all possible worlds and you'd still be born in one where we didn't.

This argument has already been criticized (here, here). In criticizing it myself, I first leaned on reasoning about large universes (e.g. ones where there are 100 worlds with low nuclear risk and 100 with high nuclear risk in the same universe) in a way that implies similar conclusions to SIA, thinking that SSA in a small, single-world universe would endorse anthropic shadow. I realized I was reasoning about SSA incorrectly, and actually both SSA and SIA agree in rejecting anthropic shadow, even in a single-world universe.

Recapping the Doomsday Argument

To explain SSA and SIA, I'll first recap the Doomsday Argument. Suppose, a priori, that it's equally likely that there will be 1 billion humans total, or 1 trillion; for simplicity, we'll only consider these two alternatives. We could number humans in order (numbering the humans 1, 2, ...), and assume for simplicity that each human knows their index (which is the same as knowing how many humans there have been in the past). Suppose you observe that you are one of the first 1 billion humans. How should you reason about the probability that there will be 1 billion or 1 trillion humans total?

SSA reasons as follows. To predict your observations, you should first sample a random non-empty universe (in proportion to its prior probability), then sample a random observer in that universe. Your observations will be that observer's observations, and, ontologically, you "are" that observer living in that universe.

Conditional on being in a billion-human universe, your probability of having an index between 1 and 1 billion is 1 in 1 billion, and your probability of having any other index is 0. Conditional on being in a trillion-human universe, your probability of having an index between 1 and 1 trillion is 1 in 1 trillion, and your probability of having any other index is 0.

You observe some particular index that does not exceed 1 billion; say, 45,639,104. You are 1000 more times likely to observe this index conditional on living in a billion-human universe than a trillion-human universe. Hence, you conclude that you are in a billion-human universe with 1000:1 odds.

This is called the "doomsday argument" because it implies that it's unlikely that you have a very early index (relative to the total number of humans), so humans are likely to go extinct before many more humans have been born than have already been born.

SIA implies a different conclusion. To predict your observations under SIA, you should first sample a random universe proportional to its population, then sample a random observer in that universe. The probabilities of observing each index are the same conditional on the universe, but the prior probabilities of being in a given universe have changed.

We start with 1000:1 odds in favor of the 1-trillion universe, due to its higher population. Upon observing our sub-1-billion index, we get a 1000:1 update in favor of a 1-billion universe, as with SIA. These exactly cancel out, leaving the probability of each universe at 50%.

As Bostrom points out, both SSA and SIA have major counterintuitive implications. Better anthropic theories are desired. And yet, having some explicit anthropic theory at all helps to reason in a principled way that is consistent across situations. My contention is that anthropic shadow reasoning tends not to be principled in this way, and will go away when using SSA or SIA.

Analyzing the Cold War Scenario

Let's analyze the cold war scenario as follows. Assume, for simplicity, that there is only one world with intelligent life in the universe; adding more worlds tends to shift towards SIA-like conclusions, but is otherwise similar. This world may be one of four types:

  1. High latent nuclear risk, cold war happens
  2. Low latent nuclear risk, cold war happens
  3. High latent nuclear risk, no cold war happens
  4. Low latent nuclear risk, no cold war happens

"High latent nuclear risk" means that, counterfactual on a cold war happening, there's a high (99%) risk of extinction. "Low latent nuclear risk" means that, counterfactual on a cold war happening, there's a low (10%) risk of extinction. Latent risk could vary due to factors such as natural human tendencies regarding conflict, and the social systems of cold war powers. For simplicity, assume that each type of world is equally likely a priori.

As with the doomsday argument, we need a population model. If there is no extinction, assume there are 1 billion humans who live before the cold war, and 1 billion humans who live after it. I will, insensitively, ignore the perspectives of those who live through the cold war, for the sake of simplicity. If there is extinction, assume there are 1 billion humans who live before the cold war, and none after.

The anthropic shadow argument asserts that, upon observing being post-cold-war, we should make no update in the probability of high latent nuclear risk. Let's check this claim with both SSA and SIA.

SSA first samples a universe (which, in this case, contains only one world), then samples a random observer in the universe. It samples a universe of each type with ¼ probability. There are, however, two subtypes of type-1 or type-2 universes, namely, ones with nuclear extinction or not. It samples a nuclear extinction type-1 universe with ¼ * 99% probability, a non nuclear extinction type-1 universe with ¼ * 1% probability, a nuclear extinction type-2 universe with ¼ * 10% probability, and a non nuclear extinction type-2 universe with ¼ * 90% probability.

Conditional on sampling a universe with no nuclear extinction, the sampled observer will be prior to the cold war with 50% probability, and after the cold war with 50% probability. Conditional on sampling a universe with nuclear extinction, the sampled observer will be prior to the cold war with 100% probability.

Let's first compute the prior probability of high latent nuclear risk. SSA believes you learn nothing "upon waking up" (other than that there is at least one observer, which is assured in this example), so this probability matches the prior over universes: type-1 and type-3 universes have high latent nuclear risk, and their probability adds up to 50%.

Now let's compute the posterior. We observe being post cold war. This implies eliminating all universes with nuclear extinction, and all type-3 and type-4 universes. The remaining universes each have a 50% chance of a sampled observer being post cold war, so we don't change their relative probabilities. What we're left with are non-nuclear-extinction type-1 universes (with prior probability ¼ * 1%) and non-nuclear-extinction type-2 universes (with prior probability ¼ * 90%). Re-normalizing our probabilities, we end up with 90:1 odds in favor of being in a type-2 universe, corresponding to a 1.1% posterior probability of high latent nuclear risk. This is clearly an update, showing that SSA rejects the anthropic shadow argument.

Let's try this again with SIA. We now sample universes proportional to their population times their original probability. Since universes with nuclear extinction have half the population, they are downweighted by 50% in sampling. So, the SIA prior probabilities for each universe are proportional to: ¼ * 99% * ½ for type-1 with nuclear extinction, ¼ * 1% for type-1 without nuclear extinction, ¼ * 10% * ½ for type-2 with nuclear extinction, ¼ * 90% for type-2 without nuclear extinction, ¼ for type-3, and ¼ for type-4. To get actual probabilities, we need to normalize these weights; in total, type-1 and type-3 (high latent risk) sum to 43.6%. It's unsurprising that this is less than 50%, since SIA underweights worlds with low population, which are disproportionately worlds with high latent risk.

What's the posterior probability of high latent risk under SIA? The likelihood ratios are the same as with SIA: we eliminate all universes with nuclear extinction or with no cold war (type-3 or type-4), and don't change the relative probabilities otherwise. The posterior probabilities are now proportional to: ¼ * 1% for type-1 without nuclear extinction, and ¼ * 90% for type-2 without nuclear extinction. As with SSA, we now have 90:1 odds in favor of low latent nuclear risk.

So, SSA and SIA reach the same conclusion about posterior latent risk. Their updates only differ because their priors differ; SIA learns less about latent risk from observing being after the cold war, since it already expected low latent risk due to lower population conditional on high latent risk.

Moreover, the conclusion that they reach the same posterior doesn't depend on the exact numbers used. The constants used in the odds calculation (probability of type-1 (¼), probability of type-2 (¼), probability of survival conditional on type-1 (1%), probability of survival conditional on type-2 (90%)) could be changed, and the SSA and SIA formulae would produce the same result, since the formulae use these constants in exactly the same combination.

To generalize this, SSA and SIA only disagree when universes having a non-zero posterior probability have different populations. In this example, all universes with a non-zero posterior probability have the same population, 2 billion, since we only get a different population (1 billion) conditional on nuclear extinction, and we observed being post cold war. They disagreed before taking the observation of being post cold war into account, because before that observation, there were possible universes with different populations.

Someone might disagree with this conclusion despite SSA and SIA agreeing on it, and think the anthropic shadow argument holds water. If so, I would suggest that they spell out their anthropic theory in enough detail that it can be applied to arbitrary hypothetical scenarios, like SSA and SIA. This would help to check that this is actually a consistent theory, and to check implications for other situations, so as to assess the overall plausibility of the theory. SSA and SIA's large counterintuitive conclusions imply that it is hard to formulate a consistent anthropic theory with basically intuitive conclusions across different hypothetical situations, so checking new theories against different situations is critical for developing a better theory.

Michael Vassar points out that the anthropic shadow argument would suggest finding evidence of past nuclear mass deaths in the fossil record. This is because nuclear war is unlikely to cause total extinction, and people will rebuild afterwards. We could model this as additional population after a nuclear exchange, but less than if there were no nuclear exchange. If this overall additional population is high enough, then, conditional on being in a world with high nuclear risk, a randomly selected observer is probably past the first nuclear exchange. So, if anthropic shadow considerations somehow left us with a high posterior probability of being in a high-latent-risk world with cold wars, we'd still expect not to be before the first nuclear exchange. This is a different way of rejecting the conclusions of ordinary anthropic shadow arguments, orthogonal to the main argument in this post.

Probability pumping and time travel

The LHC post notes the theoretical possibility of probability pumping: if the LHC keeps failing "randomly", we might conclude that it destroys the world if turned on, and use it as a probability pump, turning it on in some kind of catastrophic scenario. This is similar to quantum suicide.

I won't explicitly analyze the LHC scenario; it's largely similar to the cold war scenario. Instead I'll consider the implications of probability pumping more generally, such as the model of time turners in Yudkowsky's post "Causal Universes".

In this proposal, the universe is a cellular automaton, and the each use of a time turner creates a loop consisting of an earlier event where some object "comes from the future", and a later event where some object is later "sent back in time". Many possible cellular automaton histories are searched over to find ones with only consistent time loops: ones where the same object is sent back in time as comes from the future.

The probability distribution over universe histories can't be modeled as a causal Bayesian network; instead, it can be modeled as a factor graph. To form this factor graph, first create factors for each variable determined causally (not coming back from time), in the usual way of converting a Bayesian network into a factor graph. Then, add a factor that is 1 if the object going back in time matches the object coming from the future.

For simplicity, I'll assume that the objects are strings of bits (e.g. writing on a paper), and that when using the time turner, you request some number of bits from the future. The most trivial case is when requesting zero bits; in this case, there are no additional variables in the factor graph (other than, perhaps, a constant variable, which makes no difference to the calculation), and the factor is always 1, since the empty string sent back in time matches the empty string gotten from the future.

What if we request one bit from the future, and send it back exactly? We add a binary variable to the causal graph, and note that the factor is always 1. We've doubled the number of possible worlds without changing their weight (product of all factors; probability is proportional to weight). If requesting n bits and sending them back exactly, we multiply the weight by 2^n in the branch where this time turner request is made.

Suppose we flip a coin to determine whether to use a time turner, and the time turner is only used in this scenario. If the coin comes up heads, we request 10 bits from the future and send them back exactly. In the factor graph, all possibilities (tails, heads and got 0000000000, heads and got 0000000001, ...) have equal weight; therefore, the vast majority have heads. Accordingly, we would expect to observe the coin come up heads.

Conversely, imagine we request a non-zero number of bits from the time turner, flip all the bits, and send this bitwise negation back to the past. This is necessarily a different bit string, and so the factor will be 0. This means the weight of each universe is 0, and we can't get a probability distribution by re-normalizing.

If we flip a coin to determine whether to never use the time turner or to use it once to request bits that are flipped before being sent back, then the coin will always come up such that the time turner is not used, since the other possible universe histories have weight 0.

Overall, this implies that stochastic events will tend to turn out such that the time turner is used often to send the same bit-string back, and never to send a different bit-string back. This could lead to, for example, a universe where time turners are only invented by agents (such as AGIs) that have enough security to ensure that the time turners are used often to send back the same bit string and never to send back a different bit string. Without security, it's likely that some time turner, somewhere, would be used to send back a different bit string. We should expect the invention of time turners under insecure conditions to tend to result in universe destruction (setting the weight to 0), since the weights from different time turner uses are multiplicative, and anything multiplied by 0 is 0.

And so, given that we aren't in a world with enough security to ensure no time turner universe destruction events, it is entirely unsurprising that we see no time turners around, even under the assumption that they're physically possible!

Conclusion

Reasoning about non-causal factor graphs has a teleological feel to it: stochastic events arrange themselves such that time turners will tend to be used some ways and not other ways in the future. Anthropics involves similar non-causal probabilistic reasoning; if there could be 0 or 1 observers, SSA and SIA agree that we will only observe being in a universe with 1 observer (they disagree about the weighting between 1 observers and more observers, however), which means early universe events are more or less likely depending on the future. SSA additionally implies probability pumping from a subjective scenario, as in the Adam and Eve thought experiment. Chris Langan's CTMU generalizes anthropic reasoning to more general teleological principles for modeling the universe, implying the existence of God. The theory is a bit too galaxy-brained for me to accept at this time, although it's clearly onto something with relating anthropics to teleology.

Philosophically, I would suggest that anthropic reasoning results from the combination of a subjective view from the perspective of a mind, and an objective physical view-from-nowhere. The Kantian a priori (which includes the analytic and the a priori synthetic) is already subjective; Kantian spacetime is a field in which experiential phenomena appear. In reasoning about the probabilities of various universes, we imagine a "view from nowhere", e.g. where the universe is some random stochastic Turing machine. I'll call this the "universe a priori". Put this way, these a prioris are clearly different things. SSA argues that we don't learn anything upon waking up, and so our subjective prior distribution over universes should match the universe a priori; SIA, meanwhile, argues that we do learn something upon waking up, namely, that our universe is more likely to have a higher population. SSA's argument is less credible when distinguishing the Kantian a priori from the universe a priori. And, these have to be different, because even SSA agrees that we can't observe an empty universe; upon waking up, we learn that there is at least one observer.

Teleological reasoning can also show up when considering the simulation hypothesis. If the average technological civilization creates many simulations of its past (or, the pasts of alternative civilizations) in expectation, then most observers who see themselves in a technological but not post-singularity world will be in ancestor simulations. This is immediately true under SIA and is true under SSA in universe sufficiently large to ensure that at least one civilization creates many ancestor simulations. While there are multiple ways of attempting to reject the simulation argument, one is especially notable: even if most apparently pre-singularity observers are in ancestor simulations, these observers matter less to how the future plays out (and the distribution of observers' experiences) than actually pre-singularity observers, who have some role in determining how the singularity plays out. Therefore, pragmatically, it makes sense for us to talk as if we probably live pre-singularity; we have more use for money if we live pre-singularity than if we live in an ancestor simulation, so we would rationally tend to bet in favor of being pre-singularity. This reasoning, however, implies that our probabilities depend on how much different agents can influence the future, which is a teleological consideration similar to with non-causal factor graphs. I'm not sure how to resolve all this yet, but it seems important to work out a more unified theory.

New to LessWrong?

New Comment


38 comments, sorted by Click to highlight new comments since:

SSA and SIA are definitely framed as needing a physicalist view-from-nowhere, but I don't think it's necessary for anthropics or even the best way to go about it. Treat the self as fixed and the outside universe as uncertain, and you get anthropic reasoning in a much more natural (imo) way.

Exactly this. The problem with the current anthropic schools of thought is using this view-from-nowhere while simultaneously using the concept of "self" as a meaningful way of specifying a particular observer. It effectively jumps back and forth between the god's eye and first-person views with arbitrary assumptions to facilitate such transitions (e.g. treating the self as the random sample of a certain process carried out from the god's eye view). Treating the self as a given starting point and then reasoning about the world would be the way to dispel anthropic controversies. 

SIA implies a different conclusion. To predict your observations under SIA, you should first sample a random universe proportional to its population, then sample a random observer in that universe. The probabilities of observing each index are the same conditional on the universe, but the prior probabilities of being in a given universe have changed.

We start with 1000:1 odds in favor of the 1-trillion universe, due to its higher population. Upon observing our sub-1-billion index, we get a 1000:1 update in favor of a 1-billion universe, as with SIA. These exactly cancel out, leaving the probability of each universe at 50%.

I found this is a somewhat confusing/counterintuitive way to explain SIA. (Because why start out with 1000:1 odds when it is stated that the two universes are equally likely a priori?)

How I'd explain it: Suppose I want to put a credence on the hypothesis T Trillion observer universe, starting from a 50% prior on each of the the two hypotheses. I now observe my index being, say, 100.  According to my prior, what’s the expected fraction of observers that observe having index 100 who would be correct to believe T? It’s 50%.

"Fraction of observers having index 100 who would be correct to believe T" depends on both universes existing simultaneously, e.g. due to a big universe or a multiverse. But I see what you're getting at, SIA selects observers from the whole distribution over observers taken as a single bag.

But I see what you're getting at, SIA selects observers from the whole distribution over observers taken as a single bag.

That phrasing sounds right, yeah.

I wrote expected fraction in the previous comment in order to circumvent the requirement of both universes existing simultaneously. But I acknowledge that my intuition is more compelling when assuming that they all exist, or (in Sleeping Beauty or God's extreme coin toss) that the experiment is repeated many times. Still, it seems odd to expect an inconsistency between the "it just happened once" and the "it happens many times" cases..

Also, petition to officially rename anthropic shadow to anthropic gambler's fallacy XD.

This reasoning, however, implies that our probabilities depend on how much different agents can influence the future, which is a teleological consideration similar to with non-causal factor graphs. I'm not sure how to resolve all this yet, but it seems important to work out a more unified theory.

Are you aware of anthropic decision theory? It's not complicated, it appears to me to be pretty much what you're saying here nailed down a little better.

Yeah, that's a good reference.

Philosophically, I would suggest that anthropic reasoning results from the combination of a subjective view from the perspective of a mind, and an objective physical view-from-nowhere.

Note that if you only use the "objective physical view-from-nowhere" on its own, you approximately get SIA. That's because my policy only matters in worlds where Christopher King (CK) exists. Let X be the value "utility increase from CK following policy Q". Then

E[X] = E[X|CK exists]
E[X] = E[X|CK exists and A] * P(A | CK exists) + E[X|CK exists and not A] * P(not A | CK exists)

for any event A.

(Note that how powerful CK is also a random variable that affects X. After all, anthropically undead Christopher King is as good as gone. The point is that if I am calculating the utility of my policy conditional on some event (like my existence), I need to update from the physical prior.)

That being said, Solomonoff induction is first person, so starting with a physical prior isn't necessarily the best approach.

Reminds me of fully non-indexical conditioning; the probability that someone with your exact observations exists is in general higher in a universe with more population. SSA gets around this with "reference classes", although it's underdetermined how to construct one's reference class.

EDIT: But also, see Stuart Armstrong's critique about how it's reflectively inconsistent.

EDIT: But also, see Stuart Armstrong's critique about how it's reflectively inconsistent.

Oh, well that's pretty broken then! I guess you can't use "objective physical view-from-nowhere" on its own, noted.

I would also point out that FNC is not strictly a view-from-nowhere theory. The probability updates it proposes are still based on an implicit assumption of self-sampling. 

I really don't like the pragmatic argument against the simulation hypothesis. It demonstrates a common theme in anthropics which IMO is misleading the majority of discussions. By saying pre-simulation ancestors have impacts on how the singularity plays out therefore we ought to make decisions as if we are real pre-simulation people, it subtly shifts the objective of our decisions. Instead of the default objective of maximizing reward to ourselves, doing what's best for us in our world, it changes the objective to achieve a certain state of the universe concerning all the worlds, real and simulations. 

These two objectives do not necessarily coincide. They may even demand conflicting decisions. Yet it is very common for people to argue that self-locating uncertainty ought to be treated a certain way because it would result in rational decisions with the latter objective. 

SIA implies a different conclusion. To predict your observations under SIA, you should first sample a random universe proportional to its population, then sample a random observer in that universe. The probabilities of observing each index are the same conditional on the universe, but the prior probabilities of being in a given universe have changed.

We start with 1000:1 odds in favor of the 1-trillion universe, due to its higher population.

 

Can you elaborate on why under SIA we sample a universe proportional to its population? Is this because it's like taking one sample from all these universes together uniformly, as if you'd indexed everyone together? Wouldn't that kind of imply we're in the universe with infinite people, though?

It's like selecting a random observer from all possible universes taken as a single bag. E.g. if there are 2 possible universes with equal initial probability, and one has twice the population of the other, then if you select a random person across universes, you end up in the higher population one with higher probability. The doomsday argument motivates one reason why this might make sense. Also, if you imagine these alternative universes as actually existing due to some kind of big universe theory (just being very large, or many-worlds, or multiversal), then SSA and SIA will tend to agree.

SIA doesn't handle cases with infinite universes in a well-defined manner. For that you might need some modification like selecting a universe with higher probability if it has more observers per unit computation, or similar. In general, the presumptuous philosopher problem is a counterintuitive implication of SIA.

SIA can be considered (IMO more naturally) as randomly sampling you from “observers in your epistemic situation”, so it’s not so much “increasing the prior” but rather “caring about the absolute number of observers in your epistemic situation” rather than “caring about the proportion of observers in your epistemic situation” as SSA does.

This has the same end result as “up-weighting the prior then using the proportion of observers in your epistemic situation”, but I find it to be much more intuitive than that, as the latter seems to me to be overly circuitous by multiplying by population then dividing by population (as part of taking the proportion of the reference class that you comprise), rather than just taking the number we care about (number of observers in your epistemic situation) in the first place.

How to get anthropic shadow:

  • Assume that the universe is either type 1 or type 2, and that planets with both subtypes (extinction or non-extinction) exist in parallel.
  • Use SSA.

At this point, I believe you will get some difference between SSA and SIA. For maximizing the size of the shadow, you can add:

  • If you wake up after nuclear war has become possible: choose to use the reference class "people living after nuclear war became possible".

(I didn't read the whole post, sorry if you address this somewhere. Also, I ultimately don't agree with the anthropic shadow argument.)

If enough planets exist, SSA and SIA agree. If you have only type 1 and type 2 universes, you can't learn that the cold war happened, because the cold war necessarily happened. What you can learn is that you're a post cold war observer. That's more likely in type 2 than type 1 universes, since type 2 universes have more observers post cold war. So you still get an update towards type 2.

That argument only works for SSA if type 1 and type 2 planets exist in parallel.

I was talking about a model where either every planet in the multiverse is type 1, or every planet in the multiverse is type 2.

But extinction vs non-extinction is sampled separately on each planet.

Then SSA gives you an anthropic shadow.

(If your reference class is "all observers" you still get an update towards type 2, but it's weaker than for SIA. If your reference class is "post-nuclear-weapons observers", then SSA doesn't update at all.)

Suppose there are 100 planets and in type 1, 99 are destroyed, and in type 2, 10 are destroyed. Suppose your reference class is "all observers". Then, conditional on type 1, you're about 0.5% likely to observe being post cold war, and conditional on type 2, you're about 45% likely to observe being post cold war. These likelihoods are proportional to the SSA posteriors (due to equal priors).

Under SIA, you start with a ~19:10 ratio in favor of type 2 (in the subjective a priori). The likelihood ratios are the same as with SSA so the posteriors are equally weighted towards type 2. So the updates are of equal magnitude in odds space under SSA and SIA.

If your reference class is "post nuclear weapons observers" then I agree that SSA doesn't update at all. SIA also doesn't update but starts with a 90:1 prior in favor of type 2. I think this is an odd choice of reference class, and constructing your reference class to depend on your time index nullifies the doomsday argument, which is supposed to be an implication of SSA. I think choices of reference class like this will have odd reflective behavior because e.g. further cold wars in the future will be updated on by default.

Then, conditional on type 1, you're about 0.5% likely to observe being post cold war, and conditional on type 2, you're about 45% likely to observe being post cold war.

I would have thought:

  • p(post cold war | type-1) = 1/101 ~= 1%.
  • p(post cold war | type-2) = 10/110 ~= 9%.

I don't think this makes a substantive difference to the rest of your comment, though.

Under SIA, you start with a ~19:10 ratio in favor of type 2 (in the subjective a priori). The likelihood ratios are the same as with SSA so the posteriors are equally weighted towards type 2. So the updates are of equal magnitude in odds space under SSA and SIA.

Oh, I see. I think I agree that you can see SIA and SSA as equivalent updating procedures with different priors.

Nevertheless, SSA will systematically assign higher probabilities (than SIA) to latent high probabilities of disaster, even after observing themselves to be in worlds where the disasters didn't happen (at least if the multiverse + reference class is in a goldilocks zone of size and inclusivity). I think that's what the anthropic's shadow is about. If your main point is that the action is in the prior (rather than the update) and you don't dispute people's posteriors, then I think that's something to flag clearly. (Again — I apologise if you did something like this in some part of the post I didn't read!)

I think this is an odd choice of reference class, and constructing your reference class to depend on your time index nullifies the doomsday argument, which is supposed to be an implication of SSA. I think choices of reference class like this will have odd reflective behavior because e.g. further cold wars in the future will be updated on by default.

I agree it's very strange. I always thought SSA's underspecified reference classes were pretty suspicious. But I do think that e.g. Bostrom's past writings often do flag that the doomsday argument only works with certain reference classes, and often talks about reference classes that depend on time-indices.

Type-2 worlds have a 10% chance of being destroyed, not a 90% chance. So P(post cold war | type 2) = 90 / 190.

If your main point is that the action is in the prior (rather than the update) and you don’t dispute people’s posteriors, then I think that’s something to flag clearly.

My point is that the difference between SSA and SIA is in the prior. I dispute anthropic shadow posteriors since both SSA and SIA posteriors overwhelmingly believe type 2, but anthropic shadow arguers say you can't update towards type 2 using the evidence of being post cold war at all.

I won't explicitly analyze the LHC scenario; it's largely similar to the cold war scenario.


Is it? If you define your universe distribution and sampling rules the same way, you can make the math come out the same in a toy example. But consider actually living in-universe through many cold wars vs. living through many collider failures, and the kind of updates you would make after more and more cold wars that turned out fine vs. more and more failures to turn on the collider.

After living through enough cold wars that look (on public surface-level appearances) like they very narrowly avoided nuclear extinction, perhaps you look into a few of them in more detail, going beyond the easily-publicly-available historical accounts.

Upon investigating more deeply, you might find evidence that, actually, cold wars among humans aren't all that likely to lead to extinction, for ordinary reasons. (Maybe it turns out that command of the world's nuclear powers is dense with people like Stanislav Petrov for predictable reasons, or that Petrov-like behavior is actually just pretty common among humans under extreme enough duress. Or just that there are actually more safeguards in place than public surface level history implies, and that cooler heads tend to prevail across a wide variety of situations, for totally ordinary reasons of human nature, once you dig a bit deeper.)

OTOH, suppose you start investigating past collider failures in more detail, and find that all of them just happened to fail or be canceled for (what looks like) totally innocuous but independent reasons, no matter how hard you dig. Before observing a bunch of failures, you start out with a pretty high prior probability that giant particle accelerators are expensive and pretty difficult to build reliably, so it's not surprising to see a few mechanical failures or project cancellations in a row. After enough observations, you might start updating towards any of the following hypotheses:

  • Building a collider is a really hard mechanical challenge, but there's something in human nature that causes the physicists who work on them to have a blind spot and reason incorrectly that it will be easier than it actually is.
  • There's some kind of cabal of physicists / collider-building construction workers to purposely fail, in order to keep getting more grants to keep trying again. (This cabal is strong enough to control both the surface level evidence and the evidence you observe when digging deeper.)
  • Some kind of weird anthropic thing is going on (anthropic principle kicking in, you're inside a simulation or thought experiment or fictional story, etc.).
     

Suppose news reports and other surface level information aren't enough to distinguish between the three hypotheses above, but you decide to dig a bit, and start looking into the failures on your own. When you dig into the mechanical failures, you find that all the failures tend to have happened for totally independent and innocuous physical reasons. (Maybe one of the failures was due to the lead engineer inconveniently getting a piano dropped on his head or something, in a way that, on the surface, sounds suspiciously like some kind of cartoon hi-jinxes. But on closer investigation, you find that there was a totally ordinary moving company moving the piano for totally ordinary reasons, that just happened to drop it accidentally for perfectly predictable reasons, once you know the details. For another failure you investigate, maybe you learn enough about the engineering constraints on particle accelerators to conclude that, actually, avoiding some particular mechanical failure in some particularly finicky part is just really difficult and unlikely.) 

Also, you start doing some theoretical physics on your own, and developing your own theories and very approximate / coarse-grained simulations of high-energy physics. You can't be sure without actually running the experiments (which would require building a big collider), but it starts looking, to you, like it is more and more likely that actually, colliding particles at high enough energy will open a black hole and consume the planet, with high probability.

Given these observations and your investigations, you should probably start updating towards the third hypotheses (weird anthropics stuff) as the explanation for why you're still alive. 

The point is, independent of any arguments about anthropics in general, the way you update in-universe depends on the actual kind and quality of the specific observations you make. In both cases (cold war vs. collider failures), the actual update you make would depend on the specific mechanical failures / nuclear war close-calls that you observe. Depending on how trustworthy and detailed / gears-level your understanding of these failures and non-wars is, you would make different updates. But at some point (depending on your specific observations), you will need to start considering weird anthropics hypotheses as having high probability, even if those hypotheses aren't directly supported by your individual observations considered independently. On priors, it feels to me like the LHC scenario is one where the anthropics hypotheses could start rising in probability faster and in more worlds than the cold war scenario, but I could see an argument for the other side too.

Depending on the complexity of the scenario and your evidence, doing "actual math" might get difficult or intractable pretty quickly. And personally, I don't think there's anything that's happened anywhere in our actual world so far that makes any kind of weird anthropics hypothesis more than epsilon likely, compared to more ordinary explanations. If that changes in the future though, having detailed mathematical models of anthropic reasoning seems like it would be very useful!

Note that LHC failures would never count as evidence that the LHC would destroy the world. Given such weird observations, you would eventually need to consider the possibility of an anthropic angel. This is not the same as anthropic shadow; it is essentially the opposite. The LHC failures and your theory about black holes implies that the universe works to prevent catastrophes, so you don't need to worry about it.

Or if you rule out anthropic angels apriori, you just never update; see this section. (Bayesianists should avoid completely ruling out logically possible hypotheses though.)

I think you're saying that with LHC you could possible attain more certainty about latent risk. With SSA and SIA I think you still reject probability pumping for the some reasons as with the cold war scenario. So eventually you get Bayesian evidence in favor of alternative anthropic theories. The problem is that it's hard to get generalizable conclusions without specifying that anthropic theory, and the alternative theory to SSA and SIA that gets probability pumping has not been specified.

I think you're saying that with LHC you could possible attain more certainty about latent risk.

Maybe? It's more like, for either case, if you're actually living through it and not just in a toy example, you can investigate the actual evidence available in more detail and update on that, instead of just using the bare prior probabilities. And it's conceivable that investigation yields evidence that is most easily explained by some kind of simulation / probability pumping / other weird anthropics explanation - even if I can't write down a formal and generalizable theory about it, I can imagine observing evidence that convinces me pretty strongly I'm e.g. in some weird probability-pumped universe, fictional story, simulation, etc.

To be clear, I don't think observing surface-level reports of a few repeated mechanical failures, or living through one cold war is enough to start updating towards these hypotheses meaningfully, and I think the way you propose updating under either SSA or SIA in the examples you give is reasonable. It's just that, if there were enough failures, and then I took the time to actually investigate them, and the investigation turned up (a priori very unlikely) evidence that the failures happened for reasons that looked very suspicious / strange, when considered in aggregate, I might start to reconsider...

So eventually you get Bayesian evidence in favor of alternative anthropic theories.

The reasoning in the comment is not compatible with any prior, since bayesian reasoning from any prior is reflectively consistent. Eventually you get bayesian evidence that the universe hates the LHC in particular.

You should take into account that, assuming material-onlyism, it is far easier for anthropic probability pumps to target neurons than to target bigger structures like LHC copper wires. A few neurons is sufficient to permanently change the world timeline from LHC to non-LHC. Whereas it would take change after change of copper wires or pipes etc to maintain the non-LHC timeline.

Conversely, if you maintain the LHC targeting mechanic over the neuron targeting mechanic, you necessarily have to bite either of whe following bullets, non-materialism (free will is immune to anthropic probability pumping), or that it takes more "probability juice" to target neurons than the LHC itself (ie, the locating difficulty of neurons outweighs the magnitude difficulty of the LHC).

SSA rejection of anthropic shadow could be illustrated by the following thought experiment: imagine that there are two universes: 

One has anthropic shadow, so from 100 planets 99 planets died in LHC catastrophe. 

Another universe has no anthropic shadow, so from 100 planets all will survive. 

In this situation anthropic shadow is completely compensated by SSA, as I am 100 times more likely to find my self in the second universe with no anthropic shadow.

However, it works only if proportion of the universes with anthropic shadow to universes without such shadow is 1 : 1. If anthropic-shadow-universes are significantly more numerous, I will still more likely to be in the universe with anthropic shadow. 

How does SSA compensate? I thought SIA would do that sort of thing?

Yes, it is more like SIA

A SSA counterargument would be similar to Doomsday argument: imagine the universe with strong anthropic shadow, in which the number of habitable planets quickly declining. 

In that case I am more likely to find myself near the bottom of the pyramid of all observers (earlier in time), so before the anthropic shadow effect. However, this works only for contemporary anthropic shadow like LHC, but not for shadows which happened before most qualified observers was born (cold war, climate change).

In the cold war example in my post, consider changing the post cold war population to higher than the pre cold war population (e.g. 5 billion); the conclusion still goes through.

An interesting thing is that if we use "natural reference class" - that is I am selected only from observers who think about anthropics, when most of them have started thinking about anthropics in 1990s or later – that is, after Cold war has ended. 

However, if nuclear war would happen during Cold war, it would target university centers, so there will be less anthropics-conscious people now.  This, in my view, rebuilds "anthropic shadow".

Both universes in this example are real, so it is SSA applied to the whole metaverse

[This comment is no longer endorsed by its author]Reply

Conditional on being in a billion-human universe, your probability of having an index between 1 and 1 billion is 1 in 1 billion, and your probability of having any other index is 0. Conditional on being in a trillion-human universe, your probability of having an index between 1 and 1 trillion is 1 in 1 trillion, and your probability of having any other index is 0.

Ehm.. Huh? I would say that:

Conditional on being in a billion-human universe, your probability of having an index between 1 and 1 billion is 1, and your probability of having any other index is 0. Conditional on being in a trillion-human universe, your probability of having an index between 1 and 1 trillion is 1, and your probability of having any other index is 0. Also, conditional on being in a trillion-human universe, your probability of having an index between 1 and 1 billion is 1 in a thousand.

That way, the probabilities respect the conditions, and add up to 1 as they should.

SSA first samples a universe (which, in this case, contains only one world), then samples a random observer in the universe. It samples a universe of each type with ¼ probability. There are, however, two subtypes of type-1 or type-2 universes, namely, ones with nuclear extinction or not. It samples a nuclear extinction type-1 universe with ¼ * 99% probability, a non nuclear extinction type-1 universe with ¼ * 1% probability, a nuclear extinction type-2 universe with ¼ * 10% probability, and a non nuclear extinction type-2 universe with ¼ * 90% probability.

This is also confusing to me, as the resulting "probabilities" do not add up.

Read "having an index" as "having some specific index", e.g. 42.

The probabilities add up because typed 1,2,3,4 are each probability 1/4 in total.

For new readers: SSA = Self-sampling assumption. Read that in Bostrums "Anthropic Bias". SIA might mean "Sampling independence assumption" but I am just guessing.

It means self indication assumption.