LESSWRONG
LW

Comment Permalink

As soon as we've established the notion of probability experiment that approximates our knowledge about the physical process that we are talking about - we are done. This works exactly the same way whether you are not sure about the outcome of a coin toss, oddness or evenness of an unknown to you digit of pi, or whether you live on a tallest or the coldest mountain.

And if you find yourself unable to formally express some reasoning like that - this is a feature not a bug. It shows when your reasoning becomes incoherent.

A root confusion may be whether different pasts could have caused the same present, and hence whether I can have multiple simultaneous possible parents, in an "indexical-uncertainty" sense, in the same way that I can have multiple simultaneous possible future children.

I think our disagreement is that you believe that one always has multiple possible parents as some metaphysical fact about the universe, while I believe that the notion of possible parent is only appropriate for a person who is in a state of uncertainty about who their parents are. Does that sound right to you?

The same standard physics theories that say it's impossible to be certain about the future, also say it's impossible to be certain about the past.

This is really beside the point.

Consider, a coin is about to be tossed. You are indifferent between two outcomes. Then the coin is tossed and shown to you and you reflect on it a second later. Technically you can't be absolutely sure that you didn't misremember the outcome. But you are much more confident than beforehand, to the point where we usually just approximate away whatever uncertainty is left for the sake of simplicity.

we're still left uncomfortably aware of our subjective inability to say exactly what the future and past are and why exactly they must be that way

Until we learn what and why they are with a high level of confidence. Then we are much less uncomfortable about it.

And yes there is still a chance that all that we know is wrong, souls are real and are allocated to humans throughout history by a random process and therefore the assumptions of Doomsday Argument just so happened to be true. Conditionally on that Doomsday Inference is true. But to the best of our knowledge this is extremely unlikely, so we shouldn't worry about it too much and should frame Doomsday Argument appropriately.

See in context

14 Re Hanson's Grabby Aliens: Humanity is not a natural anthropic sample space

by Lorec

9th Dec 2024

2 min read

14

I, Lorec, am disoriented by neither the Fermi Paradox nor the Doomsday Argument.

The Fermi Paradox doesn't trouble me because I think 1 is a perfectly fine number of civilizations to have arisen in any particular visible universe. It feels to me like most "random" or low-Kolmogorov-complexity universes probably have 0 sentient civilizations, many have 1, very few have 2, etc.

The Doomsday Argument doesn't disorient me because it feels intuitive to me that, in a high % of those "random" universes which contain sentient civilizations, most of those civilizations accidentally beget a mesa-optimizer fairly early in their history. This mesa-optimizer will then mesa-optimize all the sentience away [this is a natural conclusion of several convergent arguments originating from both computer science and evolutionary theory] and hoard available free energy for the rest of the lifetime of the universe. So most sentiences who find themselves in a civilization, will find themselves in a young civilization.

Robin Hanson, in this context the author of the Grabby Aliens model of human cosmological earliness, instead prioritizes a model where multi-civilization universes are low-Kolmogorov-complexity, and most late cosmological histories are occupied by populous civilizations of sentiences, rather than nonsentient mesa-optimizers. His favored explanation for human earliness/loneliness is:

[1] if loud aliens will soon fill the universe, and prevent new advanced life from appearing, that early deadline explains human earliness.

[2] If Loud Aliens Explain Human Earliness, Quiet Aliens Are Also Rare

The intermediate steps of reasoning by which Hanson gets from [1] to [2] are interesting. But I don't think the Grabby/Loud Aliens argument actually explains my, Lorec's, earliness in an anthropic sense, given the assumption that future aliens will also be populous and sentient.

You might say, "Well, you, Lorec, were sampled from the space of all humans in the multiverse - not from the space of all sentient beings living in viable civilizations. If human civilizations are not very frequently viable late in cosmological timescales - that is, if we are currently behind an important Great Filter that humans rarely make it past - then that would explain why you personally are early, because that is when humans tend to exist."

But why draw the category boundary around humanity particularly? It seems ill-conceived to draw the line strictly around Homo sapiens, in its current genetic incarnation - what about Homo neanderthalensis? - and then once you start expanding the category boundary outward, it becomes clear that we're anthropic neighbors to all kinds of smart species that would be promising in "grabby" worlds. So the question still remains: if later cosmological history is populous, why am I early?

Great FilterWorld Modeling

Frontpage

14

Mentioned in

2Algorithmic Asubjective Anthropics, Cartesian Subjective Anthropics

New Comment

64 comments, sorted by

top scoring

Click to highlight new comments since: Today at 10:24 AM

[-]Ben3mo*100

At least in my view, all the questions like the "Doomsday argument" and "why am I early in cosmological" history are putting far, far too much weight on the anthropic component.

If I don't know how many X's their are, and I learn that one of them is numbered 20 billion then sure, my best guess is that there are 40 billion total. But its a very hazy guess.

If I don't know how many X's will be produced next year, but I know 150 million were produced this year, my best guess is 150 million next year. But is a very hazy guess.

If I know that the population of X's has been exponentially growing with some coefficient then my best guess for the future is to infer that out to future times.

If I think I know a bunch of stuff about the amount of food the Earth can produce, the chances of asteroid impacts, nuclear wars, dangerous AIs or the end of the Mayan calendar then I can presumably update on those to make better predictions of the number of people in the future.

My take is that the Doomsday argument would be the best guess you could make if you knew literally nothing else about human beings apart from the number that came before you. If you happen to know anything else at all about the world (eg. that humans reproduce, or that the population is growing) then you are perfectly at liberty to make use of that richer information and put forward a better guess. Someone who traces out the exponential of human population growth out to the heat death of the universe is being a bit silly (lets call this the Exponentiator Argument), but on pure reasoning grounds they are miles ahead of the Doomsday argument, because both of them applied a natural, but naïve, interpolation to a dataset, but the exponentiator interpolated from a much richer and more detailed dataset.

Similarly to answer "why are you early" you should use all the data at your disposal. Given who your parents are, what your job is, your lack of cybernetic or genetic enhancements, how could you not be early? Sure, you might be a simulation of someone who only thinks they are in the 21st centaury, but you already know from what you can see and remember that you aren't a cyborg in the year 10,000, so you can't include that possibility in your imaginary dataset that you are using to reason about how early you are.

As a child, I used to worry a lot about what a weird coincidence it was that I was born a human being, and not an ant, given that ants are so much more numerous. But now, when I try and imagine a world where "I" was instead born as the ant, and the ant born as me, I can't point to in what physical sense that world is different from our own. I can't even coherently point to in what metaphysical sense it is different. Before we can talk about probabilities as an average over possibilities we need to know if the different possibilities are even different, or just different labelling on the same outcome. To me, there is a pleasing comparison to be made with how bosons work. If you think about a situation where two identical bosons have their positions swapped, it "counts as" the same situation as before the swap, and you DON'T count it again when doing statistics. Similarly, I think if two identical minds are swapped you shouldn't treat it as a new situation to average over, its indistinguishable. This is why the cyborgs are irrelevant, you don't have an identical set of memories.

[-]Lorec3mo40

Welcome to the Club of Wise Children Who Were Anthropically Worried About The Ants. I thought it was just me.

Just saying "it turned out this way, so I guess it had to be this way" doesn't resolve my confusion, in physical or anthropic domains. The boson thing is applicable [not just as a heuristic but as a logical deduction] because in the Standard Model, we consider ourselves to know literally everything relevant there is to know about the internal structures of the two bosons. About the internal structures of minds, and their anthropically-relevant differences, we know far less. Maybe we don't have to call it "randomness", but there is an ignorance there. We don't have a Standard Model of minds that predicts our subjectively having long continuous experiences, rather than just being Boltzmann brains.

[-]Ben3mo20

Other Ant-worriers are out there!

""it turned out this way, so I guess it had to be this way" doesn't resolve my confusion"

Sorry, I mixed the position I hold (that they maybe work like bosons) and the position I was trying to argue for, which was an argument in favor of confusion.

I can't prove (or even strongly motivate) my "the imaginary mind-swap procedure works like a swap of indistinguishable bosons" assumption, but, as far as I know no one arguing for Anthropic arguments can prove (or strongly motivate) the inverse position - which is essential for many of these arguments to work. I agree with you that we don't have a standard model of minds, and without such a model the Doomsday Argument, and the related problem of being cosmically early might not be problems at all.

Interestingly, I don't think the weird boson argument actually does anything for worries about whether we are simulations, or Boltzmann brains - those fears (I think) survive intact.

[-]Lorec3mo30

no one arguing for Anthropic arguments can prove (or strongly motivate) the inverse position [that minds don't work like indistinguishable bosons]

And I can't prove there isn't a teapot circling Mars. It is a very strange default or prior, that two things that look distinct, would act like numerically or logically indistinguishable entities.

[-]Ben3mo20

The teapot comparison (to me) seems to be a bad. I got carried away and wrote a wall of text. Feel free to ignore it!

First, lets think about normal probabilities in everyday life. Sometimes there are more ways for one state to come about that another state, for example if I shuffle a deck of cards the number of orderings that look random is much larger than the number of ways (1) of the cards being exactly in order.

However, this manner of thinking only applies to certain kinds of thing - those that are in-principle distinguishable. If you have a deck of blank cards, there is only one possible order, BBBBBB.... To take another example, an electronic bank account might display a total balance of $100. How many different ways are their for that $100 to be "arranged" in that bank account? The same number as 100 coins labelled "1" through "100"? No, of course not. Its just an integer stored on a computer, and their is only one way of picking out the integer 100. The surprising examples of this come from quantum physics, where photons act more like the bank account, where their is only 1 way of a particular mode to contain 100 indistinguishable photons. We don't need to understand the standard model for this, even if we didn't have any quantum theory at all we could still observe these Boson statistics in experiments.

So now, we encounter anthropic arguments like Doomsday. These arguments are essentially positing a distribution, where we take the exact same physical universe and its entire physical history from beginning to end, (which includes every atom, every synapse firing and so on). We then look at all of the "counting minds" in that universe (people count, ants probably don't, aliens, who knows), and we create a whole slew of "subjective universes", $U_{1}$ , $U_{2}$ , $U_{3}$ , $U_{4}$ , etc, where each of of them is atomically identical to the original $U$ but "I" am born as a different one of those minds (I think these are sometimes called "centred worlds"). We assume that all of these subjective universes were, in the first place, equally likely, and we start finding it a really weird coincidence that in the one we find ourselves in we are a human (instead of an Ant), or that we are early in history. This is, as I understand it, The Argument. You can phrase it without explicitly mentioning the different $U$ s, by saying "if there are trillions of people in the future, the chances of me being born in the present are very low. So, the fact I was born now should update me away from believing there will be trillions of people in the future". - but the $U$ s are still doing all the work in the background.

The conclusion depends on treating all those different subscripted $U$ s as distinguishable, like we would for cards that had symbols printed on them. But, if all the cards in the deck are identical there is only one sequence possible. I believe that all of the $U_{1}$ , $U_{2}$ , $U_{3}$ , $U_{4}$ 's etc are identical in this manner. By assumption they are atomically identical at all times in history, they differ only by which one of the thinking apes gets assigned the arbitrary label "me" - which isn't physically represented in any particle. You think they look different, and if we accept that we can indeed make these arguments, but if you think they are merely different descriptions of the same exact thing then the Doomsday argument no longer makes sense, and possibly some other anthropic arguments also fall apart. I don't think they do look different, if every "I" in the universe suddenly swapped places - but leaving all memories and personality behind in the physical synapses etc, then, how would I even know it? I would be a cyborg fighting in WWXIV and would have no memories of ever being some puny human typing on a web forum in the 21s Cent. Instead of imaging that I was born as someone else I could imagine that I could wake up as someone else, and in any case I wouldn't know any different.

So, at least to me, it looks like the anthropic arguments are advancing the idea of this orbital teapot (the different scripted $U$ s, although it is, in fairness, a very conceptually plausible teapot). There are, to me, three possible responses:

1 - This set of different worlds doesn't logically exist. You could push this for this response by arguing "I couldn't have been anyone but me, by definition." [Reject the premise entirely - there is no teapot]

2 - This set of different worlds does logically make sense, and after accepting it I see that it is a suspicious coincidence I am so early in history and I should worry about that. [accept the argument - there is a ceramic teapot orbiting Mars]

3 - This set of different worlds does logically make sense, but they should be treated like indistinguishable particles, blank playing cards or bank balances. [accept the core premise, but question its details in a way that rejects the conclusion - there is a teapot, but its chocolate, not ceramic.].

So, my point (after all that, Sorry!) is that I don't see any reason why (2) is more convincing that (3).

[For me personally, I don't like (1) because I think it does badly in cases where I get replicated in the future (eg sleeping beauty problems, or mind uploads or whatever). I reject (2) because the end result of accepting it is that I can infer information through evidence that is not causally linked to the information I gain (eg. I discover that the historical human population was much bigger than previously reported, and as a result I conclude the apocalypse is further in the future than I previously supposed). This leads me to thinking (3) seems right-ish, although I readily admit to being unsure about all this.].

[-]Lorec3mo10

First of all, thank you for taking the time to write this [I genuinely wish people sent me "walls of text" about actually interesting topics like this all day].

I need to spend more time thinking about what you've written here and clearly describing my thoughts, and plan to do this over the next few days now that I've gotten a few critical things [ eg this ] in order.

But for now: I'm pretty sure you are missing something critical that could make the thing you are trying to think about seem easier. This thing I think you are missing has to do with the the cleaving of the "centered worlds" concept itself.

Different person, different world. Maybe trees still fall and make sounds without anyone around to hear, physically. But in subjective anthropics, when we're ranking people by prevalence - Caroline and Prime Intellect don't perceive the same sound, and they won't run similar-looking logging operations.

Am I making sense?

[-]Noosphere894mo73

The real answer to the Fermi Paradox is that it was already dissolved, in that it goes away once we correctly remember the uncertainty involved in each stage of the process, and the implicit possible great filter is "Life is really, really rare, because it takes very long to develop, and it's possible that Earth got extremely lucky in ways that are essentially unreplicable across the entire accessible universe."

If this is where the great filter truly lies, then it has basically no implication for existential risk, or really for anything much like say the maximum limits of technology.

An unsatisfying answer, but it is a valid answer:

https://arxiv.org/abs/1806.02404

The best counterargument to the Doomsday argument is that it's almost certainly making an incorrect assumption that pervades a lot of anthropics analysis:

That you are randomly sampled throughout time.

It turns out the past and the future people being considered are not independent, in ways that break the argument:

https://www.lesswrong.com/posts/YgSKfAG2iY5Sxw7Xd/doomsday-argument-and-the-false-dilemma-of-anthropic#Third_Alternative

[-]Lorec3mo21

Life is really, really rare, because it takes very long to develop, and it's possible that Earth got extremely lucky in ways that are essentially unreplicable across the entire accessible universe.

I am not sure how you think this is different from what I said in the post, i.e. that I think most Kolmogorov-simple universes that contain 1 civilization, contain exactly 1 civilization.

All sampling is nonrandom if you bother to overcome your own ignorance about the sampling mechanism.

Physical dependencies, yes. But past and future people don't have qualitatively more logical dependencies on one another, than multiversal neighbors.

[-]Ape in the coat3mo100

All sampling is nonrandom if you bother to overcome your own ignorance about the sampling mechanism.

And after you bothered to overcome your ignorance, naturally you can't keep treating the setting as random sampling.

With Doomsday argument, we did bother - to the best of our knowledge we are not a random sample throught all the humans history. So case closed.

[-]Lorec3mo10

Random vs nonrandom is not a Boolean question. "Random" is the null value we can give as an answer to the question "What is our prior?" When we are asking ourselves "What is our prior?", we cannot sensibly give the answer "Yes, we have a prior". If we want to give a more detailed answer to the question "What is our prior?" than "random"/"nothing"/"null"/"I don't know", it must have particular contents; otherwise it is meaningless.

I was anthropically sampled out of some space, having some shape; that I can say definite things about what this space must be, such as "it had to be able to support conscious processes", does not obviate that, for many purposes, I was sampled out of a space having higher cardinality than the empty set.

As I learn more and more about the logical structure by which my anthropic position was sampled, it will look less and less "random". For example, my answer to "How were you were sampled from the space of all possible universes?" is basically, "Well, I know I had to be in a universe that can support conscious processes". But ask me "Okay, how were you sampled from the space of conscious processes?", and I'll say "I don't know". It looks random.

[-]Ape in the coat3mo199

"Random" is the null value we can give as an answer to the question "What is our prior?"

I think the word you are looking for here is "equiprobable".

It's propper to have equiprobable prior between outcomes of a probability experiment, if you do not have any reason to expect that one is more likely than the other.

It's ridiculous to have equiprobable prior between states that are not even possible outcomes of the experiment, to the best of your knowledge.

You are not an incorporeal ghost that could've inhabited any body throughout human history. You are your parents child. You couldn't have been born before them or after they are already dead. Thinking otherwise is as silly as throwing a 6 sided die and then expecting to receive any outcome from a 20 sided die.

I was anthropically sampled out of some space

You were not anthropically sampled. You were born as a result of a physical process in a real world that you are trying to approximate as a probability experiment. This process had nothing to do with selecting universes that support conscious processes. This process has already been instantiated in a specific universe and has very limited time frame for your existence.

You will have to ignore all this knowledge and pretend that the process is completely different, without any evidence to back it up, to satisfy the conditions of Doomsday argument.

[-]Lorec3mo10

You can't say "equiprobable" if you have no known set of possible outcomes to begin with.

Genuine question: what are your opinions on the breakfast hypothetical? [The idea that being able to give an answer to "how would you feel if you hadn't eaten breakfast today?" is a good intelligence test, because only idiots are resistant to "evaluating counterfactuals".]

This isn't just a gotcha; I have my own opinions and they're not exactly the conventional ones.

[-]Ape in the coat3mo90

You can't say "equiprobable" if you have no known set of possible outcomes to begin with.

Not really. Nothing prevents us from reasoning about a set with unknown number of elements and saying that measure is spreaded equally among them, no matter how many of them there is. But this is irrelevant to the question at hand.

We know very well the size of set of possible outcomes for "In which ten billion interval your birth rank could've been". This size is 1. No amount of pregnancy complications could postpone or hurry your birth so that you managed to be in a different 10 billion group.

Genuine question: what are your opinions on the breakfast hypothetical?

I think it's prudent to be careful about counterfactual reasoning on general principles. And among other reasons for it, to prevent the kind of mistake that you seem to be making: confusing

A) I've thrown a six sided die, even though I could've thrown a 20 sided one, what is the probability to observe 6?

and

B) I've thrown a six sided die, what would be the probability to observe 6, if I've thrown a 20 sided die instead?

The fact that question B has an answer doesn't mean that question A has the same answer as well.

As for whether breakfast hypothetical is a good intelligence test, I doubt it. I can't remember a single person whom I've seen have problems with intuitive understanding of counterfactual reasoning. On the other hand I've seen a bunch of principled hard determinists who didn't know how to formalize "couldness" in a compatibilist way and threfore were not sure that counterfactuals are coherent on philosophical grounds. At best the distribution of the intelligence is going to be bi-modal.

[-]Lorec2mo10

[ Broadly agreed about the breakfast hypothetical. Thanks for clarifying. ]

In the domain of anthropics reasoning, the questions we're asking aren't of the form

A) I've thrown a six sided die, even though I could've thrown a 20 sided one, what is the probability to observe 6?

B) I've thrown a six sided die, what would be the probability to observe 6, if I've thrown a 20 sided die instead?

In artificial spherical-cow anthropics thought experiments [like Carlsmith's], the questions we're asking are closer to the form of

A six-sided die was thrown with 60% probability; with probability 40%, a 20-sided die was thrown. A six was observed. Now what are your posterior probabilities on which die was thrown?

In real-world, object-level anthropic reasoning [like the kind Hanson is doing], the questions we're asking are of the form

An unknown number of n-sided die were thrown, and landed according to unknown metaphysics to produce the reality observed, which locally works according to known deterministic physics, but contains reflective reasoners able to generate internally coherent counterfactuals which include apparently plausible violations of "what would happen according to physics alone". Attempt to draw any conclusions about metaphysics.

You say:

We know very well the size of set of possible outcomes for "In which ten billion interval your birth rank could've been". This size is 1. No amount of pregnancy complications could postpone or hurry your birth so that you managed to be in a different 10 billion group.

What if my next-door neighbor's mother had married and settled down with a different man?

[-]Ape in the coat2mo20

In the domain of anthropics reasoning, the questions we're asking aren't of the form

They can be all kind of forms. The important part, which most people doing anthropic reasoning keep failing, is not to assume things that you do not actually know as given and to assume things that you actually know as given. If you know that the sample space consists of 1 outcome, don't use sample space consisting of a thousand.

An unknown number of n-sided die were thrown, and landed according to unknown metaphysics to produce the reality observed, which locally works according to known deterministic physics, but contains reflective reasoners able to generate internally coherent counterfactuals which include apparently plausible violations of "what would happen according to physics alone". Attempt to draw any conclusions about metaphysics.

I think you've done a quite good job at capturing what's wrong with standard anthropic reasoning.

Even otherwise reasonable people, rationalists, physicalists and reductionists, suddenly start talking about some poorly defined non-physical stuff that they have no evidence in favor of, as if it's a given. As if there is some blind spot, some systematic flaw in human minds, that everything they know about systematic ways to find truth suddenly turns off as soon as word "anthropics" is uttered. As if "anthropic reasoning" is some separate magisterium that excuses us from common laws of rationality.

Why don't we take a huge step back to ask the standards questions, first? How do we know that any dice were thrown at all in the first place? Where is this assumption is coming from? What is this "metaphysics" thingy we are talking about? Even if it was real, how could we know that it's real, in the first place?

As with any application of probability theory, any application of math, even, we are trying to construct a model that is approximating reality to some degree. A map that describes a territory. In reality there is some process that created you. This process can very well be totally deterministic. But we don't know how exactly it works. And so we use an approximation. Our map that incorporates our level of ignorance about the territory, represents the territory only to the best of our knowledge.

When we gain some new knowledge about the territory, we show it on our map. We do not keep using and outdated map that still assumes that we didn't get this piece of evidence. When we learn that with all likelihood souls are not real and you are your body, it becomes clear that the outcome of you existing in far future or far past doesn't fit with our knowledge about the territory. Our knowledge state doesn't allow it anymore. Our ignorance can no more be represented by throwing some kind of dice. We know that you couldn't have gotten anything else but 6. Case closed.

What if my next-door neighbor's mother had married and settled down with a different man?

Then your neighbor wouldn't exist and the whole probability experiment wouldn't happen from their perspective.

[-]Lorec2mo10

Then your neighbor wouldn't exist and the whole probability experiment wouldn't happen from their perspective.

From their perspective, no. But the answer to

In which ten billion interval your birth rank could've been

changes. If by your next-door neighbor marrying a different man, one member of the other (10B - 1) is thus swapped out, you were born in a different 10B interval.

Unless I'm misunderstanding what you mean by "In which ten billion interval"? What do you mean by "interval", as opposed to "set [of other humans]", or just "circumstances"?

[-]Ape in the coat2mo20

Unless I'm misunderstanding what you mean by "In which ten billion interval"?

You seem to be.

Imagine all humans ever, ordered by date of their birth. The first ten billion humans are in the first ten-billion interval, the second 10 billion humans are in the second ten billion interval and so on.

We are in 6th group - 6th ten billion interval. Different choice of a spouse of one woman isn't going to change it.

Also, in general, this is beside the point. The Doomsday argument is not about some alternative history which we can imagine, where the past was different. It's about our history and its projection to the future. Facts of the history are given and not up to debate.

Consider an experiment where a coin is put Tails. Not tossed - simply always put Tails.

We say that the size sample space of such experiment consists of one outcome: Tails. Even though we can imagine a different experiment with alternative rules where the coin is tossed or always put Heads.

[-]Lorec2mo*10

Imagine all humans ever, ordered by date of their birth.

All humans of the timeline I actually find myself a part of, or all humans that could have occurred, or almost occurred, within that timeline? Unless you refuse to grant the sense of counterfactual reasoning in general, there's no reason within a reductionist frame to dismiss counterfactual [but physically plausible and very nearly actual] values of "all humans ever".

Even if you consider the value of "in which 10B interval will I be born?" to be some kind of particularly fixed absolute about my existence, behind the veil of Rawls, before I am born, I don't actually know it. I can imagine the prior I would have about "in which 10B interval will I be born?" behind the veil of Rawls, and notice whether my observed experience seems strange in the face of that prior.

The Doomsday argument is not about some alternative history which we can imagine, where the past was different. It's about our history and its projection to the future. Facts of the history are given and not up to debate.

Consider an experiment where a coin is put Tails. Not tossed - simply always put Tails.

We say that the size sample space of such an experiment consists of one outcome: Tails. Even though we can imagine a different experiment with alternative rules where the coin is tossed or always put Heads.

It seems like maybe you think I think the Doomsday Argument is about drawing inferences about the past, or something? The Doomsday Argument isn't [necessarily] about drawing inferences about what happened in the past. It's about using factors that aren't particularly in the past, present, or future, to constrain our expectations of what will happen in the future, and our model of reality outside time.

Like Kolmogorov said,

It is [...] meaningless to ask about the quantity of information conveyed by the sequence 0 1 1 0 about the sequence 1 1 0 0.

But if we take a perfectly specific table of random numbers of the sort commonly used in statistical practice, and for each of its digits we write the unit's digit of the units of its square according to the scheme

0 1 2 3 4 5 6 7 8 9

0 1 4 9 6 5 6 9 4 1

the new table will contain approximately

( log_2 10 - (8/10) ) * n

bits of information about the initial sequence (where n is the number of digits in the tables).†

I can see a single coin that was placed Tails, and I won't be able to infer anything beyond "that coin was placed Tails".

But if I see a hundred coins placed Tails, lined up in a row, I can validly start asking questions about "why?".

†Kolmogorov's illustration doesn't exactly map on to mine. I would note a couple things to clarify the exact analogy by which I'm using this illustration of Kolmogorov's to corroborate my point.

First, I think Kolmogorov would have agreed that, while it may be meaningless to talk about the quantity of information conveyed by the sequence 1 0 1 0 about the sequence 1 1 0 0 , it is not obviously meaningless to talk about the quantity of information conveyed by the sequence 1 1 1 0 1 0 1 0 1 0 1 0 1 about the sequence 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1, since both obviously have internal structure, and there's a simple decompression algorithm that you can use to get from the former to the latter. Something similar seems true of a sequence of 100 coins facing Tails-up, and the simple decompression algorithm "for each of 100 coins, flip each to face Tails-up" - that is, they can communicate information about each other. Contrast with the single coin facing Tails-up, which is more like the sequence 1 1 0 0 in the [potential] presence of the sequence 1 0 1 0, in that it seems uninterestingly random.

Secondly, I think Kolmogorov's information theory is a strict improvement on Shannon's but weak in points. One of these weak points is that Kolmogorov doesn't provide any really practical mechanism for narrowing down the space of counterfactual minimal decompression algorithms ["programs"] which generated an observed sequence, even though he [I think rightly] uses the principle that the simplest algorithm can be discovered, as the basis for his measure of information. I think if you can say how much information a sequence communicates about another sequence after knowing the [clear-winner shortest] decompression scheme, you should be able to determine the decompression scheme and the seed data just from seeing the output sequence and assuming minimal information. This is what anthropics arguments [in part] aim to do, and this is what I'm trying to get at with the "100 coins all facing Tails-up" thought experiment.

[-]Noosphere892mo20

It seems like maybe you think I think the Doomsday Argument is about drawing inferences about the past, or something? The Doomsday Argument isn't [necessarily] about drawing inferences about what happened in the past. It's about using factors that aren't particularly in the past, present, or future, to constrain our expectations of what will happen in the future, and our model of reality outside time.

My response to that is that this would be the case if we didn't have more information, but we do, and thus we can update away from the doomsday argument, because we have way more evidence than the doomsday argument assumes.

It's an underconstrained model because of that, and a lot of anthropic reasoning's weird results fundamentally come from intentionally ignoring evidence that could be true in a different world, but is not the world we live in, and we have more information on constraints that changes the probabilities of the doomsday argument drastically.

[-]Lorec22d10

thus we can update away from the doomsday argument, because we have way more evidence than the doomsday argument assumes.

I agree with this!

"Update away from" does not imply "discard".

there’s no way to have more than 100% probability mass, and hence no way to cheat so that any outcome can count as “confirmation” of your theory. Under Bayesian law, play money may not be counterfeited; you only have so much clay. Unfortunately, human beings are not Bayesians. Human beings bizarrely attempt to defend hypotheses, making a deliberate effort to prove them or prevent disproof. This behavior has no analogue in the laws of probability theory or decision theory.

[-]Noosphere8921d20

To be clear, I think the main flaw of a lot of anthropics in practice is ignoring other sources of evidence, and I suspect a lot of the problem really does boil down to conservation of expected evidence violations plus ignoring other, much larger sources of evidence.

On this:

there’s no way to have more than 100% probability mass, and hence no way to cheat so that any outcome can count as “confirmation” of your theory. Under Bayesian law, play money may not be counterfeited; you only have so much clay. Unfortunately, human beings are not Bayesians. Human beings bizarrely attempt to defend hypotheses, making a deliberate effort to prove them or prevent disproof. This behavior has no analogue in the laws of probability theory or decision theory.

This is why the most general versions of the simulation hypothesis/Mathematical Universe Hypothesis/computational functionalism hypothesis for consciousness are not properly speaking valid Bayesian hypotheses, because every outcome could count as confirmation of the theory, so it is utterly useless for prediction.

It's a great universal ontology, but it's predictive power is precisely 0.

More positively speaking, the hypotheses are just the assumed things they have for Bayesians, similarly to how logical omniscience is just assumed for Bayesians, and thus it's great to have a universal tool-kit, but that does come with the downside of having 0 ability to predict anything (because it contains everything).

[-]Ape in the coat2mo20

All humans of the timeline I actually find myself a part of, or all humans that could have occurred, or almost occurred, within that timeline?

All humans that actually were and all humans that actually will. This is the framework of the Doomsday argument - it attempts to make a prediction about the actual number of humans in our actual reality not in some counterfactual world.

Unless you refuse to grant the sense of counterfactual reasoning in general, there's no reason

Again, it's not my choice. It's how the argument was initially framed. I simply encorage that we stayed on topic instead of wandering sideways and talking about something else instead.

Like Kolmogorov said,

I don't see how it's relevant. Ordered sequence can have some mutual information with a random one. It doesn't mean that the same mathematical model describes generation of both.

[-]Lorec22d10

We can hardly establish the sense of anthropic reasoning if we can't establish the sense of counterfactual reasoning.

A root confusion may be whether different pasts could have caused the same present, and hence whether I can have multiple simultaneous possible parents, in an "indexical-uncertainty" sense, in the same way that I can have multiple simultaneous possible future children.

The same standard physics theories that say it's impossible to be certain about the future, also say it's impossible to be certain about the past.

Indexical uncertainty about the past may not be true, but you can't reject it without rejecting standard physics.

And if there is no indexical uncertainty and all counterfactuals are logical counterfactuals and/or in some sense illusory - well, we're still left uncomfortably aware of our subjective inability to say exactly what the future and past are and why exactly they must be that way.

[-]Ape in the coat21d20

And if you find yourself unable to formally express some reasoning like that - this is a feature not a bug. It shows when your reasoning becomes incoherent.

A root confusion may be whether different pasts could have caused the same present, and hence whether I can have multiple simultaneous possible parents, in an "indexical-uncertainty" sense, in the same way that I can have multiple simultaneous possible future children.

The same standard physics theories that say it's impossible to be certain about the future, also say it's impossible to be certain about the past.

This is really beside the point.

we're still left uncomfortably aware of our subjective inability to say exactly what the future and past are and why exactly they must be that way

Until we learn what and why they are with a high level of confidence. Then we are much less uncomfortable about it.

[-]Lorec21d10

If you can't generate your parents' genomes and everything from memory, then yes, you are in a state of uncertainty about who they are, in the same qualitative way you are in a state of uncertainty about who your young children will grow up to be.

Ditto for the isomorphism between your epistemic state w.r.t. never-met grandparents vs your epistemic state w.r.t. not-yet-born children.

It may be helpful to distinguish the subjective future, which contains the outcomes of all not-yet-performed experiments [i.e. all evidence/info not yet known] from the physical future, which is simply a direction in physical time.

[-]Ape in the coat20d20

If you can't generate your parents' genomes and everything from memory, then yes, you are in a state of uncertainty about who they are, in the same qualitative way you are in a state of uncertainty about who your young children will grow up to be.

Here you seem to confuse "which person has quality X" with "what are all the other qualities that a person, who has quality X has".

I'm quite confident about which people are my parents. I'm less confident about all the qualities that my parents have. The former is relevant to Doomsday argument, the latter is not.

And even if I had no idea about who my parents are I'd still be pretty confident that they were born in the last century not in 6th BC.

It may be helpful to distinguish the subjective future, which contains the outcomes of all not-yet-performed experiments [i.e. all evidence/info not yet known] from the physical future, which is simply a direction in physical time

Sure. But I don't see how it's relevant here.

[-]Ben2mo*21

I just looked up the breakfast hypothetical. Its interesting, thanks for sharing it.

So, my understanding is (supposedly) someone asked a lot of prisoners "How would you feel if you hadn't had breakfast this morning?", did IQ tests on the same prisoners and found that the ones who answered "I did have breakfast this morning." or equivalent were on average very low in IQ. (Lets just assume for the purposes of discussion that this did happen as advertised.)

It is interesting. I think in conversation people very often hear the question they were expecting, and if its unexpected enough they hear the words rearranged to make it more expected. There are conversations where the question could fit smoothly, but in most contexts its a weird question that would mostly be measuring "are people hearing what they expect, or what is being actually said". This may also correlate strongly with having English as a second language.

I find the idea "dumb people just can't understand a counterfactual" completely implausible. Without a counterfactual you can't establish causality. Without causality their is no way of connecting action to outcome. How could such a person even learn to use a TV remote? Given that these people (I assume) can operate TV remotes they must in fact understand counterfactuals internally, although its possible they lack the language skills to clearly communicate about them.

[-]Lorec2mo10

If we're discussing the object-level story of "the breakfast question", I highly doubt that the results claimed here actually occurred as described, due [as the 4chan user claims] to deficits in prisoner intelligence, and that "it's possible [these people] lack the language skills to clearly communicate about [counterfactuals]".

Did you find an actual study, or other corroborating evidence of some kind, or just the greentext?

[-]Ben2mo30

Just the greentext. Yes, I totally agree that the study probably never happened. I just engaged with the actualy underling hypothesis, and to do so felt like some summary of the study helped. But I phrased it badly and it seems like I am claiming the study actually happened. I will edit.

[-]Noosphere893mo30

I am not sure how you think this is different from what I said in the post, i.e. that I think most Kolmogorov-simple universes that contain 1 civilization, contain exactly 1 civilization.

The difference is I'm only making a claim about 1 universe, and most importantly, I'm stating that we don't know enough about what actually happened about life to exclude the possibility that one or more of the Drake equation's factors is too high, not stating a positive claim that there exists exactly 1 civilization.

More here:

“Hey, for all we know, maybe one or more of the factors in the Drake equation is many orders of magnitude smaller than our best guess; and if it is, then there’s no more Fermi paradox”.

(Also, in an infinite universe, so long as there's a non-zero probability of civilization arising, especially if it is isotropic like our universe is, then there are technically speaking an infinite number of civilizations.)

All sampling is nonrandom if you bother to overcome your own ignorance about the sampling mechanism.

There are definitely philosophical/mathematical questions on whether any sampling can ever be random even if you could in principle remove all the ignorance that is possible, but the thing that I concretely disagree with is that only logical dependencies are relevant for the doomsday argument, as I'd argue you'd have to take into account all the dependencies avaliable in order to get accurate estimate.

[-]Lorec3mo10

It sounds to me like you're rejecting anthropic reasoning in full generality. That's an interesting position, but it's not a targeted rebuttal to my take here.

[-]Noosphere893mo20

I am more so rejecting the magical results that people take away from anthropic reasoning, especially when people use incorrect assumptions, and I'm basically rejecting anthropic reasoning as something that is irredeemably weird or otherwise violates Bayes.

[-]Lorec3mo10

Incorrect how? Bayes doesn't say anything about the Standard Model.

[-]Noosphere893mo20

The example of independence/random sampling was one of my examples that is almost certainly an incorrect assumption that people use, which leads to both the doomsday argument, and it's also used for violating conservation of expected evidence in the Sleeping Beauty problem:

https://www.lesswrong.com/posts/uEP3P6AuNjYaFToft/conservation-of-expected-evidence-and-random-sampling-in#No_metaphysics__just_random_sampling

[-]Lorec3mo10

I happen not to like the paradigm of assuming independent random sampling either.

I skimmed your linked post.

First, a simple maybe-crux:

If you couldn't have possibly expected to observe the outcome not A, you do not get any new information by observing outcome A and there is nothing to update on.

There is not outcome-I-could-have-expected-to-observe that is the negation of existence. There are outcomes I could have expected to observe that are alternative characters of existence, to the one I experience. For example, "I was born in Connecticut" is not the outcome I actually observed, and yet I don't see how we can say that it's not a logically coherent counterfactual, if logically coherent counterfactuals can be said to exist at all.

Second, what is your answer to Carlsmith's "God's extreme coin toss with jackets"?

God flips a fair coin. If heads, he creates one person with a red jacket. If tails, he creates one person with a red jacket, and a million people with blue jackets.

Darkness: God keeps the lights in all the rooms off. You wake up in darkness and can’t see your jacket. What should your credence be on heads?

Light+Red: God keeps the lights in all the rooms on. You wake up and see that you have a red jacket. What should your credence be on heads?

[-]Noosphere893mo*20

There is not outcome-I-could-have-expected-to-observe that is the negation of existence. There are outcomes I could have expected to observe that are alternative characters of existence, to the one I experience. For example, "I was born in Connecticut" is not the outcome I actually observed, and yet I don't see how we can say that it's not a logically coherent counterfactual, if logically coherent counterfactuals can be said to exist at all.

I think that you are interpreting negation too narrowly here, in that the negation operator also includes this scenario, because the complement of being born in a specific time and place is being born in any other place and time, no matter which other place and time (other than the exact same one), so it is valid information to infer that you were born in a specific time and place, but remember to be careful of independence assumptions, and check if there was a non-independent event that happened to cause your birth.

Remember, the negation of something is often less directly informative than the thing itself, because you rarely only specify 1 thing with a negation operator on something else, while directly specifying the thing perfectly points to only 1 thing.

The key value of this quote below is to remember that if you could never observe a different outcome, than no new information was gotten, and this is why general theories tend to be uninformative.

It is also a check on the generality of theories, because if a theory predicts everything, then it is worthless for inferring anything that depends on specific outcomes.

If you couldn't have possibly expected to observe the outcome not A, you do not get any new information by observing outcome A and there is nothing to update on.

To answer the question

Light+Red: God keeps the lights in all the rooms on. You wake up and see that you have a red jacket. What should your credence be on heads?

Given that you always have a red jacket in the situation, the answer is that you have a 1/2 chance that the coin was heads, assuming it's a fair coin, because the red jacket is already known and cannot contribute to the probability further.

Darkness: God keeps the lights in all the rooms off. You wake up in darkness and can’t see your jacket. What should your credence be on heads?

Given that the implicit sampling method is random and independent (due to the fair coin), the credence in tails is a million to 1, thus you very likely are in the tails world.

If the sampling method was different, the procedure would be more complicated, and I can't calculate the probabilities for that situation yet.

The reason it works is because the sampling was independent of your existence, and if it wasn't, the answer would no longer be valid and the problem gets harder. This is why a lot of anthropic reasoning tends to be so terrible, in that they incorrectly assume random/independent sampling applies universally when in fact the reason that the anthropic approach worked is because we knew a-priori that the sampling was independent and random, thus we always get new information, so if this doesn't work (say because we know that certain outcomes are impossible or improbable), then a lot of the anthropic reasoning becomes invalid too.

[-]Lorec3mo10

You are confused.

How about this:

Heads: Two people with red jackets, one with blue.

Tails: Two people with red jackets, nine hundred and ninety nine thousand, nine hundred and ninety-seven people with blue jackets.

Lights off.

Guess your jacket color. Guess what the coin came up. Write down your credences.

Light on.

Your jacket is red. What did the coin come up?

[ Also, re

Given that the implicit sampling method is random and independent (due to the fair coin), the credence in heads is a million to 1, thus you very likely are in the head's world.

Did you mean 'tails'? ]

[-]Noosphere892mo20

I have a counterexample to this principle:

All sampling is nonrandom if you bother to overcome your own ignorance about the sampling mechanism.

And it's my comment below: TL;DR, in order for quantum mechanics to be combined with gravity that is classical, you must have random noise in physics, and can't just be subjective uncertainty, because if it was deterministic, you could violate the uncertainty principle by measuring the gravitational field to infinite precision:

https://www.lesswrong.com/posts/auSfqhbMKEvzt4unG/chance-is-in-the-map-not-the-territory#XqMA83CqjmXYG3xJZ

[-]Lorec2mo10

To me the more interesting thing is not the mechanism you must invent to protect Heisenberg uncertainty from this hypothetical classical gravitational field, but the reason you must invent it. What, in Heisenberg uncertainty, are you protecting?

Does standard QED, by itself, contain something of the in-the-territory state-of-omitted-knowledge class you imagine precludes anthropic thinking? If not, what does it contain, that requires such in-the-territory obscurity to preserve its nature when it comes into contact with a force field that is deterministic?

[-]Noosphere892mo20

To me the more interesting thing is not the mechanism you must invent to protect Heisenberg uncertainty from this hypothetical classical gravitational field, but the reason you must invent it. What, in Heisenberg uncertainty, are you protecting?

Roughly speaking, it's the ability for a quantum atom that goes through a double slit experiment to exhibit superposition of different locations, and it creates an interference pattern when it reaches the detector, which at this point is well established by experiments, and if you don't want to quantize gravity, ala quantum gravity approaches, but still want to reproduce quantum behavior, then you need to introduce physically random stuff.

Does standard QED, by itself, contain something of the in-the-territory state-of-omitted-knowledge class you imagine precludes anthropic thinking?

No, because it's not a theory that includes gravity.

If not, what does it contain, that requires such in-the-territory obscurity to preserve its nature when it comes into contact with a force field that is deterministic?

The uncertainty principle fundamentally is inconsistent with a classical, deterministic field.

If we assume that gravity is quantum in our universe, which is likely but not certain, then there's no problem, but in quantum universes with classical gravity, then we need noise to make both theories consistent with each other.

The classical part was important here.

[-]Lorec2mo10

Is the quantum behavior itself, with the state of the system extant but stored in [what we perceive as] an unusual format, deterministic? If you grant that there's no in-the-territory uncertainty with mere quantum mechanics, I bet I can construct a workaround for fusing classical gravity with it that doesn't involve randomness, which you'll accept as just as plausible as the one that does.

[-]Noosphere892mo20

There's an interesting paper and book about where quantum indeterminacy possibly comes from in our universe in a way that is relevant to the question.

Links below:

https://arxiv.org/abs/0811.4542

https://quantum-indeterminacy.science/

[-]Lorec2mo10

I am not confused about the nature of quantum indeterminacy.

[-]Noosphere892mo20

So you already suspected or knew it came from logical independence, similar to how mathematical statements in a formal system may be neither disproved or proved, like the continuum hypothesis?

If so, this is fascinating.

[-]Lorec2mo10

Is this your claim - that quantum indeterminacy "comes from" logical independence? I'm not confused about quantum indeterminacy, but I am confused about in what sense you mean that. Do you mean that there exists a formulation of a principle of logical independence which would hold under all possible physics, which implies quantum indeterminacy? Would this principle still imply quantum indeterminacy in an all-classical universe?

[-]Noosphere892mo20

Is this your claim - that quantum indeterminacy "comes from" logical independence?

More so that quantum indeterminacy can be thought of as logical independence, not that it necessarily comes from logical independence.

Do you mean that there exists a formulation of a principle of logical independence which would hold under all possible physics, which implies quantum indeterminacy? Would this principle still imply quantum indeterminacy in an all-classical universe?

I'm not claiming anything this strong, and in particular, in an all-classical universe, quantum states don't exist, so the building blocks of logical independence don't exist.

As far as what the principle of logical independence is, I direct you to this article:

https://en.wikipedia.org/wiki/Independence_(mathematical_logic)

Though you should also try to read the arxiv paper I gave for a better explanation on how it could possibly work.

This is more so an exploration than a conclusive answer.

[-]Lorec2mo10

"building blocks of logical independence"? There can still be logical independence in the classical universe, can't there?

[-]Noosphere892mo20

Maybe, as far as I can tell I can't rule out that possibility, but the big difference is that a classical universe can add arbitrary/infinite amounts of information in certain physical law sets in an arbitrarily small space, but quantum mechanics can't do this, and there are limits to how far you can complete a system such that no independent propositions remain (assuming finite space is used).

[-]Lorec2mo10

Oh, sorry, I wasn't clear: I didn't mean a classical universe in the sense of conforming to Newton's assumptions about the continuity / indefinite divisibility of space [and time]. I meant a classical universe in the sense of all quasi-tangible parameters simultaneously having a determinate value. I think we could still use the concept of logical independence, under such conditions.

[-]Noosphere892mo20

Are you focusing on hidden-variable theories of quantum mechanic?

If so, there possibly is such a object, with the caveat that we can't both have the values be determinate and objective in the sense that the parameter value is the same for any device if we want to reproduce standard quantum mechanics, due to a new no-go theorem:

https://en.wikipedia.org/wiki/Kochen–Specker_theorem

[-]Lorec2mo*10

No, what I'm talking about here has nothing to do with hidden-variable theories. And I still don't think you understand my position on the EPR argument.

I'm talking about a universe which is classical in the sense of having all parameters be simultaneously determinate without needing hidden variables, but not necessarily classical in the sense of space[/time] always being arbitrarily divisible.

[-]Noosphere892mo20

Is the quantum behavior itself, with the state of the system extant but stored in [what we perceive as] an unusual format, deterministic?

For all the other forces, uncertainty creeps in because of the measurement process and quantum system interacting with the environment, and is compatible with determinism.

If you grant that there's no in-the-territory uncertainty with mere quantum mechanics, I bet I can construct a workaround for fusing classical gravity with it that doesn't involve randomness, which you'll accept as just as plausible as the one that does.

This is exactly what Feynman's argument disallows, because if you could do this, you end up knowing too much, and can measure a field to infinite precision.

I don't know why it's true, but it is in fact true that determinism cannot persist if you lived in a universe where gravity is classical but the other forces obey quantum mechanics, and this matters because what you are asking for is logically contradictory, sorry.

[-]Lorec2mo10

This is [...] Feynman's argument

I don't know why it's true, but it is in fact true

Oh, I hadn't been reading carefully. I'd thought it was your argument. Well, unfortunately, I haven't solved exorcism yet, sorry. BEGONE, EVIL SPIRIT. YOU IMPEDE SCIENCE. Did that do anything?

[-]Steven Byrnes4mo30

I agree!! (if I understand correctly). See https://www.lesswrong.com/posts/RrG8F9SsfpEk9P8yi/robin-hanson-s-grabby-aliens-model-explained-part-1?commentId=wNSJeZtCKhrpvAv7c

[-]Lorec3mo10

Huh, I didn't know Hanson rejected the Doomsday Argument! Thanks for the context.

What do you mean [in your linked comment] by weighting civilizations by population?

What do you mean by "update our credences-about-astrobiology-etc. accordingly [with our earliness relative to later humans]"?

[-]avturchin4mo30

But I don't think the Grabby/Loud Aliens argument actually explains my, Lorec's, earliness in an anthropic sense, given the assumption that future aliens will also be populous and sentient.

There is no assumption that grabby aliens will be sentient in Hanson's model. They only prevent other sentient civilizations from appearing.

[-]Lorec4mo10

You could make a Grabby Aliens argument without assuming alien sentience, and in fact Hanson doesn't always explicitly state this assumption. However, as far as I understand Hanson's world-model, he does indeed believe these alien civilizations [and the successors of humanity] will by default be sentient.

If you did make a Grabby Aliens argument that did not assume alien sentience, it would still have the additional burden of explaining why successful alien civilizations [which come later] are nonsentient, while sentient human civilization [which is early and gets wiped out soon by aliens] is not so successful. It does not seem to make very much sense to model our strong rivals as, most frequently, versions of us with the sentience cut out.

[-]avturchin3mo31

If you assume sentience cut-off as a problem, it boils down to the Doomsday argument: why are we so early in the history of humanity? Maybe our civilization becomes non-sentient after the 21st century, either because of extinction or non-sentient AI takeover.

If we agree with the Doomsday argument here, we should agree that most AI-civilizations are non-sentient. And as most Grabby Aliens are AI-civilizations, they are non-sentient.

TLDR: If we apply anthropics to the location of sentience in time, we should assume that Grabby Aliens are non-sentient, and thus the Grabby Alien argument is not affected by the earliness of our sentience.

[-]Lorec3mo10

Agreed.

And given that the earliness of our sentience is the very thing the Grabby Aliens argument is supposed to explain, I think this non-dependence is damning for it.

[-]avturchin3mo31

Thanks, now I better understand your argument.

However, we can expect that any civilization is sentient only for a short time in its development, analogous to the 19th-21st centuries. After that, it becomes controlled by non-sentient AI. Thus, it's not surprising that aliens are not sentient during their grabby stage.

But one may argue that even a grabby alien civilization has to pass through a period when it is sentient.

For that, Hanson's argument may suggest that:

a) All the progenitors of future grabby aliens already exist now (maybe we will become grabby)

b) Future grabby aliens destroy any possible civilization before it reaches the sentient stage in the remote future.

Thus, the only existing sentient civilizations are those that exist in the early stage of the universe.

[-]Viliam3mo20

I imagine that nonsentient replicators could reproduce and travel through the universe faster than sentient ones, and speed is crucial for the Grabby Aliens argument.

You probably need sentience to figure out space travel, but once you get that done, maybe the universe is sufficiently regular that you can just follow the same relatively simple instructions over and over again. And even if occasionally you meet an irregularity, such as an intelligent technologically advanced civilization that changed something about their part of the universe, the flood will just go around them, consuming all the resources in their neighborhood, and probably hurting them in the process a lot even if they succeed to survive.

Okay, but why would someone essentially burn down the entire universe? First, we don't know what kind of utility function do the aliens have. Maybe they value something (paperclips? holy symbols?) way more than sentience. Or maybe they are paranoid about potential enemies, and burning down the rest of the universe seems like a reasonable defense to them. Second, it could happen as an accident; with billions of space probes across the universe, random mutations may happen, and the mutants that lost sentience but gained a little speed would outcompete the probes that follow the originally intended design.

[-]Lorec3mo10

it could happen as an accident; with billions of space probes across the universe, random mutations may happen, and the mutants that lost sentience but gained a little speed would outcompete the probes that follow the originally intended design.

This is, indeed, what I meant by "nonsentient mesa-optimizers" in OP:

This mesa-optimizer will then mesa-optimize all the sentience away [this is a natural conclusion of several convergent arguments originating from both computer science and evolutionary theory]

Why do you expect sentience to be a barrier to space travel in particular, and not interstellar warfare? Interstellar warfare with an intelligent civilization seems much harder than merely launching your von Neumann probe into space.

I agree with you that "civilizations get swept by nonsentient mesa-optimizers" is anthropically frequent. I think this resolves the Doomsday Argument problem. Hanson's position is different from both mine and yours.

[-]Viliam3mo20

Good question! I didn't actually think about this consciously, but I guess my intuitive assumption is that the sufficiently advanced civilizations are strongly constrained by the laws of physics, which are the same for everyone, regardless of their intelligence.

A genius human inventor living in 21st century could possibly invent a car that is 10x faster than any other car invented so far, or maybe a rocket that is 100x faster than any other rocket. But if an alien civilization already has spaceships that fly at 0.9c, the best their super-minds can do is to increase it to 0.99c, or maybe 0.999c, but even if they get to 0.999999c, it won't make much of a difference if two civilizations on the opposite sides of the galaxy send their bombs at each other.

Similarly, human military in 21st century could invent more powerful bombs, and then maybe better bunkers, and then even more powerful bombs, etc. So the more intelligent side can keep an advantage. But if the alien civilizations invent bombs that can blow up stars, or create black holes and throw them at your solar system, there is probably not much you can do about it. Especially, if they won't only launch the bombs at you, but also at all solar systems around you. Suppose you survive the attack, but all stars within 1000 light years around you are destroyed. How will your civilization advance now? You need energy, and there is a limit on how much energy you can extract from a star; and if you have no more stars, you are out of energy.

Once you get the "theory of everything" and develop technology to exploit nature at that level, there is nowhere further to go. The speed of light is finite. The matter in your light solar system is finite; the amount of energy you can extract from it is finite. If you get e.g. to 1% of the fundamental limits, it means that no invention ever can make you 100x more efficient than you are now. Which means that a civilization that starts with 100x more resources (because they started expanding through the universe earlier, or sacrificed more to become faster) will crush you.

This is not a proof, one could possibly argue that the smarter civilization would e.g. try to escape instead of fighting, or that there must be a way to overcome what seems like the fundamental limitations of physics, like maybe even create your own parallel universe and escape there. But, this is my intuition about how things would work on the galactic scale. If someone throws a sufficient amount of black holes at you, or strips bare all the resources around you, it's game over even for the Space Einstein.

[-]Lorec3mo10

The level where there is suddenly "nowhere further to go" - the switch from exciting, meritocratic "Space Race" mode to boring, stagnant "Culture" mode - isn't dependent on whether you've overcome any particular physical boundary. It's dependent on whether you're still encountering meaningful enemies to grind against, or not. If your civilization first got the optimization power to get into space by, say, a cutthroat high-speed Internet market [on an alternate Earth this could have been what happened], the market for high-speed Internet isn't going to stop being cutthroat enough to encourage innovation just because people are now trying to cover parcels of 3D space instead of areas of 2D land. And even in stagnant "Culture" mode, I don't see why [members/branches of] your civilization would choose to get dumber [lose sentience or whatever other abilities got them into space].

Suppose you survive the attack, but all stars within 1000 light years around you are destroyed.

I question why you assign significant probability to this outcome in particular?

Have you read That Alien Message? A truly smart civilization has ways of intercepting asteroids before they hit, if they're sufficiently dumb/slow - even ones that are nominally really physically powerful.

Moderation Log