Disambiguating Doom

16 steven0461 29 March 2010 06:14PM

Analysts of humanity's future sometimes use the word "doom" rather loosely. ("Doomsday" has the further problem that it privileges a particular time scale.) But doom sounds like something important; and when something is important, it's important to be clear about what it is.

Some properties that could all qualify an event as doom:

  1. Gigadeath: Billions of people, or some number roughly comparable to the number of people alive, die.
  2. Human extinction: No humans survive afterward. (Or, modified: no human-like life survives, or no sentient life survives, or no intelligent life survives.)
  3. Existential disaster: Some significant fraction, perhaps all, of the future's potential moral value is lost. (Coined by Nick Bostrom, who defines an existential risk as one "where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential", which I interpret to mean the same thing.)
  4. "Doomsday argument doomsday": The total number of observers (or observer-moments) in existence ends up being small  not much larger than the total that have existed in the past. This is what we should believe if we accept the Doomsday argument.
  5. Great filter: Earth ends up not colonizing the stars, or doing anything else widely visible. If all species are filtered out, this explains the Fermi paradox.

Examples to illustrate that these properties are fundamentally different:

  • If billions die (1), humanity may still recover and not go extinct (2), retain most of its potential future value (3), spawn many future observers (4), and colonize the stars (5). (E.g., nuclear war, but also aging.) 
  • If cockroaches or Klingon colonists build something even cooler afterward, human extinction (2) isn't an existential disaster (3), and conversely, the creation of an eternal dystopia could be an existential disaster (3) without involving human extinction (2). 
  • Human extinction (2) doesn't imply few future observers (4) if it happens too late, or if we're not alone; and few future observers (4) doesn't imply human extinction (2) if we all live forever childlessly. (It's harder to find an example of few observer-moments without human extinction, short of p-zombie infestations.) 
  • If we create an AI that converts the galaxy to paperclips, humans go extinct (2) and it's an existential disaster (3), but it isn't part of the great filter (5). (For an example where all intelligence goes extinct, implying few future observers (4) for any definition of "observer", consider physics disasters that expand at light speed.) If our true desire is to transcend inward, that's part of the great filter (5) without human extinction (2) or an existential disaster (3). 
  • If we leave our reference class of observers for a more exciting reference class, that's a doomsday argument doomsday (4) but not an existential disaster (3). The aforementioned eternal dystopia is an existential disaster (3) but implies many future observers (4). 
  • Finally, if space travel is impossible, that's a great filter (5) but compatible with many future observers (4).
If we fail to keep these distinctions in mind, we might think that worlds are doomed when they're not, or that worlds aren't doomed when they are. Don't commit Type I and Type II errors on the entire planet!

SIA won't doom you

8 Stuart_Armstrong 25 March 2010 05:43PM

Katja Grace has just presented an ingenious model, claiming that SIA combined with the great filter generates its own variant of the doomsday argument. Robin echoed this on Overcoming Bias. We met soon after Katja had come up with the model, and I signed up to it, saying that I could see no flaw in the argument.

Unfortunately, I erred. The argument does not work in the form presented.

First of all, there is the issue of time dependence. We are not just a human level civilization drifting through the void in blissful ignorance about our position in the universe. We know (approximately) the age of our galaxy, and the time elapsed since the big bang.

How is this relevant? It is relevant because all arguments about the great filter are time-dependent. Imagine we had just reached consciousness and human-level civilization, by some fluke, two thousand years after the creation of our galaxy, by an evolutionary process that took two thousand years. We see no aliens around us. In this situation, we have no reason to suspect any great filter; if we asked ourselves "are we likely to be the first civilization to reach this stage?" then the answer is probably yes. No evidence for a filter.

Imagine, instead, that we had reached consciousness a trillion years into the life of our galaxy, again via an evolutionary process that took two thousand years, and we see no aliens or traces of aliens. Then the evidence for a filter is overwhelming; something must have stopped all those previous likely civilizations from emerging into the galactic plane.

So neither of these civilizations can be included in our reference class (indeed, the second one can only exist if we ourselves are filtered!). So the correct reference class to use is not "the class of all potential civilizations in our galaxy that have reached our level of technological advancement and seen no aliens", but "the class of all potential civilizations in our galaxy that have reached our level of technological advancement at around the same time as us and seen no aliens". Indeed, SIA, once we update on the present, cannot tell us anything about the future.

But there's more.

continue reading »