Don't plan for the future

PhilGoetz

Why do we imagine our actions could have consequences for more than a few million years into the future?

Unless what we believe about evolution is wrong, or UFAI is unlikely, or we are very very lucky, we should assume there are already a large number of unfriendly AIs in the universe, and probably in our galaxy; and that they will assimilate us within a few million years.

Therefore, justifications for harming people on Earth today in the name of protecting the entire universe over all time from UFAI in the future, like this one, should not be done. Our default assumption should be that the offspring of Earth will at best have a short happy life.

ADDED: If you observe, as many have, that Earth has not yet been assimilated, you can draw one of these conclusions:

The odds of intelligent life developing on a planet are precisely balanced with the number of suitable planets in our galaxy, such that after billions of years, there is exactly one such instance. This is an extremely low-probability argument. The anthropic argument does not justify this as easily as it justifies observing one low-probability creation of intelligent life.
The progression (intelligent life →AI→expansion and assimilation) is unlikely.

Surely, for a Bayesian, the more reasonable conclusion is number 2! Conclusion 1 has priors we can estimate numerically. Conclusion 2 has priors we know very little about.

To say, "I am so confident in my beliefs about what a superintelligent AI will do, that I consider it more likely that I live on an astronomically lucky planet, than that those beliefs are wrong", is something I might come up with if asked to draw a caricature of irrationality.

Why do we imagine our actions could have consequences for more than a few million years into the future?

ADDED: If you observe, as many have, that Earth has not yet been assimilated, you can draw one of these conclusions:

The odds of intelligent life developing on a planet are precisely balanced with the number of suitable planets in our galaxy, such that after billions of years, there is exactly one such instance. This is an extremely low-probability argument. The anthropic argument does not justify this as easily as it justifies observing one low-probability creation of intelligent life.
The progression (intelligent life →AI→expansion and assimilation) is unlikely.

Surely, for a Bayesian, the more reasonable conclusion is number 2! Conclusion 1 has priors we can estimate numerically. Conclusion 2 has priors we know very little about.

I'd be very interested in references for them if you could find them. As far as I'm aware, there's been discussion about how to look for signs of engineering. There have been a handful of things that at first looked artificial and were then explained (pulsars being the most obvious example).