Personally, I think the most likely scenario is either that Earth is somehow special and intelligent life is rarer than we give it credit for; or that alien UFAIs are generally not interested in interstellar/intergalactic travel.
Given the sort of numbers thrown about in Fermi arguments, believing the former would suggest you are outrageously overconfident in your certainty that your beliefs are correct about the likely activities of AIs. Surely the second conclusion is more reasonable?
Er... yes. But I don't think it undermines my point that we are unlikely to be assimilated by aliens in the near future.
Why do we imagine our actions could have consequences for more than a few million years into the future?
Unless what we believe about evolution is wrong, or UFAI is unlikely, or we are very very lucky, we should assume there are already a large number of unfriendly AIs in the universe, and probably in our galaxy; and that they will assimilate us within a few million years.
Therefore, justifications for harming people on Earth today in the name of protecting the entire universe over all time from UFAI in the future, like this one, should not be done. Our default assumption should be that the offspring of Earth will at best have a short happy life.
ADDED: If you observe, as many have, that Earth has not yet been assimilated, you can draw one of these conclusions:
Surely, for a Bayesian, the more reasonable conclusion is number 2! Conclusion 1 has priors we can estimate numerically. Conclusion 2 has priors we know very little about.
To say, "I am so confident in my beliefs about what a superintelligent AI will do, that I consider it more likely that I live on an astronomically lucky planet, than that those beliefs are wrong", is something I might come up with if asked to draw a caricature of irrationality.