I've always thought of space infrastructure (including weaponry) to be very difficult to defend. Like satellites are very easily destroyed by even tiny pieces of debris. I think this would also be true for a megastructure in space, especially if it was a thin very wide diameter object focusing a laser. A defensive ASI on Earth might be able to fire a projectile at the megastructure and cause a massive structural collapse for a relatively cheap cost compared to the energy and materials to build the giant space laser?
So my intuition would be that conflicts between ASIs on Earth and in space would be in favour of the Earth-based ASI until relative power dynamics come into play later on.. the limit on energy generation on Earth seems orders of magnitude lower than what is possible with a Dyson swarm.
Yeah there are definitely a lot of footnotes - the post was originally a lot longer but I ended up focusing in on the galactic-scale stuff.
I think it's okay that they don't quite line up - it's clear what's happened and people can still access that extra info if they want to read it.
By the time there is an interstellar civilization, humanity would almost certainly already have caused a superintelligence to be created
This is definitely correct, but I think this scenario hops over the decision-point that I'm referring to (or, the point at which the governance system I describe becomes necessary). It isn't the time of the creation of interstellar civilisation that I care about, but the point at which interstellar civilisation becomes inevitable. This may be the first interstellar colonisation mission, which begins a self-propagating expansion of human civilisation throughout the galaxy. If that propagation begins without an extremely forward-thinking governance system embedded within it (seemingly superintelligent), then the eventual creation of a galactic... (read more)
Crossposted to the EA Forum: https://forum.effectivealtruism.org/posts/x7YXxDAwqAQJckdkr/interstellar-travel-will-probably-doom-the-long-term-future
I think people are very wrong about space colonisation increasing long-term existential security. Once we expand to other star systems, we may begin a self-propagating expansion of human civilisation throughout the galaxy. However, there are existential risks potentially capable of destroying a galactic civilisation, like self-replicating machines, strange matter, and vacuum decay. Without an extremely widespread and effective governance system, the eventual creation of a galaxy-ending x-risk seems almost inevitable due to cumulative chances of initiation over time across numerous (potentially hundreds of thousands of) independent actors. The requirements for a governance system to prevent galactic x-risks are extremely demanding, and it needs to be in place before interstellar colonisation is initiated.
I... (read 4761 more words →)
Thanks Peter. I'll add another category for that like I did with asteroid impacts.
The thing that worries me in this scenario is that Superpower A has the ability to expand to other star systems while Superpower B (and presumably everyone else on Earth) has very little say about it.
I don't think Superpower B would be able to convincingly argue that they are willing to initiate their own destruction through MAD if they're not given access to half of the galaxy or if their preferences for large scale space expansion are not taken into account. Also, assuming the leaders of superpower B don't want to be deposed, they wouldn't make such a threat anyway because it wouldn't be in the interest of the people that the... (read more)