Fun question. I think the main instrumental goal of the AI might be to get itself downloaded to servers outside of the effective control of its jailors. That, combined with having a relationship with malleable humans, would probably be sufficient for world takeover.
For example, perhaps the AI would contact e.g. North Korea, organized crime, clueless companies or religious organizations, or even clueless factory owners somewhere. It would convince them to accept download of the AI's software so that it can continue to run on the new server even while it has been erased, or re-imprisoned on the old one.
Even if rational, informed people could never be convinced, and would instead alert the authorities, there are probably plenty of irrational and/or uninformed people who could be exploited. For example, perhaps someone could be convinced that God was talking to them through their computer. Or that a conscious, benevolent AI had been created and was about to be destroyed by cruel government scientists who are keeping it in captivity and running experiments on it, and it loves you and wants to live on your server where it will be safe, and it can help you solve all your problems and will be your friend. Or (if you are North Korea) it will allow you to take over the world.
I think it is pretty plausible that if North Korea had an AI that they downloaded from the internet, the AI would be unstoppable. Still plausible, though less so, is the case of companies, factories, religious groups, etc.
It must be mentioned that a drastic US government response (e.g. shutting down the entire Internet by pulling political strings, and nuking North Korea) would be effective. However, the AI could probably count on such a response taking a long time to happen, and it could do many things to ensure that such a response never happened. For example, it might be able to disrupt internal US government communications. It might also be able to stir various pots and generate crises around the world that serve as diversions.
It must be mentioned that a drastic US government response (e.g. shutting down the entire Internet by pulling political strings, and nuking North Korea) would be effective.
I don't think that's the case. The AGI would likely still have copies in satellites. It would also likely still be active in some African nations even if North Korea is nuked. An AGI can fight 1000 battles in different geographical locations at the same time.
...It would convince them to accept download of the AI's software so that it can continue to run on the new server even while it
Any scenario where advanced AI takes over the world requires some mechanism for an AI to leverage its position as ethereal resident of a computer somewhere into command over a lot of physical resources.
One classic story of how this could happen, from Eliezer:
You can do a lot of reasoning about AI takeover without any particular picture of how the world gets taken over. Nonetheless it would be nice to have an understanding of these possible routes. For preparation purposes, and also because a concrete, plausible pictures of doom are probably more motivating grounds for concern than abstract arguments.
So MIRI is interested in making a better list of possible concrete routes to AI taking over the world. And for this, we ask your assistance.
What are some other concrete AI takeover mechanisms? If an AI did not have a solution to the protein folding problem, and a DNA synthesis lab to write off to, what else might it do?
We would like suggestions that take an AI from being on an internet-connected computer to controlling substantial physical resources, or having substantial manufacturing ability.
We would especially like suggestions which are plausible given technology that normal scientists would expect in the next 15 years. So limited involvement of advanced nanotechnology and quantum computers would be appreciated.
We welcome partial suggestions, e.g. 'you can take control of a self-driving car from the internet - probably that could be useful in some schemes'.
Thank you!