Question: When is an AI considered to have taken over the world?
Because there is a hypothetical I am pondering, but I don't know if it would be considered a world takeover or not, and I'm not even sure if it would be considered an AI or not.
Assume only 25% of humans want more spending on proposal A, and 75% of humans want more spending on proposal B.
The AI wants more spending on proposal A. As a result, more spending is put into proposal A.
For all decisions like that in general, it doesn't actually matter what the majority of people want, the AI's wants dictate the decision. The AI also makes sure that there is always a substantial vocal minority of humans that are endorsing it.
However, the vast majority of people are not actually explicitly aware of the AI's presence, because the AI works better when people aren't aware of it. Anyone suggesting there is a an AI controlling humans is dismissed by almost everyone as a crackpot, since the AI operates in such a distributed manner that there isn't any one system or piece of software that can be pointed to as a controller, and so it seems like there isn't an AI in place, just a series of dumb components.
In a case like that, is an AI considered to have taken over the world, and is the system described above actually an AI?
An AI is considered to haven taken over the world when it has total control. If it can divert the entire world's production capabilities to making paperclips (even if it doesn't), then it has taken over the world. If it can get a paperclip subsidy passed, that's not taking over the world.
Any scenario where advanced AI takes over the world requires some mechanism for an AI to leverage its position as ethereal resident of a computer somewhere into command over a lot of physical resources.
One classic story of how this could happen, from Eliezer:
You can do a lot of reasoning about AI takeover without any particular picture of how the world gets taken over. Nonetheless it would be nice to have an understanding of these possible routes. For preparation purposes, and also because a concrete, plausible pictures of doom are probably more motivating grounds for concern than abstract arguments.
So MIRI is interested in making a better list of possible concrete routes to AI taking over the world. And for this, we ask your assistance.
What are some other concrete AI takeover mechanisms? If an AI did not have a solution to the protein folding problem, and a DNA synthesis lab to write off to, what else might it do?
We would like suggestions that take an AI from being on an internet-connected computer to controlling substantial physical resources, or having substantial manufacturing ability.
We would especially like suggestions which are plausible given technology that normal scientists would expect in the next 15 years. So limited involvement of advanced nanotechnology and quantum computers would be appreciated.
We welcome partial suggestions, e.g. 'you can take control of a self-driving car from the internet - probably that could be useful in some schemes'.
Thank you!