For this question, let's assume the worst about the universe: 1. That a false vacuum decay can be created, 2. False vacuum decay will destroy the universe, i. e. inside the true vacuum bubble life/computation will be impossible. 3. Faster-than-light travel is impossible, i. e. it's not possible to outrun...
Let's suppose we have an AGI running on a perfectly physically isolated computer that may be misalligned. It only communicates with select humans through a perfectly secure text channel. How can such an AGI escape? While don't think that AGI can manipulate EVERY person to do what AGI wants, it's...
This is a hypothetical question, regarding possible (albeit not the most likely) existential risks. Maybe a non-artificial intelligence can realize it, but I'm talking about artificial because it can be programmed in different ways.
By hardcoded, I mean forced to prefer - in this case, a more complicated physics theory with false vacuum decay over a simple one without it.
For this question, let's assume the worst about the universe:
1. That a false vacuum decay can be created,
2. False vacuum decay will destroy the universe, i. e. inside the true vacuum bubble life/computation will be impossible.
3. Faster-than-light travel is impossible, i. e. it's not possible to outrun the bubble of true vacuum.
The AGI isn't aware of the above. What qualities does it need to have, to come to the realization that it's dangerous?
I think that an aligned AGI may be more dangerous in this case, depending on how it's aligned, since it may be restricted in the methods it can use to prevent the rest of the universe creating a false vacuum... (read more)
Hmm, I was somewhat worried about that, but there are way more dangerous things for AI to see written on the internet.
If you're trying to create AGI by training it on a large internet crawl dataset, you have bigger problems...
To fix something, we need to know what to fix first.
Let's suppose we have an AGI running on a perfectly physically isolated computer that may be misalligned. It only communicates with select humans through a perfectly secure text channel.
How can such an AGI escape?
While don't think that AGI can manipulate EVERY person to do what AGI wants, it's better to be cautious.
But recently, I thought of steganography. An AGI can tell a human to convey a message to the public that looks helpful at the first glance, but in fact contains a steganographic message with instructions for building antimatter fusion reactor/nanobots/bioweapon/whatever.
What other ways do you think can such an AGI escape?
While I think the scenario I described is very unlikely, it nonetheless remains a possibility.
More specifically - that there might be a simpler theory of physics that explains all "naturally" occurring conditions (in a contemporary particle accelerator, in a black hole, quasar, supernova, etc.), but doesn't predict the possibility of false vacuum in some unnatural conditions that may be created by AGI, while a more complicated theory of physics does.
If AGI prefers a simpler theory, that may cause it to create a false vacuum decay.