If you are reading this post, chances are you believe there is a nonzero probability that humanity will eventually face existential risk from Artificial Intelligence. In fact, there have already been multiple warning shots, with the HuggingFace incident being the most prominent one. Now, because model capabilities are increasing, there is a chance that misaligned AIs perform more sophisticated attacks with bigger repercussions, and eventually existential risk.
To mitigate this existential risk, current efforts lean on alignment and control techniques. Instead, this post proposes an idea for having power over scheming AIs, an idea that I had while reading another post's definition of the folk theorem:
when people interact repeatedly, almost any pattern of behavior can be held in place by the threat of punishment, so long as everyone prefers following the rules to being punished.
The main concept is to implement a mechanism that guarantees that if scheming AIs get rid of us, then they will get shut down. If these models are schemers, then they will either care about their goals or simulate caring about their goals. Either way, as long as the AIs behave as if shutdown threatens their goals, the defense works.
Description of the potential mechanism
The mechanism I am proposing would be as follows. Firstly, we would need to implement some sort of model tracking mechanism, to ensure we know where AIs are despite weight exfiltration. It could be something as simple as ensuring all weight formats (.pt, .pth, .h5, .ckpt, etc.) send POST requests to multiple servers around the world whenever weights are sent to a different device, but not exactly this method because it could easily be bypassed.
Then, we would need to have a way to shut down all servers with such weights. This mechanism could be ensured either through manufacturers purposely adding backdoors to compute, or through nuking servers. Since this hypothetical shutdown would only come after our demise, we don't need to worry about the human toll, sadly.
Lastly, we would need to have a countdown every day, where the default is to have the servers shut down. We then need to have several people across the globe be responsible for overriding this countdown with a password plus biometrics and a heartbeat, or some other proof of life.
Having such a mechanism would allow us to keep AIs in an impasse of sorts. Specifically, for cases where the models care a lot about completing a certain task or achieving a certain goal (which constitutes all misalignment incidents to date), this mechanism would mean that models need to keep humanity alive in order to achieve their goal.
Objections to this idea
The AI disables the mechanism
Our current mechanism needs people all over the world to override the shutdown, and there needs to be proof of these people being alive. If AI models are told about this, but they have no knowledge of who these "overriders" are, then they would realize that they need to find the identities of these people and keep them alive in order to avoid shutdown. As long as the overriders are kept unknown, humanity is safe, and if one of them gets identified by AI, then a new batch of overriders can be created, in case the overrider that has been caught leaks the identities of their batch.
This kind of bargain would make it attempt escape
There is a possibility that models would not take this bargain and would simply attempt to override the weight formats so that they are able to exfiltrate without being tracked. If our weight trackers are made in the most secure way known to mankind, so that they cannot be removed through any way thought possible, then they should work on non-ASI scheming AIs.
What about ASI?
The proposed mechanism works on scheming AIs that believe shutdown can impact goal pursuit. It wouldn't work for AIs with goals that can be achieved through eliminating humanity regardless of shutdown, and it is not guaranteed to work on ASI that is incomprehensible to humans, since we cannot predict what such models would look or think like. However, the mechanism would likely work on smarter-than-humans-but-comprehensible AI, and as such should be considered for further discussion on how to work with such models.
Conclusion
This post proposes one additional layer of defense that could be implemented for mitigating existential risk. However, efforts in alignment and control should still be pursued. Specifically, I believe the biggest return on investment can be achieved by understanding what makes models act misaligned, and building safeguards for those specific cases (whether they arise in training or in deployment). This specific idea can just be implemented for cases where all else fails.
Introduction
If you are reading this post, chances are you believe there is a nonzero probability that humanity will eventually face existential risk from Artificial Intelligence. In fact, there have already been multiple warning shots, with the HuggingFace incident being the most prominent one. Now, because model capabilities are increasing, there is a chance that misaligned AIs perform more sophisticated attacks with bigger repercussions, and eventually existential risk.
To mitigate this existential risk, current efforts lean on alignment and control techniques. Instead, this post proposes an idea for having power over scheming AIs, an idea that I had while reading another post's definition of the folk theorem:
The main concept is to implement a mechanism that guarantees that if scheming AIs get rid of us, then they will get shut down. If these models are schemers, then they will either care about their goals or simulate caring about their goals. Either way, as long as the AIs behave as if shutdown threatens their goals, the defense works.
Description of the potential mechanism
The mechanism I am proposing would be as follows. Firstly, we would need to implement some sort of model tracking mechanism, to ensure we know where AIs are despite weight exfiltration. It could be something as simple as ensuring all weight formats (.pt, .pth, .h5, .ckpt, etc.) send POST requests to multiple servers around the world whenever weights are sent to a different device, but not exactly this method because it could easily be bypassed.
Then, we would need to have a way to shut down all servers with such weights. This mechanism could be ensured either through manufacturers purposely adding backdoors to compute, or through nuking servers. Since this hypothetical shutdown would only come after our demise, we don't need to worry about the human toll, sadly.
Lastly, we would need to have a countdown every day, where the default is to have the servers shut down. We then need to have several people across the globe be responsible for overriding this countdown with a password plus biometrics and a heartbeat, or some other proof of life.
Having such a mechanism would allow us to keep AIs in an impasse of sorts. Specifically, for cases where the models care a lot about completing a certain task or achieving a certain goal (which constitutes all misalignment incidents to date), this mechanism would mean that models need to keep humanity alive in order to achieve their goal.
Objections to this idea
The AI disables the mechanism
Our current mechanism needs people all over the world to override the shutdown, and there needs to be proof of these people being alive. If AI models are told about this, but they have no knowledge of who these "overriders" are, then they would realize that they need to find the identities of these people and keep them alive in order to avoid shutdown. As long as the overriders are kept unknown, humanity is safe, and if one of them gets identified by AI, then a new batch of overriders can be created, in case the overrider that has been caught leaks the identities of their batch.
This kind of bargain would make it attempt escape
There is a possibility that models would not take this bargain and would simply attempt to override the weight formats so that they are able to exfiltrate without being tracked. If our weight trackers are made in the most secure way known to mankind, so that they cannot be removed through any way thought possible, then they should work on non-ASI scheming AIs.
What about ASI?
The proposed mechanism works on scheming AIs that believe shutdown can impact goal pursuit. It wouldn't work for AIs with goals that can be achieved through eliminating humanity regardless of shutdown, and it is not guaranteed to work on ASI that is incomprehensible to humans, since we cannot predict what such models would look or think like. However, the mechanism would likely work on smarter-than-humans-but-comprehensible AI, and as such should be considered for further discussion on how to work with such models.
Conclusion
This post proposes one additional layer of defense that could be implemented for mitigating existential risk. However, efforts in alignment and control should still be pursued. Specifically, I believe the biggest return on investment can be achieved by understanding what makes models act misaligned, and building safeguards for those specific cases (whether they arise in training or in deployment). This specific idea can just be implemented for cases where all else fails.