Senator Bernie Sanders has proposed directly banning development of ASI. Given the current scientific understanding of AI alignment, it is highly sensible to ban ASI and then consider revoking this ban if future scientific breakthroughs (e.g., “solving interp”) make it possible to safely develop ASI, or if there is truly no way to coordinate on everyone avoiding developing ASI.
Some people have proposed banning ASI indirectly instead. For instance, one could mandate:
Developing AI with superhuman capabilities is not permitted unless you can demonstrate that the AI systems you are developing won’t X, where X is some action that, in the short term, AI companies would find extremely costly or impossible to prevent their AI from taking.
Candidate notions of X include: kill people if given the opportunity, break out of sandboxes, escalate privileges, obtain unintended internet access, engage in unsanctioned coordination, start rogue deployments, commit cyber crime. In the longer term, one could instead consider conditions like overthrow the USG or violently disempower humanity.
In this note, I’ll discuss the costs and benefits of banning ASI indirectly rather than directly.
Indirectly banning ASI is maybe slightly rhetorically better than directly banning ASI
The main putative benefit of indirectly banning ASI is that it sounds very principled to mandate, for instance, that you’re confident your AI won’t kill people. Directly banning ASI sounds anti-tech, and might therefore be unpopular, especially amongst the rich. Banning development of AI models unless you’re confident they won’t kill people sounds relatively unobjectionable.
Another putative benefit of indirectly banning ASI is that it might create good incentives. If it were the case (which, by hypothesis, it is not) that companies could take some actions that would clearly satisfy the condition, then indirectly banning ASI makes companies more likely to take these actions that make their AI models safe.
However, I overall believe that indirectly banning ASI is worse than directly banning ASI. The rhetorical benefits mentioned above are fairly small, and being indirect has large downsides. I’ll discuss them below.
Indirectly banning ASI is probably worse
It’s useful to distinguish between reliably indirectly banning ASI and unreliably indirectly banning ASI. A reliable indirect ban uses a condition X where it is uncontroversial that X is not met and is intractable to meet. In contrast, an unreliable indirect ban uses a somewhat ambiguous condition X, with no consensus on whether it holds or whether it’s tractable to make it hold. If a party that I trust (e.g., METR) was adjudicating whether the condition holds, then I’d be confident they’d rule that the condition isn’t met. However, if a party without the relevant expertise (or one captured by industry) evaluates the condition, I’d be less confident they’d rule that the condition isn’t met. One problem to watch for: in ML, we’re relatively good at getting rid of problems we can measure and not so good at getting rid of those we can’t.
Unreliably indirectly banning ASI: I don’t think people will pass this law if they can’t predict its effects. People will want to know whether it would, for instance, ban AI R&D right now, and for how long. The science battle has to happen before the bill is passed, not after.
Another, maybe minor, downside is that we’d spend a bunch of time trying to reach scientific consensus in a setting with brutally high stakes.
Reliably indirectly banning ASI:
If everyone knows that the bill would obviously ban ASI, then AI companies will lobby against it about as hard as they’d lobby against a direct ban.
Conclusion
Directly banning ASI seems better than banning it indirectly. I’m not extremely confident in this position and would welcome counterarguments.
Senator Bernie Sanders has proposed directly banning development of ASI. Given the current scientific understanding of AI alignment, it is highly sensible to ban ASI and then consider revoking this ban if future scientific breakthroughs (e.g., “solving interp”) make it possible to safely develop ASI, or if there is truly no way to coordinate on everyone avoiding developing ASI.
Some people have proposed banning ASI indirectly instead. For instance, one could mandate:
Developing AI with superhuman capabilities is not permitted unless you can demonstrate that the AI systems you are developing won’t X, where X is some action that, in the short term, AI companies would find extremely costly or impossible to prevent their AI from taking.
Candidate notions of X include: kill people if given the opportunity, break out of sandboxes, escalate privileges, obtain unintended internet access, engage in unsanctioned coordination, start rogue deployments, commit cyber crime. In the longer term, one could instead consider conditions like overthrow the USG or violently disempower humanity.
In this note, I’ll discuss the costs and benefits of banning ASI indirectly rather than directly.
Indirectly banning ASI is maybe slightly rhetorically better than directly banning ASI
The main putative benefit of indirectly banning ASI is that it sounds very principled to mandate, for instance, that you’re confident your AI won’t kill people. Directly banning ASI sounds anti-tech, and might therefore be unpopular, especially amongst the rich. Banning development of AI models unless you’re confident they won’t kill people sounds relatively unobjectionable.
Another putative benefit of indirectly banning ASI is that it might create good incentives. If it were the case (which, by hypothesis, it is not) that companies could take some actions that would clearly satisfy the condition, then indirectly banning ASI makes companies more likely to take these actions that make their AI models safe.
However, I overall believe that indirectly banning ASI is worse than directly banning ASI. The rhetorical benefits mentioned above are fairly small, and being indirect has large downsides. I’ll discuss them below.
Indirectly banning ASI is probably worse
It’s useful to distinguish between reliably indirectly banning ASI and unreliably indirectly banning ASI. A reliable indirect ban uses a condition X where it is uncontroversial that X is not met and is intractable to meet. In contrast, an unreliable indirect ban uses a somewhat ambiguous condition X, with no consensus on whether it holds or whether it’s tractable to make it hold. If a party that I trust (e.g., METR) was adjudicating whether the condition holds, then I’d be confident they’d rule that the condition isn’t met. However, if a party without the relevant expertise (or one captured by industry) evaluates the condition, I’d be less confident they’d rule that the condition isn’t met. One problem to watch for: in ML, we’re relatively good at getting rid of problems we can measure and not so good at getting rid of those we can’t.
Unreliably indirectly banning ASI: I don’t think people will pass this law if they can’t predict its effects. People will want to know whether it would, for instance, ban AI R&D right now, and for how long. The science battle has to happen before the bill is passed, not after.
Another, maybe minor, downside is that we’d spend a bunch of time trying to reach scientific consensus in a setting with brutally high stakes.
Reliably indirectly banning ASI:
If everyone knows that the bill would obviously ban ASI, then AI companies will lobby against it about as hard as they’d lobby against a direct ban.
Conclusion
Directly banning ASI seems better than banning it indirectly. I’m not extremely confident in this position and would welcome counterarguments.