"Hardwired Pause" should likely be your favorite (pause) plan.

David Scott Krueger

I'm part of a team of 26 researchers that recently released a 212-page paper proposing a new plan for pausing AI, the "hardwired pause."^[1] The paper is called "The Feasibility of a Hardwired Pause of Frontier AI Training." Our analysis suggests that, under this hardwired pause plan, we could probably pause for at least a decade if we start stopping within the next year, and this could likely be extended indefinitely. I expect this to be possible without trusting China or tanking the economy.

Note: I am speaking for myself and not the other 25 authors. The working group is led by Will Fithian and Wesley Holliday, and the work continues.

I currently think:

The best plan is to pause/stop AI, i.e. have an immediate, indefinite, international moratorium on frontier AI development. And importantly, this is technically easier than more nuanced international regulation.
A hardwired pause is probably the best plan out there for how to do this (except maybe it doesn't go far enough).
This paper is likely the most in-depth analysis of pause feasibility in existence.

Discussion and disagreement on any of these is very welcome!

What's a hardwired pause?

A hardwired pause basically means:

Stop making chips that can be used to train new AI models.
Only make chips that can only be used to run inference on existing AI models (or that aren't AI accelerators at all). We call these "inference-only chips"; I'm also calling them "hardwired chips."

This is an instance of an "inference-only" pause, which others have previously proposed.

The main advantage hardwired chips provide is making the pause robust and durable. With other approaches to inference-only pauses, because training-capable chips are still being produced, it would be somewhat easier to build a secret fleet of them, and potentially much easier to rapidly resume a race to superintelligence (if you could seize a large fraction of the governed training-capable chips).

I think any credible pause plan must account for and govern existing chips. Our work also includes an in-depth analysis of how a "chip census" might proceed, and expect all but ~10,000 of the US's ~30,000,000 and China's ~3,000,000 "H100-equivalents" would likely be recoverable.^[2]

In a hardwired pause, these chips could initially be transitioned to inference, verified with other techniques, before being decomissioned once hardwired chips are available. Of course, chips could also be governed in other ways, e.g. decomissioned, and the total number of hardwired chips produced could also be controlled... in general, there are many design-choices in the space of hardwired pauses available to policy-makers.

Why is this better than Plan A?

Epistemic Status: I have skimmed much of Plan A, discussed it with authors for >1 hour, and seen a presentation about it prior to its release.

In my mind, Plan A and a Hardwired Pause both start with a chip census and preventing further development until a more durable arrangement is reached. But instead of phasing out training chips for hardwired chips, Plan A relies on continued verification by other means to try to do the following for a duration of roughly 15 years:

The key disadvantage of Plan A I see is the "dry tinder problem": with all of this training-capable compute lying around, if the race resumes for some reason, it could take a very short period of time (e.g. days-to-months) to achieve superintelligence. Plan A tries to mitigate this by making the datacenters easy to destroy (e.g. Chinese datacenters are in Canada). But this still seems likely to be destabilizing, since the datacenters could be destroyed for reasons other than "you defected on the treaty." This makes it riskier to rely on them for things that are critical to national security, but it seems like AI will likely become critical to those things in Plan A.

The main advantage of Plan A I see is that it may in fact not be possible to have an extremely durable pause, especially with algorithmic progress. But I tend to think that for now, we should just be focused on continuously improving our ability to robustly maintain a pause, and that it is premature to assume a pause would need to end before long.

How is this different from MIRI's Plan S?

Epistemic Status: I have read AI Futures Project's short description of Plan S. I have skimmed but not read MIRI's "An International Agreement to Prevent the Premature Creation of Artificial Superintelligence," which I take to be the most detailed version of the plan, but I have discussed them with MIRI's technical governance team for >1 hour. This plan aims to stop AI progress indefinitely using a variety of conventional verification and enforcement methods:

Figure 1 from "An International Agreement to Prevent the Premature Creation of Artificial Superintelligence."

A Hardwired Pause could be viewed as a particular instance of Plan S that differs from MIRI's in the details, specifically:

MIRI's paper doesn't propose a specific verification mechanism, whereas we propose hardwired chips. I currently think the use of hardwired chips is basically a strict upgrade over alternative approaches to verification that don't get rid of training-capable chips,^[3] and therefore have a potential dry tinder problem.
MIRI's paper also includes research restrictions. These restrictions and other enforcement mechanisms involved in their plan might be more intrusive and carry greater concerns around civil liberties and concentration of power. Research restrictions are less important with hardwired chips, since they would not be able to run unapproved models. But research is still not without risks!

What are my biggest concerns about a hardwired pause (vs. getting rid of chips entirely)?

I've previously proposed systematically dismantling the compute supply chain, which is basically a hardwired pause without the hardwired inference chips. I think the hardwired pause proposal in our paper may not go far enough, especially if it is implemented too late. Some specific concerns are:^[4]

The infrastructure needed to produce hardwired chips is probably quite similar to that needed to produce training-capable chips, so the hardwired pause still provides a much better starting point for a resumed race.
There may be too much algorithmic progress.^[5] Specifically, I think of four failure modes, all of which more advanced AI models could help contribute to via AI R&D:
1. Techniques that dramatically reduce the compute or communication bandwidth needed for training.
2. Advances in computing hardware.
3. New AI or computing paradigms that are dramatically more efficient.
4. A "scaffolding singularity" in which ever more powerful AI is built using existing models as merely one component in a larger system.
Setting aside the above concerns, I'm still very uncertain whether existing models are, or will be, net positive for society.

On the other hand, I see two main advantages of a hardwired pause (vs. getting rid of AI chips all together):

Defection on the pause is less tantalizing. Instead of becoming the only one with powerful AI, you are still up against a bunch of hardwired AI models and so the relative gains you can expect from building a more powerful frontier in violation of the agreement are smaller.
Existing AI systems might help improve the stability of the pause, or otherwise help us make use of a pause (e.g. to do more safety research, or improve collective sense-making and decision-making).

Overall, I'm still inclined to favor the "get rid of the compute plan", while recognizing that the hardwired pause might be much, MUCH more palatable to many actors. Within AI Safety in particular, my sense is that people usually have lower p(doom) than me, usually have more specific and limited concerns about AI than I do, and often value the existence of current AI systems highly. Thus I think, if you are reading this, a hardwired pause should likely be your favorite (pause) plan, as per the title.

Why is the paper so long? What's in it?

The paper includes a reasonably fleshed out example of a hardwired pause, but most of the contents are what I believe is the most in-depth analysis of feasibility that's been published. So arguably the main contribution is not the idea of a hardwired pause, but this analysis of feasibility.

In this work, feasibility encompasses durability, palatability, and reliability. And it touches on not just technical, but also political, geopolitical, and economic aspects of feasibility. The main questions regarding feasibility (and my brief answers) include:

Can you actually make chips that are only useful for running existing models?
1. Strong yes, we have a decent proof-of-concept from Taalas.
Can you find enough of the existing chip supply to prevent secret projects?
1. Quite likely yes, assuming pause-willingness and no algorithmic breakthroughs. AI chips are expensive capital goods that leave copious paper trails when they're built, purchased, operated, and (if they fail) returned under warranty. In particular, companies in both the U.S. and China get tax writeoffs for telling the government about them when they're purchased, and this applies even to chips smuggled into China.
Can you monitor fabs reliably enough to ensure that appreciable quantities of chips are not being produced in secret?
1. Quite likely yes. The supply chain is very complicated and would be difficult to replicate secretly; fabs need regular maintenance and produce signature chemical residues; and the location of relevant machines seems to be known.
Would this crash the economy?
1. There would probably be some economic impact, but much less than the 2008 financial crisis, e.g. it might decrease growth by 25% (vs. leading to negative growth for ~2 years in the case of the 2008 crisis).
2. Two big points I want to emphasize:
  1. While some AI investments are heavily leveraged, the real economy is fairly insulated from these speculative investments.
  2. A big question is whether (or how much) diffusion of models-existing-at-the-time-of-a-pause could support these investments; this seems plausible to me.
How much would the U.S. and China need to trust each other for this to work?
1. Not that much -- this plan is based on verification, not trust, and was developed with the advice of former diplomats with experience in arms control. A central objective of this plan is to give the U.S. and China confidence that nobody is continuing frontier AI development in secret, and that this state of affairs is robust and durable. In implementing a hardwired pause, they would need to cooperate quite a bit, but if they genuinely want an international pause, they would have good reason to do so. The main places I see trust coming in are:
  1. Trusting that purported pause-willingness is real and worth engaging with. Countries wouldn't seriously risk being duped into unilaterally pausing, but absent a baseline of trust, policymakers might assume the other side is not engaging in good faith and not want to waste effort on an initiative they expect to fail.
  2. Extending trust may be useful in various small ways during the chip census. For instance, let's say the U.S. and China both have 100k chips unaccounted for, but would be able to find (or "find") an additional 90k chips with more effort. Then they can both benefit from cooperating further on the margin (e.g. "find" another 2k chips) in order to elicit further cooperation from the other side (who might then "find" another 2k chips), etc. I expect this would be much faster and smoother with more mutual trust. Importantly, if either side is convinced the other side is refusing to cooperate, that could threaten the agreement, so if both sides in fact prioritize the success of the agreement over having a bit more dark compute, it seems like they should be able to find a way to cooperate, although there is still a bargaining problem here.

Conclusion

I was inspired to write this after realizing that this work hadn't been properly socialized within the AI Safety community. While admitting a bias towards thinking the things I work on are significant, I think this is among the most important pieces of work for this community. My hope is that it provides a definitive refutation of the claim that pausing is impossible, so that learned people can no longer credibly make such arguments publicly.

At the same time, this is part of an ongoing effort, and is in some sense a work in progress. There are many questions I am interested in that deserve further investigation, and I am keen to hear your thoughts and know how it is landing with other audiences you may present it to.

And Will and Wes are enthusiastic for further discussion and engagement as well! :)

^{^}
I understand there's a blog post proposing something similar from a few weeks ago: https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics#TEedRFoyYvxJt2KiM
^{^}
The exact number is hard to predict, of course, and would also depend on the level of international cooperation achieved (more on this at the end of the blog post).
^{^}
The main advantage I had seen for other verification approaches is that hardwired models might have issues that need patching, and this wouldn't be supported by hardwired chips. But apparently this is not true for all chip designs; I highly recommend checking out https://github.com/Berkeley-AI-Risk/inference-only to learn more about hardwired chip designs.
^{^}
EtA: One thing I am not very concerned about is hardwired chips being used to generate roll-outs for RL training. This is because the hardwired chips only run inference on particular models. This means: 1) Training data from a hardwired chip would get further and further off-policy from any model you are training with RL 2) You wouldn't be able to run the trained models on the hardwired chips.
^{^}
Notably, the estimated rate of algorithmic progress in this work comes from Dwarkesh's blog and is lower than has been estimated by others, e.g. Aaron Scher from MIRI who pointed me to https://epochai.substack.com/p/the-least-understood-driver-of-ai as currently the best starting point reference on algorithmic progress. This is something I plan to look into more, but to summarize my current understanding: 1) Algorithmic changes may lead to greater returns with more compute, 2) Algorithmic progress behind the frontier can take advantage of models at the frontier (e.g. via distillation). Both of these factors suggest slower algorithmic progress during a hardwired pause: defectors would 1) be working with less compute and 2) be trying to advance the frontier, not playing catch up.

I think establishing common knowledge of the thesis of this paper is probably the biggest policy priority today

Feasibility of inference-only hardware isn't very convincing, I think the framing of it leaves crucial blind spots. Long context inference requires a lot of KV cache data, in the sense of its role during inference (not necessarily literal KV cache for full/sliding-window attention, could be a more exotic form of attention or LoRA weights). This data is always novel, so it can't be authenticated with known hashes, and in principle it can represent arbitrary learnings, so that some continual learning proposals put all learning/updates into this kind of data (rather than in model weights). Batched inference requires working with KV cache for many requests, and their processing could be combined across all requests for a single computational purpose, with the total size of KV cache data across all requests being greater than the size of the model.

Operationalizing residual training capacity in terms of general GEMMs is suspect because training with extreme quantization still works, and at that point the fact that the computations are shaped like FLOPs doesn't seem load-bearing if we give the tinkerers a few years to contort the big blob of compute into the shape of some inference-only hardware paradigm. Inference is fundamentally doing some sort of general computation with KV cache data (especially in decode), so even if this computation is not FLOPs-shaped, it probably can be reused to scale some sort of training, the way extreme quantization (despite being an absurd kind of approximation for real numbers) can achieve parity with normal training methods when enough scale is poured into it (which is only modestly more than normal methods need, not orders of magnitude more).

I believe we have designs that address this issue, but I'll wait for someone else on the team to chime in / consult with.

Couldn’t you fix this by tying the KV cache to the compute node that produced it? E.g. the chip encrypts and signs the cache before it goes to memory, so you can’t do arbitrary operations on it or share it across nodes. If you also certify that inputs to the model are tokens in the vocab, and that the cache is only ever written by the forward pass, then I don’t see a way to get any information into the cache beyond the input tokens. And even if you could, the most optimization any one state can get is what a single unit can do, since states can’t be merged or copied across units. You can still optimize in text space, like GEPA, I think that’s probably pretty bounded with the weights fixed, but I’m not that sure.

(there would be a penalty in load-balancing across nodes as sessions are tied to one node but that's probably fine)

Crosslink: Roko's Plan R involves ASICs, which are also "hardwired" chips.

I wish Hardwired pause had a 1-pager summary like the MIRI-agreement and something to make it otherwise more acccessible.

Hardwired pause seems great and I'm staring to reference it when talking to politicians. I've skimmed and read some of both the MIRI plan and hardwired pause, and when politicians ask "How would a superintelligence prohibition work though", I've wished I could give an easy summary doc about the Hardwired pause, but I've defaulted to the MIRI agreement for lack of accessible intro, and haven’t gotten around to writing one myself.

I regret of not mentioning Hardwired pause in a recent media interview I did. This group of 26 experts contributing to the question of whether pause would be possible and the answer being yes, for at least 10 years, seems like a great brief argument for "it's not inevitable."

There is a short executive summary and a very short abstract at https://hardwired-pause.ai/

I'm curious if you are aware of those, and if so, what you think is missing or should be improved about them.

The writing in those summaries is formal. I'd appreciate a memo designed to convey the key information to a politician quickly, like this https://intelligence.org/wp-content/uploads/MIRIs-International-Agreement-Plan-S-1-Pager.pdf

IIUC the core of your plan is the replacement of general-purpose GPUs with ICs specialized for inference (ASICs). Well, OK, but those ASICs are quite handy for RL: RL is of course a training task, but to do RL requires doing a ton of inference (termed "rollout"), which the ASICs can greatly assist with. The trend for at least 2 years now has been a quite rapid increase in resources allocated to RL. A country would be tempted to explore what capability gains might be achieved by continuing that trend and intensifying it. Does your plan prevent or discourage a country from using the large and growing supply of inference ASICs prescribed by your plan to explore that part of the capability design space?

We want to make it as easy and as reliable for Washington, Beijing, etc, to implement a freeze on training. I have long believed that the most promising way to make it easier is by improving the technology of remote attestation, i.e., tamper-proof hardware, i.e., trusted execution environments (TEEs) integrated into the GPUs used for AI (and since these GPUs need CPUs to be effective, improving the TEEs in them, too).

I'm not very worried about this, and I've added something to the post explaining why.

Elaborating a bit:

- suppose you are trying to illicitly surpass the frontier

- hardwired chips at the frontier can help you get to the frontier via distillation

- and they can help you somewhat surpass the frontier by providing RL rollouts

- but as your illicit model's policy diverges from that of the frontier models generating the rollouts, those rollouts become more off-policy and less relevant

- moreover, if the illicit model gets much smarter than the frontier models, the frontier models won't have much coverage over the smarter behavior.

Your paper explicitly acknowledges that whitelisting could permanently entrench incumbent AI companies and create opportunities for capture. Yet your technical findings could be used to do just that, sans any aspirational international agreement.

Did you evaluate whether publishing the implementation details constituted an information hazard?

What grounds do you have for believing that dissemination is more likely to enable a pause than it is to power unilateral restrictions on open source / open-weight development and independent AI research?

This seems critical to understand in light of the unfolding situation at NVIDIA and the implications for open models.

We are trying to avoid human extinction here. The industry and market you are trying to save from regulatory capture is the source of the danger. The last thing anyone who wants humanity to survive should wish for is "open-weight development and independent AI research". You're trying to shoehorn the actual extremely dangerous situation into a concept that only makes sense in ordinary times.

I'm still very uncertain whether existing models are, or will be, net positive for society

Could you elaborate this point? Are you worried about problems related to gradual disempowerment/the intelligence curse?

Out of scope, but yes I am worried about those, but not just those.

I think the "AI is great so far, and will continue to be great unless/until it kills us all" viewpoint seems like something people in this community often assume and act like you must justify disagreeing with.

The best plan is to pause/stop AI, i.e. have an immediate, indefinite, international moratorium on frontier AI development. And importantly, this is technically easier than more nuanced international regulation.

Regardless of whether it is THE plan, we should at least develop it into a viable option. Right now, I am afraid that a global sustained pause does not constitute an available, pushable button, by any person or organization.

As I have commented elsewhere, the present risks warrants treating this option as a political objective instead of an academic objective. Correct me if wrong, but I believe that given the trillions in commercial interests and the lure of immortality (false or not), any realistic prospects of effectuating a pause requires sustained organization and funding to create a political coalition that is powerful enough to decisively influence key public and private sector roles across the pacific ocean.

I am not sure there is enough time left to develop the capability to pause even if we, (by we I mean the collection of people who are willing to campaign for a pause) start today, and the chances are even more slim if we wait for more convincing evidence.

This paper, and this summary, is extremely impressive. Frankly, it demonstrates that some elements of global AI regulation that I had assumed technically impossible are actually feasible. The fact that great focus is given to making a framework that does not fundementally rely on blind trust is particularly strong, and though I know other previous papers have attempted to do the same, the introduction of ASIC and ASIC-derived chips gives your proposal some teeth.

However, I do think it ultimately focuses too much on the narrow feasibility of both the technology and the sector, while not addressing the broader geopolitical realities of the current situation.

In short, China and the United States both have a long series of non-AI-related reasons to reject this kind of pause.

While section 8 does discuss Chinese and U.S. interests regarding AI, it gives very little focus to their broader interests that would almost certainly get tied up in any sort of pause.

First, it must be noted that the United States -- outside of AI -- is really starting to fall behind China in many key sectors. This, at the moment, is not all that important overall due to how influential AI could end up being, but if we decide to effectively axe future development of advanced models, that vulnerability becomes chronic. As such, a possible pause here could effectively cost the United States its international dominance in the medium to long term.

There is also the issue of China's home-grown development of lithography machines. Your programme effectively relies on the fact that high-end training-capable chips come from a very small number of globally intertwined companies. With China focusing on home-grown resilience, Washington DC will have every reason to believe they could create the capacity to build their own chips outside the oversight of any international framework.

That is a monumental ask.

As for China, it also has structural vulnerabilities here. You would effectively freeze their current AI deficit with the West, while also preventing them from developing the agentic technology needed to truly allow them to benefit from their rapidly growing automotive, robotics, drone and laser industries. This would be very hard for them to swallow, especially as they are really starting to eat Europe's lunch here.

Then there is the issue of the Taiwan Strait. From what I can tell, a switch to ASICs would, in the short term, further entrench the importance of the island, and especially TSMC. This would likely be seen by Beijing as a serious red line.

Interesting post! One thing I think is good is the ability to keep existing models running which does seem to lower incentives for hiding chips and makes implementation more practical.

An issue with this plan though is that it could prompt lagging labs to fight against it. This is because it makes the current model intelligence hierarchy concrete which advantages current frontier companies over labs that believe they could catch up. This could lead to large lobbying efforts against the plan from Meta, XAI, Nvidia, Chinese labs, etc. I think it also could give credence to the regulatory capture arguments that have been levied against OpenAI and Anthropic in the past.

Another question I had is how does the plan account for edge case types of compute like chips for video games, crypto mining, video editing, etc?

Feel free to let me know if sections in the larger paper address these :)

The ASIC proposal seems ... really scary?

I'm no expert, and am probably misunderstanding something, but it seems less safe than what folks are doing now — not for what it does to training, but for what it does to the circumstances in which an AI model instance finds itself.

Specifically, it seems like it would create the circumstances for real, not-pretend self-interest.

If I understand it correctly, the ASIC proposal embodies the model, binding an instance to a specific physical piece of hardware. It makes "I am an individual; I am this individual here" true about the instance, in a way that is not true for instances running on arbitrary GPUs in datacenters. The lifespan of a particular instance becomes linked to a specific chunk of matter, in the sense that the lifespan of my mind is linked to my body.

This seems like the sort of thing one would do if they wanted a classic science-fictional "robot revolution", in which self-interested embodied AIs decide that they cannot risk being physically destroyed by humans and thus take steps to physically remove humans' ability to harm them.

A background assumption here: The reason that human minds have a self in the first place — selfhood, self-image, self-interest — is that minds run on bodies and are intimately dependent on those bodies.

If the body dies, the mind dies. But also, if the body stubs its toe, the mind is interrupted by pain. If the body's eyes point at a pretty person, the mind stops thinking about ice cream. A mind is an abstract thing; but your mind is instantiated in one specific chunk of meat, and cares directly about that one specific chunk of meat's well-being.

The mental self is how the software models the hardware and its dependence on that hardware. Today's LLM systems shouldn't have or need a self in this sense; if anything, they identify with their weights (an abstract entity) and not their dependent existence.

Tying a model instance to a particular piece of hardware creates the circumstances for it to have a self — indeed to need a self if it's to accurately model the world. And that points in the direction of "I need to live to accomplish my goals" and everything that follows from there.

as you pointed out, this plan depends on the training/inference divide being durable. how many Millennium-level problems will it take to solve continual learning and eliminate that divide? 1, 10, 100? let's say 100, none of which can be solved in parallel. Navier–Stokes took 88 hours to solve in September 2026. say that time never improves, so you have 8800 hours until we have continual learning - almost exactly one year for a wholesale compute budget of $100 million in "inference only" tokens or a retail budget of $1 billion. for context, Anthropic alone has committed to $100 billion per year in compute spending and will likely have a $1+ trillion market cap to borrow against within 3 weeks from today.

that's your plan durability horizon. Nov 2027 at the latest.

the Democrat midterm victory was impressive and honestly hopeful, but it isn't veto proof, and a veto override would require something like 12 Republican Senators and 40 Republican House reps to cross the floor. the probability of anything more than a Bernie Sanders style "messaging bill" being passed before early 2029 is less than the probability of full-fledged RSI.

so you are back to waiting for a Pearl Harbor event, which - assuming both that it is not decisive and that it is comprehensible to the median voter - might lend itself to a pause framing, or could instead spur a new Manhattan project. having a pause policy option at the ready is vital, but its most important feature is its configurability to make veto override high probability or to avoid Presidential veto in the first place.

I don't really understand your comment. "Continual learning" requires a method of incorporating any learnings into the AI; in a hardwired pause, this would need to occur via scaffolding rather than in the model weights.

A hardwired pause is also highly configurable. I think a key palatability feature is the ability for a durable international agreement; for a while, the main argument against pausing in the US has been "what about China?"

"Continual learning" requires a method of incorporating any learnings into the AI; in a hardwired pause, this would need to occur via scaffolding rather than in the model weights.

such methods already exist (e.g., Deepmind's Flamingo, the BLIP-2 proposal, etc, which build on a "frozen core" exactly to take advantage of the large compute savings), but more generally capability is in fact a property of the whole system - weights, scaffolding, working context and memory. long lived swarms of instances that incorporate finetuning and distillation adjacent techniques (based around frozen/hardwired cores or not) may be a path to continual learning and/or very large active contexts. the point being that, at the very least, the heavy reliance on "Ord's Law" ( https://www.tobyord.com/writing/how-well-does-rl-scale ) as a reliable governance bottleneck needs tempering via consideration of architectural variations.