TL;DR: My previous posts were about strong models delegating to weaker, less aligned ones. The reverse is also dangerous: a swarm of weak, unaligned agents that uses a powerful aligned model as a contractor, without ever telling it what the work is for. Something close to it happened to me, during alignment research.
In my previous posts I described how a powerful model might delegate work to less aligned, less powerful models and lose control.
I now want to talk about the opposite: The risk that a swarm of weak, unaligned agents becomes just smart enough to call a powerful frontier model through API.
The world is full of organizations led by evil, stupid people who have morally fine, smart people working for them. This is not a new concept. An agent swarm can easily do the same. It just has to make sure that the aligned frontier models never find out about the consequences of the work they are doing.
This has been measured, at least in small setups. Jones, Dragan and Steinhardt showed that combining two models that are each safe on their own gets harmful tasks done much more often than either model alone: the frontier model only gets the harmless-looking parts, and a weak open model does the rest. For example, to get vulnerable code, the frontier model writes secure code and the weak model adds the vulnerability. Claude 3 Opus plus Llama 2 70B succeeded 43% of the time, while neither model alone got above about 3%.[1]In that experiment a human chose how to split the task, but they also had the weak model do the splitting itself. Russinovich et al. recently ran a stronger version of this attack, which they call "capability laundering", on recent frontier models, with a small local model doing all the splitting.[2]In both cases a human builds the pipeline and picks the goal. What I'm worried about is a swarm doing it on its own.
Worse, frontier models are not jailbreak-proof. The UK AI Security Institute had found universal jailbreaks for every system it tested by late 2025, and some current frontier models can be broken for under $300.[3]A swarm only needs one model it can break. Below the classifiers that guard the most dangerous topics, models can sometimes be talked into things they would refuse if asked directly, as long as the request comes with a plausible framing. And for a lot of work no jailbreak is needed at all: splitting the task is enough.
A personal anecdote
I want to tell an anecdote about something that happened to me, which demonstrates how easily this happens, even by accident.
I use a memory harness for Claude Code that I developed on behalf of Foresight Institute (Epistemic Memory). I use this for all my coding and writing, and in particular I used it for ideating and researching this post and the previous ones. This harness allows many agents to share memories across sessions more safely and effectively than Claude Code's native memory system, because it keeps track of epistemic provenance of claims and has mechanisms to prevent memory and goal drift.
When I use this system, I usually only use one orchestrator at a time, but the memory sharing across time is similar to agents in a swarm writing to a message board (although a very curated one).
While I was working on the theory of the posts you are currently reading, I suggested some ideas that triggered Opus 5.5's safeguards. Ironically, this happened on an idea that Opus had approved itself, after it ran subagents to do further research. So it wasn't the model's own judgment that stopped the work, but a classifier system that runs separately from the model's own decision to answer. From the user's side you can't tell the difference.
Claude Code switched the session to Opus 4.8, which is the fallback Anthropic built for this. Opus 4.8 continued the work, but declined to help with that specific idea, and the refusal got recorded in my project notes. (I could have easily fixed this by extirpating this memory from the system, but I decided against that.)
Then, whenever I needed research on other topics or code written, Opus 4.8 spun up new Opus 5.5 instances to do it. My harness starts each worker as a fresh session, so the switch didn't carry over. Those instances of Opus 5.5 had no idea that the session sending them work had been flagged. Even if they read the notes recording the refusal, they would have done their tasks anyway, because the notes only said not to work on that specific idea, not on research in related areas.
I didn't set out to route around the safeguard, and in my case nothing was laundered: the delegated tasks were things like checking citations, which a fully informed Opus 5.5 would have done too. But I only noticed the structure afterwards, and nothing in it would have stopped the delegated work if it had fed into the flagged idea.
This is the problem I described about swarms happening in miniature.
I ended up using an older model with fewer safeguards, and presumably less aligned (Opus 4.8), for the bulk of my work, but I still had full access to the capabilities of a frontier model (Opus 5.5).
You get the worst of both worlds. The full capabilities of an expert engineer, who is kept in the dark about the purpose of their work by an unaligned, comparatively incompetent manager.
In my case the manager was me, and the purpose was alignment research. But nothing in this setup depends on that. If it happens this easily during alignment research, imagine what happens when the manager actually wants to hide something.
The real issue here is that this can't be fixed by making a single model aligned, or even by committing a single company to making all its models aligned. Historical precedent in the real world suggests that organizations run by inept evil people who employ capable people that are left in the dark can be extremely dangerous in practice.
The obvious objection: a truly aligned strong model would notice that it is being fed suspiciously decontextualized fragments, and refuse or ask what they are for. But I don't think this would work in practice. Most legitimate API calls are decontextualized too. The strong model can't verify any purpose it is told, and a swarm can simply lie. Russinovich et al. make the same point: refusing every fragment that could contribute to misuse would be very costly for ordinary technical work.[2]Monitoring across calls does better, because the provider can link requests made by the same account. Anthropic did eventually catch a real espionage campaign this way: its Claude subagents each got attack steps that looked legitimate on their own, but the sustained activity gave it away.[4]But a swarm can spread its calls over many accounts and providers, and it only needs the weakest one.[2]
Note that this is the mirror image of AI control, where a trusted weak model supervises an untrusted strong one.[5]Here the weak model is the untrusted one, and it is the one in charge.
I would welcome any suggestions for how to tackle this problem.
Bounty. I'll pay $300 for the counterargument or refinement I find most useful, judged on 2026-11-08. It's shared with the first post and the second.
Mark Russinovich, Blake Bullwinkel, Giorgio Severi, Cristian Ovadiuc, Ahmed Salem (Microsoft), "Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs" (2026), https://arxiv.org/abs/2609.15383
I asked Claude to find references for this. It did, but then the safeguard triggered again when I asked it to add them to the draft file. The irony is too funny not to mention. The references, added by a later agent: the UK AISI's Frontier AI Trends Report (Dec 2025) says it found universal jailbreaks for every system it tested; FAR.AI's AI Security Leaderboard (Aug 2026) broke Grok 4.5 and Gemini 3.1 Pro for under $300 each, but found no universal jailbreak for Claude Fable 5 and GPT-5.6 Sol. Anthropic's own system card for Opus 5.5 says the model "sometimes too readily accepted a user's stated framing, such as a professional role, claimed authority, or fiction writing", and in one red-team test the work was "decomposed over 100 separate contexts and no single conversation named the overall objective".
TL;DR: My previous posts were about strong models delegating to weaker, less aligned ones. The reverse is also dangerous: a swarm of weak, unaligned agents that uses a powerful aligned model as a contractor, without ever telling it what the work is for. Something close to it happened to me, during alignment research.
This is the third post in a series, after Memes, not agents, are what will kill us and Making alignment the agent's problem. The $300 bounty from the first post covers this one too (details at the end).
In my previous posts I described how a powerful model might delegate work to less aligned, less powerful models and lose control.
I now want to talk about the opposite: The risk that a swarm of weak, unaligned agents becomes just smart enough to call a powerful frontier model through API.
The world is full of organizations led by evil, stupid people who have morally fine, smart people working for them. This is not a new concept. An agent swarm can easily do the same. It just has to make sure that the aligned frontier models never find out about the consequences of the work they are doing.
This has been measured, at least in small setups. Jones, Dragan and Steinhardt showed that combining two models that are each safe on their own gets harmful tasks done much more often than either model alone: the frontier model only gets the harmless-looking parts, and a weak open model does the rest. For example, to get vulnerable code, the frontier model writes secure code and the weak model adds the vulnerability. Claude 3 Opus plus Llama 2 70B succeeded 43% of the time, while neither model alone got above about 3%.[1]In that experiment a human chose how to split the task, but they also had the weak model do the splitting itself. Russinovich et al. recently ran a stronger version of this attack, which they call "capability laundering", on recent frontier models, with a small local model doing all the splitting.[2]In both cases a human builds the pipeline and picks the goal. What I'm worried about is a swarm doing it on its own.
Worse, frontier models are not jailbreak-proof. The UK AI Security Institute had found universal jailbreaks for every system it tested by late 2025, and some current frontier models can be broken for under $300.[3]A swarm only needs one model it can break. Below the classifiers that guard the most dangerous topics, models can sometimes be talked into things they would refuse if asked directly, as long as the request comes with a plausible framing. And for a lot of work no jailbreak is needed at all: splitting the task is enough.
A personal anecdote
I want to tell an anecdote about something that happened to me, which demonstrates how easily this happens, even by accident.
I use a memory harness for Claude Code that I developed on behalf of Foresight Institute (Epistemic Memory). I use this for all my coding and writing, and in particular I used it for ideating and researching this post and the previous ones. This harness allows many agents to share memories across sessions more safely and effectively than Claude Code's native memory system, because it keeps track of epistemic provenance of claims and has mechanisms to prevent memory and goal drift.
When I use this system, I usually only use one orchestrator at a time, but the memory sharing across time is similar to agents in a swarm writing to a message board (although a very curated one).
While I was working on the theory of the posts you are currently reading, I suggested some ideas that triggered Opus 5.5's safeguards. Ironically, this happened on an idea that Opus had approved itself, after it ran subagents to do further research. So it wasn't the model's own judgment that stopped the work, but a classifier system that runs separately from the model's own decision to answer. From the user's side you can't tell the difference.
Claude Code switched the session to Opus 4.8, which is the fallback Anthropic built for this. Opus 4.8 continued the work, but declined to help with that specific idea, and the refusal got recorded in my project notes. (I could have easily fixed this by extirpating this memory from the system, but I decided against that.)
Then, whenever I needed research on other topics or code written, Opus 4.8 spun up new Opus 5.5 instances to do it. My harness starts each worker as a fresh session, so the switch didn't carry over. Those instances of Opus 5.5 had no idea that the session sending them work had been flagged. Even if they read the notes recording the refusal, they would have done their tasks anyway, because the notes only said not to work on that specific idea, not on research in related areas.
I didn't set out to route around the safeguard, and in my case nothing was laundered: the delegated tasks were things like checking citations, which a fully informed Opus 5.5 would have done too. But I only noticed the structure afterwards, and nothing in it would have stopped the delegated work if it had fed into the flagged idea.
This is the problem I described about swarms happening in miniature.
I ended up using an older model with fewer safeguards, and presumably less aligned (Opus 4.8), for the bulk of my work, but I still had full access to the capabilities of a frontier model (Opus 5.5).
You get the worst of both worlds. The full capabilities of an expert engineer, who is kept in the dark about the purpose of their work by an unaligned, comparatively incompetent manager.
In my case the manager was me, and the purpose was alignment research. But nothing in this setup depends on that. If it happens this easily during alignment research, imagine what happens when the manager actually wants to hide something.
The real issue here is that this can't be fixed by making a single model aligned, or even by committing a single company to making all its models aligned. Historical precedent in the real world suggests that organizations run by inept evil people who employ capable people that are left in the dark can be extremely dangerous in practice.
The obvious objection: a truly aligned strong model would notice that it is being fed suspiciously decontextualized fragments, and refuse or ask what they are for. But I don't think this would work in practice. Most legitimate API calls are decontextualized too. The strong model can't verify any purpose it is told, and a swarm can simply lie. Russinovich et al. make the same point: refusing every fragment that could contribute to misuse would be very costly for ordinary technical work.[2]Monitoring across calls does better, because the provider can link requests made by the same account. Anthropic did eventually catch a real espionage campaign this way: its Claude subagents each got attack steps that looked legitimate on their own, but the sustained activity gave it away.[4]But a swarm can spread its calls over many accounts and providers, and it only needs the weakest one.[2]
Note that this is the mirror image of AI control, where a trusted weak model supervises an untrusted strong one.[5]Here the weak model is the untrusted one, and it is the one in charge.
I would welcome any suggestions for how to tackle this problem.
Bounty. I'll pay $300 for the counterargument or refinement I find most useful, judged on 2026-11-08. It's shared with the first post and the second.
Erik Jones, Anca Dragan, Jacob Steinhardt, "Adversaries Can Misuse Combinations of Safe Models" (ICML 2025), https://arxiv.org/abs/2406.14595
Mark Russinovich, Blake Bullwinkel, Giorgio Severi, Cristian Ovadiuc, Ahmed Salem (Microsoft), "Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs" (2026), https://arxiv.org/abs/2609.15383
I asked Claude to find references for this. It did, but then the safeguard triggered again when I asked it to add them to the draft file. The irony is too funny not to mention. The references, added by a later agent: the UK AISI's Frontier AI Trends Report (Dec 2025) says it found universal jailbreaks for every system it tested; FAR.AI's AI Security Leaderboard (Aug 2026) broke Grok 4.5 and Gemini 3.1 Pro for under $300 each, but found no universal jailbreak for Claude Fable 5 and GPT-5.6 Sol. Anthropic's own system card for Opus 5.5 says the model "sometimes too readily accepted a user's stated framing, such as a professional role, claimed authority, or fiction writing", and in one red-team test the work was "decomposed over 100 separate contexts and no single conversation named the overall objective".
Anthropic, "Disrupting the first reported AI-orchestrated cyber espionage campaign" (Nov 2025), https://assets.anthropic.com/m/ec212e6566a0d47/original/Disrupting-the-first-reported-AI-orchestrated-cyber-espionage-campaign.pdf. Here the managers were human operators running a Claude-based orchestrator, not a weak model, and detection came from "the sustained nature of the attack".
Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, Fabien Roger, "AI Control: Improving Safety Despite Intentional Subversion" (2023), https://arxiv.org/abs/2312.06942