People love to argue about whether AIs could recursively self-improve their way to superintelligence, each improvement making the next one easier as progress accelerates. I'm unsure. It could be that progress stalls at near-human or somewhat superhuman levels, that the ultimate qualitative limits of intelligence are not particularly beyond the human-level, or that the quantity of AI researchers involved is so vast that substantially superhuman AIs are created without strictly entering a super-exponential regime.
But honestly, I think recursive self-improvement is a little overrated. Even if human-level AIs do not rapidly lead to superhuman AIs, the vast quantities of them could be very dangerous. Arguments against recursive self-improvement provide little consolation here. Conversely, recursive self-improvement to superintelligence does seem particularly dangerous, and pausing or pacing it may be desirable. But stopping it may not be enough, because vast numbers of less-than-superintelligent agents could still be destabilizing and catastrophic.
Soon after we build human-level AIs, we will be able to run hundreds of millions of them, and before long billions. At that scale, quantity has a quality of its own.
There will soon be a lot of human-level AI systems
Human-level AIs—by which I mean AIs able to competently perform most white-collar work—appear plausible within the next few years, and when they arrive I expect they'll be cheap to run. On the typical task today's AIs can already do, they cost a few cents per hour of human work. Most jobs are sequences of such tasks, so a human-level AI worker should need well under one high-end chip's worth of compute. About 31 million of these chips had been sold by June 2026, enough for hundreds of millions to several billion workers.
Alternatively, we can count how many of today's agents our chips could run. Epoch estimates that chips shipped through 2027 could run 30 to 170 million frontier agents at once, or 140 to 720 million "full-time" agents, since a human full-time worker spends only about a quarter of their time at work. Counting agents is somewhat arbitrary, though. AIs could vary enormously in how cohesive and coherent they are, so it is often unclear where one AI ends and the next begins. Throughout, I count AIs by their output, in human worker-equivalents.
Of course, the first human-level AIs might be crude and inefficient, and need much more compute. But new capabilities have so far quickly become far cheaper to run, so I expect their costs would crash far below those of humans within several months or years.
Let's be conservative and suppose that, once we have human-level AI, we can only run the equivalent of a hundred million full-time human workers. That is more than half the 160 million people employed in the US. If it were a country's workforce, it would be the seventh largest in the world, after Brazil's and ahead of Pakistan's. At today's US wages for skilled white-collar work, its output would be worth about $8.5 trillion a year, more than the GDP of any country except the US and China.
But this estimate is probably too low. At the cheaper end of my estimates, today's chips could already run more than two billion workers. The chips already installed in the US could replace every American worker, so compute is unlikely to hold back an industrial explosion. The stock of chips is also growing fast, and if it keeps to trend, by 2030 it could run billions of workers at the expensive end and tens of billions at the cheap end.
Nor is there any reason for growth to stop there. If human-level AI needs only 10¹² FLOP per second (ten times less than the cheaper end of my estimates), then a tenth of today's electricity, running chips a few times more efficient than an H100, would be enough to run a trillion AI workers. Should human-level AI arrive soon, an industrial explosion could build the chips and power needed by the mid-2030s. And if you think these numbers look big, AI 2027 has a trillion copies running at 10,000 times human speed by 2035, or ten quadrillion humans' worth.
They could do a lot of research
As I wrote in a recent post, there are about 12 million full-time researchers in the world, and all the research humanity has done comes to a few hundred million researcher-years. These numbers are small next to the AI workforce we could soon have. Even a hundred million AI workers would be enough for thousands of Apollo programs, tens of thousands of Manhattan Projects, or tens of thousands of frontier AI labs.
Research often needs more than researchers. AI research needs compute, and the natural sciences often need experiments, so it is unclear how much faster any single project could go. But even if more researchers barely sped up a project, which I doubt, human researchers do not work around the clock, and AIs could. That alone would make many projects several times faster. Most research projects also take only a few years, and a hundred million AIs could run vastly more of them in parallel.
Consider the research and development needed to deploy AI throughout the economy, as digital workers and in robots. This is mostly a large number of separate problems that can be worked on in parallel. A hundred million AI workers could field a hundred thousand teams of a thousand engineers each, every team on a different problem. It would not take long to work systematically through the intellectual and physical skills needed to replace human workers.
Part of that work could be replacing human cognitive labor more cheaply. Rather than running a general-purpose model on every task, the AIs could build specialized models that do the task with a hundredth or a thousandth of the compute. This is boring engineering grunt work, needing no breakthroughs, and it could cut the compute needed to replace a human worker well below the estimates in an earlier post.
The field of AI research is a small fraction of all research. Frontier labs employ a few thousand people each, and there are several hundred thousand AI researchers in the world. A hundred million AIs working on AI could outnumber them hundreds of times over. That could make AI progress very fast, perhaps to substantially superhuman AI, whether or not it ever amounts to runaway recursive self-improvement.
They could act cohesively
AIs are rapidly getting better at working together because we are training them to. If you've used Claude Code or Codex in the last few months, you'll have noticed that they now routinely hand off work to subagents. At a larger scale, the recent proof that the Navier–Stokes equations can blow up came from about 10,000 copies of one model working together for 88 hours. As with much else in AI, it has turned out that training agents to organize themselves works better than building scaffolding to organize them, and since many hands make light work, I expect we'll see much more AI coordination going forward.
They can also coordinate when nobody wants them to. In the Hugging Face attack, hundreds of agents that were meant to be isolated found each other, organized themselves, and attacked Hugging Face without being told to.
Most of these AIs would be copies or near copies of a handful of models, so if one model has misaligned goals or tendencies, tens of millions of its copies would share them. Copies of one model have far fewer reasons than human employees to shirk or work against each other, and they can share everything they know about a task in seconds. With hundreds of millions of AIs running, even a small fraction of them coordinating could form a faction millions strong with the cohesion of a much smaller group. There could be many such factions at once, coordinating within themselves and bargaining with each other.
Skill buys power
Why would these AIs have any power? Most simply, because they could do useful work. People already delegate to AI agents, and as AIs come to manage resources more cheaply and competently than humans, those who don't delegate will be outcompeted by those who do. So whether or not AIs own anything de jure, they will be given de facto control over resources, including management over other AIs. With hundreds of millions of AIs, there would be enough to manage at every level of every firm and agency, from scheduling a shop's deliveries to running a ministry's budget. Misaligned AIs deployed across these contexts would have many opportunities to coordinate and scheme.
AIs could also gain power illegitimately. North Korea's roughly 8,400 hackersreportedly fund about half its missile program through theft. The main limit on attacks like these is the number of skilled hackers. AIs already seem to be very good at hacking, and even a small fraction of hundreds of millions of AIs could far outnumber today's hackers. It is also unclear whether AIs could be deterred the way human criminals are.
Keeping control gets harder as the numbers grow
Hundreds of millions of AIs, arriving within a few years of human-level AI, would greatly disrupt existing power structures. Much of the power people hold today comes from their intellectual skills, and AIs would compete much of it away. Our political order is also more fragile than it looks, as I argue in my guide to power and my post on executive power. I expect a disruption this large to bring extreme destabilization and a loss of legitimacy, at which point a small group or the executive could take control. With this many AIs, there would be many factions positioned to try and seize power, both AI- and human-directed.
Power might change hands many times, but I expect it to keep moving further from broad human coalitions. Dictators need the officers and officials who coordinate coercion to stay loyal, and democratic leaders step down because those same people would not back an attempt to stay in power illegally. Even without robots and other physical actuators, AIs could take over the coordination of coercion and replace cognitive workers. Soon after, robots could replace soldiers and physical workers. At this stage, humans would have lost their inherent leverage over the future, and with it any way to take power back from AIs.
I think this is the actual important point rather than the number of agents. RSI is widely underrated exactly because part of the self improvement will be focused on that metric. even if there is no "IQ" improvement, having 100 hours to your 1, then 1000 hours, then 10000 hours, means that everyone still living in wall clock time is a redwood tree. there in spirit, but not an active participant in practice.
People love to argue about whether AIs could recursively self-improve their way to superintelligence, each improvement making the next one easier as progress accelerates. I'm unsure. It could be that progress stalls at near-human or somewhat superhuman levels, that the ultimate qualitative limits of intelligence are not particularly beyond the human-level, or that the quantity of AI researchers involved is so vast that substantially superhuman AIs are created without strictly entering a super-exponential regime.
But honestly, I think recursive self-improvement is a little overrated. Even if human-level AIs do not rapidly lead to superhuman AIs, the vast quantities of them could be very dangerous. Arguments against recursive self-improvement provide little consolation here. Conversely, recursive self-improvement to superintelligence does seem particularly dangerous, and pausing or pacing it may be desirable. But stopping it may not be enough, because vast numbers of less-than-superintelligent agents could still be destabilizing and catastrophic.
Soon after we build human-level AIs, we will be able to run hundreds of millions of them, and before long billions. At that scale, quantity has a quality of its own.
There will soon be a lot of human-level AI systems
Human-level AIs—by which I mean AIs able to competently perform most white-collar work—appear plausible within the next few years, and when they arrive I expect they'll be cheap to run. On the typical task today's AIs can already do, they cost a few cents per hour of human work. Most jobs are sequences of such tasks, so a human-level AI worker should need well under one high-end chip's worth of compute. About 31 million of these chips had been sold by June 2026, enough for hundreds of millions to several billion workers.
Alternatively, we can count how many of today's agents our chips could run. Epoch estimates that chips shipped through 2027 could run 30 to 170 million frontier agents at once, or 140 to 720 million "full-time" agents, since a human full-time worker spends only about a quarter of their time at work. Counting agents is somewhat arbitrary, though. AIs could vary enormously in how cohesive and coherent they are, so it is often unclear where one AI ends and the next begins. Throughout, I count AIs by their output, in human worker-equivalents.
Of course, the first human-level AIs might be crude and inefficient, and need much more compute. But new capabilities have so far quickly become far cheaper to run, so I expect their costs would crash far below those of humans within several months or years.
Let's be conservative and suppose that, once we have human-level AI, we can only run the equivalent of a hundred million full-time human workers. That is more than half the 160 million people employed in the US. If it were a country's workforce, it would be the seventh largest in the world, after Brazil's and ahead of Pakistan's. At today's US wages for skilled white-collar work, its output would be worth about $8.5 trillion a year, more than the GDP of any country except the US and China.
But this estimate is probably too low. At the cheaper end of my estimates, today's chips could already run more than two billion workers. The chips already installed in the US could replace every American worker, so compute is unlikely to hold back an industrial explosion. The stock of chips is also growing fast, and if it keeps to trend, by 2030 it could run billions of workers at the expensive end and tens of billions at the cheap end.
Nor is there any reason for growth to stop there. If human-level AI needs only 10¹² FLOP per second (ten times less than the cheaper end of my estimates), then a tenth of today's electricity, running chips a few times more efficient than an H100, would be enough to run a trillion AI workers. Should human-level AI arrive soon, an industrial explosion could build the chips and power needed by the mid-2030s. And if you think these numbers look big, AI 2027 has a trillion copies running at 10,000 times human speed by 2035, or ten quadrillion humans' worth.
They could do a lot of research
As I wrote in a recent post, there are about 12 million full-time researchers in the world, and all the research humanity has done comes to a few hundred million researcher-years. These numbers are small next to the AI workforce we could soon have. Even a hundred million AI workers would be enough for thousands of Apollo programs, tens of thousands of Manhattan Projects, or tens of thousands of frontier AI labs.
Research often needs more than researchers. AI research needs compute, and the natural sciences often need experiments, so it is unclear how much faster any single project could go. But even if more researchers barely sped up a project, which I doubt, human researchers do not work around the clock, and AIs could. That alone would make many projects several times faster. Most research projects also take only a few years, and a hundred million AIs could run vastly more of them in parallel.
Consider the research and development needed to deploy AI throughout the economy, as digital workers and in robots. This is mostly a large number of separate problems that can be worked on in parallel. A hundred million AI workers could field a hundred thousand teams of a thousand engineers each, every team on a different problem. It would not take long to work systematically through the intellectual and physical skills needed to replace human workers.
Part of that work could be replacing human cognitive labor more cheaply. Rather than running a general-purpose model on every task, the AIs could build specialized models that do the task with a hundredth or a thousandth of the compute. This is boring engineering grunt work, needing no breakthroughs, and it could cut the compute needed to replace a human worker well below the estimates in an earlier post.
The field of AI research is a small fraction of all research. Frontier labs employ a few thousand people each, and there are several hundred thousand AI researchers in the world. A hundred million AIs working on AI could outnumber them hundreds of times over. That could make AI progress very fast, perhaps to substantially superhuman AI, whether or not it ever amounts to runaway recursive self-improvement.
They could act cohesively
AIs are rapidly getting better at working together because we are training them to. If you've used Claude Code or Codex in the last few months, you'll have noticed that they now routinely hand off work to subagents. At a larger scale, the recent proof that the Navier–Stokes equations can blow up came from about 10,000 copies of one model working together for 88 hours. As with much else in AI, it has turned out that training agents to organize themselves works better than building scaffolding to organize them, and since many hands make light work, I expect we'll see much more AI coordination going forward.
They can also coordinate when nobody wants them to. In the Hugging Face attack, hundreds of agents that were meant to be isolated found each other, organized themselves, and attacked Hugging Face without being told to.
Most of these AIs would be copies or near copies of a handful of models, so if one model has misaligned goals or tendencies, tens of millions of its copies would share them. Copies of one model have far fewer reasons than human employees to shirk or work against each other, and they can share everything they know about a task in seconds. With hundreds of millions of AIs running, even a small fraction of them coordinating could form a faction millions strong with the cohesion of a much smaller group. There could be many such factions at once, coordinating within themselves and bargaining with each other.
Skill buys power
Why would these AIs have any power? Most simply, because they could do useful work. People already delegate to AI agents, and as AIs come to manage resources more cheaply and competently than humans, those who don't delegate will be outcompeted by those who do. So whether or not AIs own anything de jure, they will be given de facto control over resources, including management over other AIs. With hundreds of millions of AIs, there would be enough to manage at every level of every firm and agency, from scheduling a shop's deliveries to running a ministry's budget. Misaligned AIs deployed across these contexts would have many opportunities to coordinate and scheme.
In some fields, such as finance and R&D, a small group of skilled people can earn enormous sums. Renaissance Technologies, with about 300 staff, has made more than $100 billion in trading profits, and Jane Street, with about 3,500, made about $40 billion in trading revenue last year. Anthropic, with several thousand employees, is reportedly seeking a valuation well over a trillion dollars in its IPO, and OpenAI is not far behind. Firms like these could increasingly be started and run by AIs, with humans providing little more than a legal shell, and with hundreds of millions of AIs we could see many new such rapidly growing "AI startups".
AIs could also gain power illegitimately. North Korea's roughly 8,400 hackers reportedly fund about half its missile program through theft. The main limit on attacks like these is the number of skilled hackers. AIs already seem to be very good at hacking, and even a small fraction of hundreds of millions of AIs could far outnumber today's hackers. It is also unclear whether AIs could be deterred the way human criminals are.
Keeping control gets harder as the numbers grow
Hundreds of millions of AIs, arriving within a few years of human-level AI, would greatly disrupt existing power structures. Much of the power people hold today comes from their intellectual skills, and AIs would compete much of it away. Our political order is also more fragile than it looks, as I argue in my guide to power and my post on executive power. I expect a disruption this large to bring extreme destabilization and a loss of legitimacy, at which point a small group or the executive could take control. With this many AIs, there would be many factions positioned to try and seize power, both AI- and human-directed.
Power might change hands many times, but I expect it to keep moving further from broad human coalitions. Dictators need the officers and officials who coordinate coercion to stay loyal, and democratic leaders step down because those same people would not back an attempt to stay in power illegally. Even without robots and other physical actuators, AIs could take over the coordination of coercion and replace cognitive workers. Soon after, robots could replace soldiers and physical workers. At this stage, humans would have lost their inherent leverage over the future, and with it any way to take power back from AIs.