METR researchers spent three months inside Anthropic to investigate whether AI is accelerating Anthropic's research, and how much. Their answers are yes, and by a lot.
They estimate that between October 2025 and October 2026, Anthropic's model capabilities improved at a 2x faster growth rate than between March 2024 and October 2025. They attribute about 80% of that acceleration to AI, and 20% to non-AI factors. If true, this is the strongest evidence yet that AI is measurably speeding up frontier AI development itself – the feedback loop behind worries about recursive self-improvement leading to runaway AI progress.
Despite its urgency, METR’s report is hard to assess, because Anthropic redacted the majority of METR’s analysis. As a result, I’ve seen plenty of criticism that we can’t really learn anything from this report.
This is wrong. There is plenty of public information we can use to triangulate METR’s claims, and figure out what we should actually learn from it. Specifically, I use public data to answer three questions:
Is the acceleration real? (yes.)
Is the acceleration because of AI? (probably.)
Is the acceleration going to continue? (who the fuck knows?)
Let’s take each of them in turn.
Is the acceleration real?
The majority of METR's unredacted analysis focuses on AECI, Anthropic's internal version of Epoch's Capabilities Index (ECI). Both indices do the same thing: they take a model's scores on many benchmarks and boil them down to a single capability number, after estimating how hard and how discriminating each benchmark is. The difference is that AECI is fit to mostly internal benchmarks. On AECI, Anthropic's models were improving at about 15 points a year before October 2025, and about 29 points a year after, a difference of 1.9x.
At first, this seems wrong. The frontier of public ECI shows no break at all in October 2025. But that is the frontier across all labs (not Anthropic's specific trend), and using Epoch's latest methodology. Once you make the comparison like-for-like, public data reproduces almost all of METR's conclusion. In sequence, we can:
Focus only on the trend for Anthropic models. This is what anyone can see on Epoch's site today. It already shows a break: Anthropic's progress after October 2025 is 1.42x as fast as before.
Refit ECI according to METR's method. Epoch's published scores now come from an updated Bayesian methodology, whereas AECI uses Epoch's older frequentist fit.[1] Refitting the same public data with Epoch's previous methodology raises the speedup to 1.64x.
Compare to METR's AECI on the same models and dates. We don't know what benchmarks AECI includes, but METR does, and we can compare their fits on Anthropic's models to AECI fits on those models. AECI gives a 1.76x speedup. That is within about 0.1x of the least-squares ECI.
Add Anthropic's internal models. METR plots the results from Anthropic's internal models, which could affect the trend break even further, since their capabilities are higher than public models. This replicates METR’s 1.91x speedup.
The first two steps represent the trend we can estimate with public data, while the last two steps represent additional information from Anthropic's private data.
The last two steps take the acceleration from 1.64x to 1.9x, METR's headline, but the majority of the work is in the first two steps, which can be reproduced from public data. In other words, there is nothing mysterious about the gap between public ECI and Anthropic's results, and the public data supports a trend break in Anthropic's AI capabilities after October 2025. In other words, you don't need to trust Anthropic's internal data to believe METR's headline result of acceleration at Anthropic.
The one wrinkle is that you do need Anthropic's internal model data to distinguish between a trend break and a one-time jump. In the Opus 5.5 system card, Anthropic analyzed recent capability progress, and found that a one-time jump in AI capabilities at Mythos Preview (with the same trend as before) fits the AECI data better than a trend break. But METR finds the opposite – a trend break fits the data better than a one-time jump.
The difference is that METR's report includes Anthropic's unreleased models, while the Opus 5.5 system card did not. So the evidence favoring a trend break over a one-time jump does rely on Anthropic's internal models, and we can't scrutinize their AECI scores to know if we should actually believe a one-time jump over a trend break.
Is the acceleration because of AI?
Maybe Anthropic's progress is really 1.9x faster than it was through October 2025. It's still possible that non-AI factors explain that acceleration. METR guesses that 80% of the current acceleration is due to AI accelerating AI research; is this likely to be true?
Redactions make this a hard question, but two sources of evidence line up with METR's assessment:
Triangulation based on AI uplift. Staff-reported AI uplift converts to roughly the acceleration METR attributes to AI.
Triangulation based on when the trend break is. The trend break arrived around when Anthropic researchers got Opus 4.5, the first excellent coding model.
Uplift. Anthropic runs informal Slack polls asking staff how much faster they work with the latest model, and the average estimate reached 12x in October. In the Mythos Preview system card, Anthropic said that the roughly 4x uplift staff reported then implied an overall progress multiplier below 2x, and that reaching 2x would need uplift to be an order of magnitude larger (so, 40x). If the conversion rate from uplift to AI progress is a power law, that statement pins the power law exponent at about 0.19 (400.19 = 2x). So 12x labor uplift converts to about a 120.19 = 1.6x speedup in overall R&D, very close to the 1.61x implied by METR's 80% attribution. (If 80% of the 1.9x speedup is due to AI, progress without AI would have been 1.18x its old rate, so AI multiplies the rate by 1.9/1.18 = 1.61x.)
Anthropic's method is relatively opaque – we don't know what went into the statement that 40x labor uplift would be needed for 2x acceleration – but you can also do the same analysis with Ryan Greenblatt's model of AI progress (here, here). In his model, a 7x labor acceleration from AI would be enough to replicate a 1.61x speedup from AI:
AI speeds up each researcher's work, but labor isn't the only input to research, so research speed rises less than one-for-one: software research speed is proportional to (labor acceleration)0.55. So if experiment compute grows on trend[2], a 7x labor acceleration makes software research 70.55 = 2.9x faster.
Faster research runs into diminishing returns sooner, because each extra year of software progress makes the next one harder. Greenblatt estimates that each year a lab gets ahead of its old trend halves how much a given research effort achieves. Greenblatt estimates that Anthropic is now about 0.6 years ahead, so research has to run 20.6 = 1.5x faster just to keep up, and 2.9x faster research gives about 2.9/1.5 = 1.9x faster software progress.
Software is only about two-thirds of AI progress, while the other third comes from scaling training compute. So 1.9x faster software progress means 2/3 × 1.9 + 1/3 × 1 = 1.61x faster progress overall, which is roughly the AI-attributable speedup implied by METR's 80% guess.
Put differently, AI only needs to make researchers 7x more productive for Greenblatt’s model to predict 1.61x speedup from AI – and the staff survey reports a higher than 7x uplift, suggesting the speedup can more than adequately be attributed to AI uplift.
Of course, Slack polls of Anthropic staff have little credibility, and METR doesn’t put much weight on them. But this makes the convergence between these Slack polls and structural estimates of AI progress all the more surprising. Overall, this triangulation reassures me that AI is really responsible for Anthropic’s acceleration.
Timing. METR did not impose that the trend break happened in October 2025 – that is the trend break that fits the data best. But October 2025 is surprisingly close to November 2025, when Claude Opus 4.5 became widely available to Anthropic researchers. Opus 4.5 was widely considered the first model that was really excellent at software engineering. So if the acceleration was caused by some non-AI factor, it would be quite a coincidence that this factor kicked in around the same time that Anthropic researchers gained access to the first excellent coding model.
The obvious problem is that October is before November, and that Opus 4.5-based uplift would surely take at least a month, leaving two months of unexplained progress. But both METR and Anthropic acknowledge that late-2025 progress was driven by human research. Mythos Preview’s system card (section 2.3.6) confidently attributes late-2025 gains to human research rather than AI uplift, and METR’s report also distinguishes late-2025 gains from its 80% AI attribution. So unspecified human research breakthroughs were behind late-2025 gains, tiding over the two months without Opus 4.5-induced progress.
Thus, these two independent sources of evidence point to AI playing a strong role in the acceleration. In addition, I find non-AI hypotheses for where this acceleration came from to be implausible:
Training compute? No. METR's only definitive statement in the report is that the acceleration was not due to faster growth in training compute. I don't think they would make a statement that definitive without strong evidence, so we can probably rule that out.
Experimental compute? Maybe. Experimental compute scaling was notably not ruled out by METR’s analysis, and it is totally possible for faster-than-trend experimental compute growth to lead to an acceleration. But if experimental compute is complementary with AI labor, experimental compute scaling is most valuable if you also scale AI labor, making it murky whether you would count this as a separate explanation from AI automation.
Data? Doubtful. On its face, this is the strongest alternative hypothesis – Anthropic is scaling their data pipeline so aggressively that it could lead to a large acceleration without AI. But Anthropic's S-1 puts vendor spending on data quality and evaluation at $330-392M in 2025, or 5-6% of R&D – this is too small of an R&D input to explain large accelerations. To be precise, growth accounting assumes the cost share of an input is the elasticity of output to that input. Even if the data elasticity was 0.1, larger than its cost share, for data spending to lead to a 1.5x acceleration would require data spending to grow 1.51/0.1 = 58x in 2026, which is implausible.[3]
A specific breakthrough? Doubtful. We already know that late-2025 progress was driven by human research breakthroughs. Maybe 2026 progress is the same? Perhaps Anthropic researchers made an unreleased algorithmic breakthrough whose importance is comparable to the invention of RLHF or reasoning models, that set them on a new trend. After all, their stated rationale for redactions on METR's report was that the redacted information could accelerate other AI developers, which would certainly be consistent with this story. But – and this is just reading tea leaves – if such a breakthrough existed, I would expect it to be a strong enough alternative hypothesis that METR would put much less than 80% weight on AI as an explanation, and I'd expect Anthropic staff to put more weight on this alternative hypothesis too. But only two of the eleven interviewed Anthropic researchers disagreed with the AI hypothesis, and only one of those two cited a redacted alternative hypothesis. That redacted factor could be exactly such a breakthrough, so this is weak counter-evidence, but it is counter-evidence nonetheless.
Hiring? Doubtful. One of the two Anthropic staff members who disagreed with the AI hypothesis cited hiring as the alternative source of productivity increases. But Anthropic's hiring has not actually accelerated enough to support this as a reason for a trend break. Their S-1 says headcount grew more than 5x from December 2024 to September 2026, and 2.7x in the last year of that. That implies headcount was growing about 2.7x a year both before and after the break – no acceleration at all, so hiring can't explain a trend break in capability progress.[4]
Is the acceleration going to continue?
So maybe the 1.9x acceleration is real, and maybe it is mostly because of AI. How worried should we be? Does this mean the acceleration will continue, triggering a runaway loop of recursive self-improvement that leads to superintelligence? Or could it be stopped by bottlenecks?
Here, unfortunately, we have nothing to go on. METR says they "did not identify any convincing bottlenecks" to continued rapid progress. But they also say that their view of bottlenecks changed several times as they got more information, so we don't have to put too much weight on the statement that there were no convincing bottlenecks. In particular, there's one bottleneck that could be consistent with the current acceleration but would still rule out runaway recursive self-improvement – verifiability.
Throughout the report, METR is careful to say that they are measuring progress only on well-specified tasks. We don't know what tasks are in AECI, but we can imagine they overrepresent tasks that can be graded by AIs and have clear rubrics, which also makes them easier to train models to perform. But this also makes them less representative of messy research tasks that might dominate human effort. For example, Anthropic's August risk report (section 3.4) says its models do not yet substitute for its researchers' "research taste", and that researchers often hold back from using more AI because they are bottlenecked on steps they don't trust the models to do.
Imagine AI research has two kinds of work: verifiable tasks (e.g. debugging a training run) and non-verifiable tasks (e.g. deciding which ideas are worth testing). Suppose AI dramatically speeds up the verifiable work and does little for the rest. At first you'd see exactly what METR sees, a trend break. But as the verifiable work gets cheaper and cheaper, the non-verifiable work becomes the binding constraint, and the acceleration stops. If the two kinds of work are complements, the acceleration is capped by how much of research is non-verifiable. This story would predict a trend break in AI capabilities – but it would also predict that the new trend is constant, rather than a prelude to even faster acceleration in the future.
A theory in which explosive growth is bottlenecked by some factor (whether verifiability or something else) predicts a trend break that is capped at 2.2x without leading to runaway acceleration. In contrast, a model without any bottleneck fits the data worse, since the data is pretty close to a linear trend break.
Is this scenario representative of AI progress? I have no idea. As a check, I had a research agent classify ECI's benchmarks by how verifiable they are. I then re-scored each Anthropic model, once using only the more-verifiable benchmarks and once using only the less-verifiable benchmarks.[5] The trend break is about the same on both: 1.67x on verifiable benchmarks, and 1.64x on less verifiable ones. So this analysis gives no support to a differential speedup of more-verifiable tasks, no support to the idea that less-verifiable tasks could bottleneck AI progress.
So we have no way to assess METR's claim that diminishing returns are unlikely to bind in the future, and bottlenecks are consistent with the trend break we’ve seen. But my leading hypothesis for a bottleneck to AI progress does not seem to be supported in the data. That said, this is the weakest part of my analysis by far, since "more verifiable" vs "less verifiable" is a dubious distinction between ECI benchmarks. (They’re all basically verifiable; that’s why they’re benchmarks!) We just don’t know how to forecast the bottlenecks to AI progress, and so there isn’t much public that we can compare METR’s assessment to.
Diminishing returns might be surprisingly weak?
I don't read this report as evidence that AI makes AI researchers more productive. Anybody who has used AI to write code knows that already. What is surprising is that this productivity effect shows up as a trend break in progress.
There have been tons of innovations in AI, and most of them have not led to a trend break. In 2024, reasoning models arrived. That was a hugely consequential innovation that defined AI research – but it did not create a new trend in frontier capability growth. Reasoning models were just a necessary innovation to stay on the old trend, fighting the effect of diminishing returns. That is what diminishing returns look like: you have to invest more and more just to keep progress going at the same pace.
So the question for models that are good at coding has always been: are coding models just another improvement we need to feed into the fire to maintain constant progress? The major surprise from this report is that the answer is no, that coding models could buy us progress on top of the existing trend. In other words, even if Opus 4.5 and the models after it had been bad at coding, could Anthropic have kept up its old rate of progress through simple compute/data scaling? Maybe. That is evidence that diminishing returns are weaker than I thought, that AI progress might not need crazy new breakthroughs to scale to superintelligence.
Of course, an equally plausible story is that AI uplift was much higher than I would have expected. For example, the triangulation exercise with Ryan Greenblatt's model already assumed a diminishing returns parameter that is historically grounded, and didn't need weak diminishing returns to fit the data. It just needed AI to make Anthropic's researchers “only” 7x faster, an enormous productivity effect by any historical standard. So the trend break could just as well reflect a very large productivity shock running into ordinary diminishing returns. Either way, the story in which AI productivity uplift is just barely enough to offset diminishing returns is likely false.
Conclusion
I’ve seen a lot of people on Twitter dismiss the METR report because of its redactions, and the difficulty of assessing evidence without knowing how it was generated. But dismissing the report based on its redactions is a mistake. It contains enough concrete details that we can assess its validity against public data.
I started this analysis expecting to cast doubt on METR’s analysis, but I ended it believing them. Will this acceleration continue to hold, and will it lead to recursive self-improvement? I guess we’ll find out.
METR never says outright that AECI uses Epoch's older least-squares fit, but it is implied. Their robustness checks vary "L2 regularization" (making it 10x weaker or stronger, removing it, re-centring it on a time trend), and that kind of regularization belongs to the least-squares fit. Epoch switched its published index to a Bayesian fit in late September 2026, deep into METR’s engagement with Anthropic.
Note that experiment compute could be growing above trend. In that case, the serial labor acceleration required to fit the ECI trend break would be even lower than 7x.
The S-1 doesn't report data spending as a single number. It reports that "vendor costs related to data quality and performance evaluation services" rose $165M in 2024, $165M in 2025 and $251M in the first half of 2026, which together with the 2023 category total bounds the levels. The line bundles evaluation vendors with data, may not include RL environments, and excludes in-house data teams.
The S-1 reports approximately 940 employees as of December 31, 2024 and approximately 5,300 as of September 30, 2026, a 5.6x increase over 21 months. It also reports that headcount grew approximately 170% (2.7x) from September 30, 2025 to September 30, 2026. That leaves 5.6/2.7 = 2.1x growth over the nine months from December 2024 to September 2025, which annualizes to 2.112/9 = 2.7x a year – the same rate as after the break.
I label a benchmark "less verifiable" when its answers are graded by a rubric, an AI judge or human raters, rather than checked automatically against a known answer, a unit test or a numeric score. These are the tasks where it is hardest to tell whether a model got it right, which is also what makes them hard to train on. Nine of ECI's benchmarks count as less verifiable: APEX-Agents, CL-bench, CL-bench Life, DeepResearch Bench, FrontierCode, Furniture Assembly, GDPval, LMCA and Remote Labor Index. The other 39 count as verifiable. Each model's score on a subset is estimated holding every benchmark's difficulty and discriminability at their values in the full least-squares fit.
METR researchers spent three months inside Anthropic to investigate whether AI is accelerating Anthropic's research, and how much. Their answers are yes, and by a lot.
They estimate that between October 2025 and October 2026, Anthropic's model capabilities improved at a 2x faster growth rate than between March 2024 and October 2025. They attribute about 80% of that acceleration to AI, and 20% to non-AI factors. If true, this is the strongest evidence yet that AI is measurably speeding up frontier AI development itself – the feedback loop behind worries about recursive self-improvement leading to runaway AI progress.
Despite its urgency, METR’s report is hard to assess, because Anthropic redacted the majority of METR’s analysis. As a result, I’ve seen plenty of criticism that we can’t really learn anything from this report.
This is wrong. There is plenty of public information we can use to triangulate METR’s claims, and figure out what we should actually learn from it. Specifically, I use public data to answer three questions:
Let’s take each of them in turn.
Is the acceleration real?
The majority of METR's unredacted analysis focuses on AECI, Anthropic's internal version of Epoch's Capabilities Index (ECI). Both indices do the same thing: they take a model's scores on many benchmarks and boil them down to a single capability number, after estimating how hard and how discriminating each benchmark is. The difference is that AECI is fit to mostly internal benchmarks. On AECI, Anthropic's models were improving at about 15 points a year before October 2025, and about 29 points a year after, a difference of 1.9x.
At first, this seems wrong. The frontier of public ECI shows no break at all in October 2025. But that is the frontier across all labs (not Anthropic's specific trend), and using Epoch's latest methodology. Once you make the comparison like-for-like, public data reproduces almost all of METR's conclusion. In sequence, we can:
The first two steps represent the trend we can estimate with public data, while the last two steps represent additional information from Anthropic's private data.
The last two steps take the acceleration from 1.64x to 1.9x, METR's headline, but the majority of the work is in the first two steps, which can be reproduced from public data. In other words, there is nothing mysterious about the gap between public ECI and Anthropic's results, and the public data supports a trend break in Anthropic's AI capabilities after October 2025. In other words, you don't need to trust Anthropic's internal data to believe METR's headline result of acceleration at Anthropic.
The one wrinkle is that you do need Anthropic's internal model data to distinguish between a trend break and a one-time jump. In the Opus 5.5 system card, Anthropic analyzed recent capability progress, and found that a one-time jump in AI capabilities at Mythos Preview (with the same trend as before) fits the AECI data better than a trend break. But METR finds the opposite – a trend break fits the data better than a one-time jump.
The difference is that METR's report includes Anthropic's unreleased models, while the Opus 5.5 system card did not. So the evidence favoring a trend break over a one-time jump does rely on Anthropic's internal models, and we can't scrutinize their AECI scores to know if we should actually believe a one-time jump over a trend break.
Is the acceleration because of AI?
Maybe Anthropic's progress is really 1.9x faster than it was through October 2025. It's still possible that non-AI factors explain that acceleration. METR guesses that 80% of the current acceleration is due to AI accelerating AI research; is this likely to be true?
Redactions make this a hard question, but two sources of evidence line up with METR's assessment:
Uplift. Anthropic runs informal Slack polls asking staff how much faster they work with the latest model, and the average estimate reached 12x in October. In the Mythos Preview system card, Anthropic said that the roughly 4x uplift staff reported then implied an overall progress multiplier below 2x, and that reaching 2x would need uplift to be an order of magnitude larger (so, 40x). If the conversion rate from uplift to AI progress is a power law, that statement pins the power law exponent at about 0.19 (400.19 = 2x). So 12x labor uplift converts to about a 120.19 = 1.6x speedup in overall R&D, very close to the 1.61x implied by METR's 80% attribution. (If 80% of the 1.9x speedup is due to AI, progress without AI would have been 1.18x its old rate, so AI multiplies the rate by 1.9/1.18 = 1.61x.)
Anthropic's method is relatively opaque – we don't know what went into the statement that 40x labor uplift would be needed for 2x acceleration – but you can also do the same analysis with Ryan Greenblatt's model of AI progress (here, here). In his model, a 7x labor acceleration from AI would be enough to replicate a 1.61x speedup from AI:
Put differently, AI only needs to make researchers 7x more productive for Greenblatt’s model to predict 1.61x speedup from AI – and the staff survey reports a higher than 7x uplift, suggesting the speedup can more than adequately be attributed to AI uplift.
Of course, Slack polls of Anthropic staff have little credibility, and METR doesn’t put much weight on them. But this makes the convergence between these Slack polls and structural estimates of AI progress all the more surprising. Overall, this triangulation reassures me that AI is really responsible for Anthropic’s acceleration.
Timing. METR did not impose that the trend break happened in October 2025 – that is the trend break that fits the data best. But October 2025 is surprisingly close to November 2025, when Claude Opus 4.5 became widely available to Anthropic researchers. Opus 4.5 was widely considered the first model that was really excellent at software engineering. So if the acceleration was caused by some non-AI factor, it would be quite a coincidence that this factor kicked in around the same time that Anthropic researchers gained access to the first excellent coding model.
The obvious problem is that October is before November, and that Opus 4.5-based uplift would surely take at least a month, leaving two months of unexplained progress. But both METR and Anthropic acknowledge that late-2025 progress was driven by human research. Mythos Preview’s system card (section 2.3.6) confidently attributes late-2025 gains to human research rather than AI uplift, and METR’s report also distinguishes late-2025 gains from its 80% AI attribution. So unspecified human research breakthroughs were behind late-2025 gains, tiding over the two months without Opus 4.5-induced progress.
Thus, these two independent sources of evidence point to AI playing a strong role in the acceleration. In addition, I find non-AI hypotheses for where this acceleration came from to be implausible:
Is the acceleration going to continue?
So maybe the 1.9x acceleration is real, and maybe it is mostly because of AI. How worried should we be? Does this mean the acceleration will continue, triggering a runaway loop of recursive self-improvement that leads to superintelligence? Or could it be stopped by bottlenecks?
Here, unfortunately, we have nothing to go on. METR says they "did not identify any convincing bottlenecks" to continued rapid progress. But they also say that their view of bottlenecks changed several times as they got more information, so we don't have to put too much weight on the statement that there were no convincing bottlenecks. In particular, there's one bottleneck that could be consistent with the current acceleration but would still rule out runaway recursive self-improvement – verifiability.
Throughout the report, METR is careful to say that they are measuring progress only on well-specified tasks. We don't know what tasks are in AECI, but we can imagine they overrepresent tasks that can be graded by AIs and have clear rubrics, which also makes them easier to train models to perform. But this also makes them less representative of messy research tasks that might dominate human effort. For example, Anthropic's August risk report (section 3.4) says its models do not yet substitute for its researchers' "research taste", and that researchers often hold back from using more AI because they are bottlenecked on steps they don't trust the models to do.
Imagine AI research has two kinds of work: verifiable tasks (e.g. debugging a training run) and non-verifiable tasks (e.g. deciding which ideas are worth testing). Suppose AI dramatically speeds up the verifiable work and does little for the rest. At first you'd see exactly what METR sees, a trend break. But as the verifiable work gets cheaper and cheaper, the non-verifiable work becomes the binding constraint, and the acceleration stops. If the two kinds of work are complements, the acceleration is capped by how much of research is non-verifiable. This story would predict a trend break in AI capabilities – but it would also predict that the new trend is constant, rather than a prelude to even faster acceleration in the future.
A theory in which explosive growth is bottlenecked by some factor (whether verifiability or something else) predicts a trend break that is capped at 2.2x without leading to runaway acceleration. In contrast, a model without any bottleneck fits the data worse, since the data is pretty close to a linear trend break.
Is this scenario representative of AI progress? I have no idea. As a check, I had a research agent classify ECI's benchmarks by how verifiable they are. I then re-scored each Anthropic model, once using only the more-verifiable benchmarks and once using only the less-verifiable benchmarks.[5] The trend break is about the same on both: 1.67x on verifiable benchmarks, and 1.64x on less verifiable ones. So this analysis gives no support to a differential speedup of more-verifiable tasks, no support to the idea that less-verifiable tasks could bottleneck AI progress.
So we have no way to assess METR's claim that diminishing returns are unlikely to bind in the future, and bottlenecks are consistent with the trend break we’ve seen. But my leading hypothesis for a bottleneck to AI progress does not seem to be supported in the data. That said, this is the weakest part of my analysis by far, since "more verifiable" vs "less verifiable" is a dubious distinction between ECI benchmarks. (They’re all basically verifiable; that’s why they’re benchmarks!) We just don’t know how to forecast the bottlenecks to AI progress, and so there isn’t much public that we can compare METR’s assessment to.
Diminishing returns might be surprisingly weak?
I don't read this report as evidence that AI makes AI researchers more productive. Anybody who has used AI to write code knows that already. What is surprising is that this productivity effect shows up as a trend break in progress.
There have been tons of innovations in AI, and most of them have not led to a trend break. In 2024, reasoning models arrived. That was a hugely consequential innovation that defined AI research – but it did not create a new trend in frontier capability growth. Reasoning models were just a necessary innovation to stay on the old trend, fighting the effect of diminishing returns. That is what diminishing returns look like: you have to invest more and more just to keep progress going at the same pace.
So the question for models that are good at coding has always been: are coding models just another improvement we need to feed into the fire to maintain constant progress? The major surprise from this report is that the answer is no, that coding models could buy us progress on top of the existing trend. In other words, even if Opus 4.5 and the models after it had been bad at coding, could Anthropic have kept up its old rate of progress through simple compute/data scaling? Maybe. That is evidence that diminishing returns are weaker than I thought, that AI progress might not need crazy new breakthroughs to scale to superintelligence.
Of course, an equally plausible story is that AI uplift was much higher than I would have expected. For example, the triangulation exercise with Ryan Greenblatt's model already assumed a diminishing returns parameter that is historically grounded, and didn't need weak diminishing returns to fit the data. It just needed AI to make Anthropic's researchers “only” 7x faster, an enormous productivity effect by any historical standard. So the trend break could just as well reflect a very large productivity shock running into ordinary diminishing returns. Either way, the story in which AI productivity uplift is just barely enough to offset diminishing returns is likely false.
Conclusion
I’ve seen a lot of people on Twitter dismiss the METR report because of its redactions, and the difficulty of assessing evidence without knowing how it was generated. But dismissing the report based on its redactions is a mistake. It contains enough concrete details that we can assess its validity against public data.
I started this analysis expecting to cast doubt on METR’s analysis, but I ended it believing them. Will this acceleration continue to hold, and will it lead to recursive self-improvement? I guess we’ll find out.
METR never says outright that AECI uses Epoch's older least-squares fit, but it is implied. Their robustness checks vary "L2 regularization" (making it 10x weaker or stronger, removing it, re-centring it on a time trend), and that kind of regularization belongs to the least-squares fit. Epoch switched its published index to a Bayesian fit in late September 2026, deep into METR’s engagement with Anthropic.
Note that experiment compute could be growing above trend. In that case, the serial labor acceleration required to fit the ECI trend break would be even lower than 7x.
The S-1 doesn't report data spending as a single number. It reports that "vendor costs related to data quality and performance evaluation services" rose $165M in 2024, $165M in 2025 and $251M in the first half of 2026, which together with the 2023 category total bounds the levels. The line bundles evaluation vendors with data, may not include RL environments, and excludes in-house data teams.
The S-1 reports approximately 940 employees as of December 31, 2024 and approximately 5,300 as of September 30, 2026, a 5.6x increase over 21 months. It also reports that headcount grew approximately 170% (2.7x) from September 30, 2025 to September 30, 2026. That leaves 5.6/2.7 = 2.1x growth over the nine months from December 2024 to September 2025, which annualizes to 2.112/9 = 2.7x a year – the same rate as after the break.
I label a benchmark "less verifiable" when its answers are graded by a rubric, an AI judge or human raters, rather than checked automatically against a known answer, a unit test or a numeric score. These are the tasks where it is hardest to tell whether a model got it right, which is also what makes them hard to train on. Nine of ECI's benchmarks count as less verifiable: APEX-Agents, CL-bench, CL-bench Life, DeepResearch Bench, FrontierCode, Furniture Assembly, GDPval, LMCA and Remote Labor Index. The other 39 count as verifiable. Each model's score on a subset is estimated holding every benchmark's difficulty and discriminability at their values in the full least-squares fit.