METR came out with an extensive report on how much AI is accelerating AI R&D at Anthropic, and thus how close we might be to recursive self-improvement (RSI) and how far that might go.
This is one of the most important questions in the world.
Their answer: Things are already accelerating on the order of a 2x speedup.
This is several months ahead of the AI 2027 timeline. A naive extrapolation says we may be less than a year away from full RSI. There are no clear bottlenecks in the way.
This report updated many towards shorter timelines.
This is consistent with what OpenAI and Anthropic themselves have been saying, and with extrapolation from the progress we are observing. We are running out of time.
That doesn’t have to be the way this plays out. We could stop the labs. The labs could choose to stop. Or the estimates could be wrong, or some bottlenecks could be hit, or the results could be less impactful than we expect. Indeed do many things come to pass. And if it does happen this quickly, one would expect it to go quite poorly, and likely to cause everyone to die, but perhaps it goes well.
It doesn’t look good.
The central conclusions are:
Capabilities are advancing at roughly twice the rate they did from March 2024 to October 2025 (CI ~1.3x-3x), with wide uncertainty. And that a majority of that acceleration is probably due to AI doing AI R&D. Since October 2025, Anthropic has made ~2x progress on the ECI versus periods prior to October 2025.
Yes, this seems like what has been obviously happening.
There are no observed bottlenecks or strong diminishing returns that would prevent advancement from accelerating further.
Unfortunately, Anthropic redacted the analysis of various bottlenecks.
METR: These conclusions are based on key empirical findings from the full report.
Some of the key findings that most informed our conclusions about risk were redacted.
Ajeya Cotra (METR): For the last few months, some of my colleagues at METR have been working on a deep-access investigation to understand how much Anthropic is being accelerated by its AI systems. From my reading of their report and Anthropic’s RSP, it looks like the “dramatic acceleration” threshold has already been crossed.
I do not find either of these results surprising. I made minor updates towards faster outcomes and fewer bottlenecks, but not major ones. I had already made major ones.
The situation looks rather grim. But this is not a surprise.
The information from inside the labs makes you think the situation is grim and timelines are short, but those with that information were already shouting from the rooftops that their secret information meant the situation is grim and timelines were short. Why did you not think this was the standard reaction to their information?
If you are making major updates in response to this report, that is probably good and wise, but indicates you had previously made a mistake in your estimations.
If you only look at public models, you can make a case that there is a step change at Mythos but that otherwise the rate of progress has not changed. I was always skeptical of that, but it was a valid claim.
Add in the orange data points from internal models and you cannot tell that story. When we include the recent additional internal releases, this takes the estimate from the ~1.7x in the Haiku system card up to a new estimate of ~1.9x.
Peter Wildeford: It seems important to realize that:
– AI companies have unreleased internal models that are more powerful than the public ones we plebs have access to
– Only when you account for these internal models, you can see that AI progress is actually accelerating faster than before
– AI companies use their AIs with like 10,000x more compute than us plebs, so they can do much more than we can even with the same AIs
You could also have gotten to a similar result by extrapolating the automation graph:
Benjamin Todd: Extrapolating Anthropic’s internal AI R&D data results in Claude leading internal research by mid-2027, and plausibly near full automation by the end of 2027, lining up with other methods. (h/t @luc_feron )
The information METR is relying on most is largely redacted. The public report centers on AECI and other estimates. METR is implying that such arguments are merely the legible case they are allowed to make, and that there is a stronger case.
Kudos to Anthropic for allowing this investigation.
METR: Anthropic offered unprecedented access to internal data on their AI R&D process, including a variety of metrics on pretraining, reinforcement learning (RL), training compute in each domain, benchmark scores for internal models, AI speedup analyses, forecasts of a future model’s metrics, and interviews with relevant staff.
This is all highly sensitive stuff, so there are important redactions involved. I would have liked to see less of this, but if this is what it takes to get the report then so be it.
As of October 2026, Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025. This estimate is given with high uncertainty; we would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.
A majority of this acceleration today is likely attributable to Anthropic’s extensive use of AI to do R&D, though there is significant uncertainty and other causes likely played a large role in acceleration in late 2025.
The lack of observed bottlenecks or strong diminishing returns means there is a significant risk that further capabilities progress could lead to even faster acceleration, though, as we say below, we may be missing important bottlenecks.
Here are their top empirical findings:
Since October 2025, benchmark scores (measured via AECI, Anthropic’s version of the Epoch Capabilities Index) for Anthropic’s models have been growing at an estimated 1.9x the pre-October 2025 rate [95% confidence interval: 1.5–2.8x], implying that Anthropic has made roughly two years of historical progress over the last year.
We are confident that the acceleration was not due to an acceleration in the scaling of training compute.
Interviews and surveys are consistent with AI usage causing a doubling in the pace of capabilities progress. Anthropic staff self-report dramatic uplift from use of AI systems in interviews and surveys, although these are challenging to interpret. We were told that R&D is accelerated primarily via AI assistance rather than fully autonomous contributions, but there are some signs that fully autonomous contributions will play a larger role soon.
We did not see evidence for significant bottlenecks which would prevent further capabilities progress due to limits on algorithmic progress, training compute, experiment compute, or data in the next 12 months. However, our understanding of the status of different bottlenecks changed several times as we received more detailed information, so we are not confident in our current assessment.
Data Access and Redactions
The goal of the report was to measure acceleration of Anthropic’s internal growth rate, how much of that acceleration is due to AI R&D and whether there exist any important bottlenecks to further acceleration. This requires a lot of sensitive data, and they got most of what they requested.
They could access:
A detailed description of the data we had access to was redacted. A summarized version is as follows:
Internal benchmark scores and AECI for internal models, plus sample tasks, rubric answers and agent expenditure on those tasks.
Historical data on algorithmic efficiency and overall model strength in pretraining and RL, as well as improvements to compute utilization, inference efficiency and data efficiency.
Headcount and attribution of progress to specific staff.
Training compute metrics in many relevant domains.
Internal analysis on relevant open questions.
Internal surveys of self-reported AI uplift by team.
Lines of code and pull requests across the company and some research teams.
AI agent inference spend over time.
Internal documents on autonomous research contributions.
~15 interviews with key staff, covering a predefined but wide scope that excluded particularly sensitive topics.
In a small number of cases, Anthropic did not provide relevant information when they deemed it too sensitive.
Anthropic also offered extensive staff time and expertise to this project. We believe the two staff we worked most closely with are the two highest-context researchers on AI R&D acceleration measurement at Anthropic. Both of these staff met with us frequently for live discussion, and spent substantial time collecting data, doing analysis, and providing feedback.
In exchange, Anthropic could redact anything in the report. They did so, aggressively.
METR’s only recourse is that it can then share its level of access, that things had been redacted, and whether findings depend on redacted information. Which they do. The majority of the most informative evidence is redacted.
METR: We submitted a ~100-page full report on our assessment to Anthropic. For the public version, Anthropic redacted a majority of the evidence we found most informative. As a consequence, the public must rely on our judgment more than we consider ideal.
Some of these redactions seem clearly appropriate. Others seem questionable.
METR: While we were able to freely share our highest-level conclusions about risk, Anthropic redacted a majority of the evidence we found most informative for these conclusions. Anthropic described the redactions as “aimed at removing confidential information that could accelerate other AI developers.”
Some of these redactions are themselves challenging to describe clearly without revealing sensitive information we have been asked to redact.
We share a table of important evidence we were and were not able to discuss given the redactions in our process section below. All redactions that were important for our conclusions are covered by our descriptions in that table or other in-line descriptions in the report below.
Much of the redacted evidence we viewed had multiple plausible interpretations, and both METR and Anthropic researchers have significant uncertainty about how to synthesize the evidence into a holistic picture. This makes it hard for the public to judge this evidence themselves, reconcile it with evidence from other sources, and come to their own conclusions (i.e. this report does not achieve ‘transparency about evidence’). As a consequence, the public has to rely on METR’s judgment much more than would be ideal, given the weight of the questions answered here.
METR: Anthropic’s redactions also meant that we could not communicate:
Our more detailed estimates of acceleration and attribution to AI usage for key areas of the model training pipeline.
Our more detailed breakdown of progress attributable to compute, data and algorithmic progress, other than the key takeaway that “We are confident that the acceleration was not due to acceleration in training compute.”
More detailed answers to the limitations questions in our proposal, which would require more discussion of methodological details.
Forecasts of metrics for future models.
They also made some other more minor redactions throughout, which we attempt to describe in each section of this report.
Anthropic described the redactions as “aimed at removing confidential information that could accelerate other AI developers.”
They also could not show us any use of METR’s core accounting framework. We know the framework looks like this, but that’s all we get:
The Pace of Progress
This is one of the more annoying redactions. What are these non-AI factors? Candidates include compute scaling, hiring, data improvements, algorithmic discoveries and resolving other bottlenecks. Compute has been ruled out. But we don’t know how they view the other factors.
The acceleration in late 2025 was likely influenced by non-AI factors. We investigated a variety of factors which could have contributed to the observed acceleration in late 2025, but details of many of these factors have been redacted.
We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI. This judgment is given with substantial uncertainty, but is based on a variety of lines of evidence on contributions from AI and other causes. We believe AI’s contribution has grown relatively more important over time, and do not think the other factors we investigated would explain the sustained acceleration. The evidence of speedup from the usage of AI we describe below also seems compatible with a large contribution.
The non-AI acceleration continues, supplemented by the AI-driven one:
Thus they illustrate how to think about all three numbers: The historic trend, the counterfactual no-AI trend and the observed trend that includes use of AI.
1, 2, 3, 4
The Risk Report introduced Model 1 and Model 2. Now we meet 3 and 4, and also 5.
This includes the following models (these are the codenames shared with us; we understand they have different internal codenames):
Unreleased 2022, which finished training in 2022
Model 2, which finished training in July 2026
Model 3, which finished training in August–September 2026
Model 4, which finished training in September–October 2026
METR’s Full Report includes a forecast for a future model, but this was redacted.
Timing Is Everything
If this is consistent, it seems like the clear correct way:
METR: We use the date at which the model could be used by most Anthropic employees instead of public release dates; this allows us to make apples-to-apples comparisons with unreleased models. These in general show slightly higher degrees of acceleration as Anthropic is holding onto more recent models for slightly longer before their public release.
OpenAI used to hold onto models for a longer time. Then it got shorter. Whereas after Mythos, various forces are now holding models back for longer, including the desire to keep an advantage in R&D.
There are different reasons one might want to measure ‘leads’ or rates of progress. If you are measuring for commercial purposes, you care mostly about practical questions of what a public customer can do when. If you are measuring to predict RSI, you care about a very different measurement, and what you care about are internal capabilities, and ability to use them to make progress at the frontier.
By that measure, things are picking up faster, and also the gaps are wider. If you believe in what this report is telling us, these are the measurements that count.
What Could Possibly Have Gone Wrong?
There are a number of ways this could be wrong, or be less than ideally accurate.
The biggest issue is limited access to data. Not all data requests were accepted and there are many documents and other information sources that were not available.
METR might have missed further acceleration, or AECI could underestimate capability growth, or automation could be closer.
On the other hand, perhaps there are non-AI factors that were missed that explain past acceleration, or historical growth was unusually low creating the illusion of fundamental acceleration. Or perhaps acceleration could only work on a narrow set of tasks.
I notice that the upside risks and downside risks of the estimate are asymmetrical. We have three in each direction but they don’t match (Opus 5.5 says some of these do cancel out, but I don’t think they fully do, and others clearly don’t):
Acceleration might be missed but not illusory.
AECI could underestimate effective growth but not overestimate it, although elsewhere they do discuss error in both directions.
Automation could be closer but not farther away.
Historical growth might be unusually low, but not unusually high.
Acceleration could be narrower, but not broader.
Non-AI factors could be underestimated but not overestimated.
Curves on such things do not have to be symmetrical, so long as the average results of all of them are incorporated. This doesn’t have to be an error. I do still wonder.
AECI
The main metric METR uses for this report is the Anthropic fork of the Epoch Capabilities Index, AECI.
AECI is a measure of practical capability. Growth in AECI is multiplicative of odds of success on a task rather than linear, so linear progress on AECI means more than you might think:
Charles Foster (METR): It’s easy to mistake linear gains in ECI for linear gains in model performance. Each additional point actually *multiplies* a model’s odds of success on every benchmark. For example, historically each 5-point increase in ECI corresponded to roughly a doubling in time horizon.
METR focused a lot on the question of whether the metrics are biased or consistent.
METR: These investigations included attempts to understand:
Different methodological variants of internal metrics. We spent multiple weeks on these investigations, and this is our greatest source of uncertainty.
The beliefs of staff on the relevant teams about key internal evidence and estimates of acceleration.
Whether other parts of the model training pipeline could be biasing our estimates of acceleration, and where adjustments could be applied.
Whether Anthropic could be overfitting to internal metrics.
The measurements are also robust to choice of included benchmarks, and to which model snapshots are included, and to regularization.
If non-Anthropic models are included, then the estimate of acceleration goes down to 1.4x.
METR suggests as another potential justification for lower acceleration rates that Anthropic’s models underperformed on benchmarks in the earlier period relative to their true capabilities, which matches my understanding.
I would also suggest that if Anthropic was previously behind, then being behind makes progress easier via fast following, even if you cannot distill.
There are some potential issues with AECI:
We go into detail on potential problems with AECI such as:
Anthropic’s internal benchmarks are increasingly saturated: Model 4 achieves greater than 50% on all the AECI benchmarks on which it was tested. However, measured acceleration appears reasonably robust to saturation concerns.
Token spend may be too low to maximally elicit models.
The logistic link function could be misspecified on a few benchmarks. However, on the whole we think it is a reasonable approximation.
There are various reasons why an acceleration in benchmark scores could overstate or understate an acceleration in real-world impacts.
If AECI is going to serve as a measure of the capabilities of models, the first part of this must be true:
AECI assumes that a model’s score on a benchmark is a function of the model’s latent strength, and specifically that the function is logistic (shown below in black).
Any given metric then has to worry about S-curves and saturation points.
This can be interpreted as an assumption on the distribution of difficulty of tasks in a benchmark, including how many tasks are broken or what fraction can be guessed. The following figure illustrates a few different link functions. The logistic function assumes a symmetric distribution with exponentially declining tails. The log-log function assumes a long tail of hard tasks; the complementary log-log function assumes a long tail of easy tasks.
Depending on what type of curve you think this is, you get different measurements of the rate of acceleration. Unfortunately basically all the curve fits are similarly good at Out-of-sample R2. It makes sense to default to standard logistic, but this illustrates that in theory you can explain or justify a wide range of measurements.
METR says they have no reason to prefer one of these over others. I would say they make distinct claims over the distribution of tasks. So if we want to know which is best, we should look at the tasks.
The Public Information Alone Could Suggest Slower Speedup
In particular, if you are relying only on the AECI, one can selectively edit out the Mythos jump or consider part of this catching up on relative compute allocation.
Toby Ord: I don’t think 2x is the best interpretation of this data. It assumes zero level shift from the new Mythos pre-train. If you allow for a level shift from that as well as a slope change, the slope change drops to about 1.4x.
METR has released a very interesting report suggesting a recent acceleration in AI capabilities this year at Anthropic due to RSI.
But they can’t confidently reject an alternative explanation: the step change you get with the larger pre-train of Mythos.
Woody Lee: It looks like Anthropic was running behind the composite frontier so the speed up in relation to that is 1.4x not sure if this changed any analysis, but it shows that using just the ant baseline may give a different number.
There are two responses to this. One is that there is private redacted information that is factoring into the reports.
Buck Shlegeris: Toby’s analysis totally misses the point here IMO. He’s right that the graph of capabilities over time could be explained as a one-off jump in capabilities from the Mythos pretrain. But that graph is not the main reason why METR thinks AI R&D is now sped up 1.7x by AI. The graph is just the only part of their analysis that wasn’t redacted.
They say that “a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI”, based on “a variety of lines of evidence on contributions from AI and other causes”, after saying that “[t]he acceleration in late 2025 was likely influenced by non-AI factors.”
METR’s report makes it clear that they considered Toby’s hypothesis and rejected it for redacted reasons, rather than failing to consider it, in the appendix “New internal models make a permanent slope change look more plausible than a one-time jump at Mythos Preview.”
It totally makes sense to worry that METR’s analysis is wrong. It’s definitely unfortunate that the main evidence they were allowed to share does not suffice to demonstrate their point, for this reason among others. But I don’t think it makes sense to consider Toby’s argument a knockdown argument against the report’s conclusion, because the conclusion wasn’t mostly derived from the observations whose interpretation Toby disputes. See [here].
The other response is that such modifications are selective. Yes, if you take away whatever the biggest discontinuous improvement is over a period, you will see slower returns. Yes, Anthropic’s position in compute consumption relative to others has shifted. But once we go down such roads, why do we think similar shifts will not happen again?
There is always temptation to extrapolate a future without such jumps. You don’t get to do that. The baseline scenario contains outliers and jumps.
We can only reject the Mythos upshift if we have strong reason to believe we won’t be able to do any similar things again. I don’t see why that would be true.
Well, That Escalated Quickly
It is poised to soon escalate far more quickly. This fact is overdetermined. There are even ‘signs of life’ of areas of potentially full automation. The details are less clear.
METR: We define AI speedup as the ratio between a staff member or team’s output with AI and without AI, noting that output can often be difficult to measure or define.
We now briefly survey the overall evidence for AI speedup, concluding:
There is some evidence for high speedup, though the precise amount of speedup is highly uncertain.
Fully autonomous contributions to AI R&D are not currently a large contributor to overall progress, but they may be in the near future.
… We have many lines of evidence relevant to AI’s contribution to progress in different parts of the model training pipeline, including interviews, self-reported uplift, headcount, lines of code, experiment usage, and qualitative reflections from leadership. We highlight three important unredacted sources.
Both METR and Anthropic interviewed many researchers in detail about the degree of overall acceleration in their area of work attributable to AI use (see details in the appendix below). Our overall impression was a quite wide range of estimates, but centered around a 2x acceleration.
Anthropic also shared data on self-reported uplift due to AI from a variety of sources. The most recent data is from a series of Slack polls. … The responses reflect low-effort consideration. Nevertheless, we observe consistent growth in these self-reports, and very high levels (reaching 12x in October 2026), as shown in the chart below.
Researchers described significant “signs of life” implying [highly significant and fully autonomous] contributions could occur soon.
The estimates here are a lot more than 2x in uplift for the employees, offset by slowness elsewhere, balancing out to 2x. Coding is dramatically more productive now.
Ajeya Cotra (METR): Importantly, if AI is roughly doubling capabilities progress overall, it must be speeding up human researchers by much more than that.
Ajeya Cotra (METR): researcher labor is only one input to discovering these compute multipliers
… If we assume a very simple Cobb-Douglas model where experiment compute and labor are similarly important, that means the uplift to labor roughly needs to be the square of the acceleration in compute multipliers.
… This would imply that we would need a (~2-3x)^2 = ~4-9x speed up in labor to make overall AI capabilities go twice as fast.
They call this ‘broadly support AI doubling the growth rate’ but if we are going purely by the Claude sentiment analysis then the median answer was ‘borderline/unclear’:
Both METR and Anthropic interviewed many researchers (see details in the following section). We used Claude to classify the quotes according to whether they support 2x or more acceleration from AI in their area of work, relative to working without AI. We interpret the results as mildly in favor of 2x or greater acceleration:
Yes: 2
Leans yes: 3
Borderline/unclear: 4
Leans no: 0
No: 2
The quotes, with sensitive details redacted, are in the following section.
The massive personal speedups are necessary to maintain the overall slower pace, so there is no inconsistency here. That’s the frame for looking at the researcher quotes.
They roughly rank the researchers from the most gung-ho (A) to the least, and offer quotes. We start with the two ‘Yes’ votes.
A: I would definitely endorse >2 years of progress in 1 year and I don’t think 4 is crazy.
I think there’s some truth to the ‘we just started trying harder’ story, but IMO the bulk of the effect is recent models.
Like 80% reduction in time between idea and experiment, other 20% some mix of making fewer mistakes and being able to measure fiddly things that would’ve required too much one-off work before.
I think there was a medium bump when it was good enough to code at all and a big bump with Opus-4.5/Opus 4.6 and an even bigger bump with mythos-preview.
B: It is hard to see how I am not being 3x more productive. I feel like I have postdocs and grad students, and we are working towards a larger project. I think the simplest explanation is Claude—just from my individual perspective that feels sensible.
B is talking about personally being more productive, where yes I presume 3x is a floor. Then there is A who is talking explicitly about years of progress.
The consensus is that uplift was not that big before Opus 4.5, then Mythos Preview sent it into overdrive, and other improvements from public models were incremental. There were some voices for Model 2 or Model 4 as additional large jumps.
C, D and E are ‘lean yes’:
C: this is probably the biggest factor imo. When people present experiments, they most often just point to an [agent scaffold] thread. Experimentation overhead is driven towards zero super fast.
D: It is just much easier to [perform] lots of [redacted] experiments that would have taken days. People don’t try [redacted] things bc it takes 1-2 days to implement per idea, but now its ~free with mythos-preview
I personally think this is a large lift and has led to me trying far more ideas.
E: I do genuinely think Claude (claude code, [agent-scaffold]) has been fairly significant. I’ve seen us discovering very promising directions ~entirely because people can spam 10 ideas at [agent-scaffold] and see what sticks.
A lot of [redacted] research work is just implementation, babysitting, etc. Opus-4.5 in claude code is already pretty helpful with this, even if it mostly can’t do end to end.
The enthusiastic set have a clear idea of their jobs and how to get AI progress.
The main loop is:
You come up with a bunch of experiments.
You do the experiments.
You analyze the results and go to step 1.
The AIs can run the experiments, which at the limit makes the limiting factor coming up with more new experiments worth running, which Researcher E currently says are ‘nearly free’ but at minimum picking the highest quality ones seems like a key part of the value add of a researcher. We’re not there yet, it sounds like, and certainly I feel like I am constantly coming up with good ideas for experiments. But it is close.
The other part, once you’ve decided what to do or while you’re running the experiment, is ‘implementation, babysitting, etc,’ and operational excellence.
A similar story is told elsewhere, only with less confidence on magnitude of change and more openness to alternative sources of improvement.
F: The slope change is 50-80% Claude.
Change is especially notable for this manager, who can de-risk ideas with Claude enough to get someone else excited, and would not have been able to do that at all as a manager of 10 before.
G: Inflection point matches his vibes as to what’s been happening. Two biggest contributors here are Claude automation and hiring.
Claude lets researchers try many things quickly, most of which won’t work.
H: I think it’s true [that] we’re now trying a lot more experiments at the same time because of claudification.
J: Before Opus-4.5 -> Claude not useful for autonomous code / Opus-4.5 -> first time useful for autonomous code / mythos-preview + [agent-scaffold] -> first time useful for autonomous experiment
Then we get to the two skeptics.
K thinks it was hiring. If it was hiring, we should be able to identify that based on either finding particular superstars or looking at a graph of hiring, and from what I can tell we have neither.
L thinks it was [redacted] while thinking Opus 4.5 couldn’t write code, with inflection points at Mythos Preview and the mysterious Model 2. Even then this is talking price.
K: IMO [hiring is] probably the biggest reason …
L: Guesses 15-20% is due to AI. Don’t believe in self-reported productivity increase.
My favored interpretation: [redacted non-AI factor]
(1) opus45 couldn’t write code; (2) opus46 could write a little; (3) Mythos Preview a sharp inflection; (4) Model 2 is another really large improvement in how useful these things are.
definitely the case that we’re accelerated by claude .. Even though this rate of progress by end of year will be even faster … my attribution [mostly not to Claude] and uncertainty remain.
One impact is SWE acceleration, which is extremely substantial. We’re definitely benefiting from that differentially between 2025 and 2026.
This Infinite Series Likely Has a Finite Sum
If we follow the natural implications of the trend lines we see, each stage is sufficiently faster than the previous stage that you finish an unlimited number of steps in a finite time, plausibly as soon as next year, also known as a singularity. The report found no clear bottlenecks blocking that from happening in the next 12 months.
Whether or not this goes fully vertical depends on various factors, such as the change in the diminishing returns rate, compute-labor substitutability and the true pace of acceleration, and the shape of the future tech tree and implications of physics. It could go either way.
I don’t think full RSI and passing the relevant points of no return is a favorite to happen in 2027, but at this point I would no longer be surprised if that happened.
Ajeya Cotra (METR): If we take it at face value that AI is now nearly doubling the rate of AI capabilities progress and assume acceleration from AI is smooth, this naively implies that we would reach extreme superintelligence within a year.
We Cannot Say What We Cannot Say
Throughout, METR had to consider Anthropic’s ability to pull access for this or future projects, and its need to negotiate with and get cooperation from other labs. They are warning us that they may have had to make invisible compromises.
My presumption is that this pushes METR in the direction of underreporting potential acceleration and danger, rather than the other way around.
Study of Anthropic’s Compute Allocation Was Postponed
Initially this study was also going to analyze Anthropic’s compute allocation, but this was at least postponed, allegedly due to concerns about focus. Another obvious hypothesis is that Anthropic realized this information was rather sensitive.
We do not have a timeline for when we can expect this information.
As usual, dangling something like that and then taking it away upsets people. Knowing the compute allocation is important, because it plays directly into governance proposals that attempt to pace the frontier via compute allocation.
Remember that Anthropic’s September 17 post called compute ‘among the most verificable inputs to the AI R&D process’ and identified it as a potential key lever for future pacing, and that it has self-reported such allocations in the past.
I urge Anthropic to allow this part of the report to proceed. If they do not, it is hard not to interpret this as Anthropic not wanting to take compute allocation restrictions seriously as a policy mechanism.
Daniel Kokotajlo: @AnthropicAI: what the fuck? The compute allocation assessment is the thing we’ve been pushing for the last few months. It’s very important, because it is the foundation on which pacing verification depends. “to keep the focus on risk assessment” is a non-answer, a non-excuse. Do you want to pace or not?
Security Mindset
Anthropic has had remarkable success avoiding many kinds of leaks of information, including with one notably rare exception of its memos, despite some rather inadequate security practices like ‘put everything short of the model weights on Slack.’
The danger is that Anthropic is ending up in a maximally bad spot, where due to other security flaws we have the worst of both worlds. If the redactions do not protect information due to other security lapses, all they are doing is making us unable to properly analyze or trust the reports.
This is still far better than everyone else’s strategy of ‘don’t have METR do a report.’
Nathan Calvin: One irony about Anthropic’s emphasis on redactions here is that they (and the other frontier AI companies) have (by their own admission?) swiss cheese level security for ~everything other than model weights so it seems extremely likely that if China wanted to they could easily have the information redacted here (see e.g. Cathedral having access to Anthropic Slack).
In other words, I expect that the redactions are not enough to actually stop sensitive Anthropic IP from leaking in high stakes adverserial settings, but they are enough to stop good faith observers, skeptics, and governments from being as likely to believe Anthropic’s claim about how imminent the risks are.
That seems like a bad tradeoff! At least give us the descriptions of the redactions or the categories of information that informed the assessment (e.g. AI accelerating algorithmic improvements).
I am hesitant to criticize too hard here because of course the other companies didn’t do a partnership with METR at all, but on the other hand Anthropic tries to present itself as different from other companies to the point that responsible folks should be rooting for them to win even at the expense of worsening race dynamics, so it also seems appropriate to hold them to a higher standard.
It’s a Secret to Everyone
It is difficult to have a high quality discussion about a report and estimation whose major sources of justification have been redacted.
From METR’s perspective, the redactions are the cost of creating the report at all. From Anthropic’s perspective, this is the only way to avoid leaking key info, and that has to include some amount of glomarization around what info can be shared (e.g. you have to sometimes hide information that did not need to be hidden, to avoid the fact of redaction itself leaking too much info).
Did Anthropic strike the right balance? My guess is no, and they redacted too aggressively, but again it is great that the report exists at all and we take what we can get. It is tough.
Ajeya Cotra (METR): Our colleagues found that Anthropic’s capabilities progress has doubled, mainly due to AI. Ryan’s modeling from public info suggests the whole company’s R&D would be going 4-5x faster. Labor uplift would be higher still. This is a major update to very short timelines.
Seán Ó hÉigeartaigh: I have a great deal of trust in the quality and rigor of METR’s analysis, which means that their conclusion alarms me.
But as a member of a scientific body tasked with informing policymakers (and yes, they are interested in this topic), it is difficult to convey how frustrating it is to have to perform our role without being able to point to the actual evidence, or even confirm that we have seen and analysed it ourselves. I trust the METR team. But it’s not a good epistemic situation when the ask is for decision-makers to put their trust in my trust of the METR team.
Ryan Greenblatt is in an even tougher position, since his role relies on trust that he can keep confidential information confidential, and there have been accusations (which he vehemently denies, and which I believe are untrue) that he did leak info.
Ryan is working only from the public report in this first section, finding it to be a major update versus his expectations for 2026, although it is not clear that he had not already updated those expectations.
Ryan Greenblatt (METR): This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027.
I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind.
I haven’t seen an unredacted version of this report; this is purely about the public report. Because so much of the underlying evidence was redacted from the public report, I think conclusions about things other than the observed 2x faster rate of progress on benchmarks depend significantly on trust for the authors of the report. (E.g., conclusions about whether this speed up is from RSI depend on trust.) I do trust the authors of the report a lot to be careful and rigorous and generally do a good job. (Note that I also work at METR.)
Less extensive redactions would greatly reduce this need for trust.
What is Ryan’s new view? He concluded that is something he cannot share, and instead must do his best to only use information available to the public, including his own prior statements.
He does have to do some form of this. Glomarization is hard.
Since in my current role I may have access to other confidential information that could be relevant to timelines one way or another, I unfortunately cannot state my all-things-considered views on ASI timelines or the probability of a software intelligence explosion right now. Here, I’ve done some analysis of the implications using my previous public statements.
A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 4.5x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027.
Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger:
– Not all of AI progress is from better software (some is from scaling up compute in training runs). I’ve previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 3x faster software progress.
– Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred.
We can estimate the effects of diminishing returns by first determining how many additional years of software progress we’ve seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I’ve previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed and based on the report, it seems as though we’ve perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6).
This combines to 3 * 2^(~0.6) = ~4.5. (Relevant public modeling and numbers I’m using [here] and [here].
Alexander Berger (Quoting Ryan): “Since … I may have access to other confidential information that could be relevant to timelines one way or another, I unfortunately cannot state my all-things-considered views on ASI timelines or the probability of a software intelligence explosion right now”
roon (OpenAI): I think he should state his all things considered views on ASI timelines. it seems implausible that that leaks any bits of info on training IP
Nathan Calvin: yeah, lab employees already vaguely talk about short timelines from private info frequently. If Ryan isn’t doing it because he thinks the cos would be mad they should explicitly make clear that its fine for him to do it (and I agree its silly for them to be mad about that.)
Daniel Kokotajlo: Totally agree. I think METR is being persnickety to the point of self-sabotage here, going way beyond normal confidentiality norms.
In this case, I believe the vast majority of the unnecessary secrecy is due to Anthropic. That can include ‘Anthropic created ambiguity on necessary secrecy levels.’
Back in the 2010s, in order to avoid potentially sharing insights that advanced capabilities, MIRI (where Jessica Taylor once worked) had an extreme amount of secrecy, including within MIRI, in ways that in hindsight were highly counterproductive. Knowing what we know now, the warnings that AI was dangerous were themselves far more dangerous than any technical knowledge.
Reading something like the above raises the question of whether we are making that mistake once again. It certainly seems bizarre to hold back overall timeline conclusions. Would that really leak sensitive information?
Jessica Taylor: MIRI glomarizing & secrecy culture has spread to other AI safety orgs.
Kelsey Piper: entirely wrong diagnosis IMO: Greenblatt was alleged in the WSJ to have leaked secrets to the Information (almost certainly false) which would make a lot of people more cautious about saying things people might take as a leak
Jessica Taylor: This is a good point for Ryan specifically, and if it were just Ryan doing this, there would be less justification for this diagnosis, however, there are other signs here.
Bubbling Creek: I really do not think METR is the one pushing for this level of secrecy, it’s Anthropic.
Jessica Taylor: Yeah good point, this seems more Anthropic than METR. Anthropic is relatively secretive for an AI company.
The central claim of that risk report is that at the time there was low risk of directly automating R&D in key domains. Key reasoning for this was redacted from the public version. METR got to look at the private version.
METR’s conclusion is that the line probably was not crossed, but that confidence that we had not seen 2x acceleration by that point is not high and that the report understated the level of acceleration present.
The report basically sold us the ‘Mythos-class models were a step change and rates of acceleration are otherwise not so high’ model, in ways that were somewhat misleading, and in ways that do not seem to reflect Anthropic’s private information.
I am pretty not happy about this one. The threshold was 2, they reported the answer was ‘less than 2’ and it turned out the answer was 1.97, well within any reasonable margin of error, and also known as ‘approximately 2’ with METR’s error bounds being something like [1.3, 2.8]:
METR: We believe a reader of the public report would interpret the phrase “less than a factor of 2” to imply an estimate meaningfully below 2; thus it is surprising that the private risk report’s figures imply 1.97. A better description would seem to be “around 2.”
Claim 2 (Private Risk Report): Overall we expect that end-to-end growth in [redacted] is slightly less than twice as fast as it would be without AI assistance.
This creates a wording ratchet going forward. If I see ‘less than a factor of [X]’ I will assume that simply means that the point estimate is some amount less than [X], but that it could be very close to [X], and [X] could easily be within the margin of error. Going forward, if they want to communicate ‘no seriously this is not close to [X]’ they will have to say that it is not close.
Again, the comparison here seems stark:
Redacting information is fine, but the public version is framed to imply little progress, where the private verison is framed to imply expectations of more progress. The whole point of the risk report is to share exactly this intuition. Not cool.
A Word From Our Cofounder
Here is Anthropic cofounder Sam McCandlish’s take on the report. He agrees with many of its conclusions, and admits that a lot of information was redacted to avoid giving competitors a roadmap to go faster.
Sam McCandlish (Cofounder, Anthropic): I’m Sam, one of Anthropic’s co-founders, and I lead much of our safety work. I’m going to have more to share on that work over the coming months.
I thought I’d start by sharing a few thoughts on the latest report from @METR_Evals .
The speed of progress in model capabilities is one of the most important questions in AI safety, and METR’s work here provides the public with an independent, external view into how this is happening inside Anthropic. I’ll talk more about some of the details below, but first some context for those new to this work.
How quickly AI progress is accelerating has many important safety implications, since it determines how quickly the risks from advanced models increase. But, because most research happens within the frontier labs, it’s difficult for external observers and policy makers to get a strong sense of what that rate is. We kicked off the process with METR that led to this report earlier in the year, in anticipation of progress continuing to accelerate.
For this report, we gave METR access to a range of sensitive internal data, including benchmark scores for unreleased models, estimates of how models are progressing, and interviews with our researchers. They had complete independence to form their own conclusions.
Many of their conclusions match my own. AI is speeding up our research, but it’s hard to say exactly how much. While they conclude AI can’t yet make the most important contributions to model training on its own, they think the pace of development could, by default, accelerate further.
This work began well before and is separate from the embedded third-party evaluators that are part of our commitment to pacing the frontier, but it’s a good illustration of why we think evaluators are a good idea. The more access reviewers have, the clearer a picture they can form of the situation and the better-positioned they are to highlight situations like this one.
We redacted a lot, and I accept that this makes it harder for others to check METR’s work. But we determined that sharing exactly what is driving the speedup isn’t the right thing to do, both because it’s commercially sensitive and because it risks handing out a roadmap for others to go faster. That would make the risks METR is worried about worse. That’s a tradeoff. I think we made the right call, though we’re continuing to assess the pros and cons here.
In the coming weeks, we plan to share a report with our own analysis of the speed of AI progress within Anthropic, including where we stand against our Responsible Scaling Policy, plus new steps we’re taking to strengthen our safety and security practices as AI keeps improving quickly.
Daniel Kokotajlo: It is pretty rich of you to cite “others could go faster” as a reason to redact so much important information when you yourself do not appear to be slowing down much at all. Nor are you throwing yourself behind efforts to pace the frontier with nearly the gusto that you would if it was your top priority. Not to mention “commercial sensitivity” you yourself have said human extinction is a plausible outcome why are you even talking about commercial factors (well, I guess honesty is good, and since it IS apparently a factor in your decision…)
To all my friends at Anthropic: Your company appears to be recklessly gambling with our lives. You fancy yourselves the more responsible alternative to OpenAI (as they fancied themselves compared to Google) but judging by actions rather than words the similarities between the three of you seem stronger than the differences.
Some of the redactions can be justified as avoiding giving competitors a roadmap, but I do not see how the full extent of the redactions can be justified that way. Time estimates for future acceleration by both METR in general and Ryan Greenblatt in particular are being redacted. How would those numbers provide a roadmap?
More importantly, you believe all this is accurate, and you intend to keep going?
We also still have data points like this:
Theo Bearman: ANTHROPIC’S RESPONSIBLE SCALING OFFICER: “In the coming weeks, we plan to share a report with our own analysis of the speed of AI progress within Anthropic, including where we stand against our Responsible Scaling Policy, plus new steps we’re taking to strengthen our safety and security practices as AI keeps improving quickly.”
Nathan Calvin: If Anthropic wants me to think these steps are real/credible, they should put it in their SB 53/RAISE Act Frontier Compliance Framework.
State law gave Anthropic and OpenAI a mechanism for making credible statements about their behavior, and they mostly choose not to use it on the risks they are most worried about (neither company includes anything in their compliance frameworks about internal deployment RSI risk, though Anthropic says maybe at some point in the future they will. Now seems like as good a time as any???!!!???)
ANTHROPIC: “AI is extremely perilous and should be carefully regulated”
CALIFORNIA GOV: “Ok, we will regulate you.”
ANTHROPIC: “No! Good regulation is really hard to get right. We know so little about what to do. Instead we should let companies carefully iterate over time.”
CALIFORNIA GOV: “Oh ok. How about we do a thing where you get to write what the rules are but we will hold you to them? Then you can make your own commitments as you think is best but they will be true commitments!”
ANTHROPIC: “Nah, we’d rather make fake commitments.”
CALIFORNIA GOV:
Nathan Calvin: Worth reading. Dario says that one of the ways that Anthropic can justify remaining at the frontier (with the associated risks) is because the company is effective at pushing for strong AI regulation. I think Anthropic’s track record here is more complex than he implies.
Max Winga: The lawyers here really held @DarioAmodei’s feet to the fire and it’s interesting to see how he squirmed under oath about AI extinction risk. Under penalty of perjury he’s reaffirmed his belief that AI is the greatest threat humanity has ever faced.
Peter Wildeford: Anthropic will really be like “you believe in slowing down? that pales in effectiveness to my strategy, racing so fast that we have credibility to push for strong regulation” and then not push for strong regulation
Daniel Tan overstates a bit here, but I agree that Anthropic’s actions that supposedly ‘race to the top’ often do not look so different from what they would do if they were instead racing to superintelligence.
People Update Their Timelines and Expectations
Andrew Curran: Judging by the posts on my timeline, today is the big update on both RSI and ASI for many people in this community. I went through this in November 2022, so I know exactly how you feel. I know nobody drinks anymore, but I would suggest having a drink.
Trevor Levin: This report and reactions from people I trust (like Jake) are making me physically ill. As far as I can tell, Anthropic’s central plan is to continue this acceleration of AI capabilities until we’re getting decades of AI capabilities growth in a year. Given our very limited ability to understand or control AI systems, this would be extremely reckless and completely unacceptable, and it should be the single top priority of civil society and the US government to stop them and other companies from doing this.
Dane Sherburn (former OpenAI): I feel sick after reading today’s METR report. I’m worried for my friends and family. I don’t know what evidence people will need to see before they start taking these risks seriously.
If you’re undecided about helping, please act; there’s many opportunities to help.
tldr:
(1) capabilities speeding up significantly
(2) speedup is not driven by growth in compute
(metr estimate overall speedup of approx. 2x on well-specified tasks by Oct 26 vs Mar 24 to Oct 25)
there are a bunch of qualifications/caveats worth reading in the thread itself.
Jake Mendel (Coefficient Giving): I’ve always had substantially longer timelines than my friends at Anthropic — I’ve been concerned that they’re in an echo chamber and believing the hype about their own models too much, and that they’re underestimating bottlenecks and unknown unknowns. And there have definitely been specific occasions when they have overstated how much uplift they were getting from their models. Nonetheless, if METR is correct here (and I think they are among the most trustworthy sources on this topic), then my Anthropic friends have been much more right about timelines than me.
This report is also some evidence that a software-only singularity is plausible. I now think that we are more than 50% likely to have full blown superintelligence in the next 18 months, absent a slowdown. I need to call my family.
I think the US government should make them stop.
If you already believed the labs that have been screaming similar things for months, and do not trust information gated by redactions, perhaps you did not update so much.
Dylan Bowman (Apollo Research): I have finished this report and learned approximately nothing (because of redactions) and updated a tiny tiny bit towards longer timelines.
Tiny tiny bit towards longer timelines because of this sentence: “Anthropic’s internal benchmarks are increasingly saturated”. This sentence is about AECI evals but maybe is also true of their production evals (low conf, hence “tiny tiny”)
I also expected more than 2x uplift. I think Apollo’s internal uplift is ~1.5-2x.
The report also glosses over the biggest plausible alternative to researcher uplift driving the acceleration, which is data uplift (like RL environment pipelines and automated quality control)
Peter Wildeford attributes a lot of the gains to Anthropic catching up on RL if you look only at public information, and remains skeptical that there is a full ~2x speedup happening, but trusts METR’s analysis of private info.
Peter Wildeford: The RL shift could plausibly explain much of the benchmark acceleration. RL’s share grew fastest before METR’s break point (4% to 24% from 2024Q1 to 2025Q3), but if you add an additional assumption for a 2-3 quarter lag from compute to released models, that lines up with a ~1.4-2.2x speedup, which is what we see. Also, I think catch-up on RL probably explains some of why Anthropic is accelerating more than other labs.
Before Oct 2025, Anthropic’s pace was below the overall frontier’s. OpenAI went RL-heavy first (o1, Sep 2024) and Anthropic followed later (Claude 3.7 Sonnet, Feb 2025).
So part of Anthropic’s 1.9x could be a late RL ramp from a low baseline. METR flags this in the chart subtitle and I find it plausible. But the all-developer frontier still accelerated ~1.4x, so catch-up doesn’t explain all of it.
Additionally, this “RL is doing the work” makes a testable prediction, however… since RL share is plateauing (50% to 53% to 55% over the last three quarters), this means benchmark growth should slow by mid-2027. If it doesn’t, something else is driving it. Worth watching. To be clear, 1.4x faster benchmark progress is very different from engineers being 1.4x faster. Per METR’s report, engineers can be sped up ~12x while overall progress only speeds up ~1.4x, because of compute constraints and Amdahl’s law.
I should clarify that I personally believe AI is responsible for like at least 1.4x acceleration, though it sounds like this is more based on METR having private information and me trusting METR than us having clear independent public information.
To be clearer, METR says that they have “received convincing evidence that the acceleration was not due to an acceleration in the scale of training compute” but that “further details on this evidence were redacted” and I believe METR’s analysis even though it requires trusting private information and cannot be confirmed with public information.
You Are Not Ready
Even if we don’t worry about alignment, the bare minimum requirement for attempting to build a superintelligence is not to leak all your algorithmic secrets or model weights. The AI industry is not ready to meet even that bar.
We are even less prepared than that to build an aligned or inherently safe superintelligence, especially if we try to build it in 2027 or 2028.
Ajeya Cotra (METR): Very strong security, both for model weights and algorithmic secrets themselves, is a minimum prerequisite for undergoing an intelligence explosion next year without running very large risks of killing billions of people.
Ryan Greenblatt: I recommend @ajeya_cotra ‘s post about implications of the AI R&D report. Ajeya argues:
– A 2x speed-up is dramatic.
– 2x faster progress implies a much larger effective “time warp bubble” speed for staff.
– This could compound extremely fast.
– We aren’t prepared.
If (If Anyone Builds It, Everyone Dies) Then Don’t Let Anyone Build It
If they do believe the full report, they should act accordingly.
Kelsey Piper: Of course, the report might be wrong, and Anthropic’s employees might be deluding themselves about their progress. So I want Congress to read the unredacted report. I want lawmakers to hire brilliant RSI believers and brilliant RSI skeptics, and have those experts pore over all the evidence and tell us whether these companies are experiencing some kind of mass delusion or actually reporting on an imminent disaster for humankind.
And if it’s the latter, I want Congress to ban RSI. Make the companies demonstrate where they’re spending their money, and don’t let them spend it on this.
It’s ridiculous for our elected representatives to be stuck reading the tea leaves from heavily redacted reports about a technology that could determine humanity’s future.
This is an urgent matter. Again, if we let them build it, there is a double digit chance that by the end of 2027 it will be too late to stop it from being built, where ‘it’ is recursively-self improving AI that will become true superintelligence.
Neil Chilson: I see little innovation harm from an RSI ban, for my definition of RSI. Little innovation harm because RSI, properly defined, is a practice too stupid to need banning. But I’m worried Alex means something different than me.
Neil Chilson: Something like “fully relinquishing AI R&D and deployment to AI, with no ability to monitor and stop the process.”
Which by the Sixth Law of Human Stupidity means we will definitely do it at the first opportunity if allowed to do so, also the AI companies are screaming that they are going to do it at the first opportunity if allowed to do so.
Neil and I agree it would be deeply stupid, but for very different reasons.
How Can We Legally Define RSI?
You cannot ban that which you cannot define.
Banning an overly narrow definition, such as Neil’s above, won’t make us not dead.
If you so narrowly define and ban RSI as meaning only a 100% relinquishing with zero ability to monitor, what you get is a 90% then 99% then 99.9% automated process with some illusion that you can monitor and stop the process. Then you find out you cannot stop the process and that your human is not in any of the loops that matter. Then, probably, you die.
Legal text has to be robust to various workarounds, including in very stupid ways, as in Critch might well have a point here no matter how stupid it sounds when you say it:
Andrew Critch: This ban [on RSI] needs to be expanded to “Recursively Self-Modifying”, to avoid unfortunate debates about whether killing or driving humans to extinction counts as “Improvement”. (Answer: it doesn’t, but many people disturbingly claim it does.)
The problem with most other known definitions is they don’t work as legal text.
Alex Bores: Every AI company is trying to fully automate AI research; to allow the AIs to build the next AI.
This is called Recursive Self Improvement (RSI).
… The exact mechanism can be debated. The ban doesn’t have to be permanent either (see below for both). This is not the only thing we have to do in regards to AI, far from it! But it is urgent. Congress must ban RSI.
Woody Lee (responding to Chilson from above): [Alex Bores] defines what he means [by RSI] in the tweet.
Neil Chilson: That’s not a definition that works in legislation.
Dave Kasten: I often disagree with Neil, so I want to flag here that I STRONGLY agree with him: we desperately need a definition of RSI that has gone through a bunch of rounds of lawyers kicking its tires.
You’re a law student, want to get cited for forever? Write this as your journal note.
Saul Munn: wdyt of @testingham et al’s definition of ‘self-sustaining acceleration’?
Dave Kasten: Seems interesting, but not leg text by a long shot
It is probably going to be impossible to come up with a definition that will stop RSI at an early enough stage that such a law stays relevant to what happens next, and also avoids hitting anything that is not RSI. The blast radius will not be zero.
The good news is that the blast radius can be things that RSI skeptics mostly say will not happen either way. If they are in good faith, they should be happy to draw lines in the sand that they do not expect anyone to be able to cross.
This is similar to the talk of ‘pacing the frontier.’ Those calling to Pace the Frontier want to go at High Speed rather than Ludicrous Speed. Those who are against it want to ensure we go at Medium Speed rather than Low Speed, would love High Speed, and are scared of pausing entirely. It is easy to see how a deal there could be win-win.
Now more than ever, we need to prepare for scenarios where the critical events happen in 2027 or 2028. It might not play out that way, but it very well might.
METR came out with an extensive report on how much AI is accelerating AI R&D at Anthropic, and thus how close we might be to recursive self-improvement (RSI) and how far that might go.
This is one of the most important questions in the world.
Their answer: Things are already accelerating on the order of a 2x speedup.
This is several months ahead of the AI 2027 timeline. A naive extrapolation says we may be less than a year away from full RSI. There are no clear bottlenecks in the way.
This report updated many towards shorter timelines.
This is consistent with what OpenAI and Anthropic themselves have been saying, and with extrapolation from the progress we are observing. We are running out of time.
That doesn’t have to be the way this plays out. We could stop the labs. The labs could choose to stop. Or the estimates could be wrong, or some bottlenecks could be hit, or the results could be less impactful than we expect. Indeed do many things come to pass. And if it does happen this quickly, one would expect it to go quite poorly, and likely to cause everyone to die, but perhaps it goes well.
It doesn’t look good.
The central conclusions are:
I do not find either of these results surprising. I made minor updates towards faster outcomes and fewer bottlenecks, but not major ones. I had already made major ones.
The situation looks rather grim. But this is not a surprise.
The information from inside the labs makes you think the situation is grim and timelines are short, but those with that information were already shouting from the rooftops that their secret information meant the situation is grim and timelines were short. Why did you not think this was the standard reaction to their information?
I did find it surprising that so many found this surprising, including METR. Their team did not expect substantial acceleration. Ryan Greenblatt expected only 1.15x-1.3x speedup this year. Six weeks ago in the Opus 5.5 system card the estimate was at least a 1.5x speedup.
If you are making major updates in response to this report, that is probably good and wise, but indicates you had previously made a mistake in your estimations.
If you only look at public models, you can make a case that there is a step change at Mythos but that otherwise the rate of progress has not changed. I was always skeptical of that, but it was a valid claim.
Add in the orange data points from internal models and you cannot tell that story. When we include the recent additional internal releases, this takes the estimate from the ~1.7x in the Haiku system card up to a new estimate of ~1.9x.
You could also have gotten to a similar result by extrapolating the automation graph:
The information METR is relying on most is largely redacted. The public report centers on AECI and other estimates. METR is implying that such arguments are merely the legible case they are allowed to make, and that there is a stronger case.
Kudos to Anthropic for allowing this investigation.
This is all highly sensitive stuff, so there are important redactions involved. I would have liked to see less of this, but if this is what it takes to get the report then so be it.
I would also suggest to Congress that they should want to see the full report along with everything else Anthropic and Dario Amodei know and really think about this, and to do that they should follow Kelsey Piper’s suggestion and subpoena Anthropic.
Table of Contents
Top Conclusions
This is how they describe their subjective conclusions.
Here are their top empirical findings:
Data Access and Redactions
The goal of the report was to measure acceleration of Anthropic’s internal growth rate, how much of that acceleration is due to AI R&D and whether there exist any important bottlenecks to further acceleration. This requires a lot of sensitive data, and they got most of what they requested.
They could access:
In exchange, Anthropic could redact anything in the report. They did so, aggressively.
METR’s only recourse is that it can then share its level of access, that things had been redacted, and whether findings depend on redacted information. Which they do. The majority of the most informative evidence is redacted.
Some of these redactions seem clearly appropriate. Others seem questionable.
They also could not show us any use of METR’s core accounting framework. We know the framework looks like this, but that’s all we get:
The Pace of Progress
This is one of the more annoying redactions. What are these non-AI factors? Candidates include compute scaling, hiring, data improvements, algorithmic discoveries and resolving other bottlenecks. Compute has been ruled out. But we don’t know how they view the other factors.
The non-AI acceleration continues, supplemented by the AI-driven one:
Thus they illustrate how to think about all three numbers: The historic trend, the counterfactual no-AI trend and the observed trend that includes use of AI.
1, 2, 3, 4
The Risk Report introduced Model 1 and Model 2. Now we meet 3 and 4, and also 5.
Timing Is Everything
If this is consistent, it seems like the clear correct way:
OpenAI used to hold onto models for a longer time. Then it got shorter. Whereas after Mythos, various forces are now holding models back for longer, including the desire to keep an advantage in R&D.
There are different reasons one might want to measure ‘leads’ or rates of progress. If you are measuring for commercial purposes, you care mostly about practical questions of what a public customer can do when. If you are measuring to predict RSI, you care about a very different measurement, and what you care about are internal capabilities, and ability to use them to make progress at the frontier.
By that measure, things are picking up faster, and also the gaps are wider. If you believe in what this report is telling us, these are the measurements that count.
What Could Possibly Have Gone Wrong?
There are a number of ways this could be wrong, or be less than ideally accurate.
The biggest issue is limited access to data. Not all data requests were accepted and there are many documents and other information sources that were not available.
METR might have missed further acceleration, or AECI could underestimate capability growth, or automation could be closer.
On the other hand, perhaps there are non-AI factors that were missed that explain past acceleration, or historical growth was unusually low creating the illusion of fundamental acceleration. Or perhaps acceleration could only work on a narrow set of tasks.
I notice that the upside risks and downside risks of the estimate are asymmetrical. We have three in each direction but they don’t match (Opus 5.5 says some of these do cancel out, but I don’t think they fully do, and others clearly don’t):
Curves on such things do not have to be symmetrical, so long as the average results of all of them are incorporated. This doesn’t have to be an error. I do still wonder.
AECI
The main metric METR uses for this report is the Anthropic fork of the Epoch Capabilities Index, AECI.
AECI is a measure of practical capability. Growth in AECI is multiplicative of odds of success on a task rather than linear, so linear progress on AECI means more than you might think:
METR focused a lot on the question of whether the metrics are biased or consistent.
The measurements are also robust to choice of included benchmarks, and to which model snapshots are included, and to regularization.
If non-Anthropic models are included, then the estimate of acceleration goes down to 1.4x.
METR suggests as another potential justification for lower acceleration rates that Anthropic’s models underperformed on benchmarks in the earlier period relative to their true capabilities, which matches my understanding.
I would also suggest that if Anthropic was previously behind, then being behind makes progress easier via fast following, even if you cannot distill.
There are some potential issues with AECI:
If AECI is going to serve as a measure of the capabilities of models, the first part of this must be true:
Any given metric then has to worry about S-curves and saturation points.
Depending on what type of curve you think this is, you get different measurements of the rate of acceleration. Unfortunately basically all the curve fits are similarly good at Out-of-sample R2. It makes sense to default to standard logistic, but this illustrates that in theory you can explain or justify a wide range of measurements.
METR says they have no reason to prefer one of these over others. I would say they make distinct claims over the distribution of tasks. So if we want to know which is best, we should look at the tasks.
The Public Information Alone Could Suggest Slower Speedup
In particular, if you are relying only on the AECI, one can selectively edit out the Mythos jump or consider part of this catching up on relative compute allocation.
There are two responses to this. One is that there is private redacted information that is factoring into the reports.
The other response is that such modifications are selective. Yes, if you take away whatever the biggest discontinuous improvement is over a period, you will see slower returns. Yes, Anthropic’s position in compute consumption relative to others has shifted. But once we go down such roads, why do we think similar shifts will not happen again?
There is always temptation to extrapolate a future without such jumps. You don’t get to do that. The baseline scenario contains outliers and jumps.
We can only reject the Mythos upshift if we have strong reason to believe we won’t be able to do any similar things again. I don’t see why that would be true.
Well, That Escalated Quickly
It is poised to soon escalate far more quickly. This fact is overdetermined. There are even ‘signs of life’ of areas of potentially full automation. The details are less clear.
The estimates here are a lot more than 2x in uplift for the employees, offset by slowness elsewhere, balancing out to 2x. Coding is dramatically more productive now.
They call this ‘broadly support AI doubling the growth rate’ but if we are going purely by the Claude sentiment analysis then the median answer was ‘borderline/unclear’:
The massive personal speedups are necessary to maintain the overall slower pace, so there is no inconsistency here. That’s the frame for looking at the researcher quotes.
They roughly rank the researchers from the most gung-ho (A) to the least, and offer quotes. We start with the two ‘Yes’ votes.
B is talking about personally being more productive, where yes I presume 3x is a floor. Then there is A who is talking explicitly about years of progress.
The consensus is that uplift was not that big before Opus 4.5, then Mythos Preview sent it into overdrive, and other improvements from public models were incremental. There were some voices for Model 2 or Model 4 as additional large jumps.
C, D and E are ‘lean yes’:
The enthusiastic set have a clear idea of their jobs and how to get AI progress.
The main loop is:
The AIs can run the experiments, which at the limit makes the limiting factor coming up with more new experiments worth running, which Researcher E currently says are ‘nearly free’ but at minimum picking the highest quality ones seems like a key part of the value add of a researcher. We’re not there yet, it sounds like, and certainly I feel like I am constantly coming up with good ideas for experiments. But it is close.
The other part, once you’ve decided what to do or while you’re running the experiment, is ‘implementation, babysitting, etc,’ and operational excellence.
A similar story is told elsewhere, only with less confidence on magnitude of change and more openness to alternative sources of improvement.
Then we get to the two skeptics.
K thinks it was hiring. If it was hiring, we should be able to identify that based on either finding particular superstars or looking at a graph of hiring, and from what I can tell we have neither.
L thinks it was [redacted] while thinking Opus 4.5 couldn’t write code, with inflection points at Mythos Preview and the mysterious Model 2. Even then this is talking price.
This Infinite Series Likely Has a Finite Sum
If we follow the natural implications of the trend lines we see, each stage is sufficiently faster than the previous stage that you finish an unlimited number of steps in a finite time, plausibly as soon as next year, also known as a singularity. The report found no clear bottlenecks blocking that from happening in the next 12 months.
Whether or not this goes fully vertical depends on various factors, such as the change in the diminishing returns rate, compute-labor substitutability and the true pace of acceleration, and the shape of the future tech tree and implications of physics. It could go either way.
I don’t think full RSI and passing the relevant points of no return is a favorite to happen in 2027, but at this point I would no longer be surprised if that happened.
We Cannot Say What We Cannot Say
Throughout, METR had to consider Anthropic’s ability to pull access for this or future projects, and its need to negotiate with and get cooperation from other labs. They are warning us that they may have had to make invisible compromises.
My presumption is that this pushes METR in the direction of underreporting potential acceleration and danger, rather than the other way around.
Study of Anthropic’s Compute Allocation Was Postponed
Initially this study was also going to analyze Anthropic’s compute allocation, but this was at least postponed, allegedly due to concerns about focus. Another obvious hypothesis is that Anthropic realized this information was rather sensitive.
We do not have a timeline for when we can expect this information.
As usual, dangling something like that and then taking it away upsets people. Knowing the compute allocation is important, because it plays directly into governance proposals that attempt to pace the frontier via compute allocation.
Remember that Anthropic’s September 17 post called compute ‘among the most verificable inputs to the AI R&D process’ and identified it as a potential key lever for future pacing, and that it has self-reported such allocations in the past.
I urge Anthropic to allow this part of the report to proceed. If they do not, it is hard not to interpret this as Anthropic not wanting to take compute allocation restrictions seriously as a policy mechanism.
Security Mindset
Anthropic has had remarkable success avoiding many kinds of leaks of information, including with one notably rare exception of its memos, despite some rather inadequate security practices like ‘put everything short of the model weights on Slack.’
The danger is that Anthropic is ending up in a maximally bad spot, where due to other security flaws we have the worst of both worlds. If the redactions do not protect information due to other security lapses, all they are doing is making us unable to properly analyze or trust the reports.
This is still far better than everyone else’s strategy of ‘don’t have METR do a report.’
It’s a Secret to Everyone
It is difficult to have a high quality discussion about a report and estimation whose major sources of justification have been redacted.
From METR’s perspective, the redactions are the cost of creating the report at all. From Anthropic’s perspective, this is the only way to avoid leaking key info, and that has to include some amount of glomarization around what info can be shared (e.g. you have to sometimes hide information that did not need to be hidden, to avoid the fact of redaction itself leaking too much info).
Did Anthropic strike the right balance? My guess is no, and they redacted too aggressively, but again it is great that the report exists at all and we take what we can get. It is tough.
Ryan Greenblatt is in an even tougher position, since his role relies on trust that he can keep confidential information confidential, and there have been accusations (which he vehemently denies, and which I believe are untrue) that he did leak info.
Ryan is working only from the public report in this first section, finding it to be a major update versus his expectations for 2026, although it is not clear that he had not already updated those expectations.
What is Ryan’s new view? He concluded that is something he cannot share, and instead must do his best to only use information available to the public, including his own prior statements.
He does have to do some form of this. Glomarization is hard.
In this case, I believe the vast majority of the unnecessary secrecy is due to Anthropic. That can include ‘Anthropic created ambiguity on necessary secrecy levels.’
Back in the 2010s, in order to avoid potentially sharing insights that advanced capabilities, MIRI (where Jessica Taylor once worked) had an extreme amount of secrecy, including within MIRI, in ways that in hindsight were highly counterproductive. Knowing what we know now, the warnings that AI was dangerous were themselves far more dangerous than any technical knowledge.
Reading something like the above raises the question of whether we are making that mistake once again. It certainly seems bizarre to hold back overall timeline conclusions. Would that really leak sensitive information?
Review of the Risk Report from August 2026
Along with the main report METR also reviewed the August 2026 Anthropic Risk Report, which in turn covered events up to July 15. That seems so quaint now.
The central claim of that risk report is that at the time there was low risk of directly automating R&D in key domains. Key reasoning for this was redacted from the public version. METR got to look at the private version.
METR’s conclusion is that the line probably was not crossed, but that confidence that we had not seen 2x acceleration by that point is not high and that the report understated the level of acceleration present.
The report basically sold us the ‘Mythos-class models were a step change and rates of acceleration are otherwise not so high’ model, in ways that were somewhat misleading, and in ways that do not seem to reflect Anthropic’s private information.
I am pretty not happy about this one. The threshold was 2, they reported the answer was ‘less than 2’ and it turned out the answer was 1.97, well within any reasonable margin of error, and also known as ‘approximately 2’ with METR’s error bounds being something like [1.3, 2.8]:
This creates a wording ratchet going forward. If I see ‘less than a factor of [X]’ I will assume that simply means that the point estimate is some amount less than [X], but that it could be very close to [X], and [X] could easily be within the margin of error. Going forward, if they want to communicate ‘no seriously this is not close to [X]’ they will have to say that it is not close.
Again, the comparison here seems stark:
Redacting information is fine, but the public version is framed to imply little progress, where the private verison is framed to imply expectations of more progress. The whole point of the risk report is to share exactly this intuition. Not cool.
A Word From Our Cofounder
Here is Anthropic cofounder Sam McCandlish’s take on the report. He agrees with many of its conclusions, and admits that a lot of information was redacted to avoid giving competitors a roadmap to go faster.
Some of the redactions can be justified as avoiding giving competitors a roadmap, but I do not see how the full extent of the redactions can be justified that way. Time estimates for future acceleration by both METR in general and Ryan Greenblatt in particular are being redacted. How would those numbers provide a roadmap?
More importantly, you believe all this is accurate, and you intend to keep going?
We also still have data points like this:
Daniel Tan overstates a bit here, but I agree that Anthropic’s actions that supposedly ‘race to the top’ often do not look so different from what they would do if they were instead racing to superintelligence.
People Update Their Timelines and Expectations
If you already believed the labs that have been screaming similar things for months, and do not trust information gated by redactions, perhaps you did not update so much.
Peter Wildeford attributes a lot of the gains to Anthropic catching up on RL if you look only at public information, and remains skeptical that there is a full ~2x speedup happening, but trusts METR’s analysis of private info.
You Are Not Ready
Even if we don’t worry about alignment, the bare minimum requirement for attempting to build a superintelligence is not to leak all your algorithmic secrets or model weights. The AI industry is not ready to meet even that bar.
We are even less prepared than that to build an aligned or inherently safe superintelligence, especially if we try to build it in 2027 or 2028.
If (If Anyone Builds It, Everyone Dies) Then Don’t Let Anyone Build It
At minimum, we need to know what they fully think, and that can start by having Congress read the unredacted report and decide if they believe the full report.
If they do believe the full report, they should act accordingly.
This is an urgent matter. Again, if we let them build it, there is a double digit chance that by the end of 2027 it will be too late to stop it from being built, where ‘it’ is recursively-self improving AI that will become true superintelligence.
Alex Bores used this report to call for a ban on RSI.
The ‘don’t let anyone build it’ faction certainly knows how to respond to this situation.
Actually it’s more like this:
Even Neil Chilson sees little innovation harm in ‘banning RSI’ because, by his definition, no one would be so stupid as to do it and not keep humans ‘in the loop.’
Which by the Sixth Law of Human Stupidity means we will definitely do it at the first opportunity if allowed to do so, also the AI companies are screaming that they are going to do it at the first opportunity if allowed to do so.
Neil and I agree it would be deeply stupid, but for very different reasons.
How Can We Legally Define RSI?
You cannot ban that which you cannot define.
Banning an overly narrow definition, such as Neil’s above, won’t make us not dead.
If you so narrowly define and ban RSI as meaning only a 100% relinquishing with zero ability to monitor, what you get is a 90% then 99% then 99.9% automated process with some illusion that you can monitor and stop the process. Then you find out you cannot stop the process and that your human is not in any of the loops that matter. Then, probably, you die.
Legal text has to be robust to various workarounds, including in very stupid ways, as in Critch might well have a point here no matter how stupid it sounds when you say it:
The problem with most other known definitions is they don’t work as legal text.
It is probably going to be impossible to come up with a definition that will stop RSI at an early enough stage that such a law stays relevant to what happens next, and also avoids hitting anything that is not RSI. The blast radius will not be zero.
The good news is that the blast radius can be things that RSI skeptics mostly say will not happen either way. If they are in good faith, they should be happy to draw lines in the sand that they do not expect anyone to be able to cross.
This is similar to the talk of ‘pacing the frontier.’ Those calling to Pace the Frontier want to go at High Speed rather than Ludicrous Speed. Those who are against it want to ensure we go at Medium Speed rather than Low Speed, would love High Speed, and are scared of pausing entirely. It is easy to see how a deal there could be win-win.
Now more than ever, we need to prepare for scenarios where the critical events happen in 2027 or 2028. It might not play out that way, but it very well might.