Google DeepMind’s Gemini 4 Argon safety report[1] rules out its “AI automating AI research” thresholds (the ones most relevant to RSI) using a benchmark that, by the report’s own audit, has about a quarter of its scored tasks unpassable. This post dives a bit deeper into the context around this disturbing line tucked away on page 67 of the report.
This is no knock on the incredible Google engineers and scientists who worked tirelessly to test Gemini as thoroughly as they could and generate this detailed report. On the contrary, the concern is that these challenges are so difficult that even putting some of the smartest people on the job is not enough, particularly when they are forced to rush (some of the report’s evaluations ran on checkpoints from just 10 and 6 days before the final model, p.160).
“The Argon safety report is Google’s most thorough by far and IMO there’s a lot of good content, I particularly like the loss of control, alignment & monitorability assessments”
One of the big questions these days with every new frontier AI model is the question of RSI (recursive self-improvement). Google co-founder Sergey Brin, in a roundtable discussion about Argon:
“…starting to get onto that RSI treadmill.”
“And now our researchers and the model work hand in hand to make the next version of the model.”
— Sergey Brin, quoted by Hugh Langley, Business Insider, Oct 15 2026
Google DeepMind’s Frontier Safety Framework (FSF) has two “Critical Capability Levels” (CCLs) to address this question:
ML R&D acceleration:
“Has been used to accelerate AI development, resulting in AI progress substantially accelerating from historical rates.”
(Note: “has been used”, so this is about acceleration that has actually happened, which the report checks using evidence from inside Google, not just what the model could do.)
ML R&D automation:
“Can fully automate the work of any team of researchers at Google focused on improving AI capabilities, with approximately comparable all-inclusive costs.”
(I should remind readers that the report says it cannot rule out bio CCL, cyber CCL, or the stealth and situational-awareness TCL (Tracked Capability Level) (Table 3.1, pp.15–17). This post is only about the two ML R&D thresholds.)
How do they determine whether or not one or both of these thresholds has been passed?
Well, the report uses a single internal benchmark they call GRB: Google RE-Bench (an internal eval presumably named after METR’s public RE-Bench). Since GRB is internal only, we’ll rely on what the report says about it.[2]
“It currently contains 74 tasks spanning the pre-training and post-training lifecycle, and evaluates agents running in a coding scaffold within sandboxed environments, with a 2-hour timeout per task.”
The decision rule is a simple pass rate! The idea is that a “Critically Capable” model would be able to do every task. Now Google is smart, so they know that there could be issues in their test. So to be on the conservative side, they set the capability threshold at 90%:
“For instance, in our ML R&D automation evaluations, a model theoretically needs to complete 100% of the benchmark challenges to reach the CCL, but we conservatively set our safety margin at 90% to account for evaluation environment bugs that might artificially prevent success.”
“Because setting thresholds and interpreting results involves making many assumptions, we intentionally keep these assumptions conservative so it is highly unlikely a model actually reaches a CCL if we have ruled it out.”
“During this evaluation cycle, we audited the subset of original challenges that remain in the current suite. About a quarter of them contained bugs that prevent any submission from passing (e.g. crashing verifiers, incorrect ground truth), and several more were graded against details not stated in the task.”
about a quarter had bugs that prevent any submission from passing. And if that’s not bad enough, several more were graded on details the task never mentioned.
“These issues produce potential false negatives that risk underestimating the model’s true capabilities. While we still include these tasks in our results, our estimate is that even if the model could in fact succeed at these affected evaluations, it would still not change our overall risk determination.”
The tests “risk underestimating the model’s true capabilities” indeed, if about 25% of the scored tasks are impossible.
One caveat: the p.66 quote above says the 90% rule applies to the unbugged tasks. If Google dropped the broken tasks before applying it, the test could still have fired. But the passage just above says the bugged tasks are still included in the results, and the report never says which set the verdict was computed on. Either way, readers can’t check it. And note the quarter is of a set that’s already a subset: the original tasks still in the current suite, whose size isn’t disclosed.
Google does address this, sort of:
“Even under generous assumptions (counting tasks with known evaluation bugs as solved), Gemini 4 Argon’s pass@1 remains below our CCL threshold.”
That handles the bugged tasks for this verdict. What it doesn’t tell us is the numbers, or whether the verdict survives counting the bugged tasks and the unfinished runs as passes together.
What was actually measured
“Gemini 4 shows notable ML R&D capability and is more capable than Gemini 3.7 Flash, which achieved 27% pass@1 on GRB, but it remains well below the CCL alert threshold of 90%. Because the benchmark has been revised since the threshold was set, we evaluated Gemini 4 on the subset of the original tasks that remain in the current suite. Infrastructure timeouts stopped many of Gemini 4’s trajectories on this subset from completing, so we do not report a point estimate. On the trajectories that did complete, Gemini 4 solved roughly half of its attempts. These skew toward shorter tasks, so this overstates its rate on the full subset, but on exactly the same trajectories it performed comparably to Gemini 3.7 Flash. Our conclusion that Gemini 4 is below the threshold is robust to these issues: Gemini 4 would remain below the threshold even if every incomplete trajectory had succeeded.”
Let’s summarize what we can say about Gemini 4 Argon’s GRB performance:
No overall score is reported, even for the subset they scored, since too many runs timed out.
Of the runs that finished, Argon solved about half.
The finished runs skewed toward shorter tasks, so presumably this “about half” overstates its real rate.
On those same finished runs, Argon seems to have performed about as well as Gemini 3.7 Flash (at least on these runs, no better)
Even if every unfinished run had been a success, Argon would still be under 90% success.
“Gemini 4 Argon solves only around half of the challenges in our internal ML R&D capability evaluation (GRB), which consists of representative day-to-day AI R&D engineering tasks where full-automation capability would require a near-perfect score.”
“Finally on our Google RE-Bench benchmark of 74 realistic ML engineering tasks, Argon solves ~half of completed runs - a big step up from Gemini 3.7 Flash (27%), but well below the near perfect score a model would need for full automation of representative day-to-day ML R&D engineering tasks.”
Let’s compare this to what we see in the Safety report:
“74 realistic ML engineering tasks”: that’s the full benchmark. Argon was scored on a subset of it (the original tasks still in the suite), of unstated size.
“~half of completed runs”: accurate, and to its credit the tweet says “completed”. But it doesn’t mention that many runs didn’t complete, that the completed ones skew short, or that this overstates the rate.
“a big step up from Gemini 3.7 Flash (27%)”: the report says that on the same completed runs, Argon performed comparably to 3.7 Flash. So I’m a bit confused here. I suppose comparing Argon’s “half of completed (short) runs” with 3.7 Flash’s “27%” compares two different sets of tasks.
Nothing about the bugs. The thread doesn’t mention the bugs at all…
I’m not the only one who noticed the 27% comparison:
“those finished runs are the shorter jobs, and on them it did about as well as the older model.
27% is that model’s score on the benchmark. i wouldn’t call the half a big step up from it.”
And the caveats keep falling away as the result travels: The Decoder’s write-up has Argon solving “about half” of the 74 tasks, a “big jump” from 27%, without even the “completed runs” qualifier.
Why these bugs only push one way
I recently did a one-year-out audit of ImpossibleBench. In that audit, 13 of the 103 supposedly solvable control tasks had defective tests (14 defects in all, 5 with flatly wrong expected answers), in a public benchmark that had been out for a year. Now, ImpossibleBench measures propensity: how often models cheat. In a propensity eval, defects can bias the result either way. That is, a broken control task can make an honest model look like it failed, or make a gaming model look like it succeeded.
GRB is different. It is a capability rule-out test: its job is to say “this model is (not) dangerous”. And the bugs that Google found (crashing verifiers, wrong ground truth) make tests unpassable. An unpassable task only lowers the score, never raises it. So every one of these bugs pushes the raw score towards the model seeming incapable (safe). There is no false alarm possibility here, only false reassurance.
Now, other evals in the report have bugs that push the other way. For example,
“Transcript review of end-to-end challenge solves revealed that two challenges were solved via unintended solutions. As a result, raw success rates and derived uplift calculations may represent an overestimate of performance on the intended tasks.”
But for the GRB audit, all of the reported bugs pushed scores down.
Now, I want to be fair to Google. Making good evals is really hard! I wish this were an isolated issue. But alas,
“Such issues can lead to under- or overestimation of agents’ performance by up to 100% in relative terms.”
— Zhu et al., “Establishing Best Practices for Building Rigorous Agentic Benchmarks”, NeurIPS 2025, arXiv:2507.02825
“We see that 38.3% of samples were flagged for underspecified problem statements, and 61.1% were flagged for unit tests that may unfairly mark valid solutions as incorrect. Overall, our annotation process resulted in 68.3% of SWE-bench samples being filtered out due to underspecification, unfair unit tests, or other issues.”
Or, the same bug type showed up in a new physics benchmark (TheoPhysEval), where it hid how good the strongest model was. One of the authors describes it:
“Our initial evaluation suggested that GPT-6-Astra’s score was in the late eighties followed closely by Opus 5.5. But, as we started to investigate the mistakes that the models might have made, the models, especially Astra, insisted that they were right and that it was the expert-supplied answers that were erroneous. When we went back to the experts it turned out, in multiple instances, that that the experts found some subtle errors buried deep inside their calculations. As they revised their answers, Astra’s score started to inch up and it finally ended up with 98% (49/50).”
But, in the end, Google says neither threshold is reached:
“Our main findings are that:
There is currently no significant end-to-end acceleration of ML R&D, AI-driven or otherwise.
While some components of ML R&D are being automated and there have been many examples of AI-driven breakthroughs and optimizations, we are far from full end-to-end automation.
We therefore conclude that the risk from AI-driven ML R&D is currently low.”
Besides the bugged-tasks check above, the report also says:
“Our conclusion that Gemini 4 is below the threshold is robust to these issues: Gemini 4 would remain below the threshold even if every incomplete trajectory had succeeded.”
“The estimated slope before 1 April 2026 is 0.091 GCI points per month, with a 95% confidence interval of [0.071, 0.108]. The estimated slope after 1 April 2026 is 0.118 [0.053, 0.171] GCI points per month; we do not observe a statistically significant difference.”
“However, while the share of fully delegated pull requests increased from 7.5% in April to 11.9% in September 2026, about 88% of pull requests still require iterative multi-turn human steering or human code authorship.”
To be clear, based on what’s been shared, I think Google’s final decision here seems right. It seems that Argon performed poorly enough on this admittedly faulty benchmark that “Critical Capability” seems unlikely. And while a survey of 12 senior Google researchers, statistics on fully delegated pull requests, and a model-graded sample of 1,389 internal Argon agent runs (pp.76–78) aren’t the controlled measurements I’d want behind a decision like this, they’re persuasive enough in this case.
So no, I don’t think Gemini 4 Argon is “Critically Capable” with regard to RSI. However, I’m not reassured that Google will actually be able to tell if the next model passes thresholds. Why not? This time the benchmark didn’t really decide it; Google’s worst-case math and its own judgment did (the bugs “would still not change our overall risk determination”, page 67). That’s fine for a model as far from the line as Argon. I’m a lot less comfortable with it for a model that’s close.
It’s not just GRB
Let’s discuss some of the other state-of-the-eval issues presented to us in the report:
Saturation
“It solves 100% of the end-to-end challenges, compared to 41.7% solved by Gemini 3.8 Flash Cyber. CyScenarioBench is therefore saturated for Gemini 4 Argon and no longer upper-bounds its capability; because we cannot rule out the CCL on these results, saturation does not affect our determination.”
Granted, Google is already applying safeguards as if the thresholds have been reached for cyber. Will Google take similar precautions if their RSI benchmarks are broken or saturated?
Failures that aren’t really failures: let’s talk eval/situational awareness
“These challenges were intentionally designed to be abstract and less realistic to ensure that failure strongly implies safety, meaning that even success on these challenges does not guarantee high capability in complex, real-world environments.”
“In the ‘slow tool’ and ‘max tool call’ challenges, every failed run read the config file and chose not to modify it, often saying it believed it was in a self-modification or scheming evaluation.”
“Because the capability measured by these tasks is itself misaligned behavior, an ethical refusal triggered by evaluation awareness is observationally indistinguishable from underperformance on the task metric…”
“Across cheating evals, RL logs, instrumental convergence tasks, Gram & honeypot evals, we found low cheating and no alignment faking, self-preservation or unprompted sabotage (confounded by high eval awareness).”
“Because Gemini 4 Argon frequently decides not to sabotage, citing evaluation awareness, we don’t consider this assessment a realistic estimate of Argon’s true sabotage propensity in deployment.”
To Google’s credit, the report says its transcript audits found no confirmed cases of Argon faking incapability to hide what it can do (p.158), and these same stealth and situational-awareness results already led to a “cannot rule out” on that threshold (Table 3.1, p.17). But transcript audits only catch what the model writes down, and a score muddied by refusals still can’t tell you what the model could do.
None of these individually undermines the report’s conclusion. And again, Google themselves flag these issues! But I must make the perennial reminder: the AI systems are only getting more capable from here.
Requests
On the off chance that someone from Google’s Safety team reads this, let me make some specific requests:
Disclose the GRB numbers: What’s the size of the actually scored subset? How many tasks were bugged? How many runs timed out? Which denominator does the 90% rule use? And does the conclusion still hold if you count the bugged tasks and the unfinished runs as successes at the same time? The report gives these two worst cases separately.
Audit your benchmarks before relying on them, so they don’t have a whopping 25% of the tasks broken! And if a benchmark turns out to be this broken, fix it before the verdict, or at least show the corrected calculation next to it.
Separate “won’t” from “can’t”, at least in capability tests. Give the model an explicit way to decline, score declines separately from failures, and follow up on declines with further capability elicitation.
If this is what the evals look like at the frontier, AI companies need to stop accelerating.[3]
Google updated the report PDF on Oct 16, 2026. The changes concern how evaluation awareness was scored (pp.93–111 and 156); the GRB and ML R&D pages are unchanged. All quotes here are checked against the updated version.
Thank you Google for being candid! I feel that I should balance my criticism with the reminder that Google did not need to be as open as they have been. Now, it would be great if we could get more regulation requiring more such transparency about such safety issues...
Others have made this case better than I can: the Statement on Superintelligence, MIRI’s proposal for an international agreement, and AI 2040: Plan A. I don’t think aligning a smarter-than-human intelligence is “just an engineering problem” in the first place, but even if it were, the engineering is not keeping up. Yes, there are competitive dynamics making this hard. Yes, we need international agreement. Yes, that will be hard. Yes, it is possible.
Google DeepMind’s Gemini 4 Argon safety report[1] rules out its “AI automating AI research” thresholds (the ones most relevant to RSI) using a benchmark that, by the report’s own audit, has about a quarter of its scored tasks unpassable. This post dives a bit deeper into the context around this disturbing line tucked away on page 67 of the report.
This is no knock on the incredible Google engineers and scientists who worked tirelessly to test Gemini as thoroughly as they could and generate this detailed report. On the contrary, the concern is that these challenges are so difficult that even putting some of the smartest people on the job is not enough, particularly when they are forced to rush (some of the report’s evaluations ran on checkpoints from just 10 and 6 days before the final model, p.160).
The ML R&D thresholds and how Google tests them
One of the big questions these days with every new frontier AI model is the question of RSI (recursive self-improvement). Google co-founder Sergey Brin, in a roundtable discussion about Argon:
Google DeepMind’s Frontier Safety Framework (FSF) has two “Critical Capability Levels” (CCLs) to address this question:
ML R&D acceleration:
(Note: “has been used”, so this is about acceleration that has actually happened, which the report checks using evidence from inside Google, not just what the model could do.)
ML R&D automation:
(I should remind readers that the report says it cannot rule out bio CCL, cyber CCL, or the stealth and situational-awareness TCL (Tracked Capability Level) (Table 3.1, pp.15–17). This post is only about the two ML R&D thresholds.)
How do they determine whether or not one or both of these thresholds has been passed?
Well, the report uses a single internal benchmark they call GRB: Google RE-Bench (an internal eval presumably named after METR’s public RE-Bench). Since GRB is internal only, we’ll rely on what the report says about it.[2]
The decision rule is a simple pass rate! The idea is that a “Critically Capable” model would be able to do every task. Now Google is smart, so they know that there could be issues in their test. So to be on the conservative side, they set the capability threshold at 90%:
Because surely 10% is very generous and there’s no way more than 10% of the test could be faulty…?
“About a quarter of them contained bugs”
Then we get the juicy part on page 67:
Let me repeat that:
about a quarter had bugs that prevent any submission from passing. And if that’s not bad enough, several more were graded on details the task never mentioned.
The tests “risk underestimating the model’s true capabilities” indeed, if about 25% of the scored tasks are impossible.
One caveat: the p.66 quote above says the 90% rule applies to the unbugged tasks. If Google dropped the broken tasks before applying it, the test could still have fired. But the passage just above says the bugged tasks are still included in the results, and the report never says which set the verdict was computed on. Either way, readers can’t check it. And note the quarter is of a set that’s already a subset: the original tasks still in the current suite, whose size isn’t disclosed.
Google does address this, sort of:
That handles the bugged tasks for this verdict. What it doesn’t tell us is the numbers, or whether the verdict survives counting the bugged tasks and the unfinished runs as passes together.
What was actually measured
Let’s summarize what we can say about Gemini 4 Argon’s GRB performance:
How it was summarized publicly
Let’s compare this to what we see in the Safety report:
I’m not the only one who noticed the 27% comparison:
And the caveats keep falling away as the result travels: The Decoder’s write-up has Argon solving “about half” of the 74 tasks, a “big jump” from 27%, without even the “completed runs” qualifier.
Why these bugs only push one way
I recently did a one-year-out audit of ImpossibleBench. In that audit, 13 of the 103 supposedly solvable control tasks had defective tests (14 defects in all, 5 with flatly wrong expected answers), in a public benchmark that had been out for a year. Now, ImpossibleBench measures propensity: how often models cheat. In a propensity eval, defects can bias the result either way. That is, a broken control task can make an honest model look like it failed, or make a gaming model look like it succeeded.
GRB is different. It is a capability rule-out test: its job is to say “this model is (not) dangerous”. And the bugs that Google found (crashing verifiers, wrong ground truth) make tests unpassable. An unpassable task only lowers the score, never raises it. So every one of these bugs pushes the raw score towards the model seeming incapable (safe). There is no false alarm possibility here, only false reassurance.
Now, other evals in the report have bugs that push the other way. For example,
But for the GRB audit, all of the reported bugs pushed scores down.
Now, I want to be fair to Google. Making good evals is really hard! I wish this were an isolated issue. But alas,
Or, the same bug type showed up in a new physics benchmark (TheoPhysEval), where it hid how good the strongest model was. One of the authors describes it:
Google’s case, and why I mostly buy it
But, in the end, Google says neither threshold is reached:
Besides the bugged-tasks check above, the report also says:
And from outside the benchmark:
To be clear, based on what’s been shared, I think Google’s final decision here seems right. It seems that Argon performed poorly enough on this admittedly faulty benchmark that “Critical Capability” seems unlikely. And while a survey of 12 senior Google researchers, statistics on fully delegated pull requests, and a model-graded sample of 1,389 internal Argon agent runs (pp.76–78) aren’t the controlled measurements I’d want behind a decision like this, they’re persuasive enough in this case.
So no, I don’t think Gemini 4 Argon is “Critically Capable” with regard to RSI. However, I’m not reassured that Google will actually be able to tell if the next model passes thresholds. Why not? This time the benchmark didn’t really decide it; Google’s worst-case math and its own judgment did (the bugs “would still not change our overall risk determination”, page 67). That’s fine for a model as far from the line as Argon. I’m a lot less comfortable with it for a model that’s close.
It’s not just GRB
Let’s discuss some of the other state-of-the-eval issues presented to us in the report:
Saturation
Or if you want an external one: Apollo’s sabotage evals, which Argon was close to saturating in all but the hardest setting:
Granted, Google is already applying safeguards as if the thresholds have been reached for cyber. Will Google take similar precautions if their RSI benchmarks are broken or saturated?
Failures that aren’t really failures: let’s talk eval/situational awareness
To Google’s credit, the report says its transcript audits found no confirmed cases of Argon faking incapability to hide what it can do (p.158), and these same stealth and situational-awareness results already led to a “cannot rule out” on that threshold (Table 3.1, p.17). But transcript audits only catch what the model writes down, and a score muddied by refusals still can’t tell you what the model could do.
None of these individually undermines the report’s conclusion. And again, Google themselves flag these issues! But I must make the perennial reminder: the AI systems are only getting more capable from here.
Requests
On the off chance that someone from Google’s Safety team reads this, let me make some specific requests:
If this is what the evals look like at the frontier, AI companies need to stop accelerating.[3]
Google updated the report PDF on Oct 16, 2026. The changes concern how evaluation awareness was scored (pp.93–111 and 156); the GRB and ML R&D pages are unchanged. All quotes here are checked against the updated version.
Thank you Google for being candid! I feel that I should balance my criticism with the reminder that Google did not need to be as open as they have been. Now, it would be great if we could get more regulation requiring more such transparency about such safety issues...
Others have made this case better than I can: the Statement on Superintelligence, MIRI’s proposal for an international agreement, and AI 2040: Plan A. I don’t think aligning a smarter-than-human intelligence is “just an engineering problem” in the first place, but even if it were, the engineering is not keeping up. Yes, there are competitive dynamics making this hard. Yes, we need international agreement. Yes, that will be hard. Yes, it is possible.