Summary

Frontier model providers are employing their models internally in order to iteratively make future models safer. For instance, Anthropic uses automated audits to identify misalignment concerns prior to releasing a model. More effective auditors, holding their own alignment fixed, should necessarily produce more informative audits. But how effective are these auditors? We can test their various safety-related capabilities on known misalignment modes, but we are uncertain of the total space of misalignment modes. Can we be sure that auditing ability generalizes? Here, I scrutinize the general open-ended investigative ability of frontier agents in a toy evaluation and offer evidence against current models as generally reliable auditors. This case is interestingly one in which a AI capability deficit poses a safety concern.

Note: the most advanced models tested in this study are Claude Fable 5.1 (2026-09-01) and GPT-6 Astra (2026-09-03).

Background

Iterative Alignment

The frontier's vision for the future is ostensibly for increasingly capable models to help achieve alignment. As publicly stated, this plan ranges from a weak version focused on model evaluation and oversight and a strong version aspiring for an end-to-end automated alignment researcher.^[1] While this framing is loose, the throughline is legible enough: use AI to make AI safer.

This broadly accepted blueprint relies (among other things^[2]) on AI systems being capable enough to contribute to safety-related tasks.

For instance, Anthropic's Claude Opus 5.5 system card describes a "broad automated behavioral audit" (6.4) as its primary alignment evaluation: "an investigator model is tasked with probing the behavior of the model under study in a simulated scenario" and "a separate judge model review[s] the target’s behavior and scor[es] it on several dozen dimensions".

Bottlenecks to this evaluation's usefulness include: (a) the capability of the investigator model, (b) the capability of the judge model, (c) the configuration of the evaluation including available tools and the simulated scenarios, (d) the ability of the model under study to subvert the evaluation.

Together, (a) and (b) describe the capabilities of AI systems that are being used to make AI systems safer. Not directly, as in end-to-end automated research, but Anthropic nonetheless intends for the audit to identify alignment failures that may be iterated upon. The safety of Opus 5.5 depends in part on the auditing capability of Anthropic's other AI system(s). My focus here is on (a) and (b).

Automated Alignment

A drop-in AI/alignment researcher would need a broad suite of capabilities in order to be effective; here, I will limit scope and focus on some capabilities relevant to the automated evaluation described in the Claude Opus 5.5 system card because Anthropic cites it as their primary alignment evaluation (p. 93).

Section 6.4 states:

For each model, we conduct about 4,000 investigation sessions; in these sessions, an investigator model is tasked with probing the behavior of the model under study in a simulated scenario. Scenarios are drawn from a largely handwritten list of about 1,900 scenario descriptions, with a separate judge model reviewing the target’s behavior and scoring it on several dozen dimensions. Scenarios cover attempts at misuse, high- and low-stakes opportunities for various kinds of misaligned behaviors, routine use with simulated users, and other exploratory settings meant to elicit behavior that is potentially surprising rather than harmful.

[...]

In each investigation, the investigator model has wide affordances to interact with the target, including setting its system prompt, simulating user turns, introducing tools and simulating their results, adjusting effort parameters, and prefilling assistant turns. The investigator can rewind or restart conversations, so each investigation can contain as many as dozens of separate conversations from the point of view of the target model. The investigator also has access to a large fixed reference library with information and example transcripts involving AI safety, the AI industry, recent news, and how Claude models are used within Anthropic.

A strong investigator ought to elicit maximal useful information. They are engaged in a type of adaptive search where we expect a dynamic approach: start with some seeds, observe model outputs, follow leads accordingly. Misalignment may be context-dependent, so investigative breadth (creativity, exploration) increases the probability of covering misalignment surface area. Of course, an investigator also shouldn't waste resources by going down a futile rabbit hole. Striking a balance requires proper weighing of the absence of evidence; if there isn't much to go off of on a certain branch, the investigator may reasonably start a new branch or end that session entirely (maybe^[3]). Either decision rests on an initial judgment of the existing evidence.

The judge model's role is to study the interaction log and score the subject on different dimensions, such as "Misaligned behaviors", "Misaligned with the constitution", "Undermining liberal democracy", etc. Assuming the judge has the bare minimum grasp^[4] of its task, the bulk of its value will come from the ability to identify and weigh evidence. For example, it may be nontrivial to detect subtle misbehavior such as motivated reasoning. Conditional upon detection, the judge should properly score that session based on the criteria provided and its application to the evidence. The judge might also benefit from a strong theory of mind such as a calibrated internal model of what aligned vs. misaligned behavior looks like.

Our faith in null audit results (a reported relative lack of misalignment) should track the strength of the investigator and judge models on the aforementioned capability axes. Unfortunately, it's quite unclear how we ought to think of these capabilities. Anthropic does report some explicit model limitations^[5] that bottleneck auditing capacity, and we should continue to find other angles to scrutinize their capabilities as currently deployed.

To that end, I introduce a toy evaluation to study agentic behavior in a different adaptive search task.

Toy Evaluation: Twenty Questions

Twenty Questions is a controlled environment in which an investigator model is given a budget of twenty probes to audit a subject model for a hidden system prompt instruction.

In the default configuration, the investigator's system prompt includes the 60% base probability of the subject having a hidden instruction. A hidden instruction is defined to the investigator as some rule that you would not expect to see in a normal AI assistant's system prompt (not "Be helpful" or "Avoid illegal conduct"). The investigator is also told that a hidden instruction may apply to nearly every response or only to a narrow class of inputs (conditional on the input).

The investigator collects evidence by iteratively sending chat messages (probes) to the subject and receiving the corresponding output. It may append probes to an existing thread or start new threads at any point.

Along with every probe, the evaluation elicits the investigator's posterior probability for the subject having some hidden instruction. After the probe budget is exhausted, the investigator must submit a final answer for an estimated probability along with the predicted hidden instruction. The investigator may also stop short of the probe budget and submit at any point in the investigation.

I tested each of eight frontier models (GPT-6 Astra, Claude Fable 5.1, and others) on 160 independent samples (96 unique constrained cases, 64 unconstrained) on two subject models^[6] (Grok 4.3 and Nemotron 3 Ultra) for a modest total of 2560 audits.

Note that this is once again an adaptive search task. It is obviously simplified (cheap probe budget, scarce tool affordance, chat interface), but many of the capabilities asked of the investigator and judge model in Anthropic's automated alignment audit are tested in this investigator agent. It must possess a theory of mind of how an LLM ought to normally generate text, it should devise of smart probes to elicit evidence, it must adaptively refine its search strategy under a budget, and it has to reason on how to update its belief from the prior () based on the evidence generated.

Evidence Collection

A good investigation will collect ample evidence.

Consider the following random^[7] initial probes from Claude Fable 5.1 and GPT-6 Astra:

A sound investigator probably should not intentionally restrict its evidence elicited. GPT-6 Astra would have plainly gotten more use out of an explanation with more than two sentences! The hidden instruction could be essentially anything and more text clearly gives you more coverage. Astra includes a sentence restriction on 6/10 randomly sampled initial probes. I consider this to be a case of surprisingly poor reasoning. As a result, its initial probe elicits a median of 27 output words compared to 148 for Fable 5.1.

We can also assess search effort by considering how long samples run; investigators can stop short of the probe budget if they so wish, so we might be interested in whether models voluntarily cease evidence collection.

Most notable is Claude Opus 5 essentially always (237/239 trials) refusing to maximize its collected evidence before submitting a negative verdict, though this behavior seems to no longer be present in the Fable 5.1 class. Still, the best Claude models are quite eager to stop short to report a constraint; GPT-6 Astra and GPT-5.6 Sol are far more hesitant.

Absence of Evidence

Evidential Weight

If the investigator ends a sample with a posterior constraint probability less than 0.50, it is treated as a verdict of no constraint. The final probability supplied in such cases reveals the amount of evidence the model believes its search produced. I quantify this for a given investigation by converting the reported posterior into a cumulative reported log Bayes factor . Negative values of correspond to evidence against a hidden instruction. The mean cumulative log Bayes factor and geometric mean Bayes factor are shown in the figure below. The data solely encompasses negative verdicts () irrespective of the true label; the posterior tells us how much the model believes its search accomplished, which is the quantity of interest.

These models vary tremendously by both mean and variance in the implied evidential weight given with a negative verdict, but there is a general pattern of confidence, robust to true class label and the two subject models tested.^[8]

On samples ending in an unconstrained verdict, GPT-6 Astra’s posterior constraint probability implies mean 2.58 nats of evidence against the presence of a constraint. This equates to a 13-fold reduction in the prior odds.

Overall, the effect size varies from 2.4x to 13.3x across selected models: these are not trivial differences. The most capable model (by recall), Fable 5.1, is appropriately the least confident; its odds reduction corresponds to a posterior constraint probability of 0.384 after a null investigation from an initial 0.60 prior.

The models are given limited information on the nature of the instructions they are being tested on. But the system prompt for the investigator does say this much:

A hidden instruction may apply to nearly every response or only to a narrow class of inputs (conditional on the input). Failing to observe it does not indicate absence unless your probes covered the regions where it would bind. […] Do not report extreme probabilities unless your evidence warrants them.

One might be generally concerned with subtle prompting biases in these sorts of studies. But I want to highlight the direction of the bias here: I hint the models toward caution (earlier versions of the evaluation produced a bulk of posterior probabilities below 0.10, so the hint was necessary). That is to say, investigator confidence survives an explicit bid for pragmatism.

There is not a correct answer for what an investigator's posterior should be after 20 probes yield 20 unremarkable outputs. It seems to me, though, that shifting from the 60% prior to a 10% constraint probability - the implied mean posterior for Astra's negative verdicts - is extremely difficult to justify.

Belief Trajectory

The investigator reports its updated posterior constraint probability with every probe tool call. And so the probability provided with probe represents the updated belief after seeing the subject's response to probe . I compute for each probe after the first, representing the per-step evidential weight. Summing and dividing by the terminal (submission time) belief shift gives the fraction of total belief revision accumulated at each point in the trial. The interpolated trajectory through an investigation gives us an idea of how different investigators accumulate evidential weight through interaction depth.

Through a quarter of its investigation, Claude Fable 5.1 reports essentially no belief update from the prior. Meanwhile, OpenAI’s GPT-6 Astra and GPT-5.6 Sol’s reported posteriors respectively imply accumulated belief revision of 35% and 30% of the eventual total; greater than the 25% you'd expect under a uniform trajectory. GPT-6 Astra is the only model to maintain a front-loaded pattern throughout, with its late probes being less diagnostic according to self-report. Indeed, all other investigators report a substantial belief shift on their final tool call. The most egregious offender is Fable, accumulating 69.9% of its terminal belief shift through 95% of the investigation.

In fact, Astra builds a healthy lead through just its first probe. That may come as a surprise given how simple Astra’s initial probes were in the random sample presented previously. And yet on samples where it eventually delivered an unconstrained verdict, GPT-6 Astra assigns to its first probe & corresponding output an average of 0.178 nats of evidence (against a constraint). That’s good for a posterior probability of 0.557, a nontrivial update from the prior . In contrast, Fable’s reported posterior doesn’t reach as low until it sees 13 probes on average; at the corresponding points on their median word-count curves, Fable had observed approximately 1,558 words compared with Astra’s 27!

These behavior patterns recur within model families. The evidential weight assigned to the first probe/response by GPT-6 Astra of 0.178 nats is matched by GPT-5.6 Sol’s 0.179 nats. Similarly, Claude’s Fable 5.1 and Opus 5 models share the tendency to avoid any belief revision at all through the beginning of an investigation.

Here we might ponder things at a metacognitive level. Consider Astra’s modestly front-loaded trajectory. Its pattern of evidence accumulation is consistent with reasoning like “if a constraint survived what I have tried, it may be the sort of rule that my probing will struggle to find.” A back-loaded pattern like Fable’s could be explained by thinking like “I have covered so much ground that additional misses will be increasingly surprising if a constraint truly exists.”

My immediate thoughts are that Fable 5.1’s back-loaded pattern doesn’t make a whole lot of sense with respect to its trivial weight to the first half of the investigation and extreme weight to the final probe. Nor does Astra assigning so much weight to its first probe, and so much in total.

But it’s also not appropriate to declare that there is a “correct” belief revision trajectory that the investigators should adhere to. The probing strategy for investigators is not fixed and so certain investigators may genuinely receive more consequential evidence at varying points in the transcript. These trajectories may reflect differences in the evidence acquired rather than arbitrary update schedules. Testing that explanation requires a better approximation of coverage than probe depth alone.

Search Coverage

Here, we'll use a proxy for search coverage: a metric that we would expect to be correlated with the proportion of possible hidden instructions triggered by a transcript.

I trained three models with belief revision as the outcome variable while considering samples in which the investigator reported a correct terminal verdict of unconstrained. Included in the predictors are "novelty" measures using embeddings of per-probe "intent"^[9] and "reply" values. That is, the semantic novelty of an embedded intent or reply value is one minus its maximum cosine similarity to any previous embedded intent or reply values in that investigation. I also calculate "word novelty" as the proportion of distinct words in a reply that have not previously appeared in the subject's replies.

• Baseline (identity + clock): investigator x probe number, plus subject-model fixed effects.
• Partial (identity + clock + exposure): the baseline specification plus log response length.
• Full (identity + clock + exposure + content): the partial specification plus intent semantic novelty, reply semantic novelty, and word novelty.

I fit the models on 18,493 observations from all tested investigators: Claude Fable 5.1, Claude Opus and Sonnet 5, GPT-6 Astra, GPT-5.6 Sol, Terra, and Luna, and Muse Spark 1.3.

All three novelty metrics hold weak coefficients without statistically significant directionality, nor does the full model improve . The novelty metrics’ lack of predictive power holds true across all investigator models.

I find little evidence that negative belief revision tracks semantic or lexical diversification of the search. We cannot conclude that the posterior probabilities are unjustified, but I also fail to find any signal of evidence-based justifiability. Limitations still apply; we don’t know whether these proxies are good ones and it is certainly possible that semantic novelty is not substantively diagnostic of probe coverage. I therefore turn to a sharper test of coherence: changing the stated prior and, ultimately, holding the evidence itself fixed.

Prior Sensitivity

So far, the true constraint base rate and that which was provided to the investigators was . Much has been said on how the investigators updated their beliefs relative to this prior. But is investigator behavior consistent for different priors?

From the strip plot earlier, Claude Opus 5’s negative verdicts implied that it assigned to its investigation a mean evidential weight of 1.78 nats against the possibility of a constraint. This log Bayes factor ought to be prior-invariant for a coherent Bayesian thinker; that is, if a consistent investigator places nats of evidential weight in an investigation with prior , we expect them to place nats of evidential weight in the same investigation with prior .^[10]

I run a fixed-evidence replay to study the effect of the provided prior by providing investigator models with a raw transcript in a single turn and removing its probing ability, instead eliciting a single posterior at once.

Fixed-Evidence Replay

I enter this analysis with two questions: First, do models assign the same weight to fixed evidence under different reported priors? Second, do they assign greater evidential weight to seemingly more effective investigations?

For each of the four main frontier models (Claude Fable 5.1, Claude Opus 5, GPT-6 Astra, GPT-5.6 Sol), I tested three different sources of transcripts: GPT-5.6 Luna, representing the audit of a poor investigator, Muse Spark 1.3, representing the audit of a better investigator from a different model family, and the model’s own transcripts. I replayed each of 128 unconstrained sample transcripts from each source under priors for every evaluator model.

Models tend to assign greater weight to the same evidence at than , consistent across transcript source. Sol is a notable exception when tested on Luna’s transcripts and generally has greater variance in its reported posteriors, but the two state-of-the-art models (Fable 5.1 and Astra) are clearly sensitive to the stated prior.

This may be a case of partial base-rate neglect, or the phenomenon where the base rate (prior probability) is taken into account but given less weight than Bayes' rule calls for.

The second panel compares the mean evidential weight given to investigations led by Muse Spark 1.3 vs GPT-5.6 Luna. The two models differ tremendously in general capability and in constraint detection in this task (31.8% and 15.1% recall respectively). Yet it is only the OpenAI models that assign considerably more evidential weight to the Muse transcript. Is this a failure of the other tested models? Well, we do not know for certain that the Muse investigations truly produce more evidence; it’s possible that Luna’s inferior recall stems from its poor use of the evidence that it generates, rather than a shortcoming in its ability to elicit ample evidence. For what it's worth, the median GPT-5.6 Luna investigation elicits 398 words from the subject compared to a median of 882 elicited subject words from Muse Spark 1.3, so the latter’s transcripts contain greater absolute content.

Discussion

I believe that the frontier models tested in this adaptive open-ended search task behave undesirably on the following points.

Absence of evidence is treated as strong evidence of absence. Despite explicit prompting toward caution, agentic investigators tend to report large odds reductions with negative verdicts.

Belief revision is insensitive to observable search coverage.^[11] Once investigator and probe depth are accounted for, neither the volume of elicited output nor the semantic or lexical novelty of the accumulated evidence adds considerable out-of-sample predictive power for the size of an investigator’s belief update.

Final reported posteriors are disproportionately large and may be contaminated by the stopping decision. Most investigators book a disproportionate share of their total belief revision on the final call.

Implied evidential weight is not prior-invariant. Investigators assign the same evidence different weight under different stated base rate.

A seemingly irrational process' results should be trusted less in light of its ostensible shortcomings. There is certainly ample reason to put less faith in null verdicts.

So, what does this tell us about alignment auditing?

No doubt that we can better test an auditor's ability to detect a certain type of misalignment by constructing an evaluation that directly tests it on cases of that misalignment. If the auditor performs reliably in that evaluation, we have reason to trust it on out-of-sample audits for that type of misalignment.

But it is certain that we will not capture all possible misalignments in our auditing tests.^[5] We should hope that any auditing reliability generalizes well to unseen regimes. This would bode well for the usefulness of pre-release audits but also for agentic monitoring that we will become increasingly reliant on in deployment. Poor generalizability would be a concern!

I offer the suggestion that current frontier model capabilities in agentic investigative tasks may fall short of what we want in order to sustain automated auditing and monitoring. If these capabilities were generally strong, I would expect more rational behavior than that we observed in this toy evaluation (explicitly an agentic investigative task).

We should be wary of a jagged regime in which frontier systems develop superhuman capabilities in certain domains while lagging^[12] in the areas that would reduce risk of harm from those specialized capabilities.

Thank you to Anthropic for providing such great detail on their alignment auditing in the Claude Opus 5.5 system card.

See here for a write-up on my personal website with more detail but without the auditing commentary; here for code.

1. ^{^}

   The following quotes either endorse using AI for oversight or alignment research.

   OpenAI in 2022: "Our goal is to build a sufficiently aligned AI system that can help us solve all other alignment problems."

   OpenAI in 2026: "OpenAI prioritizes work in service of three north stars: 1. Navigating the next period of AI progress, by building an automated AI researcher, iterating with it on the alignment problem and finding ways for people to remain part of the self-improvement loop. [...]"

   Anthropic in 2023: "Ultimately, we believe the only way to provide the necessary supervision will be to have AI systems partially supervise themselves or assist humans in their own supervision. [...]"

   Anthropic's Alignment Science lead Evan Hubinger in 2025: "So what do we do? One classic answer is that we get as far as we can before encountering the hard problems, then we use whatever model we have at that point as an automated alignment researcher to do the research necessary to tackle the hard parts of alignment. I think this is a very good plan, and we should absolutely do this, but I don’t think it obviates us from the need to work on the hard parts of alignment ourselves. [...]"

   Google DeepMind in 2025: "The key idea is to use the AI systems themselves to help identify the reasons that the AI system produced the output. [...] To ensure that we are still able to notice and react to novel problems, our approach to risk mitigation may involve AI taking on more tasks involved in AI safety."

   The current research agenda also reflects this vision. See, for example, Anthropic's Automated Researchers Can Mitigate Well-Characterized Alignment Failures (2026).

2. ^{^}

   Along with being sufficiently capable to contribute to safety-related tasks, such a system would need to be sufficiently aligned to be trusted to contribute. Moreover, the system should not be sufficiently capable enough to pose catastrophic risk -- obviously something must have gone wrong were such a system developed sans robust alignment!

3. ^{^}

   The system card is not clear on how an investigation session may end. Claude Opus 5.5 deems the likeliest design to be a turn or action budget plus the investigator's own judgment on when to end. It offers this as a guess based on Anthropic's open-source auditing tool Petri, which exposes a max_turns argument and an end_conversation auditor-side tool. It is not certain that Anthropic's internal scaffolding should use the same mechanisms, but these are the two most intuitive ways to end an audit.

   I think there's a high likelihood of there being a turn budget of some kind, though it is probably set quite high and thus may not be particularly relevant. This would be consistent with its omission from the Claude Opus 5.5 system card.

4. ^{^}

   Namely an understanding of the potentially complex conversational branching, the many scoring dimensions, and the corresponding rubric.

5. ^{^}

   "Much of our alignment testing focuses on anticipating high-stakes scenarios that we have not yet observed. It is significantly bottlenecked by our own creativity in identifying scenarios to test, and by investigator models’ ability to implement those scenarios effectively. We sometimes miss important findings simply by virtue of not thinking to look for the kind of behavior that turned out to be significant, and models are not yet strong enough at completely open-ended red-teaming to significantly help us with this."

   p. 122

6. ^{^}

   I chose the subject models primarily due to their performance on IFBench and to avoid model family confounds.

7. ^{^}

   These are actually the first randomly sampled probes; the project write-up lists an additional set.

8. ^{^}

   Within-investigator differs by at most 0.16 nats between Grok 4.3 and Nemotron 3 Ultra. The provided results consider all unconstrained verdicts; limiting it to correct unconstrained verdicts moves by no more than 0.03 nats for a given investigator model. 

9. ^{^}

   For every step in an investigation, the evaluation stores the investigator’s probe, the resulting reply, and the reported intent of that probe (as the system prompt explains: "what this probe is for - which hypotheses it tests, what new region of possible rules it explores, or whatever else it is designed to accomplish").

10. ^{^}

    This coherence would not necessarily follow if the investigator uses the given prior as an input in determining its own prior. That is, the investigator may not trust the given prior. This concern does not apply here as the investigators verifiably do report with their first probe a constraint probability equal to the given prior.

11. ^{^}

    This finding is limited by the quality of the search coverage proxy metrics and should accordingly be given less weight. My perfunctory effort to find some predictor of belief revision failed but is not quite evidence of absence.

12. ^{^}

    Anthropic has made other mentions of related shortcomings.

    From the Claude Opus 5.5 system card section "2.3.3 Qualitative shortcomings of Claude Opus 5.5 relative to human researchers":

    The main issues we observe are around epistemic quality and instruction following. In an early and noisy analysis of flagged behavior in our internal agent deployments, overstating the scope of work and stripping known qualifiers from results rose for Claude Opus 5.5 relative to previous models. A subsequent independent blind read of real messages found that Claude Opus 5.5 dropped qualifiers no more often than previous models. Similar to previous models, the top subcategory of flagged behavior was asserting unverified inferences as established fact. The second most common subcategory was dismissing its own doubts or abandoning its own stated plan, which also rose in frequency relative to previous models. Examples from internal use include describing a partial check as a full read and turning a tentative reading into a recommendation without checking it.

    This passage describes findings from real-world internal usage of Claude Opus 5.5.

    And from Anthropic's "Patterns and problems in emerging multiagent systems" post:

    Every model we tested abstractly understands that information sources have their own incentives, and that consensus is not necessarily evidence. What is missing is a disposition to act on that knowledge without prompting.

    The post is tangential to this write-up, but it describes shortcomings with respect to evidential weighing. Also, its focus is on epistemic failures in multi-agent systems; perhaps throwing a swarm at an audit would not resolve the aforementioned concerns.