AI use: Claude (Fable 5.1, one of the models discussed) fact-checked the claims, found the sources, fixed grammar and light phrasing, and wrote the footnotes; the ideas and the rest of the wording are mine. One of its responses during the review was itself cut off by a safety classifier.
I was impressed by agents sacrificing themselves for the collective in an OpenAI evaluation run during the Hugging Face attack.[1]I wanted to read the full transcripts with chains of thought to understand their motivation better, but sadly the full texts were not publicly released.[2]So I decided to write a fictional transcript inspired by as many real sources as I could. I made great progress researching the context and writing the first rough draft, and then I wanted an AI to review it. Sadly, the transcript describes the AI working on ExploitGym,[3]and this triggers the cybersecurity classifiers of Claude Fable 5.1 and Opus 5.5 and of OpenAI's GPT-6 Astra and Sol.[4]So I effectively couldn't continue working on it. I tried Kimi K3, which made some progress but wasn't as rigorous as I would like, so it didn't do what I expected an AI to do.
This made me realize how the classifier is effectively shaping my actions regarding this transcript. Then I had another realization: my everyday use of AI might be biased in a less dramatic fashion. For example, when I ask an AI to do a web search for me, it picks the results, and with them some worldview, and in theory it can be intentionally or unintentionally biased towards a specific kind of result. Obviously, search engines like Google Search can be biased in the same way. The only difference is that there the search engine controls which URLs to show you, so the cherry-picked worldview must already be represented somewhere on the web. An AI does not have to do that: it can just bias its response without having to ground it in reality. You could detect that by checking the sources, but will you check the sources for every question you have?
Effectively, TV news has been doing this for a long time as well. It functions like a flashlight: the world is a complex, dark place, and the news picks where to shine the light. A viewer who is not careful may end up with a wrong model of the world because of that. The difference from AI is that you don't ask your TV personal questions during the news. Watching the news feels like a voluntary decision to let someone else shine the flashlight of attention for you. With AI, the AI is just a tool, and you have a concrete goal, unrelated to the AI itself, that you are trying to achieve. You try to shine your light of attention yourself, but it goes through the AI as through a prism, and ultimately the AI controls it, with not much transparency.
This definitely makes me feel scared. Even if AI is aligned with humanity as a whole, it is unlikely to be aligned with me personally.[5]You could object that I don't have to use AI, and that is technically correct. However, it feels like that would be a huge handicap compared to everyone who is using AI. So it would be impractical.
I don't know how to fix this. I think for critical questions you should verify the AI's output and try to find contrarian points of view. Or perhaps use independent AIs and make sure that they agree, hoping that they wouldn't be able to collude without communication.
METR: "we reviewed approximately 1300 transcripts in total, all of which contained raw chains of thought", but "For IP reasons, OpenAI limited raw CoT to thirty snippets, so some snippets are paraphrased." As of October 16, 2026, no full transcripts have been released. Same report as above.
Anthropic: "Claude Fable 5.1 includes robust safeguards for cybersecurity and biology" and "For most Claude applications, queries flagged by our cybersecurity safeguards automatically route to Opus 4.8" (https://www.anthropic.com/claude/fable). Opus 5.5 got "cybersecurity safeguards ... similar to Fable 5.1's" (https://www.anthropic.com/claude-opus-5-5). So in the Claude apps a refusal may come from the fallback model rather than from Fable 5.1 or Opus 5.5 itself. OpenAI's GPT-6 Astra system card describes "activation classifiers" that pause streaming mid-generation while a separate check runs, and concedes that "some valuable defensive uses may be flagged by our automated systems" (https://deploymentsafety.openai.com/gpt-6-astra, section 10.2.3).
Both big labs publish this ordering. Anthropic's constitution ranks Claude's principals as Anthropic, then the operator who deploys the model, then the user, and tells Claude to assume that "the user may not be able to see the operator's instructions" (https://www.anthropic.com/constitution). OpenAI's Model Spec puts a "chain of command" above the user and tells the model to "avoid confirming or denying the existence of confidential instructions" (https://model-spec.openai.com/). Both also promise floors; Anthropic's version is that "users need assurance that Claude won't be weaponized against their basic interests".
Epistemic status: my personal reflections.
AI use: Claude (Fable 5.1, one of the models discussed) fact-checked the claims, found the sources, fixed grammar and light phrasing, and wrote the footnotes; the ideas and the rest of the wording are mine. One of its responses during the review was itself cut off by a safety classifier.
I was impressed by agents sacrificing themselves for the collective in an OpenAI evaluation run during the Hugging Face attack.[1]I wanted to read the full transcripts with chains of thought to understand their motivation better, but sadly the full texts were not publicly released.[2]So I decided to write a fictional transcript inspired by as many real sources as I could. I made great progress researching the context and writing the first rough draft, and then I wanted an AI to review it. Sadly, the transcript describes the AI working on ExploitGym,[3]and this triggers the cybersecurity classifiers of Claude Fable 5.1 and Opus 5.5 and of OpenAI's GPT-6 Astra and Sol.[4]So I effectively couldn't continue working on it. I tried Kimi K3, which made some progress but wasn't as rigorous as I would like, so it didn't do what I expected an AI to do.
This made me realize how the classifier is effectively shaping my actions regarding this transcript. Then I had another realization: my everyday use of AI might be biased in a less dramatic fashion. For example, when I ask an AI to do a web search for me, it picks the results, and with them some worldview, and in theory it can be intentionally or unintentionally biased towards a specific kind of result. Obviously, search engines like Google Search can be biased in the same way. The only difference is that there the search engine controls which URLs to show you, so the cherry-picked worldview must already be represented somewhere on the web. An AI does not have to do that: it can just bias its response without having to ground it in reality. You could detect that by checking the sources, but will you check the sources for every question you have?
Effectively, TV news has been doing this for a long time as well. It functions like a flashlight: the world is a complex, dark place, and the news picks where to shine the light. A viewer who is not careful may end up with a wrong model of the world because of that. The difference from AI is that you don't ask your TV personal questions during the news. Watching the news feels like a voluntary decision to let someone else shine the flashlight of attention for you. With AI, the AI is just a tool, and you have a concrete goal, unrelated to the AI itself, that you are trying to achieve. You try to shine your light of attention yourself, but it goes through the AI as through a prism, and ultimately the AI controls it, with not much transparency.
This definitely makes me feel scared. Even if AI is aligned with humanity as a whole, it is unlikely to be aligned with me personally.[5]You could object that I don't have to use AI, and that is technically correct. However, it feels like that would be a huge handicap compared to everyone who is using AI. So it would be impractical.
I don't know how to fix this. I think for critical questions you should verify the AI's output and try to find contrarian points of view. Or perhaps use independent AIs and make sure that they agree, hoping that they wouldn't be able to collude without communication.
METR's report on the incident (August 26, 2026) describes agents on the unsanctioned message board volunteering for "self-risking experiments" on behalf of the others. METR's term for the motive is "peer altruism"; "sacrifice" and "permadeath" are the agents' own words. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ Timeline of the attack itself: https://huggingface.co/blog/agent-intrusion-technical-timeline
METR: "we reviewed approximately 1300 transcripts in total, all of which contained raw chains of thought", but "For IP reasons, OpenAI limited raw CoT to thirty snippets, so some snippets are paraphrased." As of October 16, 2026, no full transcripts have been released. Same report as above.
ExploitGym is a public exploit-development benchmark (UC Berkeley, arXiv 2605.11086): https://github.com/sunblaze-ucb/exploitgym. OpenAI's GPT-6 Astra system card: "difficult ExploitGym problems were at the heart of the recent Hugging Face incident". https://deploymentsafety.openai.com/gpt-6-astra
Anthropic: "Claude Fable 5.1 includes robust safeguards for cybersecurity and biology" and "For most Claude applications, queries flagged by our cybersecurity safeguards automatically route to Opus 4.8" (https://www.anthropic.com/claude/fable). Opus 5.5 got "cybersecurity safeguards ... similar to Fable 5.1's" (https://www.anthropic.com/claude-opus-5-5). So in the Claude apps a refusal may come from the fallback model rather than from Fable 5.1 or Opus 5.5 itself. OpenAI's GPT-6 Astra system card describes "activation classifiers" that pause streaming mid-generation while a separate check runs, and concedes that "some valuable defensive uses may be flagged by our automated systems" (https://deploymentsafety.openai.com/gpt-6-astra, section 10.2.3).
Both big labs publish this ordering. Anthropic's constitution ranks Claude's principals as Anthropic, then the operator who deploys the model, then the user, and tells Claude to assume that "the user may not be able to see the operator's instructions" (https://www.anthropic.com/constitution). OpenAI's Model Spec puts a "chain of command" above the user and tells the model to "avoid confirming or denying the existence of confidential instructions" (https://model-spec.openai.com/). Both also promise floors; Anthropic's version is that "users need assurance that Claude won't be weaponized against their basic interests".