AI allows us to prototype, develop, and research at unprecedented speeds. Across many tech industries, the barrier to entry to develop something new has significantly decreased. One particularly noteworthy example is at the intersection of AI and biology. As our computational capabilities increase, we now have the ability to fold, design, and predict the function of never-before-seen proteins. However, just as AI gives us the opportunity to do biological good in the world (in fact, we are just on the horizon of seeing the first AI-designed pharmaceutical drugs in the US![1]), it unfortunately opens up a terrifying possibility: could AI also give malicious actors the opportunity to do biological harm?
This sobering reality is a question that biosecurity researchers are currently trying to tackle. To be clear: the probability of a catastrophic event happening (e.g., designing a biologically harmful virus) seems unlikely to happen at least with current technology. But there have been several warning signs in the field that we may be getting close. For instance, recently, Anthropic revealed instances of malicious actors using Claude to design harmful proteins.[2] Even more concerning, Dario Amodei, Sam Altman, and Elon Musk have also called for the pace of AI development to slow down and to be regulated.[3] Additionally, building robust biosafety defenses takes time. If capable threats are even plausible with today’s technology, then we must begin to design solutions now in the face of exponential AI improvement. This begs the question: what does the landscape of biosecurity look like in the face of ever-growing AI?
The Dilemma of Open-source Models and Redefining AI Safety
The AIxBio ecosystem is heavily driven by open science. In fact, almost all of today's frontier biological protein models, from function and structure predictors to sequence generators, are released with open weights. While this massively accelerates biological discovery, it also completely fractures traditional frameworks of AI safety.
In more conventional, closed-source large language models, safety is typically managed at the model level. We can train linear probes, implement API-level prompt refusals, or use unlearning techniques to safeguard models from dangerous requests. This paradigm is fundamentally incompatible with the open-weight models of biology, whose weights are openly available to download on the internet without the need to go through an API. Settings like these require us to redefine AI safety in biology. Firstly, for future models, safeguards need to be embedded within the weights themselves, ideally ones that are robust against potential malicious fine-tuning strategies.[4] Secondly, while we may not be able to control the computational generation of proteins, an increasing viewpoint in biosecurity points to safeguarding the digital-physical interface where computational designs are actually synthesized.
The Digital-Physical Interface: Synthesis and Screening
The fundamental bottleneck of biological design, even beyond biosecurity, is the transition from a computational model to a physical sample. In practice, bringing computational designs to life typically relies on 3rd party DNA synthesis providers, where customers order DNA sequences that encode their designs, which are then commonly amplified and expressed in host systems to produce proteins with targeted functions. In principle, stopping the synthesis of potentially dangerous gene sequences is the first line of defense toward preventing biological harm.
Historically, 3rd party synthesis providers have relied on sequence-based screening—that is, checking ordered sequences against known hazards or closest database matches—to filter out dangerous sequences. However, in the age of AI where we have the ability to design functional de novo, or never-before-seen, sequences, sequence-based screening becomes increasingly unreliable.
Clearly, sequence-based screening is not enough to secure our digital-physical interface. Some potential solutions biosecurity researchers are looking to in the field are incorporating structure-based screens. Given that AI has increasing capabilities to predict the structure of proteins, we can check the structure of ordered proteins against the structure of known hazards. Additionally, current advances in mechanistic interpretability have demonstrated that biological AI models can be decomposed into interpretable features.[5][6] This may enable us to interpret the predictions of biological AI models and check for features corresponding to known hazardous motifs.
Properly Evaluating Agentic Frameworks
Beyond the need to safeguard DNA synthesis screening, the advent of agentic AI poses unique challenges toward assessing the threat level of potential biological misuse. Most notably, Anthropic recently demonstrated that Claude could act as a fully autonomous AI scientist.[7] When given access to specialized biological AI models, Claude was able to independently design de novo proteins, achieving comparable or even better experimental hit rates than those typically reported. This suggests that we may not fully understand the capabilities of today’s AI models and, consequently, we may not understand if these models are truly safe. Furthermore, this demonstrates that agentic systems may be able to interact more closely with the physical world than we thought. A longstanding assumption was that executing biological threats would always be bottlenecked by the tacit knowledge and biological intuition unique to human scientists. Instead, these advances show that agentic AI can bypass that barrier, making the need to rigorously evaluate whether agentic pipelines are truly safe vastly more urgent.
As further research is needed to assess the threat of agentic frameworks, including also designing better evaluations for these frameworks, it also creates a secondary challenge: how do we responsibly disclose our results, such that the information doesn’t become dual-use? One strategy Anthropic is taking is to restrict the dual-use biological research capabilities of Fable.[8] Alternatively, AI developers can also conduct internal research through secure, third-party auditors to red-team models in closed sandboxes. Ultimately, striking a balance between warning the community of emerging capabilities without inadvertently providing an instruction manual for misuse is essential for conducting research in biosecurity.
Conclusion
The field of AI safety for biosecurity can be best summarized as a community doing research to prevent future threats. While the probability of a catastrophic event happening with today’s AI models remains low, the evaluations and warning signs we see are based entirely on a snapshot of existing capabilities. As AI development continues to grow exponentially, our technical safeguards, screening pipelines, and evaluation frameworks must scale as fast as the models they aim to secure.
Acknowledgements
Special thanks to Chris Strahle from BlueDot Impact for providing feedback on an earlier draft of this article. Thanks to all of the Global Challenges Project Biosecurity Workshop organizers and participants for the insightful discussions. Lastly, thanks to the BlueDot Impact Technical AI Safety Course organizers and participants for the amazing conversations that provided me the background knowledge to write this article.
This article is written as part of a summary of the AI safety discussions held at the 2026 Global Challenges Project Biosecurity Workshop in Washington, D.C. All views held are mine.
Background
AI allows us to prototype, develop, and research at unprecedented speeds. Across many tech industries, the barrier to entry to develop something new has significantly decreased. One particularly noteworthy example is at the intersection of AI and biology. As our computational capabilities increase, we now have the ability to fold, design, and predict the function of never-before-seen proteins. However, just as AI gives us the opportunity to do biological good in the world (in fact, we are just on the horizon of seeing the first AI-designed pharmaceutical drugs in the US![1]), it unfortunately opens up a terrifying possibility: could AI also give malicious actors the opportunity to do biological harm?
This sobering reality is a question that biosecurity researchers are currently trying to tackle. To be clear: the probability of a catastrophic event happening (e.g., designing a biologically harmful virus) seems unlikely to happen at least with current technology. But there have been several warning signs in the field that we may be getting close. For instance, recently, Anthropic revealed instances of malicious actors using Claude to design harmful proteins.[2] Even more concerning, Dario Amodei, Sam Altman, and Elon Musk have also called for the pace of AI development to slow down and to be regulated.[3] Additionally, building robust biosafety defenses takes time. If capable threats are even plausible with today’s technology, then we must begin to design solutions now in the face of exponential AI improvement. This begs the question: what does the landscape of biosecurity look like in the face of ever-growing AI?
The Dilemma of Open-source Models and Redefining AI Safety
The AIxBio ecosystem is heavily driven by open science. In fact, almost all of today's frontier biological protein models, from function and structure predictors to sequence generators, are released with open weights. While this massively accelerates biological discovery, it also completely fractures traditional frameworks of AI safety.
In more conventional, closed-source large language models, safety is typically managed at the model level. We can train linear probes, implement API-level prompt refusals, or use unlearning techniques to safeguard models from dangerous requests. This paradigm is fundamentally incompatible with the open-weight models of biology, whose weights are openly available to download on the internet without the need to go through an API. Settings like these require us to redefine AI safety in biology. Firstly, for future models, safeguards need to be embedded within the weights themselves, ideally ones that are robust against potential malicious fine-tuning strategies.[4] Secondly, while we may not be able to control the computational generation of proteins, an increasing viewpoint in biosecurity points to safeguarding the digital-physical interface where computational designs are actually synthesized.
The Digital-Physical Interface: Synthesis and Screening
The fundamental bottleneck of biological design, even beyond biosecurity, is the transition from a computational model to a physical sample. In practice, bringing computational designs to life typically relies on 3rd party DNA synthesis providers, where customers order DNA sequences that encode their designs, which are then commonly amplified and expressed in host systems to produce proteins with targeted functions. In principle, stopping the synthesis of potentially dangerous gene sequences is the first line of defense toward preventing biological harm.
Historically, 3rd party synthesis providers have relied on sequence-based screening—that is, checking ordered sequences against known hazards or closest database matches—to filter out dangerous sequences. However, in the age of AI where we have the ability to design functional de novo, or never-before-seen, sequences, sequence-based screening becomes increasingly unreliable.
Clearly, sequence-based screening is not enough to secure our digital-physical interface. Some potential solutions biosecurity researchers are looking to in the field are incorporating structure-based screens. Given that AI has increasing capabilities to predict the structure of proteins, we can check the structure of ordered proteins against the structure of known hazards. Additionally, current advances in mechanistic interpretability have demonstrated that biological AI models can be decomposed into interpretable features.[5][6] This may enable us to interpret the predictions of biological AI models and check for features corresponding to known hazardous motifs.
Properly Evaluating Agentic Frameworks
Beyond the need to safeguard DNA synthesis screening, the advent of agentic AI poses unique challenges toward assessing the threat level of potential biological misuse. Most notably, Anthropic recently demonstrated that Claude could act as a fully autonomous AI scientist.[7] When given access to specialized biological AI models, Claude was able to independently design de novo proteins, achieving comparable or even better experimental hit rates than those typically reported. This suggests that we may not fully understand the capabilities of today’s AI models and, consequently, we may not understand if these models are truly safe. Furthermore, this demonstrates that agentic systems may be able to interact more closely with the physical world than we thought. A longstanding assumption was that executing biological threats would always be bottlenecked by the tacit knowledge and biological intuition unique to human scientists. Instead, these advances show that agentic AI can bypass that barrier, making the need to rigorously evaluate whether agentic pipelines are truly safe vastly more urgent.
As further research is needed to assess the threat of agentic frameworks, including also designing better evaluations for these frameworks, it also creates a secondary challenge: how do we responsibly disclose our results, such that the information doesn’t become dual-use? One strategy Anthropic is taking is to restrict the dual-use biological research capabilities of Fable.[8] Alternatively, AI developers can also conduct internal research through secure, third-party auditors to red-team models in closed sandboxes. Ultimately, striking a balance between warning the community of emerging capabilities without inadvertently providing an instruction manual for misuse is essential for conducting research in biosecurity.
Conclusion
The field of AI safety for biosecurity can be best summarized as a community doing research to prevent future threats. While the probability of a catastrophic event happening with today’s AI models remains low, the evaluations and warning signs we see are based entirely on a snapshot of existing capabilities. As AI development continues to grow exponentially, our technical safeguards, screening pipelines, and evaluation frameworks must scale as fast as the models they aim to secure.
Acknowledgements
Special thanks to Chris Strahle from BlueDot Impact for providing feedback on an earlier draft of this article. Thanks to all of the Global Challenges Project Biosecurity Workshop organizers and participants for the insightful discussions. Lastly, thanks to the BlueDot Impact Technical AI Safety Course organizers and participants for the amazing conversations that provided me the background knowledge to write this article.
Intuition Labs. "AI-Discovered Drugs in Clinical Trials 2026: Full Pipeline".
Anthropic. "Detecting and Countering Biological Misuse of AI." In Detecting and Countering Misuse of AI: September 2026.
BBC News. "Anthropic Boss Dario Amodei Calls for AI Development to Slow Down." September 12, 2026.
Lee Wall. "Tamper-Resistance is a Moving Target We Might Not Hit".
Adams, Etowah, Liam Bai, Minji Lee, Yiyang Yu, and Mohammed AlQuraishi. "From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models." ICML, 2025.
Tsui, Darin, Kunal Talreja, Daniel Saeedi, and Amirali Aghazadeh. "Protein Circuit Tracing via Cross-layer Transcoders." ICML, 2026.
Anthropic. "How Claude is Accelerating Protein Design and Analytical Chemistry".
Anthropic. "Improving Fable 5's Biology Safeguards".