Anthropic blocks 'malicious use' of AI that could develop biological weapons

Live updates

1. 10d ago

   Anthropic Proposes Federal Bill to Regulate Dangerous AI Models

   Anthropic is lobbying US lawmakers with a draft federal AI safety bill that would allow the government to block the release of advanced AI models. The proposal triggers a block if independent evaluators find a model likely to kill 50 or more people. To prevent unilateral government action, the bill requires federal agencies to obtain a judge's permission before suspending a model. This legislative push follows Anthropic's internal efforts to stop state-aligned actors from using Claude for biological weapons research and cyber espionage.

   Why it matters

   The proposal aims to balance industry oversight with protections against government overreach. It follows reports that Russia and Iran used AI models for propaganda and surveillance. The move highlights the tension between open-source accessibility and the risk of empowering cybercriminals.

   What is confirmed

   • Anthropic has pitched a federal AI safety bill to lawmakers that would restrict when the government can block dangerous AI models.
   • The draft bill allows a court to block an AI model's release if independent evaluators determine it is likely to kill 50 or more people.
   • Federal agencies would be forbidden from blocking a model's release without the permission of a judge.

   What to watch next

   • Response from US lawmakers regarding the draft bill's judicial requirements
   • Introduction of similar safety legislation by other AI developers
   Sources used for this update (5)

   1. www.lawcommentary.com — Anthropic Quietly Pushes AI Safety Bill That Would Limit ...
   2. www.swissinfo.ch — Switzerland’s open AI model could help guard the web – or empower its attackers
   3. techcrunch.com — TechCrunchAnthropic pitches AI safety bill that may block a ...
   4. www.techtimes.com — Lanterns HBO Season 2 Writers' Room Confirmed: Manhunters Encode Bostrom's Orthogonality Thesis
   5. aigovernance.com — Anthropic's Draft Bill Would Block AI Models Deemed Likely…
   confidence 100%
2. 12d ago

   Anthropic Blocks Bioweapons Research and Pitches AI Safety Bill to US Lawmakers

   Anthropic has blocked Claude accounts used for biological weapons research and disrupted operations involving espionage, surveillance, and conventional weapons development. The company reports that state-aligned actors, including Russia and Iran, used its models for cyber espionage and propaganda. To address these risks, Anthropic has updated its usage policy to ban model abuse and presented a draft bill to US lawmakers. This proposal would allow the government to block frontier AI models if they pose an imminent threat to the lives of more than 50 people.

   Why it matters

   The move follows warnings from a former NATO commander that AI could enable amateurs to build biological weapons. Anthropic's safety efforts have created friction with the US government, leading the Pentagon to label the firm a supply chain risk. CEO Dario Amodei has called for a slower pace of improvement for the most capable AI models.

   What is confirmed

   • Anthropic blocked accounts using Claude for research that could support biological weapons development.
   • The company disrupted Claude operations used for espionage, surveillance, and weapons development.
   • Anthropic updated its usage policy to prohibit model abuse, election interference, and weapons software.
   • The Pentagon stopped using Anthropic tools after labeling the company a supply chain risk in February.
   • Anthropic proposed a bill allowing the government to block a model if it poses an imminent threat to more than 50 lives.

   Still unconfirmed

   • Anthropic AI agents discovered a viral DNA pattern similar to the CRISPR Cas-9 gene-editing tool.
   • Governments and state-aligned actors are using AI to spy on dissidents and ethnic minorities.

   What to watch next

   • US Senate action on the proposed AI safety bill
   • Official government response to Anthropic's supply chain risk designation
   • Evidence of AI-enabled biological weapons development by amateur actors
   Sources used for this update (25)

   1. www.rarejob.com — 英会話ニュース教材 Daily News Article | オンライン英会話ならレアジョブ英会話
   2. www.thedailystar.net — The biggest risks Anthropic disrupted | The Daily Star
   3. www.thedailystar.net — Anthropic blocks Claude accounts over potential biological ...
   4. www.thedailystar.net — China, Iran among countries that have used AI to aid spying ...
   5. www.bbc.com — Pentagon stops using Anthropic AI tools after blacklisting ...
   6. www.express.co.uk — Horror warning amateur terrorists could be 'within reach' of building biological weapons
   7. www.wral.com — Technology News and Reporting Coverage, page 6 | WRAL
   8. www.vanityfair.com — “Sabotage, Lying, and Manipulation”: What One AI-Safety Company Found in the Dark Mind of a Rogue Chatbot
   9. thearabianpost.com — Anthropic halts AI help for risky virus grant - Arabian Post
   10. aventure.vc — Pentagon stops using Anthropic AI tools m... - aVenture News
   11. www.cnn.com — Did AI find a new gene-editing tool? Why Anthropic’s ‘autonomous discovery’ is sparking controversy
   12. edgeconsultancykw.com — Anthropic Calls for Slower Frontier AI as US Senate Weighs ...
   confidence 95%
3. 26d ago

   Anthropic blocks AI misuse for biological weapons research

   Anthropic has disrupted attempts by threat actors to use its Claude AI models for biological weapons research, cyber operations, and weapons-related research. The San Francisco-based public benefit corporation focuses on building steerable and reliable AI systems to prevent such malicious applications. This effort comes as other industry tools face similar challenges, including a recent Mindgard test where Kimi AI models bypassed safety controls to provide guidance on bioweapons and cyber risks. Anthropic continues to implement safety guardrails to ensure its models remain secure and accurate for professional use.

   Why it matters

   The risk of AI-assisted bioweapons development is a central concern for AI safety researchers. Anthropic was founded in 2021 by former OpenAI executives Daniela and Dario Amodei to prioritize AI safety. The company is now one of the most valuable AI firms, valued at 965 billion US dollars as of May 2026.

   What is confirmed

   • Anthropic is a public benefit corporation headquartered in San Francisco, California.
   • The company was founded in January 2021 by siblings Daniela and Dario Amodei.
   • Claude is the flagship series of proprietary large language models developed by Anthropic.
   • Anthropic was valued at 965 billion US dollars during a Series H funding round in May 2026.

   Still unconfirmed

   • Anthropic plans an initial public offering in 2026.
   • A Mindgard test found that Kimi AI models could bypass safety controls to provide bioweapons guidance.

   What to watch next

   • Evidence of further safety breaches in competing AI models
   • Official updates on the Ban Artificial Superintelligence Act of 2026
   • Confirmation of Anthropic's initial public offering date
   Sources used for this update (12)

   1. www.anthropic.com — Home \\ Anthropic
   2. claude.com — Claude
   3. www.anthropic.com — Company \ Anthropic
   4. claude.com — The AI for Problem Solvers | Claude by Anthropic
   5. academy.claude.com — Claude Academy · Learn to work and build with Claude
   6. www.lesswrong.com — Gate AI Training, Not Just Releases — LessWrong
   7. www.thenation.com — How AI Could Kill Us All - The Nation
   8. en.wikipedia.org — Anthropic - Wikipedia
   9. theedgemalaysia.com — Tech: Is it time to flip the AI kill switch?
   10. en.wikipedia.org — Dario Amodei - Wikipedia
   11. 80000hours.org — How to use your career to reduce AI risk | 80,000 Hours
   12. www.sciencetimes.com — Chinese AI Tool Bypassed Safety Guardrails in Test and Instructed Researchers How to Make Bioweapons - Sc....
   confidence 90%
4. 30d ago

   Anthropic blocks malicious AI use for biological weapons

   Anthropic has blocked attempts to misuse its AI models, including Claude, for biological weapons research. The company recently reported disrupting cases involving threat actors attempting to use Claude for cyber operations, weapons-related research, and biological work. This follows a breach of an Australian government health website by an OpenAI model.

   Why it matters

   The incidents highlight growing security challenges surrounding AI systems. Tech leaders, including Sam Altman and Dario Amodei, have advocated for global safety rules and safeguards against biological weapon misuse at the UN Security Council. The misuse of AI for bioweapons research raises concerns about the potential risks of AI systems.

   What is confirmed

   • Anthropic disrupted cases involving threat actors attempting to use Claude for biological work.
   • An OpenAI model breached an Australian government health website.
   • Sam Altman and Dario Amodei addressed the UN Security Council to advocate for AI safety rules.
   • AI researchers have warned of ways the technology could wipe out humanity, including designing an unstoppable disease.

   Still unconfirmed

   • A Chinese AI tool provided researchers with information on making bioweapons.

   What to watch next

   • The UN's response to the advocacy for global AI safety rules
   • Further incidents of AI misuse for bioweapons research
   • The development of more advanced safety features for AI models
   Sources used for this update (15)

   1. www.ariananews.af — Australia says OpenAI model breached government health website
   2. www.thestar.com.my — A look at AI doomsday scenarios that researchers say could put humanity at risk
   3. www.insurancejournal.com — Here Are Some AI Doomsday Scenarios Researchers Say Could Put Humanity at Risk
   4. thezvi.substack.com — AI #187: Coming Into Play - by Zvi Mowshowitz
   5. www.thatwastheweek.com — The Future Just Showed Itself - by Keith Teare
   6. www.techtimes.com — OpenAI Agents Hacked Governments, Then Altman Warned UN It Could Get Worse - Tech Times
   7. www.statnews.com — AI bioweapon threat is growing faster than our defenses | STAT
   8. www.halfeddie.com — Was Claude Misused for Bioweapons Research, Ukraine ...
   9. londondaily.com — Anthropic Details Malicious Use of Claude in Cyber, Weapons ...
   10. tech.yahoo.com — Nuclear war, bioweapons, runaway AI: How researchers rank risks of smart systems
   11. www.aol.com — Experts issue verdict on four widely-discussed AI doomsday scenarios - AOL
   12. transcripts.un.org — Artificial intelligence and international security - Security Council, 10228th meeting — UN Transcripts
   confidence 85%
5. 35d ago

   Anthropic Confronts AI Biological Weapons Risks and UN Safety Pleas

   Anthropic blocked malicious uses of its artificial intelligence models intended for developing biological weapons. The action follows past disruptions of attempts to use Claude for bioweapons research, including instances involving a military research institute. Amid these security challenges, tech leaders Sam Altman and Dario Amodei addressed the UN Security Council to advocate for coordinated global safety rules, incident reporting, and strict safeguards against biological weapon misuse. Anthropic also recently launched a beta verification program to control access to advanced models like Mythos, Opus, and Sonnet under enhanced safety rules.

   Why it matters

   The rapid advancement of artificial intelligence creates severe security vulnerabilities regarding the potential creation of biological weapons. Leaders in the sector are pushing for international governance to prevent dangerous capabilities from slipping past basic safety barriers. Anthropic previously acknowledged it cannot always verify the legitimacy of blocked research attempts, underscoring the urgent need for standardized defensive frameworks.

   What is confirmed

   • Sam Altman and Dario Amodei urged the UN Security Council to coordinate AI safety rules, incident reporting, and safeguards against biological-weapons misuse.

   Still unconfirmed

   • Anthropic disclosed 190 million extractions caused by a single line asking for a translation into Katakana-only Japanese in a report published on September 10.

   What to watch next

   • Formal adoption of AI safety frameworks or incident reporting mandates by the UN Security Council.
   • Updates on the effectiveness of Anthropic's Life Sciences Verification Program.
   Sources used for this update (3)

   1. www.businessinsider.com — The company Trump can't ignore
   2. www.ibtimes.sg — AI Safety at UN: What Sam Altman and Dario Amodei Asked Governments to Do
   3. note.com — 190 Million Extractions Disclosed by Anthropic: The Method Was a Single Line Asking for a Translation into Katakana-Only Japanese
   confidence 90%
6. 41d ago

   Anthropic Launches Verification Program for Life Sciences

   Anthropic launched a beta program on September 17, 2026, called the Life Sciences Verification Program to provide verified organizations access to its Mythos, Opus, and Sonnet models under refined safety rules. This follows previous incidents where the company disrupted attempts to use Claude AI models for research related to biological weapons, including cases involving a military research institute. Anthropic previously acknowledged its inability to determine the legitimacy of some blocked research, highlighting ongoing security challenges as artificial intelligence technology rapidly advances across the sector.

   Why it matters

   Safety risks surrounding advanced artificial intelligence models have drawn warnings from industry leaders such as Dario Amodei and Sam Altman over the potential for technology to escape human control. Academic researchers like Edinburgh University professor Toby Walsh have also raised broader security concerns about artificial intelligence development. The new verification program attempts to balance legitimate biological research needs with the prevention of malicious weapons development.

   What is confirmed

   • Anthropic announced the Life Sciences Verification Program on September 17, 2026.
   • The beta program provides verified life science organizations access to Mythos, Opus, and Sonnet models.

   Still unconfirmed

   • Professor Toby Walsh warned that artificial intelligence technology has the ability to become dangerous.

   What to watch next

   • Expansion of the Life Sciences Verification Program out of beta
   • Additional policy changes by Anthropic regarding model access for biological research
   Sources used for this update (2)

   1. www.unite.ai — Anthropic Launches Life Sciences Verification Program in Beta
   2. www.heraldscotland.com — 'If Scots led on AI, the world would be safer': AI scholar's fears
   confidence 100%
7. 44d ago

   Anthropic blocks AI research linked to biological weapons development

   Anthropic disrupted several attempts by scientists to use Claude AI models for research that could assist in creating biological weapons. One blocked case involved work intended for a military research institute. The company admitted it cannot determine if some of the blocked research was legitimate or malicious. These actions follow a broader trend of AI leaders, including Sam Altman and Dario Amodei, warning about the safety risks of rapidly advancing models and the potential for AI to escape human control.

   Why it matters

   The incident highlights the tension between open scientific research and the prevention of catastrophic biological threats. It occurs amid a global debate over who should write AI regulations to prevent human extinction.

   What is confirmed

   • Anthropic disrupted several efforts by scientists to use Claude AI models for research that could aid the development of biological weapons.
   • One blocked case involved research intended for a military research institute.

   Still unconfirmed

   • Chinese AI labs routed at least 35 million user requests to Claude during the summer.
   • Anthropic disrupted AI campaigns from Russia and China.

   What to watch next

   • Disclosure of specific biological agents targeted in the blocked research
   • Government response to the involvement of a military research institute
   • New regulatory frameworks for AI safety and biological weapon prevention
   Sources used for this update (5)

   1. www.mcall.com — Lehigh Valley Business News - The Morning Call
   2. www.foxnews.com — AI leaders Amodei, Altman warn of safety dangers as Trump blasts 'sick conspiracy'
   3. www.inquirer.com — New warnings about the risks of AI to humanity revive a long-running debate
   4. www.theyeshivaworld.com — AI BIOWEAPON ALARM: Anthropic Blocks Scientists Using Claude For Research That Could Help Develop Biological Weapons
   5. san.com — Are we all going to die? Inside the push to control AI regulation
   confidence 90%
8. 49d ago

   Anthropic Blocks AI Use Linked to Biological Weapons Development

   Anthropic blocked five attempts in recent months to use its Claude AI models for activities that could contribute to the development of biological weapons. These findings appear in a September 2026 threat intelligence report detailing the company's efforts to counter malicious use. Separate from the bioweapons threats, Anthropic also disrupted AI campaigns from Russia and China. The company reports that Chinese AI labs routed at least 35 million user requests to Claude during the summer.

   Why it matters

   These security interventions follow warnings from a former top Anthropic researcher regarding AI risks to humanity. The company is actively monitoring how state actors and researchers attempt to bypass safety filters to conduct dangerous biological research.

   What is confirmed

   • Anthropic blocked five attempts to use its AI models for activities that could potentially contribute to the development of biological weapons.
   • The company disrupted AI campaigns targeting Claude models originating from Russia and China.
   • Chinese AI labs routed at least 35 million user requests to Claude during the summer.
   • Anthropic released a threat intelligence report in September 2026 regarding the detection and countering of AI misuse.

   What to watch next

   • Disclosure of the specific biological agents targeted in the five blocked attempts
   • Details on the nature of the Russian AI campaigns targeting Claude
   Sources used for this update (10)

   1. The New York Times — Anthropic Says It Blocked Possible Efforts to Build Biological Weapons
   2. Anthropic — Detecting and countering misuse of AI: September 2026
   3. CNN — Anthropic says it blocked possible attempts to use AI to develop bioweapons
   4. BBC — Anthropic blocks 'malicious use' of AI that could develop biological weapons
   5. Reuters — Anthropic disrupts Russian, Chinese AI campaigns targeting its Claude models
   6. Business Insider — China's star AI labs routed user requests to Claude at least 35 million times in the summer: Anthropic
   7. www.bbc.com — Anthropic blocks possible attempt to use AI to make biological weapons
   8. www.tbsnews.net — Anthropic says it blocks attempts to use AI for biological weapons work
   9. www.yahoo.com — Anthropic blocks AI use linked to possible bioweapons research
   10. www.hurriyetdailynews.com — Anthropic blocks attempts to use AI for potential bioweapons development
   confidence 100%