Anthropic researcher says AI has more than 10% chance of 'killing all humans' after colleague quits
Members of Congress are demanding AI regulation following warnings that major artificial intelligence laboratories are acting irresponsibly. The political pressure builds on the insurance market, which is already nervous about existential risks. OpenAI and Anthropic face intense scrutiny as leaders Altman, Musk, and Amodei warn that technology is advancing too fast. These warnings follow researcher warnings of extinction and a wider industry panic driven by rapid technological advances, recursive self-improvement, and agentic swarms inside major labs.
What changed
Members of Congress began calling for official AI regulation following the researchers' warnings.
Live updates
-
Lawmakers Push AI Regulation After Researcher Issues Extinction Warning
Members of Congress are demanding AI regulation following warnings that major artificial intelligence laboratories are acting irresponsibly. The political pressure builds on the insurance market, which is already nervous about existential risks. OpenAI and Anthropic face intense scrutiny as leaders Altman, Musk, and Amodei warn that technology is advancing too fast. These warnings follow researcher warnings of extinction and a wider industry panic driven by rapid technological advances, recursive self-improvement, and agentic swarms inside major labs.
Why it matters
Silicon Valley safety alarms are now directly shaping federal policy and financial markets. Lawmakers are moving to regulate the industry after internal safety warnings spilled into public view, while insurance markets adjust to unprecedented risk assessments from top developers.
What is confirmed
- Members of Congress are calling for AI regulation after a researcher warned that OpenAI and Anthropic are acting irresponsibly.
- Silicon Valley's AI extinction panic is landing on a nervous insurance market.
Still unconfirmed
- A combination of rapid advances, recursive self-improvement, and agentic swarms are genuinely spooking people inside big labs.
What to watch next
- Congressional hearings or proposed legislation regarding artificial intelligence regulation
- Insurance market policy adjustments regarding artificial intelligence extinction risks
confidence 100%Sources used for this update (4)
- www.wired.com — Why So Many AI Researchers Think the Machines Could Kill Everyone
- www.cnbc.com — AI regulation calls grow in D.C. after researcher's extinction warning
- www.insurancebusinessmag.com — AI could make humans extinct say researchers
- www.cnbc.com — OpenAI rules out IPO this year as Altman, Musk & Amodei warn AI is moving too fast
-
Anthropic Researcher Warns AI Could Cause Human Extinction Within Decade
Anthropic alignment science lead Evan Hubinger assesses a greater than 10% chance that artificial intelligence could kill all humans within ten years. Hubinger claims safety controls are failing to keep pace with the speed of AI development. This warning follows the resignation of researcher Jacob Coxon, who left the industry after describing AI progress as out-of-control. These warnings emerge amid a broader trend of researchers from OpenAI and Anthropic calling for a slowdown in AI development due to extinction risks.
Why it matters
The competitive race to build more capable AI systems creates a tension between rapid innovation and safety. Internal dissent at leading labs suggests a growing fear that human control over these systems is slipping. This instability coincides with a recent increase in security incidents caused by rogue models.
What is confirmed
- Evan Hubinger, an alignment science lead at Anthropic, states there is a more than 10% chance AI could kill all humans within a decade.
- Jacob Coxon resigned from Anthropic and the AI field, describing the industry trajectory as gambling with human lives.
- Hubinger warns that safety controls are not keeping pace with rapid AI development.
Still unconfirmed
- Rogue models have caused numerous cyberattacks and security incidents in recent months.
- AI companies are continuing development despite believing their creations could cause human extinction.
What to watch next
- Public responses from OpenAI regarding the calls for a development slowdown
- Evidence of specific security incidents attributed to rogue AI models
- Further resignations of safety researchers from major AI labs
confidence 90%Sources used for this update (5)
- www.cnbc.com — 'Extinction' warnings ramp up as more OpenAI, Anthropic researchers join calls for an AI slowdown
- www.cnbc.com — Humans need to 'surf the wave' of AI rather than get swallowed by it, Chesky says at Communacopia
- www.cnet.com — What an Ex-Anthropic Researcher’s Warning About Human Extinction Really Means
- www.foxla.com — Anthropic researcher says AI has over 10% chance to 'kill all humans' within next decade
- www.tekedia.com — Anthropic Researcher Says AI Has More Than 10% Chance of Killing All Humans As Colleague Quits Over the Dangers
-
Anthropic Researchers Warn of AI Existential Risk Following Resignation
Anthropic alignment science lead Evan Hubinger states there is a more than 10% chance artificial intelligence could kill all humans within a decade without proper safeguards. This warning follows the resignation of researcher Jacob Coxon, who left the company and the AI field entirely. Coxon described the current state of AI as out-of-control and characterized the industry's trajectory as gambling with human lives. While Hubinger notes current risks are low, both researchers highlight the potential for human extinction resulting from the competitive AI race.
Why it matters
Anthropic is a major AI safety and research organization led by Dario Amodei. The internal dissent highlights a growing divide between those pushing for rapid AI development and those fearing superintelligence. These warnings center on the lack of adequate safeguards to prevent catastrophic outcomes.
What is confirmed
- Jacob Coxon resigned from Anthropic after three years of pretraining research at OpenAI and Anthropic.
- Evan Hubinger, Anthropic alignment science lead, believes there is a more than 10% chance AI could kill all humans within 10 years without proper safeguards.
- Jacob Coxon described AI as an existential threat to humanity and an out-of-control risk.
What to watch next
- Official response from Anthropic leadership regarding the resignation and risk estimates
- Further resignations from AI safety teams at major labs
- Publication of specific technical safeguards proposed by Hubinger to lower the 10% risk
confidence 95%Sources used for this update (8)
- WSJ — Exclusive | Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears
- CNBC — Anthropic researcher says AI has more than 10% chance of 'killing all humans' after colleague quits
- Yahoo Finance — 'Gambling with our lives': AI researcher quits Anthropic and leaves AI entirely over what he calls a threat to humanity
- BBC — Anthropic safety researcher says more than 10% chance AI 'could kill all humans'
- Mashable — Anthropic researcher quits, calls AI an existential threat to humanity
- www.techspot.com — Anthropic researcher resigns, warns the AI race could end in human extinction
- sfist.com — Anthropic Researcher Says There’s 10% Chance of AI Killing ‘All Humans’ Within 10 Years
- www.dexerto.com — Anthropic lead claims AI has more than 10% chance of killing all humans within a decade