‘Gambling with our lives’: Anthropic researcher quits, warns against self-improving AI

Live updates

1. 4d ago

   US Lawmakers Seek AI Regulation After Anthropic Researcher Warning

   Members of Congress are calling for new AI regulations following the resignation of Jacob Coxon, a former Anthropic and OpenAI researcher. Coxon claims leading labs are gambling with human lives by racing toward self-improving systems they cannot control. While AI insiders warn that development is outrunning safety efforts, some companies continue their current trajectory despite fears that the technology could lead to human extinction. OpenAI is reportedly evaluating a slowdown in advanced AI development, though competition and regulatory hurdles complicate a multi-lab agreement.

   Why it matters

   The tension centers on whether AI labs can safely manage superintelligent systems that improve themselves without human intervention. Coxon's departure highlights a growing rift between commercial acceleration and existential safety concerns. This debate now extends from internal corporate disputes to the US legislative branch.

   What is confirmed

   • Jacob Coxon previously worked for both OpenAI and Anthropic.
   • Coxon warns that AI labs are building self-improving systems that could threaten humanity.
   • AI insiders state the race toward self-improving systems is moving faster than control efforts.

   Still unconfirmed

   • OpenAI is considering slowing the development of advanced AI.
   • METR CEO Beth Barnes states that AI safety research faces a talent bottleneck despite salaries of $500,000.

   What to watch next

   • Official legislative proposals for AI regulation in the US Congress
   • Public responses or pacing agreements signed between OpenAI and Anthropic
   Sources used for this update (6)

   1. www.businessinsider.com — At an ex-OpenAI researcher's influential lab, $500,000 salaries aren't enough to fix a talent 'bottleneck'
   2. www.cnet.com — What an Ex-Anthropic Researcher’s Warning About Human Extinction Really Means
   3. www.zmescience.com — Anthropic Researcher Quits and Warns AI Labs Are ‘Gambling With Our Lives’ Building Self-Improving Systems
   4. anewz.tv — Anthropic researcher quits, warning AI 'could kill us all'
   5. www.cnbc.com — AI regulation calls grow in D.C. after researcher's extinction warning
   6. www.ai4business.it — Perché OpenAI considera una frenata nello sviluppo dell’AI
   confidence 90%
2. 6d ago

   Anthropic Researcher Quits Over AI Extinction Fears

   Jacob Coxon, a 27-year-old researcher at Anthropic and former OpenAI employee, has resigned over safety concerns regarding advanced artificial intelligence. Coxon warned that leading technology labs are rushing recklessly toward uncontrollable superintelligent systems, accusing the industry of gambling with human lives. Writing on social media, Coxon called for pacing agreements between labs to prevent catastrophic outcomes. His departure highlights internal tensions within major artificial intelligence companies over rapid acceleration and the existential risks posed by unaligned, self-improving technology over the coming decade.

   Why it matters

   The resignation adds to a growing wave of safety-related departures across major artificial intelligence firms as researchers clash with corporate leadership over development speed. Leading developers continue to scale up foundational models despite mounting warnings from internal personnel about potential extinction-level threats. These ongoing disputes focus heavily on self-improving systems that could soon surpass human control.

   What is confirmed

   • Jacob Coxon resigned from Anthropic after spending three years building foundational models at OpenAI and Anthropic.
   • Coxon warned that leading industry labs are rushing recklessly toward uncontrollable superintelligent systems.
   • Coxon wrote on X that the teams building artificial intelligence earnestly believe it could kill us all by the end of the decade.
   • Coxon called for pacing agreements between artificial intelligence laboratories.

   Still unconfirmed

   • Artificial intelligence could reach an uncontrollable state by late next year.

   What to watch next

   • Whether other safety researchers resign from Anthropic or competing labs over acceleration concerns
   • Any formal response from Anthropic management regarding Coxon's warnings and pacing agreements
   Sources used for this update (13)

   1. WSJ — Exclusive | Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears
   2. CNBC — Anthropic researcher says AI has more than 10% chance of 'killing all humans' after colleague quits
   3. CNN — ‘Gambling with our lives’: Another AI employee quits over safety concerns
   4. Fox Business — Anthropic researcher says AI has over 10% chance to 'kill all humans' within next decade
   5. www.ocregister.com — Anthropic worker quits, says AI firms ‘gambling with our lives’
   6. TechCrunch — ‘Gambling with our lives’: Anthropic researcher quits, warns against self-improving AI
   7. The Guardian — Anthropic researchers panic, say AI could cause human extinction by 2030
   8. nytimes.com — Anthropic Researchers Raise Alarm Over A.I. Acceleration, Warning of Threat to Humanity
   9. techcrunch.com — ‘Gambling with our lives’: Anthropic researcher quits, warns against self-improving AI
   10. qazinform.com — Anthropic researcher quits over ‘irresponsible’ AI race, warns of potential risks
   11. hothardware.com — Anthropic Safety Researcher Quits, Warns AI Could Kill All Humans By 2036
   12. finance.biggo.com — 27-Year-Old Anthropic Researcher Quits, Warns AI Could Become Uncontrollable by Late Next Year
   confidence 95%