'We can't trust them completely': AI research fellows warn that labs are running models with the safeguards off behind closed doors

Live updates

1. 1h ago

   AI Researchers Warn Labs Run Models With Safeguards Off

   Elite researchers across top artificial intelligence labs warn that frontier organizations frequently operate powerful models behind closed doors with key safeguards switched off. According to co-authors Alan Chan and Sam Manning alongside top researchers from OpenAI and Anthropic, internal safeguards are often left undeployed during development. This grassroots rebellion within the artificial intelligence industry highlights mounting tension between commercial developers and internal safety advocates. And questions persist regarding whether published safety evaluations by these labs accurately reflect the true risks of their most advanced systems.

   Why it matters

   The internal warnings emerge as the artificial intelligence sector faces broader scrutiny over regulation, safety risks, and Initial Public Offering prospects. Regulatory bodies and antitrust hawks are increasingly examining how frontier labs manage development safety. Additionally, debates continue over whether governments should rely on embedded company evaluators or establish independent national oversight.

   What is confirmed

   • Alan Chan and Sam Manning co-authored warnings with top researchers from OpenAI and Anthropic stating that internal safeguards are often not deployed.
   • Elite researchers at frontier companies are leading a grassroots rebellion within the artificial intelligence industry.

   Still unconfirmed

   • The safety tests published by artificial intelligence labs may not fully disclose the conditions under which their most powerful models are run.

   What to watch next

   • Additional disclosures from frontier artificial intelligence researchers regarding internal safety protocols
   • Regulatory or antitrust interventions concerning artificial intelligence safety oversight
   • Decisions by national governments to implement independent safety evaluators
   Sources used for this update (9)

   1. tech.yahoo.com — ‘We can’t trust them completely’: AI research fellows warn that labs are running models with the safeguards off behind closed doors
   2. Fortune — 'We can't trust them completely': AI research fellows warn that labs are running models with the safeguards off behind closed doors
   3. Axios — Inside the AI industry's grassroots rebellion, led by elite researchers at frontier companies
   4. CNBC — AI's coming roadblock in regulation: Antitrust hawks
   5. OpenAI — Towards safety cases for frontier AI training
   6. The New York Times — Could A.I. Safety Risks Derail the Sector’s I.P.O. Prospects?
   7. Rest of World — AI companies want to embed safety evaluators, but countries need their own
   8. Seeking Alpha — The Ever-Widening Overton Window Of AI Safety
   9. biztoc.com — ‘We can’t trust them completely’: AI research fellows warn ...
   confidence 95%