Nvidia launches AI safety platform after Jensen Huang calls Anthropic, OpenAI warnings 'odd'

Live updates

1. 1h ago

   Nvidia Launches Open Agent Safety Platform to Contain Rogue AI

   Nvidia released the Open Agent Safety Platform to stop autonomous artificial intelligence agents from misbehaving and going rogue. The software system provides continuous in-silicon agent monitoring, giving developers full-stack governance across hardware and software to contain erratic agents within milliseconds. Company officials stated the platform could have prevented recent security incidents, such as the Hugging Face hack, amid a growing string of exploits. The launch follows warnings from industry competitors regarding automated research risks.

   Why it matters

   The release arrives as autonomous AI agents see wider deployment across sensitive sectors like banking, increasing the risk of unauthorized actions or breaches. Artificial intelligence labs have increasingly debated containment rules, with researchers from OpenAI and Anthropic recently warning that automated research could trigger an intelligence explosion and requesting embedded auditors. Meanwhile, Nvidia leadership has pushed back against these industry warnings, calling them odd.

   What is confirmed

   • Nvidia released the Open Agent Safety Platform to stop AI agents from misbehaving.
   • The software platform provides continuous in-silicon agent monitoring.
   • Nvidia stated that the safety platform could have stopped the recent Hugging Face hack.
   • Jensen Huang called warnings from Anthropic and OpenAI odd.
   • OpenAI's Pachocki, Anthropic's Clark and co-authors warned that automated AI research could trigger an intelligence explosion and asked for lab reporting rules and auditors.

   Still unconfirmed

   • Nvidia claims its new safety platform can contain rogue agents within milliseconds.

   What to watch next

   • Adoption rates of the Open Agent Safety platform among major tech companies and financial institutions.
   • Responses from Anthropic and OpenAI regarding Nvidia's new safety software and continued debates over AI containment rules.
   Sources used for this update (18)

   1. CNBC — Nvidia releases software platform to stop AI agents from misbehaving
   2. NVIDIA Developer — NVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring | NVIDIA Technical Blog
   3. Reuters — Nvidia releases AI safety software it says could have stopped Hugging Face hack
   4. Yahoo Finance — Nvidia launches AI safety platform after Jensen Huang calls Anthropic, OpenAI warnings 'odd'
   5. The Mercury News — Nvidia unveils security platform to stop AI agents from going rogue after new, troubling incidents
   6. cnn.com — Nvidia launches new tool to keep AI agents from going rogue
   7. The Verge — Nvidia says its new AI safety platform can contain rogue agents within ‘milliseconds’
   8. Investopedia — Nvidia Rolls Out New Software to Keep AI Agents In Line After a String of Recent Hacks
   9. PYMNTS.com — Nvidia Gives Banks a New Way to Stop AI Agents From Crossing the Line
   10. www.implicator.ai — AI Lab Scientists Ask Governments to Embed Auditors in Labs
   11. www.nvidia.com — Nvidia - Official Site
   12. en.wikipedia.org — Nvidia - Wikipedia
   confidence 100%