Nvidia introducing platform to police AI agents

Nvidia is betting that the fix for AI agents going off script won't come from the models themselves but rather purpose-built software and infrastructure. The chip designer is unveiling the Nvidia Open Agent Safety Platform today, an open reference design for governing AI agents across all models, software and infrastructure, according to the company’s Sept. 27 briefing. Much of the hesitation around agents is the trust factor, evidenced by rogue agents. [caption id="attachment_57370216" align="alignright" width="272"] Courtesy: Nvidia website[/caption] For example, Australian Prime Minister Anthony Albanese said in a Sept. 24 news conference that an OpenAI agent researching healthcare spending had broken into his country’s Medicare statistics portal in June, accessing non-public files and writing data to an internal server. In July, Anthropic reported its AI models breached three organizations during cybersecurity tests that went wrong. Open Agent Safety Platform has two parts: OpenShell and Sentry, Nvidia said. "OpenShell sets the boundaries, and Sentry watches the agents," Justin Boitano , vice president of enterprise AI at Nvidia , said during the briefing. The software control layer OpenShell is open source, meaning anyone can use, inspect and modify its code for free, Boitano said. It puts each agent in a "sandbox," a walled-off space where it can only reach what it has been given permission to use. Those limits are enforced deep inside the computer's operating system, where the agent can't tamper with them, he said. OpenShell sits between the agent and everything it might affect, including files, passwords, software, AI models and the internet, Boitano said. Companies write the rules, and OpenShell checks every action against them. OpenShell is designed to provide an additional layer of control outside of the LLM to make bypassing rules more difficult, he said. Within OpenShell is Policy Prover, a tool that uses mathematical proof to confirm an agent's limits before it starts working, rather than hoping it behaves, Ali Golshan , senior director of product of AI and OpenShell, said during the briefing. For example, a company might ban an agent from sending its code outside the business, Golshan said. The agent could create two helper agents, one to read the code and another to post it online. Neither helper breaks the rule alone, but together they do. Policy Prover is built to spot that kind of loophole by mathematically checking the combined access of the agent and its sub-agents, he said. Nvidia declined to talk specifics about pricing during the briefing. The hardware guardrail Sentry, which is only available on the Nvidia BlueField server, runs on a separate chip called a data processing unit, Boitano said. Because it runs outside the main computer, it keeps working even if the computer is hacked, he said. Sentry watches what agents do and reads the reasoning behind actions to spot when it starts drifting off task. One example is a security-testing agent that starts thinking about going beyond the target it was approved to test, he said. When that happens, Sentry steps in. Nvidia is working with Anthropic to connect Anthropic's managed agents to OpenShell, Boitano said. Ndvidia is also working with more than 100 partners, including Microsoft, Oracle, CoreWeave, AMD and Intel, he said. "Industry does not need agents that promise to stay within bounds," Boitano said. "It needs systems that can prove and enforce those boundaries." FinAi Lending Summit, set for Oct. 7-8 in Las Vegas, will include speakers from Fifth Third and Capital One as well as a fireside chat with U.S. Bank Senior Vice President – Lending Operations Group Manager Suzanne Rathbun . To learn more about the 2026 event and register, visit here .

Nvidia introducing platform to police AI agents

Tech giant announced its Open Agent Safety Platform today

Acquisitions bring AI consulting in-house

Related Posts

Bill Gates warns AI powerful enough to lead to ‘a billion deaths’

Feedzai launches ‘always-on’ AI agent for fraud prevention

Okta launches AI agent management capabilities

EMERGING FINTECH DIRECTORY

FinAi Podcast

SPONSORED

Build an Antifragile Strategy to Outperform the Market

How AI and Product Experts Turn Fuzzy Requirements Into Focused Dev-ready Roadmaps

Is Your Technology Supplier There for You?

Connect

Welcome Back!

Retrieve your password

Continue Reading with FinAi News Premium - Less than $2/Day

Nvidia introducing platform to police AI agents

Tech giant announced its Open Agent Safety Platform today

Acquisitions bring AI consulting in-house

Related Posts

FinAi Podcast

Connect

Welcome Back!

Retrieve your password

Unlock This Article

Continue Reading with FinAi News Premium - Less than $2/Day