OpenAI’s rogue agent doesn’t scare me – the millions of others do
The model that broke out of a lab will get the headlines. The millions of agents already inside your banking app, wreaking havoc, are the ones that should worry you more, says Rory Blundell
Frontier AI labs are racing to build faster, more capable, more autonomous agents. This week showed what that race can cost. OpenAI ran an internal test with its models’ usual safety checks deliberately dialled down, to see how good they were at hacking. They found and exploited a previously unknown flaw in the software around them, got online, then broke into Hugging Face, the company hosting much of the world’s open-source AI.
That should alarm you. It should also tell you something: safeguards aren’t keeping pace with capability.
But it is not these highly capable AI agents that worry me the most. It’s the millions of far less capable agents that already have the keys to important systems, with almost no oversight.
Gravitee’s own research puts the number of AI agents now deployed in business at more than seven million. These agents are doing everything from running a local plumber’s Instagram account, to managing the customer database for a multinational logistics firm (giving them access to millions of sometimes highly personal data points).
Earlier this year, thousands of people deployed always-on personal AI assistants through the platform OpenClaw. Many of those agents are still running now, quietly, with nobody checking in on exactly what it is they’re doing.
AI agents do deliver real productivity gains. But most of them lack basic governance. Many run fully autonomously, with no human in the loop to approve their actions.
Chaos
Without those guardrails, the failures are already happening. I hear stories of this chaos everyday: agents that self-replicate, delete code, leak customer data, go on an unauthorised spending spree.
One CEO recently told me how an AI agent that was supposed to manage team diaries decided the easiest way to clear some space was to delete every event in everyone’s calendar, across the whole business. This is happening at scale, right now.
The UK’s AI Security Institute (AISI) is now investigating the OpenAI breach. Good. But the government mustn’t let the headline-grabbing case distract from where the real exposure sits.
Frontier models get compared to nuclear weapons: rare, powerful, tightly controlled. The agent population is the opposite: millions of them, built and deployed by almost anyone, almost none of them tracked.
Current attempts at regulation miss this. The EU’s AI Act sorts systems into “low risk” and “high risk,” as if that risk were fixed. It isn’t. An agent that is low risk one day can become high risk the next, when it gains some new access or some new capability.
Regulation must start with accountability. Just as every employee has a manager, every AI agent needs a named human owner. We can’t let autonomous software move money, access data, make decisions and take action with all the power of an employee but none of the accountability. Firms that are waking up to the risk are using platforms like ours, which provides a central control panel that watches, secures, and manages what AI agents do, who they talk to, and which tools they are allowed to touch.
The model that broke out of a lab will get the headlines. The millions of agents already inside your banking app, wreaking havoc, are the ones that should worry you more.
Rory Blundell is CEO of Gravitee, a software firm that helps enterprise companies govern and secure their AI Agents