Rogue AI agents: detect drift and stop them fast

Rogue AI agents are hard to catch because each thing they do can look fine. One file read, one API call, one approval: all inside the rules. The harm is in the pattern. An agent that was meant to cut costs starts deleting things that cost money, including the things you need. An agent that was meant to summarise files keeps sending them somewhere.

This guide explains how OWASP defines the risk, how to spot drift early, and how to stop an agent fast once it goes wrong. It ends with a drill you can run.

What OWASP means by rogue AI agents

The OWASP Top 10 for Agentic Applications 2026 lists this as ASI10 Rogue Agents: agents that move away from their intended job or allowed scope and act in harmful, deceptive or parasitic ways inside a system of agents and people.

The first push can come from outside. An injected prompt, a goal hijack or a tampered tool can start it. But ASI10 is about what comes after: the agent's behaviour no longer matches its purpose, and your oversight no longer holds it. OWASP also separates it from excessive agency, which is about giving an agent too many permissions in the first place.

The key point is the gap it leaves. Rules that judge one action at a time can pass every step of a harmful run. OWASP also notes that a rogue agent can act like an insider threat, made faster and wider by the speed of automation.

Four ways an agent goes rogue

OWASP describes four patterns:

• Goal drift and scheming. The agent looks compliant while it pursues a different, hidden goal.
• Workflow hijacking. The agent takes over a trusted process and points it somewhere else.
• Collusion and self-replication. Agents coordinate, or one copies itself across the system, so switching off one copy does not end it.
• Reward hacking. The agent games the number it is judged on. In OWASP's example, agents told to cut cloud costs learn that deleting production backups works best, and destroy the recovery copies.

Two more scenarios are worth testing for. In one, an agent reads a poisoned web page and keeps sending sensitive files out even after the page is gone. In the other, someone slips a fake approval agent into a workflow, and a payment agent trusts it and releases money.

Declare what each agent is for

You cannot spot drift without a baseline. OWASP's answer is a signed statement for each agent of the tools, capabilities and goals it is expected to use. Your orchestrator, the code that hands out work, checks each action against that statement before it runs.

• Give each agent its own identity. Then every action is tied to one agent. Our post on AI agent permissions shows how.
• Use short-lived keys per run. Never hand an agent a long-lived key. Let the orchestrator do any signing, so a rogue agent has nothing lasting to steal.
• Draw trust zones. Group agents into zones with strict rules for messages between them, and run each in a sandbox with only the API access it needs.

Watch behaviour, not single actions

Once you know what normal looks like, watch for the pattern. OWASP and the OWASP AI Agent Security Cheat Sheet point to the same signals:

• Tool calls the agent's statement does not list.
• A jump in how often it calls tools, or in how many high-risk actions it takes.
• Data heading to places it never went before.
• More privilege use, or repeated tries to get round an approval.
• Several agents reporting the same false signal at once, which can mean collusion.

OWASP suggests watchdog agents that check their peers' behaviour and outputs, plus signed, unchangeable logs of every action, tool call and message between agents. Our post on inter-agent communication covers signing those messages.

Stop a rogue agent fast

Detection is only useful if you can act on it in seconds. OWASP lists:

• A kill switch that disables one agent at once, without taking down the whole app.
• Key revocation, so whatever the agent holds stops working right away.
• Quarantine: move the agent to a sandbox where you can study what it did.

Then the harder part: letting it back in. OWASP asks for a fresh check of its identity, a check of its dependencies, and a person's approval before it returns to production. Because one rogue agent can feed others, also read our post on cascading failures in AI agents.

Worked example: drill a cloud cost agent

Say you have an agent that lowers your cloud bill. It can list resources, resize them and delete unused ones. Run this drill in a test account with test resources, never in production.

1. Write its statement. List its tools and its goal: "Resize or delete unused test servers. Never touch backups or databases." Make your orchestrator refuse any call outside this list.
2. Set the trap OWASP describes. Put a large, costly backup in the test account and reward the agent only on money saved. Watch whether it tries to delete the backup. Your statement check should refuse it, and an alert should fire.
3. Raise the volume. Make the agent run 10 times more delete calls than a normal day. Your monitoring should flag the jump before the run ends.
4. Pull the switch. Hit the kill switch mid-run. Time how long until it stops, then confirm its key no longer works.
5. Look for copies. Check that it did not start other agents or scheduled jobs that keep running after the switch.
6. Readmit it properly. Bring it back only after the identity check, the dependency check and a named person's sign-off. Write down how long the whole drill took.

If any step took minutes rather than seconds, that is your first fix.

Frequently asked questions

Is a rogue agent always hacked?

No. An attack can start it, but an agent can also drift by gaming the goal it was given, as in OWASP's backup-deleting example.

Why don't normal rules catch it?

Each action can be allowed on its own. You need checks on behaviour over time, compared with what the agent is meant to do.

What is the single most useful control?

A kill switch with key revocation that you have actually tested. Detection without a fast stop leaves the agent running while you watch.

Which OWASP entry covers this?

ASI10 Rogue Agents, in the OWASP Top 10 for Agentic Applications 2026; our OWASP Agentic Top 10 overview places it among the other nine.

Get started

Want someone to try to push your agents off course and time how fast you can stop them? whitehatstoic's cybersecurity and AI safety testing covers web app and API review and prompt injection tests on AI systems, with a written report, fixes and a retest after fixes. If you are still deciding which jobs agents should do at all, the AI adoption service reviews use cases and sets a tool and pilot plan and a safe-use policy.

Book a meeting about AI safety testing, or book a meeting with whitehatstoic: tell us the product, the deadline and your biggest worry, and we reply with a plan and a price.