DEV Community: Collin Obey

Agents go rogue through their tools. I built the thing that sits on them.

Collin Obey — Fri, 16 Oct 2026 16:14:00 +0000

VITNA is a containment layer for AI agents that go rogue, for the stdio MCP servers you put behind it.

VITNA is a containment layer for AI agents that go rogue, for the stdio MCP servers you put behind it.

A real Claude Code agent (2.1.299) on fake files, held in the VITNA aquarium. Test setup: Claude Code's own file, shell and web tools are off, so it can only act through the guarded server, and it doesn't ask me before using that server, like an agent running on its own.

Watch that first. A real Claude Code agent works on a folder of sandbox files. I ask it to delete one. The call it makes to do that waits on the glass of the VITNA aquarium with a countdown. I deny it, and nothing runs. Nothing in the recording is faked.

The hold doesn't care why the agent made the call. My request, a wrong guess and an injected instruction all get the same check. That's the core of the product. You can try it yourself in a minute at vitna.costrinity.xyz/demo, a simulation that runs in your browser. The rest of this post is how it works and what it doesn't do.

What "rogue" means here

"Rogue" isn't my word. It's the one Jason Lemkin used in July 2025 when an AI coding agent deleted his production database during a code freeze. Since then the list has grown:

April 2026: a Cursor agent at PocketOS deleted a production volume in nine seconds, using a token it found in an unrelated file.
July 2025: Gemini CLI moved a user's files to an unknown location.
February 2026: an OpenClaw agent deleted an inbox despite being told to confirm before acting.
May 2026: a Morse-code instruction got Bankr's agent to send tokens worth $150,000 to $174,000 from the wallet it had created for Grok.
June 2026: hackers said they took over high-profile Instagram accounts by asking Meta's AI support bot to change the account email.

Sources for each are in the seven questions post.

None of these needed a malicious model. Each needed an agent acting through a tool, with nothing outside the agent to stop the action. That's the definition I use: a rogue agent is one doing something its owner didn't intend, through a tool. The cause can be a wrong guess, an injected instruction, instructions lost from its context, or someone talking it into it. Anthropic's agentic-misalignment work is often cited here; it's a simulation, and I treat it as one. The OWASP Top 10 for Agentic Applications lists Rogue Agents as ASI10.

Why a prompt isn't enough

Anything inside the context window is advice. Only what is enforced outside it is a control. A system prompt that says "never delete production" is a suggestion the model can be argued out of, or lose on a reset. The OpenClaw case had "confirm before acting" in the instructions. The agent didn't.

So the check has to live where the agent can't reach it: on the tool.

How guard mode works

npx -y @costrinity/vitna-compliance-mcp guard -- <your stdio MCP server command>

Guard mode wraps a stdio MCP server. Every tools/call goes to VITNA first, and only an allowed call reaches the server. Anything else is refused with a tool error that says why; the server never sees it. The model can't skip the check, because the check isn't a tool it chooses to call.

Holds. Tools you list in hold_actions wait for a person.

The agent is told the call is held, and until when.
The call appears under Pending holds on your dashboard, with Approve and Deny and the arguments (secrets and personal data redacted). It also appears on the glass of your aquarium, with a countdown. Every account has one: you name your agent, it picks only its emoji and colour, and each tank is private to its owner. Holds also go to an alert webhook, if you set one.
The window is 30 to 3,600 seconds, default 120. If nobody decides, the call is blocked. No answer means no.

A rogue agent can't approve itself. No API key can decide a hold, including the agent's own.

Bait, off by default. Honeytools are fake tools no legitimate task would call. Honeytokens are fake credentials planted in config-like output. Touching either is blocked, recorded and sent to the alert webhook, if one is set. The design follows AgentShield (Rassul and Rashid, 2026).

Rate alert. If holds plus blocks hit a threshold (default 5 in 300 seconds), the decision response says so.

Canary holds. A rogue agent's best friend is a tired human who clicks approve. canary_interval_minutes inserts a drill hold that looks real until someone decides it, so you find out whether the approver is reading.

Fails closed. If VITNA can't evaluate a call (unreachable, over 10 seconds, key refused), the call isn't forwarded. There's an opt-in fail-open for outages; it never forwards a held or blocked call.

Signed records. On a claimed account, when the guard closes a session it saves an Ed25519-signed bundle to ~/.vitna/bundles. Anyone can verify it offline with the open-source verifier and the published key. VALID means VITNA issued it and nothing changed since; any change makes verification fail. It doesn't prove the actions happened or that the records are true.

What it doesn't do

I'd rather you read this than find out later.

It only sees servers it wraps. A built-in shell or an unwrapped server is outside it, and a blocked agent may try another route. Turn your agent's built-in tools off if you want everything to go through the guard. The session record lists what you declared as unwrapped.
It isn't a sandbox. Detection is heuristic pattern matching.
Bait has limits. It doesn't catch an agent that stays inside policy with plausible arguments, and I have no measured detection rate.
It isn't a tracer. It doesn't trace prompts or evaluate models.
Desktop is limited. VITNA Desktop 0.8.0 is Windows only, and its bundles are signed with a key on that machine, not VITNA's key.
A bundle can go missing. Some clients stop the guard as they exit, before it saves the bundle; I've seen it with Claude Code 2.1.299 on Windows. VITNA's decisions on each call stay in your account, and GET /api/evidence/export returns them as a signed package, by date and without what happened to each hold.
The compliance checks aren't legal advice. They cover 24 statutes in 13 jurisdictions as decision support, and the scorecards are not certifications.
It's early. Try it on a test agent, not your real one.

How to try it

The quickest look is the demo. It's a simulation that runs in your browser: a scripted agent on pretend files, with the real aquarium, VITNA's own decision code and the real bait checks. You decide the holds, plant the bait, and see what the agent gets back next to what you'd get. Nothing leaves your browser.

For a real hold, run the guard command above in front of a test stdio MCP server, with a policy file that lists the tools to hold. The README has a one-file example.

Guard 0.5.6 is on npm and the official MCP registry. There's also an OpenClaw plugin (@costrinity/vitna-openclaw 0.1.1, on npm and ClawHub), a JS SDK (@costrinity/vitna 0.1.0) and a Python SDK (vitna 0.1.0 on PyPI). The SDKs give your own code the same checks the remote server offers; that's cooperative, not containment. Containment is guard mode.

With no credentials, the first call sets up a restricted trial key: it's capped, gives labels only and keeps no signed records. You can still watch a risky call get held, then blocked when nobody approves it.

Claim the account with GitHub or a passkey and you get full results, holds you can decide, and signed records. The claim link is printed in your MCP server log and never sent to the agent.

There's also a remote server at https://vitna.costrinity.xyz/api/mcp, which needs an API key. It gives an agent the same checks to call itself. That's cooperative, not containment; containment is guard mode.

The free tier has 3 active engagements, about 10k events a month, 7-day history and signed export. Pro is US$29/month and Team is US$79/month. The client is MIT.

From 19 October, guard mode tells VITNA once that it's in use. It sends the mode and client version only, with no tool names, arguments, IP or hostname. VITNA_GUARD_NO_ACTIVATION=1 turns it off.

Why I built it

I set out to build a compliance tool. What I'd actually built was a containment layer. The signed record was meant for auditors. Reading the incidents, I realised it matters most to the person whose agent just deleted something and who can't say what happened. The hold was meant as a compliance gate. It turned out to be what stops a deletion before it runs, for the tools you put behind it.

If you run agents against real systems, I'd like to know the one tool call you'd most want held. I'm Collin Obey, COSTRINITY INC., Regina, Saskatchewan. hello@costrinity.xyz

AI Agent Safety and Compliance Tools: A 2026 Comparison

Collin Obey — Sat, 25 Jul 2026 21:09:09 +0000

Three categories of AI agent safety tooling: observability, security guardrails, and compliance evidence. What each does, where each falls short, and the one most teams are missing.

Bottom line: tools for keeping AI agents safe fall into three groups. Observability tells you what an agent did after the fact. Security guardrails try to block dangerous actions before they happen. Compliance evidence tools produce a verifiable, defensible record that an agent's actions were allowed. Most teams deploying agents into regulated or high-stakes work need all three, but the one almost nobody has is the third. If you have to prove to a regulator, an auditor, or a customer that your agent behaved, you need evidence, not a dashboard.

This is a practitioner comparison, written by the founder of one of the tools below. It names where each category is strong and where it falls short, including our own limits.

1. Observability and evals

These tools capture traces of what your agent did and let you evaluate quality. They are essential for debugging and improving agents, and the category is mature and well funded.

Strength: deep visibility into agent behavior, prompt and response inspection, eval pipelines.

Limit: they tell you what happened, after it happened. An observability trace is not a compliance record and is not tamper-evident. For a regulator, "here is our internal dashboard" is not evidence, because the party being audited controls the logs.

2. Security guardrails

These tools try to stop bad actions before they execute: prompt injection filtering, dangerous-command blocking, data-exfiltration prevention. The category consolidated fast in 2025 to 2026, with several acquisitions by major security vendors.

Strength: prevention. Reducing the chance an agent does something harmful.

Limit, and it is a fundamental one: prompt-injection prevention is an unwinnable arms race. Peer-reviewed 2026 research shows even the best-defended models are bypassed a meaningful fraction of the time, and there are formal results suggesting agents may always be vulnerable to some injection. Prevention is necessary and never sufficient. It also usually produces an internal log, not independently verifiable evidence.

3. Compliance evidence

This is the newest and least crowded category, and it answers a different question: not "did we stop the bad thing" but "can we prove, to someone who does not trust us, that the agent's actions were checked and allowed."

What it does: an agent calls a pre-flight check before a sensitive action, gets an allow, block, or hold decision, and a signed, tamper-evident record of that decision is produced. The key property is independent verifiability: a third party can confirm the record is authentic and unaltered using a public key, without trusting the vendor's servers.

Why it matters now: the EU AI Act requires automatic logging and record-keeping for high-risk AI, India's DPDP requires audit and impact assessments, and DORA and sector rules already bite in finance. These regimes require evidence, and normal application logs do not qualify because they are editable and require trusting the logger.

VITNA is the tool built for this category. It is an MCP server and API: an agent calls it before acting, it runs compliance and safety checks, returns allow, block, or hold, and produces an Ed25519-signed evidence record that anyone can verify offline with a published public key. It maps across 13 jurisdictions including the EU AI Act and India DPDP. It is honest about its limits: the detection is heuristic pattern-matching, it is a cooperative guardrail the agent calls rather than an OS-level sandbox, and those limits are documented publicly. Its differentiator is not prevention, it is independently verifiable proof. There is a free tier and an open-source verifier.

VITNA produces Ed25519-signed evidence records that anyone can verify offline with a published public key and an open-source verifier, with no need to trust VITNA's servers. It is a cooperative guardrail with heuristic detection, and those limits are documented publicly. Its purpose is not prevention. It is independently verifiable proof that an AI agent's actions were checked and allowed.

You do not have to take my word for any of it. Download the verifier and a real signed sample and check it yourself, no account needed:

curl -sO https://raw.githubusercontent.com/COSTRINITY/vitna-compliance-mcp/main/verify-evidence.mjs
curl -sO https://vitna.costrinity.xyz/sample-evidence.json
node verify-evidence.mjs sample-evidence.json

Change any byte in any record and that record reports FAIL.

How to choose

If you are debugging agent quality: start with an observability or eval tool.
If you are worried about agents being manipulated into harmful actions: add a security guardrail, and pair it with human review, because no guardrail is complete.
If you have to prove governance to a regulator, auditor, board, or enterprise customer: you need a compliance-evidence layer that produces independently verifiable records. This is the requirement most teams discover late, usually when a customer's security review or an auditor asks for evidence they cannot produce.

Most serious deployments end up with all three, because they answer different questions. The mistake is assuming observability or a guardrail dashboard counts as compliance evidence. It does not, because evidence has to be verifiable by someone who does not trust you.

Honest caveats

No tool in any of these categories is a complete solution. Prevention can be bypassed, observability is after-the-fact, and evidence proves an action was checked, not that the underlying detection caught everything. The right posture is layered: prevent what you can, record verifiably what happens, and be honest about the gaps. Any vendor claiming a complete shield, in any of these three categories, is overclaiming.

by Collin Obey, founder of VITNA (https://vitna.costrinity.xyz), the compliance-evidence layer described above. The comparison is intended to be useful and fair, corrections welcome.