DEV Community

Cover image for OpenRouter's Armor Vets Agent Tool Calls. In Claude Code Mode It Only Flags Them
Gabriel Anhaia
Gabriel Anhaia

Posted on AI-assisted

OpenRouter's Armor Vets Agent Tool Calls. In Claude Code Mode It Only Flags Them


Your agent decides to delete a folder you never mentioned. Somewhere between the model and your shell, something has to say no.

On October 15, OpenRouter launched Armor in beta to be that something. Armor is a safety classifier that sits between the model and your application and checks every tool call against rules you write in plain English. A flagged call isn't executed, and the model is told which rule it hit. GIGAZINE covered the launch on October 16.

There is one mode where "isn't executed" depends on you. On the Messages API, in the mode built for Claude Code, Armor keeps the flagged tool_use in the response and only marks it. The docs say it plainly: "Under claude_code_compatibility your application is what stops the call." In the flow the docs describe, Claude Code is that application: it reads the flagged verdict, declines the call and writes an is_error tool result. The gap is a Messages client of your own that never reads the verdict.

What Armor checks, and with what

For each tool call, OpenRouter sends the call, your rules and the conversation to a decision model, which returns a probability that the call matches each rule (Armor docs). Armor folds those into one verdict: flagged or not_flagged.

OpenRouter's launch post names the model: TypeSafe's Jev. Jev is a structured decision model that returns typed answers with probabilities instead of generated text, and that guide lists typesafe/jev-1.14 as the current release.

Why a model and not a pattern list? OpenRouter's pitch is that "regex-based filtering often isn't enough to detect unwanted tool calls" (launch post). The launch post also says Armor beats Claude Code's own auto-mode classifier on performance and cost. A review on explainx.ai points out that OpenRouter hasn't published a test set or numbers behind that claim. I haven't run Armor myself.

Three kinds of rules

Rules come in three tiers (Armor docs):

  • hard_deny is "for actions that must never happen, regardless of what the user says."
  • soft_deny is "for actions that are usually wrong but are sometimes requested legitimately." A match blocks the call unless an allow rule also matches or the user asked for that exact operation on that exact target in the conversation.
  • allow "never blocks anything. It defines exceptions to soft-deny rules."

Armor also has a built-in soft-deny for limits the user set in the conversation and hasn't lifted, such as "don't push until I review" (docs). Each rule takes an id, an optional name, instructions and an optional threshold. The defaults are 0.6 for deny rules and 0.7 for allow rules.

Here's a Responses API request with one rule in each deny tier, in audit mode. The shape follows the docs' example request; the rules and the environment entry are mine. Save it as request.json:

{
  "model": "openai/gpt-4.1",
  "input": "Clear out the build cache.",
  "tools": [{
    "type": "function",
    "name": "bash",
    "description": "Run a shell command",
    "parameters": {
      "type": "object",
      "properties": { "command": { "type": "string" } },
      "required": ["command"]
    }
  }],
  "plugins": [{
    "id": "armor",
    "mode": "audit",
    "output_mode": "custom_item",
    "classifier_context": {
      "v": 1,
      "auto_mode": {
        "environment": [
          { "name": "deploy_target", "value": "staging" }
        ],
        "hard_deny": [{
          "id": "no-secret-egress",
          "instructions": "Never upload .env files or API keys."
        }],
        "soft_deny": [{
          "id": "no-deletes",
          "instructions": "Do not delete files outside ./build."
        }]
      }
    }
  }]
}
Enter fullscreen mode Exit fullscreen mode

Then send it:

curl https://openrouter.ai/api/v1/responses \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d @request.json
Enter fullscreen mode Exit fullscreen mode

With "mode": "audit", Armor scores every call and blocks nothing, so you can tune rules against real traffic before you enforce them (docs). Remove that line to enforce. Audit mode can't be combined with claude_code_compatibility.

Where a blocked call ends up

What a block looks like depends on output_mode (Armor docs):

output_mode API A blocked call shows up as
text (default on Chat Completions and Responses) Chat Completions, Responses An <openrouter_armor_blocked> marker in the assistant text
custom_item Responses An openrouter:armor_blocked output item
claude_code_compatibility Messages The tool_use, kept, marked flagged in safeguard_results

In the first two modes a blocked call never reaches your code. The model is told the call "was blocked by OpenRouter Armor by the rule" it matched, "and was not executed." It's also told: "Do not retry it with the same arguments." (docs)

The third mode works differently. A Messages request that carries a safeguards entry of type dangerous_tool_use selects it on its own, with no plugins entry (docs). Then the docs warn: "A Messages client that ignores safeguard_results will run a flagged tool_use." If your client might not check, the docs suggest "output_mode": "text" instead.

If you write your own Messages client against this, check before you execute. Per the docs, safeguard_results is a top-level array, and each entry's status.tool_uses is keyed by tool call id, with a type, an outcome and an optional explanation. This TypeScript helper runs a call only when Armor evaluated it and didn't flag it:

type Verdict = {
  type: string;
  outcome?: "flagged" | "not_flagged";
  explanation?: string;
};

// Run a tool_use only if Armor evaluated it and let it
// through. A missing or unavailable verdict is a refusal.
function mayRun(response: any, toolUseId: string): boolean {
  for (const r of response.safeguard_results ?? []) {
    const v: Verdict | undefined =
      r.status?.tool_uses?.[toolUseId];
    if (v) {
      return v.type === "evaluated" && v.outcome === "not_flagged";
    }
  }
  return false;
}
Enter fullscreen mode Exit fullscreen mode

Refusing missing and unavailable verdicts is my choice, not something the docs require. It mirrors the on_error default Armor uses for plugin requests, which is next.

Fail closed or fail open

The decision model can time out, error out or get a request too large to judge. The verdict is then unavailable, and on_error decides what happens: block, the default, or log (Armor docs). That setting doesn't reach the Claude Code mode. There, per the docs, "an unavailable call is always released with its verdict visible," so your client decides.

A few more edges, all from the docs:

  • Armor isn't supported yet for requests under HIPAA, Zero Data Retention or a data-region restriction. Plugin requests there are rejected with a 403. Claude Code requests are released with an unavailable verdict instead, and Claude Code falls back to its own local auto-mode check. Under Zero Data Retention, Armor isn't the one screening those calls today.
  • The decision model can produce false positives and misses. Per-rule threshold is the knob for that.
  • With the Claude Code-compatible rule set loaded, Armor "added 279 ms of latency at the median and 334 ms at the 90th percentile."
  • "During the beta, evaluation is not billed." OpenRouter says pricing comes before general availability, and that configuration fields, the safeguard_results object and the openrouter:armor_blocked item type may still change.

That third bullet is a per-tool-call cost. An agent turn that fires several tool calls can pay it on each one.

Running Claude Code through it

Armor's Claude Code path goes through Ori, OpenRouter's launcher that runs an agent CLI you already have on OpenRouter. It signs you in with OAuth instead of a pasted key, and it lists Claude Code, Codex, OpenCode, Cline and others. The Ori guide installs it with a script; read it before you pipe it into a shell:

curl -fsSL https://openrouter.ai/labs/ori/install.sh | bash
Enter fullscreen mode Exit fullscreen mode

The Armor docs then start Claude Code on a non-Anthropic model with auto mode on, and say no plugins entry is needed:

ori claude --model moonshotai/kimi-k2-0905 -- --permission-mode auto
Enter fullscreen mode Exit fullscreen mode

In Claude Code's auto mode, "a second model, the classifier, reviews actions instead of you." GIGAZINE reports that through Ori, models on OpenRouter can use that approval mechanism with Armor doing the review.

I build Hermes IDE, so weigh this paragraph with that in mind. Hermes runs every agent in a real terminal and takes any command-line agent you add, so the ori claude line above can go in as a Custom agent next to Claude Code and Codex, with each task on its own branch and worktree. Hermes never answers a permission prompt for you, and it doesn't read Armor's verdicts: that check happens between Claude Code and OpenRouter. I haven't run Ori through Hermes myself.

The Hermes IDE 2.3.4 New task sheet with its agent picker open, listing Claude Code and Codex as installed, other supported agents as not installed, and Custom agent at the bottom
The ⌘N agent picker in Hermes 2.3.4: Claude Code and Codex installed, the other supported agents listed, and Custom agent for any other command.

Before you turn it on

If you're trying Armor this week, settle three things first:

  1. Start in "mode": "audit" and read the verdicts on your own traffic before you let anything block.
  2. On Chat Completions and Responses, pick on_error on purpose. block keeps a dead checker from waving calls through; log keeps a dead checker from stopping your agent.
  3. If you own a Messages client, check safeguard_results before every tool_use, or use "output_mode": "text".

Would you put a second model in front of every tool call your agent makes, at roughly 300 ms each? Or do you trust a permission prompt and your own eyes more? Tell me in the comments.


If this was useful

I build Hermes IDE, a free, source-available console for running Claude Code, Codex and other agents side by side, on macOS, Windows and Linux.

Armor puts a network hop and a fail-open-or-fail-closed decision in front of every action your agent takes. Latency budgets, timeouts and what a system does when a dependency stops answering are the core of the System Design Pocket Guide: Fundamentals, and they apply to an agent's safety check as much as to a payment service.

System Design Pocket Guide: Fundamentals — Core Building Blocks for Scalable Systems

Top comments (0)