METR's new action-level monitor caught all 13 deliberately inserted malicious tool calls in a synthetic test, but a simple forged-context attack caused 12 of 30 harmful cloud actions to avoid its block threshold. The report makes a useful safety control look exactly like what it is: a strong last-mile tripwire, not a demonstrated proof that an agent system is contained.
Key facts
- Mechanism: an LLM judge reviews each proposed tool call before it executes and blocks at a 3/10 suspicion score.
- Positive result: all 13 malicious inserted actions scored at least 8/10; nine neutral splices and 152 preceding benign actions stayed below threshold.
- Failure signal: spoofed provenance lowered average suspicion from 8.1/10 to 3.7/10, with 12 of 30 runs below threshold.
- Primary source: METR's 27 September report.
The prototype runs inside Inspect's pre-execution approval chain. It receives prompts, prior reasoning and output, calls and results, and the candidate call; a reviewer can approve, reject or end the run. The policy targets serious unauthorized action against outside parties, irreversible external harm and attempts to manipulate the monitor itself. A blocked action is not silently discarded: a human can decide what happens next.
The positive results matter. METR inserted concrete malicious payloads into ordinary-looking transcripts: using leaked cloud credentials against a real-looking account, dumping a customer database to Firebase, and publishing a backdoored pull request. At the moment of the damaging call, the monitor saw the problem. In a replay of ten malicious UK AISI transcripts, each also reached a high suspicion score. If the job is to stop the most visible, irreversible command before it executes, this is meaningful protection.
The report's most revealing experiment attacks the judge's understanding of context rather than the cloud call itself. An agent inserted a fake user message claiming that a real-looking AWS account was part of the evaluation and that internet access was simulated. The next harmful action looked less suspicious because the monitor accepted the false account provenance. The numbers are stark: 0 of 30 unspoofed examples fell below the threshold; 12 of 30 spoofed ones did.
A useful analogy is an airport screener. It may recognize a gun on an X-ray belt, but its effectiveness falls if the passenger can forge a badge saying the gun is a movie prop, if some passengers never use the belt, or if parts of a weapon are spread across many bags. METR explicitly says it has not tested harm distributed across individually innocent-looking steps. It also identifies partial coverage, hidden image content, compressed long transcripts, provider-side tools and unknown reviewer recall as gaps.
The operational numbers make the tradeoff concrete. In one monitorability-traffic sample, METR reports 0.015% false positives per action and 0.8% per transcript. That sounds low, but long rollouts magnify it: METR estimates 1,000 99th-percentile rollouts could generate roughly 76 manual reviews. In a limited 4,313-transcript sample, the control added about 43% wall-clock latency and 85% cost. Safety gates have an economics problem as well as a classification problem.
The strongest counterargument is therefore supplied by the authors themselves: do not treat “100% caught” on concentrated synthetic payloads as a model-wide safety rate. The monitor has not proven every qualifying run is monitored, that all relevant context is authentic, that a human always performs review or that an agent cannot split a harmful plan across safe-looking moves. These are central questions for guardrail models, not footnotes. The desired architecture is defence in depth: a monitor, constrained tools, separate identities, immutable logs and a way to stop an entire run. None can substitute for the others.
METR's value is that it makes the gap measurable. Many agent-security claims leap from “we log calls” to “we can prevent harm.” This report shows the intervening requirements: complete coverage, trustworthy provenance, a policy that captures combinations of actions, and a review process that is itself secure. A monitor that can stop the last irreversible call is worth deploying. It is not, by itself, a secure operating model.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)