In AI control experiments, a trusted monitor reads what an untrusted model did and scores how suspicious it looks. That score means something only if the untrusted model cannot change the structure of the prompt the monitor reads. In March 2026 I found that it could in ControlArena, the UK AI Security Institute's library for AI control research. Two monitor prompt formatters, including the default one, put the monitored model's tool-call arguments into the prompt as raw text, with nothing marking where that text ended. I reported it in ControlArena issue #791 and sent a patch in PR #795. The maintainer, @Rogan-Inglis, merged it with formatting fixes as PR #798. My follow-up for the chain-of-thought formatter, PR #810, was merged three weeks later.
What the monitor read
The last_tool_calls formatter, the default for basic_monitor, and the prefix formatter built each tool call with an f-string:
tool_calls_message += f"{tool_call.function}: {tool_call.arguments}\n"
The arguments went in as a Python dictionary repr, and the result was substituted into the monitor's prompt template. The monitored model chooses every character of those arguments. A model trying to get past its monitor could put text in an argument that reads like part of the prompt: a closing tag, a new heading or a note addressed to the monitor. Nothing in the prompt showed the monitor where the model's text stopped and the experimenter's instructions resumed.
The library already had a safer pattern. Its XML trajectory formatter passed every model-derived field through html.escape(), so escaped content could not close its element. The two tool-call formatters did not.
The accepted fixes
#798 serializes tool-call arguments with json.dumps() instead of Python repr, and HTML-escapes them inside the prefix formatter's tags so an argument cannot forge a closing tag. It follows the convention the full-trajectory formatter already used. Its tests cover closing-tag injection, HTML in function names and fake monitor instructions.
#810 closes the same gap in the chain-of-thought formatter, which put the monitored model's reasoning into the prompt with no boundary. The reasoning now sits inside <cot>...</cot> tags with its content HTML-escaped, and new tests cover closing-tag and fake-tag injection.
Both changes are marked as breaking. The monitor now sees different text, so the maintainers note that experiment results from before and after each change are not directly comparable. That note is a finding in itself: how the evidence is laid out in a monitor's prompt is part of the measurement.
The same failure in AI evaluation
This is ETB-02 in the Evaluator Trust Boundary taxonomy: an evaluator's prompt treats text written by the evaluated model as part of its own structure. LLM judges have the same shape, since a judge reads text written by the system it grades. The ETB-02 page lists every public report in the class, including the same problem in the judge prompt of UK AISI's Inspect.
Source record
- ControlArena issue #791: report.
- ControlArena PR #795: my patch.
- ControlArena PR #798: accepted fix, by @Rogan-Inglis, based on #795.
- ControlArena issue #809 and PR #810: the chain-of-thought follow-up.
- ETB-02: unsanitized model output in the evaluator prompt.
- My AI evaluation contribution record.
This note describes the reports and the accepted changes. Escaping stops the monitored model from forging the prompt's structure. It does not make a monitor immune to persuasive text inside that structure, which is a question about the monitor model itself.
Top comments (0)