The one-sentence problem
A static analyzer sees eval(x) and asks "is x tainted by SQL or an HTTP parameter?" It has no concept of "x is the model's reply."
That blind spot — model output reaching a code, shell, or deserialization sink — is the entire agent layer, and it's invisible to the tools most teams already run.
I scanned 30+ of the most-starred open-source AI-agent frameworks for exactly this, then hand-verified every high-severity finding against the source. Here's what's real — and, deliberately, what I decided not to report.
What's actually out there
~1 in 5 frameworks let model output flow into eval/exec/new Function — mostly code-writing agents by design, where the only thing between a prompt injection and RCE is a sandbox the scanner can't verify.
One is unambiguous and public: SuperAGI's eval() on the assistant's own reply, CVE-2025-51472.
73% assemble LLM request params in a dict, spread them with **kwargs, and set no max_tokens anywhere in the path (27 call sites).
Cost hygiene, not a vuln, on its own — but near-universal, and passed indirectly where most scanners can't see it.
The part nobody publishes: what I refused to flag
A scanner that cries wolf gets demoted to advisory and ignored. The hardest — and most valuable — engineering here is not flagging the things that look scary but aren't. Every one of these is a real pattern in a real, well-engineered framework that a lazy scanner reports as critical:
The rule that produces that discipline: assert HIGH only when you can see the dangerous input's source — a value assigned from an LLM call in scope, or an unambiguous name like request.body.
When the danger is only inferred from a variable's name, say so — MEDIUM, "confirm the source" — not "you have an RCE." A judge rules on the evidence in front of it.
What I'm not claiming
No precision/recall number yet — that needs a labeled benchmark with true negatives, which I'm building. This is a methodology plus hand-checked findings, plus a public account of the false positives I eliminated.
Honesty about the second is the point. (One reported finding — an unbounded tool-call loop in LightAgent — already got a maintainer fix. That's the bar.)
Read the full report / scan your own agent
Full write-up: https://release-gate.com/research.html
pip install release-gate
release-gate audit . # two honest scores + a PROMOTE/HOLD/BLOCK verdict
It's open source — the analyzer and the entire false-positive test suite are on GitHub.(https://github.com/VamsiSudhakaran1/release-gate)
Top comments (0)