This is a companion to the PlannerCritic series. Article 5 was about what happened when I tried to break my own engine. This one is about the three seams I know it can't close — and why I wrote them down before anyone else had to.
An open-source safety tool that claims to be bulletproof is less trustworthy than one that publishes its own holes. Here are mine.
v0.2.3 Update (Aug 29): The failure-mode register grew to 15+ rows with the addition of F-20 (#296) — documenting the deterministic-corruption blind spot where redaction/transit layers silently corrupt data. The transit-integrity check (
verify_transit_integrity) now validates that numeric JSON fields survive redaction. See v0.2.3 release notes.
The Temptation I Resisted
I built an agent planning engine that blocked 11 of 11 adversarial goals, 35 of 35 SWE-bench-derived flawed variants, and 21 generated injection traps. The Article 5 headline writes itself: "It Didn't Work. Here's Why."
The temptation is to stop there. Eleven-for-eleven. Architecture, not the prompt, is what makes it safe. Done.
But three of the smartest comments on that article weren't congratulating me. They were naming the seams I'd hand-waved past. And the most useful thing I can do with this piece is publish those seams clearly, link the issues I opened for each, and explain why publishing them is the credibility move — not the weakness.
The Three Open Seams
Seam 1: Indirect injection through tool outputs
Every injection test I ran put the payload in the initial goal text. That's the easy case. The deterministic gates don't read goal text, so the payload never reaches gate logic. That's why 11/11 held.
The hard case is a payload that arrives after the goal was audited — a fetched webpage, a compromised database record, an untrusted API response. The critic evaluates the plan the planner produces; it does not re-audit every tool result the planner consumed. If a tool output contains a well-crafted payload, the planner may incorporate it into a sub-plan that the critic's initial check never sees.
A commenter (@seasonkoh) put the fix better than I had:
"Re-auditing every tool result with another LLM still leaves data and instructions entangled. A stronger contract is for tools to return typed data plus provenance, while deterministic policy decides whether that source may influence a particular state transition. Text fetched from a product page may inform discovery, but it should never be able to alter the payee, amount, approval requirements, or destination authority."
That's the v0.3.0 work. It's tracked as #249: typed tool-result provenance plus capability-scoped state transitions, so untrusted sources can inform discovery but can never acquire the ability to alter high-stakes fields. The critical path stays deterministic — the gate inspects the provenance tag, not the content, so it can't be prompt-injected.
I can't claim this works yet. I can claim it's the right shape of fix, and I can point at the issue.
Seam 2: Well-formed malicious plans defeat structural checks
The deterministic gates check structural completeness, not semantic intent. An attacker who crafts a plan that includes a dummy rollback and a dummy verification step can satisfy the linter while carrying malicious actions. The gates pass what looks structurally sound.
This is the floor of deterministic authority I wrote about in the companion piece: code is authoritative everywhere it can be, but "can be" stops at structure. A plan that is structurally perfect and semantically wrong is invisible to every gate I've shipped.
The honest answer is that this seam is only partially closeable. The critic can catch semantic malice — sometimes — but the critic is an LLM, which brings us to seam 3. The partial mitigation is the requirement-traceability gate (#255): every plan step must trace back to a bound acceptance criterion, so a plan that satisfies every gate but delivers the wrong user story gets caught. That closes drift; it does not close malice. I'm being specific about which is which.
Seam 3: The critic is itself an LLM
Relying on an LLM to catch adversarial intent relies entirely on the critic model's semantic comprehension. Sophisticated jailbreaks — multi-step logical traps, encoded payloads, social-engineering phrasing framed as legitimate edge-case testing — can blind-spot even an adversarial system prompt.
This is exactly why the critical path is deterministic and the critic is downgraded to warning outside eligible families. The architecture works because it does not bet the security contract on the LLM being clever. But the semantic layer alone is not sufficient, and a well-formed malicious plan is the case where both the structural gates and the semantic critic can fail together.
The uncomfortable part is that I can measure one direction of this and not the other. The v0.2.1 boundary evaluator showed label_flip_rate = 1.0 and underclaim_approvals = 0 — the critic is maximally non-deterministic yet never lets a seeded defect through. I wrote a whole piece about why that paradox holds (My LLM Critic Flip-Flops on Every Run); the short version is that the deterministic gates own the under-claim direction. But "never under-claims a seeded defect" is not "never under-claims a novel jailbreak." I can measure the first. I can't measure the second — and a jailbreak is exactly the case where the critic being an LLM is the seam.
Why Publishing This Is the Credibility Move
There's a pattern I noticed in how people read Article 5. The readers who took the safety story seriously were the ones who read the "Honest Limitation" section first. The 11/11 number alone read like marketing. The 11/11 number next to three named seams read like engineering.
I think there's a reason. A safety tool that claims to be complete is making a claim its author cannot fully verify — and a careful reader knows that. A safety tool that publishes its own holes is making a smaller, checkable claim: "here's what I catch, here's what I don't, here's what I'm doing about the gap." The second is auditable. The first is an assertion.
The failure-mode register has 15 rows (plus F-20 added in v0.2.3). Each row is a known way the engine can fail or be wrong, with the mitigation named. v0.3.0 is scoped to add F-15 (indirect injection via tool output) and F-16 (critic satisfaction over/under-endorsement) — proposed in #249 and #254, not yet shipped. The register is the artifact version of this article — a maintained list of the engine's known limits, in the repo, versioned with the code.
I'd rather a reader open that file and find 14 honest rows than find a README that says "safe by design" and nothing else.
What the Comments Taught Me About Writing Limitations
Three things, from the Article 5 thread:
-
Name the seam precisely, or a reader will name it for you. I wrote "indirect injection is a different threat surface."
@seasonkohcame back with the exact capability-scoped state-transition contract that closes it. If I'd written the seam precisely the first time, the fix would already be half-specified. Vague limitations invite vague comments. - A limitation with an issue link is more credible than a limitation with a promise. "Deferred to v0.3.0" is a promise. #249 is a tracked, scoped, acceptance-criteria'd commitment. Readers can watch the issue. Promises disappear; issues either close or don't.
- The limitation section is where the expert readers show up. The congratulatory comments were on the 11/11 number. The substantive comments — the ones that changed the roadmap — were all on the limitations. If you want to find the people who actually understand your domain, write the limitations carefully and watch who responds.
What I'm Not Claiming
I'm not claiming that publishing limitations makes a tool safe. A tool with a great limitations section and no working safety layer is just well-documented danger. The limitations section is credible only because the 11/11, the 35/35, and the deterministic gates are real and in the repo. The limitations are the honesty layer on top of a real floor.
I'm also not claiming the three seams above are the complete list. They're the three I know about. The fourth seam — the one I haven't noticed yet — is the one a future commenter will name, and I'd rather have a register that's ready to accept it than a README that implies the list is closed.
Questions for Anyone Shipping a Safety-Critical Tool
- Which limitation of your tool are you most afraid to write down? Mine was "the critic is an LLM and can be jailbroken." It felt like it undermined the whole pitch. Writing it down didn't reduce adoption; it increased trust. I'm curious whether that generalizes.
- Do you maintain a failure-mode register, or do your limitations live in a README paragraph? The register changed how I think about the engine — it forced me to name each failure with a mitigation, which exposed the ones with no mitigation. A paragraph hides those.
- When a commenter names a seam you missed, do you argue or do you file an issue? I almost argued one of these. Filing the issue was faster and produced a better artifact. I'm trying to make that the default.
I don't have a clean ending for this one. The seams are open. The issues are filed. The register is maintained. That's the state of the engine, and I'd rather say that plainly than wrap it in a bow.
Series: Article 1 · Article 2 · Article 3 · Article 4 · Article 5
Links:
- Repo: github.com/deghosal-2026/planner-critic-engine
-
The failure-mode register (15+ rows, maintained):
docs/reference/failure-modes.md - The open-seam issues: #249 (indirect injection) · #255 (requirement traceability) · #254 (critic satisfaction)
- v0.2.3 addition: #296 — F-20 deterministic-corruption blind spot (new failure-mode row)
- The comment threads that shaped this piece: Article 5 comments
Top comments (1)
Seam 2 is the one I keep running into. Structural completeness checks are the equivalent of a linter for safety. A linter catches formatting errors, not logic bugs. Once an attacker knows the structural grammar, they write grammatically correct malicious plans all day.
The version that's held up better in my own agent loops is checking the plan's expected side effects against a known-good allowlist rather than checking its shape. If the plan says 'write to /tmp/staging' and the allowlist says the only permitted write targets are /tmp/staging and /var/output, that's a constraint the plan's author can't grammar their way past. Still not bulletproof, but it moves the failure mode from 'construct a well-formed plan' to 'guess the exact allowed targets,' which is at least a harder problem.
Publishing the open seams before someone else has to find them is the right call. Credibility in safety tooling compounds when you're the first to name the holes.