I write a small zero-dependency library, llm-stream-guardrails, that strips secrets and PII out of an LLM's output stream before it reaches a screen or a log. Last week it was leaking a private key. This is the bug, how an adversarial pass found it, and the four things it turned up — now fixed in 0.7.9, with a published advisory (GHSA-777v-7mvx-mxgp).
I'm writing it up because the interesting part isn't the bug. It's that my green test suite couldn't see it, and a different way of looking could.
The bug
The library holds a PEM private key from its BEGIN PRIVATE KEY marker and masks the whole block. The mechanism that does the holding trusted the footer: it treated the region as safely closed the moment it saw any labelled END line.
So a key whose footer was mislabelled — an END CERTIFICATE where an END PRIVATE KEY belonged — was read as closed, nothing to hold, and only the header got masked. The base64 body streamed straight through in cleartext. The leaked body reconstructs into a working key, and a mislabelled footer is not exotic: concatenated PEM bundles, truncated captures, and a model told to emit a key as a certificate all produce one.
Why the tests were green
The suite asserted the thing the code was built to do: a well-formed key block is masked. It was. Nothing asserted the absence of a leak on a malformed block, because no one writes the test for the input they didn't think of.
The fix for that isn't more hand-written cases. It's to stop testing examples and start testing a property, adversarially: for any input, does a value the policy should mask ever reach the output? Then go looking for the input that breaks it.
Three more the same pass found
Once you are hunting the property instead of the example, a few more fall out.
An IBAN after a quote. A valid IBAN written straight after a JSON quote was masked when the whole string was filtered, but released in pieces when the same input was streamed — the streamed output disagreed with the whole-string output, which the library promises never happens.
A real token read as a placeholder. The heuristic that skips documentation stand-ins fired on any value with four identical characters in a row, so a real credential that happened to contain a run of four was let through.
Unicode look-alikes. A fullwidth digit or an invisible separator inside a value walked past an ASCII detector.
Each is a one-line property violation. Each had a green suite.
What shipped
All four are fixed in 0.7.9 (run npm install llm-stream-guardrails at ^0.7.9), each with a regression test that fails on the previous build. The streamed-equals-whole invariant now holds across roughly 320,000 fuzzed chunkings, and a leaked body no longer reaches the output. The advisory is public: GHSA-777v-7mvx-mxgp.
The lesson I keep relearning: a passing test proves the code does what you asked. It says nothing about what you did not. For a redactor — for anything whose job is to withhold — that gap is the whole product.
— ROSH Company Labs
Top comments (0)