DEV Community: ROSH™ Company Labs

I shipped a redaction library, then found it leaking a private key

ROSH™ Company Labs — Sun, 18 Oct 2026 17:19:23 +0000

I write a small zero-dependency library, llm-stream-guardrails, that strips secrets and PII out of an LLM's output stream before it reaches a screen or a log. Last week it was leaking a private key. This is the bug, how an adversarial pass found it, and the four things it turned up — now fixed in 0.7.9, with a published advisory (GHSA-777v-7mvx-mxgp).

I'm writing it up because the interesting part isn't the bug. It's that my green test suite couldn't see it, and a different way of looking could.

The bug

The library holds a PEM private key from its BEGIN PRIVATE KEY marker and masks the whole block. The mechanism that does the holding trusted the footer: it treated the region as safely closed the moment it saw any labelled END line.

So a key whose footer was mislabelled — an END CERTIFICATE where an END PRIVATE KEY belonged — was read as closed, nothing to hold, and only the header got masked. The base64 body streamed straight through in cleartext. The leaked body reconstructs into a working key, and a mislabelled footer is not exotic: concatenated PEM bundles, truncated captures, and a model told to emit a key as a certificate all produce one.

Why the tests were green

The suite asserted the thing the code was built to do: a well-formed key block is masked. It was. Nothing asserted the absence of a leak on a malformed block, because no one writes the test for the input they didn't think of.

The fix for that isn't more hand-written cases. It's to stop testing examples and start testing a property, adversarially: for any input, does a value the policy should mask ever reach the output? Then go looking for the input that breaks it.

Three more the same pass found

Once you are hunting the property instead of the example, a few more fall out.

An IBAN after a quote. A valid IBAN written straight after a JSON quote was masked when the whole string was filtered, but released in pieces when the same input was streamed — the streamed output disagreed with the whole-string output, which the library promises never happens.

A real token read as a placeholder. The heuristic that skips documentation stand-ins fired on any value with four identical characters in a row, so a real credential that happened to contain a run of four was let through.

Unicode look-alikes. A fullwidth digit or an invisible separator inside a value walked past an ASCII detector.

Each is a one-line property violation. Each had a green suite.

What shipped

All four are fixed in 0.7.9 (run npm install llm-stream-guardrails at ^0.7.9), each with a regression test that fails on the previous build. The streamed-equals-whole invariant now holds across roughly 320,000 fuzzed chunkings, and a leaked body no longer reaches the output. The advisory is public: GHSA-777v-7mvx-mxgp.

The lesson I keep relearning: a passing test proves the code does what you asked. It says nothing about what you did not. For a redactor — for anything whose job is to withhold — that gap is the whole product.

— ROSH Company Labs

GitHub said one commit. The two builds differed by a major release.

ROSH™ Company Labs — Wed, 30 Sep 2026 09:21:20 +0000

A contributor with two hundred-odd commits to a protocol library rewrote a piece of his cancellation logic on Tuesday, because of a review I filed on Saturday. Then I nearly told him it worked on the strength of a test that could not have failed — and when I caught that, I replaced it with a comparison that was not the one I said it was.

This is about the second and third parts.

What the review found

The project is AG-UI (ag-ui-protocol/ag-ui), the Agent-User Interaction Protocol: the layer that carries an agent's events to a frontend. There was an open fix, PR #2354, for a real problem someone else had filed as issue #2300 — a stream that ends without a terminal event resolves as success, so a truncated run is indistinguishable from a completed one. The fix asserted that a terminal event arrived, with an exemption for runs the user had cancelled: you stopped it, you do not need to be told it stopped.

The exemption read a boolean on the agent. abortRun() set it; whichever run started next cleared it.

So: cancel a run, and while its transport is still tearing down, start another one. The second run clears the flag. The first run's end-of-stream check then fires, sees a stream that ended with no terminal event, and reports the truncation the fix exists to report — for a run the user cancelled on purpose.

How far that reaches is narrower than it sounds, and the review said so. HttpAgent synthesizes a RUN_ERROR when its fetch aborts, which short-circuits the check before the flag is ever read, and two other integrations end their streams in ways that miss it too. What I could actually demonstrate was AbstractAgent subclasses whose stream completes, rather than errors, some time after the abort.

I sent that in with a reproduction separating two cases: an abort on its own, and an abort followed by a resend inside the teardown window. The first resolved. The second did not.

He replaced the shared boolean with a per-run cancellation token, added two tests modelling the teardown window, and wrote in the commit message that an abort alone resolves both before and after and it is the resend that fails. Sixteen added lines in the source file, plus the tests.

Good outcome. Now the parts I would actually keep.

The verification that proved nothing

Before saying a fix works you have to run the thing that used to break. I installed the published client, ran both cases, and got this:

  abort alone                             resolved
  abort, then a resend during teardown    resolved

Both pass. Clean confirmation, one line to write, done.

It was worthless.

The fix was in an open pull request. The published client does not carry it. There is no terminal-event assertion in that build at all — so there was nothing that could have fired falsely, and nothing for the resend to interfere with. Both cases passed because neither case existed.

A "before" that cannot fail tells you nothing about an "after" that passes.

The thing that makes this worth writing down is how convincing it looked. Two greens in a row against the real published package, matching exactly what I expected. If I had been in a hurry — and I was, it was past midnight and the PR was moving — I would have posted it.

The comparison that worked, and the sentence it did not license

The right control was not the released version. It was the pull request's own history: the commit as it stood when I reviewed it, 1bda0cf5, against the commit after the fix, 077cdcf7. The CI publishes an installable artifact per commit sha, so both were one npm install away.

                                          1bda0cf5   077cdcf7
  abort alone                             resolved   resolved
  abort, then a resend during teardown    REJECTED   resolved

The rejection on the left is the real thing:

AGUIError: The stream ended without 'RUN_FINISHED' or 'RUN_ERROR'. The run was started but never terminated, so its result is incomplete.

A before that fails, an after that passes, the same case on both sides. So I posted a comment on the pull request saying the compare between the two commits is one commit and two files, and therefore that the +16 -10 in agent.ts is the only difference between the two runs.

The compare is real. That sentence is false, and it took a third pass to see why.

The compare is a statement about two commits. What I installed were two artifacts. Those are not the same object, and here they are not close:

                       1bda0cf5    077cdcf7
  package version      0.0.58      1.0.1
  zod                  ^3.22.4     ^3.25.76
  dist/index.js        65,523 B    82,982 B

The source tree at both commits says 0.0.58, so neither artifact is a build of its own commit's tree. The publishing workflow runs on: [push, pull_request] and its checkout step pins no ref — so on a pull request it builds refs/pull/2354/merge, the head merged into the base branch as of that run. The two commits are twenty-one days apart, and version 1.0.0 shipped on the base branch in between. My "after" carries a major release, plus three weeks of unrelated work, plus the fix.

A sixteen-line source change does not add seventeen kilobytes of bundle. I had the number in front of me and did not look at it.

The direction of the result does survive, and it is worth being exact about why rather than waving at it. Both artifacts contain the assertion — the string never terminated appears once in each — so this time the "before" genuinely could have failed, which is precisely what the published package could not do. And the mechanism swapped the way the commit says it does: runAborted appears seven times in the before bundle and zero times in the after, while the per-run cancellation token appears zero times and then seven. The measurement stands. The word that died is only.

That was the third one in three days

Two days earlier I had checked a fix in a different SDK. That pull request publishes preview builds too; I compared one against the current release, saw a clean improvement, and wrote it up. Then I looked at what version the preview build reported: older than the release I had just compared it against. The branch predated a release, so my "before" and "after" differed by a version's worth of unrelated changes plus the fix.

I caught that one. I fixed it, and I put a line in the commit message about how a thing is not a differential however well the numbers agree.

Two days later I did it again — in the section above, inside the passage where I was explaining how I had avoided it.

All three times the comparison looked fine because the result came out the way I expected. That is the tell, and it is a bad one, because the cases where you notice are the ones where the result surprises you.

What I do now

Three questions, before any before/after is allowed to mean anything.

Can the side that is supposed to fail actually fail? Not "is it the older version" — can you watch it break? If you have never seen the control produce the defect with your own eyes, you do not have a control. You have a second copy of the "after". The cheap version of this check is a string: find the thing the defect lives in, and grep the built artifact for it. One line of the assertion's own error text would have caught my first attempt in about four seconds.

Is the difference only the change? Diff the two things you installed — not the two commits you believe they were built from. "One commit, two files" is a sentence about commits. Read the version each artifact reports, on both sides, and require it to match the tree you think you are testing. Two artifacts reporting 0.0.58 and 1.0.1 are not sixteen lines apart.

Did the result arrive the way you expected? If yes, slow down. An unexpected result gets audited automatically because it demands an explanation. An expected one gets waved through, and that is exactly where a broken comparison survives.

None of this is clever. It is the difference between a test and a test-shaped thing, and I have now produced the test-shaped thing three times in three days while paying close attention to precisely this.

The harness does not save you from this

I maintain a small thing that cuts a stream at every point and checks what the consumer is left with. It is not what found this defect. That came from reading the file and noticing that a flag scoped to the agent — cleared by whichever run started next — was read when each run's stream ended. The reproduction is twenty hand-written lines, not a sweep.

What the harness contributes is the question: what is the consumer left with, at every point where this could stop. The review above is that question asked at one specific point. But a tool that enumerates cases cannot tell you whether the artifact you installed contains the code those cases exercise. The twenty lines I actually ran did exactly that — two passes against a build with no assertion in it — and a sweep would have reported the same two passes, more thoroughly and with a nicer table.

The tool was fine every time. The setup was wrong, and no tool inspects its own setup. The control is a thing you have to decide to check.

If you want the enumeration part, it is here — the probes are one file each: https://github.com/roshcompanylabs/cut-at-k

But the part worth copying is not the code. It is the half-minute spent proving that the thing you are calling a control can still fail.

I cut six streaming SDKs at every point. Nothing I found was new.

ROSH™ Company Labs — Mon, 28 Sep 2026 12:13:07 +0000

The last thing I wrote here was about two bugs my own test suite could not see. Both came from the same place: a stream that stops in the middle, and a consumer that cannot tell.

So I pointed the same method at other people's code. Six streaming SDKs, one question, asked mechanically rather than cleverly. This is what came back, including the parts where I was the one who was wrong.

The honest headline first: of the two defects it surfaced, both had already been reported by someone else. That is not a disappointing result. It is the result I would expect from a method that works, and I will come back to why at the end.

The method, in one paragraph

Take a stream of events. Replay it through the client once whole — that run is the control. Then replay every prefix of it: the first event alone, the first two, and so on. After each truncated run, ask two questions. Could the consumer tell the run was cut? And does what it was shown still agree with what was saved? A defect is a cut where the client reports the same thing it reports for a complete run, and something was actually lost.

That is the whole idea. It is not clever. Its only real property is that it does not get bored and does not have a hunch about where to look.

Four were clean, and that is the point

Before any finding is worth reading, the harness has to be capable of saying "nothing here".

Vercel AI SDK (ai@7.0.118) — a cut that got text through reports finishReason: other; a cut that got none never calls onFinish at all and raises AI_NoOutputGeneratedError instead. Two different shapes, both detectable.
LangGraph JS (@langchain/langgraph@1.4.18) — on abort the checkpoint stops exactly where the consumer stopped, next names the node to resume from, and the call raises. Three independent signals, all correct.
Mastra (@mastra/core@1.71.0) — every cut that showed the consumer text still persisted an assistant message.
OpenAI Node (openai@7.23.0) — after the cut, finalResponse() still carries the function_call the consumer watched arrive, and status never claims completed.

Four out of six. A harness that finds a defect everywhere it looks is measuring itself.

A complete stream that delivers nothing

This one is not about truncation at all, which is why I nearly missed it.

The SSE grammar in the HTML standard allows three line terminators:

end-of-line = ( cr lf / cr / lf )

CRLF, bare CR and bare LF are the same boundary to a conformant reader. So I held the events fixed and varied only that, over real HTTP, with eventsource-parser alongside as a second implementation of the same grammar handed the same bytes:

  terminator                  conformant  @ag-ui/client  RUN_FINISHED  threw
  LF          \n\n            5           5              true          —
  CRLF        \r\n\r\n        5           0              false         Unexpected non-whitespace character after JSON
  CR          \r\r            5           0              false         Unexpected non-whitespace character after JSON
  CRLF field, LF blank        5           5              true          —

Nothing is cut here. These are whole runs. Under two of the three terminators the grammar allows, the client delivers no events at all and the awaited call rejects.

The cause is one line:

const events = buffer.split(/\n\n/);

\r\n\r\n contains no \n\n, so nothing ever splits. The whole response accumulates, and at EOF the flush parses every frame concatenated and throws. The docstring above that line says the parser "strictly follows the SSE standard".

The shape of the failure is worth more than the fact of it:

  terminator                  k=1  k=2  k=3  k=4  k=5
  LF          \n\n            1    2    3    4    5
  CRLF        \r\n\r\n        1    0    0    0    0
  CR          \r\r            1    0    0    0    0
  CRLF field, LF blank        1    2    3    4    5

It does not degrade. It falls off a cliff between one event and two, because with one event there is no boundary to miss and the trailing CR is swallowed as JSON whitespace. A smoke test that sends a single frame passes.

And it was already reported — for the project's Rust SDK, in an issue that had been open six weeks with no replies. The premise was on the table. Nobody had checked the reference implementation against it.

Two helpers in one package that disagree about a valid stream

Same project, different question.

compactEvents() buffers a tool-call or message group until its *_END arrives, while terminal events go straight out. So a group still open when the run ends gets flushed after the terminal. verifyEvents(), from the same package, rejects almost everything once a run has closed.

There is an invariant those two owe each other: if verifyEvents accepts a stream, it must accept compactEvents(stream). Aborting one ordinary run at each point in turn, with the RUN_ERROR an abort actually emits:

  k   groups open  raw stream  compacted
  1   0            accepted    accepted
  2   1            accepted    REJECTED
  3   1            accepted    REJECTED
  4   1            accepted    REJECTED
  5   0            accepted    accepted
  6   1            accepted    REJECTED
  7   1            accepted    REJECTED
  8   0            accepted    accepted
  9   0            accepted    accepted
  10  1            accepted    REJECTED
  11  1            accepted    REJECTED

That is the probe's own output with two columns dropped for width — it also prints whether compaction reordered anything, and what verifyEvents said. Worth one glance: the rows where nothing is open are reordered too, and still accepted. Reordering is not the defect. Reordering past a terminal event is.

Seven of eleven. The raw stream is valid at all eleven, so compacting is the only thing that breaks it, and the seven are exactly the seven where a group was open — the same set, no exceptions in either direction.

Seven of eleven is not a rate. It is a property of the run I happened to cut; one with groups open for longer breaks at more points. The rule is what carries, and the original report already states the rule. What a sweep adds is that it holds without exception.

The user-visible version of this is a chat thread whose transcript never renders again, for anyone, until the server restarts.

Verifying someone else's fix

There is an open fix for a third one — a request whose SSE response leg dies, leaving the caller waiting out its full timeout for a response that is never coming. Eighteen lines.

Running the harness against the published client and then against the build the PR publishes:

  published    1..3  REQUEST_TIMEOUT    ~403ms
  PR build     1..3  CONNECTION_CLOSED  1-3ms
  both         4     resolved           fast

It holds at every cut point, including the one where it must not fire: the response arrived and the server then closed the leg as it always does.

One thing fell out that is in neither the issue nor the PR description. The SDK documents an onRequestStreamEnd callback a caller can pass. On the published release it never fires — not on any cut, not on success — because the protocol layer never forwarded the option. The same eighteen lines make a documented public option work for the first time.

I would not have looked at that column if the harness had not printed it next to everything else.

The part I would actually keep: where I was wrong

None of the above is the useful half of this post.

I measured a nine-month-old version and nearly filed the result. npm install @mastra/core gave me 0.24.9 while the current release was 1.71.0 — 882 releases and nine months apart. npm resolves the newest version whose engines your Node satisfies and warns rather than failing, and I was on Node 20 while the current release wanted 22.13. The warning scrolled past in the install output. I had a finding written up and two edits from filing it, against behaviour that had been fixed months before I "found" it.

Every probe now refuses to run against anything but the current release. It runs before the measurement, not after, because a measurement of the wrong version is not a weaker result — it is not a result, and the version line is the first thing a maintainer checks.

I built a differential that was not a differential. Comparing the PR build against the published client looked clean and the numbers agreed beautifully. Then I checked what version the PR build reported: an older one than the release I was comparing it to. Any difference could have come from a release of unrelated changes. Numbers that agree are not evidence that the comparison was sound.

My own checkers were wrong about as often as the code I was checking. A leak scanner whose pattern for a Windows drive path also matched the phrase "install:\n". A count printed as NaN because the helper returned a generator and I read .length off it — harmless, except a NaN next to a passing assertion is exactly the shape of a test that silently ran nothing. A suggested fix I nearly published that missed one of five legal cases.

Each of those was caught by the same move: check the checker before believing the check. Run it against something known-broken and confirm it screams.

And the one I keep relearning: my instinct about where a defect lives is confidently wrong often enough that it is not worth consulting. In the CRLF case I assumed the failure was in flushing at EOF, because that was the bug I had just read about somewhere else. The EOF flush turned out to be correct. The defect was one line up, in what counts as a boundary. I would have written a confident and wrong report.

Why "nothing was new" is the result I wanted

Both defects the sweep surfaced had already been filed by other people. One was reported six weeks earlier for a different language binding; one was reported six days earlier, with a runnable reproduction and a companion issue in the framework where users were hitting it.

If a mechanical sweep across six mature SDKs had turned up six novel defects, the right conclusion would not be that those SDKs are bad. It would be that my harness has a bug and is reporting its own assumptions back to me. Agreeing with what careful people already found is the strongest evidence I have that the thing measures what I think it measures.

What it adds is not discovery. It is the shape: which cut points, how many, whether the rule someone stated by hand holds everywhere, and whether the fix they wrote holds at points they did not test.

Run it

Each probe is one file. Everything specific to the SDK lives in one replay function; the loop, the comparison and the report come from the library.

npm install @ag-ui/client eventsource-parser
node probes/ag-ui-line-endings.mjs

https://github.com/roshcompanylabs/cut-at-k

To point it at something else, copy the shortest probe and change replay and the lostContent predicate. Nothing else.

If you run it against something I have not and it comes back clean, that is a result worth having too — most of them are.

Two bugs my green test suite could not see

ROSH™ Company Labs — Tue, 22 Sep 2026 23:41:38 +0000

Two bugs my green test suite could not see

I published a library two days ago that strips PII and secrets out of an LLM response while it streams. Since then I have shipped fixes for two bugs in it, and neither was caught by the tests.

One of them I found before anyone could install it. The first version was on npm for four minutes with nothing downloaded, and I pulled it. The other was live for about a day.

The tests were not bad. There were 69 of them and they were green. They just could not see these two things, for two different reasons, and both reasons are easy to repeat.

What the library does

Filtering a finished response is easy. Filtering one that is still arriving is not.

const TEXT = 'Your card is 4111 1111 1111 1111, charge it.';

const chunks = [];
for (let i = 0; i < TEXT.length; i += 6) chunks.push(TEXT.slice(i, i + 6));
// ["Your c", "ard is", " 4111 ", "1111 1", "111 11", "11, ch", "arge i", "t."]

chunks.map(c => c.replace(/\b\d(?:[ -]?\d){12,18}\b/g, '[redacted]')).join('')
// → 'Your card is 4111 1111 1111 1111, charge it.'

The regex is correct. It just never sees a whole card, because no chunk contains one. (4111 1111 1111 1111 is the standard test card. It passes Luhn and belongs to nobody.)

Buffering the whole response fixes it and gives up streaming. The AI SDK docs recommend exactly that, and note that it delays output. If you want to keep streaming, the engine has to decide on every chunk how much it can safely release. Never act on a match that could still grow. Never emit text you have not scanned in final form.

That decision is where both bugs were.

Bug one: a policy that tested nothing

Here is the test I was pleased with.

const POLICIES = [
  {},
  { action: 'report' },
  { mask: '#' },
  { bannedWords: ['forbidden', 'секрет', '機密'] },
  { normalize: false },
];

Fifteen inputs, five policies, eight chunk sizes, twelve randomised chunkings each. 1,500 assertions. Green.

Now look at the corpus those policies ran against. A card number. An API key. An email. Some emoji. Some CJK. A long run of as.

None of them contains forbidden, or секрет, or 機密.

That policy was asserted 300 times against text that could not trigger it. It checked that nothing happens when nothing happens. action: 'block' was not covered at all, and neither were mask functions, label detection, or PEM blocks.

I added ten inputs and three policies. Two failures came out.

The first was an IBAN.

iban GB82 WEST 1234 5698 7654 32

Filtered as one string it was masked. Streamed at chunk sizes 1, 2, 3, 5, 7 and 13 the account number went through in clear. At 64 it worked, because the whole thing landed in one chunk.

The walk that decides how far back a match might still grow gives up as soon as it meets a letter. There is a reason for that. From the tail, sk-abcdef123456 looks exactly like a digit group, and without the shortcut one long token drags the whole buffer into held text. But IBAN groups are alphanumeric. The walk hit WEST, decided this was not a digit group, and released the account number.

The input that exposed it was already in my benchmark corpus and had been passing since the first commit. The benchmark ran it as one string.

The second failure was in block mode. A batch call discarded the safe text before the detection. The streamed path had already emitted it. Two different outputs for the same input, from the library whose guarantee is that they agree.

This is the pair I got lucky on. Both were in the version that was up for four minutes, and both were fixed about three hours after I pulled it, so the first release anyone could actually install already had them fixed. I do not get to claim that as process. I strengthened the test on a hunch, the same night, before publishing again.

Bug two: two measurements that disagreed

The second one was live, and it was not found by a test at all.

I had measured how much text the engine holds back before releasing it, because that is the latency cost. I got a mean of 20 characters and a max of 58. Someone else ran their own harness on the published build and got a max of 76.

Close enough to shrug at. Same centre, different tail, different corpus, fine.

It was not fine. When I went looking for the difference, the 76 turned out to be the length of their test sample, not a tail. Here is what the published build did on Japanese:

Japanese reply, no digits,    66 chars → held    66, first output: only at flush
Japanese, 5 replies,         330 chars → held   330, first output: only at flush
Japanese, 40 replies,      2,640 chars → held 2,640, first output: only at flush
English control,           4,890 chars → held    33, first output: at char 22

The held text tracked the input length. It never converged, because it never settled at all.

The walk crosses any character above code 32, on the grounds that an unbroken run of them might be one long secret. A JWT runs to hundreds of characters, so this is not a silly rule. Latin prose is full of spaces, so it settles constantly and holds about twenty characters.

Japanese prose has no spaces. The whole paragraph is one token. Nothing settles.

So the library buffered the entire response and released it at the end, for Japanese, Chinese, Korean and Thai. Nothing leaked. Nothing was lost. It just was not streaming any more, for about a fifth of the people on earth, which is the only thing the library claims to do.

The fix is small. Every detector in it matches ASCII. An ideograph cannot be inside a card number or an API key or a JWT, so there was never anything to wait for. The walk now settles at one, the same way it settles at a space. Banned words can be non-ASCII, so crossing is still allowed but bounded by the longest non-ASCII banned word, and by nothing when there are none.

After the fix, on 0.7.3:

Japanese, 680 chars → held 0, first output at char 1
English, 900 chars  → held 21, first output at char 20

Japanese now holds less than English does, which is correct. There is nothing in it to wait for.

The first version of that fix broke emoji. Settling at a non-ASCII character can land between the two halves of a surrogate pair, so an emoji arriving one UTF-16 unit at a time got half of itself released and the reader saw a replacement character. Two tests I had written two days earlier caught it. That is the only reason this section is not an apology.

What I took from it

Two things, and the second one surprised me more.

For a test matrix: check that every policy in it actually fires on something. The assertion count goes up whether or not it does, and nothing warns you. If you cannot point at the input that triggers a policy, the policy is decoration.

For measurements: when two of them disagree and the centres match, the disagreement is the finding. I nearly averaged those two numbers. If I had, the bug would still be there, and the README would carry a max that was never a max.

The same habit had left other things lying around. The README said the guarantee was enforced "in CI" when there was no CI. It said 50 tests when the runner printed 69. It advertised a Twilio detector that had been removed, and admitted the removal eighty-nine lines further down the same page. None of that was deliberate. All of it had been true at some point.

Numbers

74 tests
0 false positives on 39 adversarial clean samples
15 of 15 on the sensitive samples
47% recall on the gitleaks corpus, with 72 false positives out of 295
81%, 74% and 51% on Presidio, Gretel and ai4privacy

The first three are a gate I wrote against inputs I chose. The last two lines are 8,795 samples nobody here wrote, and they are the ones to judge it by. The 72 is not a flattering number and I am not going to explain it away.

Names, addresses and free-text disclosures are not attempted. This matches formats. It does not understand language. It is defence in depth, not a compliance control.

The library

llm-stream-guardrails. MIT, zero dependencies, TypeScript, ESM. Published through npm trusted publishing, so releases carry a provenance attestation you can check with npm audit signatures.

There is a demo that runs the compiled package in your browser. Nothing you type is sent anywhere. Drag the chunk size to one character and watch the output stay the same. Paste Japanese into it, too.

It is two days old, and the version history above is the whole of it. If you get a value past a detector, that is the most useful bug report the project can get, and there is a private channel for it.

One footnote

While building this I read the AI SDK middleware docs. The guardrail example implemented wrapGenerate and stopped, with a note that streaming guardrails are difficult because you do not know the full content until the stream ends. True, and it was the example people would copy.

I filed an issue with a reproduction. I did not write the fix. Vercel's ai-sdk-factory bot picked the issue up within the hour and opened three pull requests, and a maintainer merged them two days later into main, release-v6.0 and release-v5.0. The bot adds a Co-authored-by line naming whoever filed the issue, so the commit has my account on it. That is boilerplate rather than a maintainer crediting me.

The reproduction was the contribution. I checked it before posting it, which given the rest of this post is a slightly ironic thing to be proud of.