Sending User Data to an LLM? Redact It First — and Order Your Defenses So the AI Can Only Make Them Stricter

The instant your code writes messages: [{ role: 'user', content: someUserString }] and posts it to OpenAI, Anthropic, or Perplexity, that string has left your trust boundary. Whatever was in it — a customer's email, a pasted Authorization: Bearer header, an internal IP, a Stripe key someone dropped into a support ticket — is now in a third party's logs, retention window, and possibly a future training set. You cannot un-send it.

Any product with an AI feature has this surface. Monitoring tools have it badly, because the raw material is the sensitive stuff: support tickets, names a user typed, the diff of a page, DNS and TLS metadata. All of it can carry PII or secrets, and all of it is on its way to a model.

We send data to LLMs in several places — summarizing a page diff, drafting a support reply, running a legal filter when a user names a resource — so we settled on two rules and, more importantly, an ordering between them. The ordering is the entire point.

Rule 1: redact with a deterministic floor, before the call

Every string headed for an external model runs through one function first. It is boring, deterministic regex — and boring is exactly what you want on the layer that must never surprise you. It replaces, in place:

• Credentials, always: JWTs (eyJ….eyJ….sig), Bearer <token>, common SaaS key prefixes (sk_, pk_, ghp_, github_pat_, …), URL query params named token/key/secret/password/auth/access_token/api_key, and PEM -----BEGIN … PRIVATE KEY----- blocks.
• Personal data, on the full depth: emails, phone numbers (E.164 and North-American NANP), IPv4 and IPv6, credit-card-shaped digit runs, SSNs, MAC addresses, and US-format dates (which leak DOBs).

const safe = redactForAi(userInput, 'full');
// -> "contact [EMAIL] from [IP], card [CARD], token [API_KEY]"

Two design choices matter more than the pattern list.

It fails closed on anything it cannot scan. A non-string input, or one larger than a fixed byte ceiling, is not "best-effort scanned" — it is replaced wholesale with a placeholder. Partial redaction is worse than no output, because it looks clean while a pattern you didn't reach sails through. Same for a regex that throws (catastrophic backtracking on hostile input): the catch returns the placeholder, never the raw string.

containsPii is defined in terms of the redactor, not a parallel ruleset. It is literally "does redaction change the string?" — redactForAi(x) !== x. One place to keep correct, and it inherits the same bias: oversize or non-string returns true (assume PII).

That's the floor. Be honest about what a floor is: regex does not understand meaning. A person's name, a street address without a clean number-and-street shape, "the patient in room 4B" — none of that matches a pattern, none of it gets redacted. Which is exactly why there's a second layer, and exactly why it's ordered the way it is.

Rule 2: an AI second opinion — ordered so it can only make the verdict stricter

For the highest-sensitivity calls we add a model-backed guard on top of the regex. The temptation is to "ask the AI if this is safe" and trust its answer. That's backwards and dangerous: now an AI hiccup can block legitimate traffic or wave through the obvious. The guard is built so the AI can only ever tighten the decision:

1. Deterministic first. Run the regex containsPii. If it flags PII, return unsafe immediately — no AI call, no spend on the obvious cases.
2. AI only on the survivors. If and only if the regex passed, ask a small fast instruct model one narrow yes/no: does this still contain PII the regex missed? Unsafe only on a clear positive.
3. Fail back to the verdict you already trust. On any AI error, timeout, or ambiguous output, return the regex verdict — which already passed.

Read what step 3 means: the AI can move a "safe" to "unsafe," never the reverse. It can't rescue an input the regex already condemned, and if the model is down, the guard degrades to exactly the deterministic behavior you had before you added it. An outage in the fancy layer can't make you less safe than the boring layer alone — and it can't start blocking real traffic on its own either.

The two layers fail in opposite directions, on purpose:

• Redaction fails closed — when unsure, redact everything.
• The AI guard fails to the already-safe verdict — when the model is unsure or unreachable, defer to the deterministic check that already ran.

The layer that decides what content leaves must never leak on uncertainty. The layer that decides whether a whole high-sensitivity flow proceeds must never take that flow down because a best-effort model timed out. Same phrase — "fail safe" — opposite behavior at the two layers. Getting that right is the difference between defense-in-depth and two ways to break.

If you take three things

1. Redact before the call, not after. The mistake is persisting the raw input "for debugging" and redacting only the copy you show a human. The copy that matters is the one crossing your boundary to the vendor.
2. Make the deterministic layer fail closed. Oversize, non-string, or a throwing regex must yield a placeholder, never the original.
3. Let the model only tighten. A regex floor plus an AI ceiling is strong; an AI floor is not a floor at all. Order them so a model outage costs you nothing you had, and a false-negative can never loosen a decision the deterministic check already made.

---

I'm building Merlonix, which monitors uptime, SSL/TLS, DNS, and answer-engine presence — and the AI features that summarize and triage all sit behind the two layers above. Free, no-signup tools if useful: the MCP server health check and domain health scan.