I've been running LiteLLM as a gateway in front of a few models, and hit two things that turned out to be more annoying than expected.
First, PII redaction. LiteLLM supports it via Presidio, but that means deploying and maintaining two extra containers alongside the proxy, plus YAML config per entity type. For a small deployment that's more moving parts than the gateway itself.
Second, audit logs. If you need a record of what was redacted and when — which you do, the moment anyone in compliance asks — that's an Enterprise feature.
So I wrote a custom guardrail that does both in-process. No sidecars, no external calls, standard library only.
yaml
guardrails:
- guardrail_name: cloakwall litellm_params: guardrail: cloakwall.guardrail.Cloakwall mode: pre_call default_on: true redaction_mode: mask audit_path: /var/log/cloakwall/audit.log in: "Patient MRN: A1234567, email bob@clinic.org, card 4111111111111111" out: "Patient , email , card " The part that actually took the time
Writing regexes for email and credit cards is easy. Making them not fire constantly is the hard bit, and it's where I'd expect most naive implementations to fail.
Two examples.
NHS numbers look exactly like US phone numbers. Both are 10 digits in a 3-3-4 grouping. 555-867-5309 matched my NHS pattern perfectly, and every phone number in my test set came out labelled as a health identifier. The fix is the modulus-11 check digit that real NHS numbers carry: weight the first nine digits 10 down to 2, sum, take mod 11, subtract from 11. Now genuine NHS numbers get labelled NHS and phone numbers don't.
Every 16-digit number looks like a credit card. Order numbers, transaction IDs, tracking references. Without a Luhn check, 1234567890123456 gets redacted and your users lose trust in the redactor within about a day. With it, only real card numbers match.
There's a third one I only caught by testing: if you replace matches as you find them, the offsets shift under every subsequent detector and your audit record points at the wrong positions. Collect all spans first, sort by start position with longer matches winning ties, then resolve overlaps in one pass. That ordering is also what stops a card number being partly eaten by the phone pattern.
Audit logging
Each entry commits to the SHA-256 of the previous one. Edit or delete any line and verification tells you which:
chain intact
line 3: entry altered after writing
One deliberate choice: the log records which entity types were found and how many, never the values. An audit log that quotes the PII it just redacted is a second copy of the thing you were protecting. I have a test asserting the card number doesn't appear in the log file.
SIEM export (Splunk HEC, Datadog) runs on a worker thread. With a dead endpoint, 20 events took 214ms and never blocked. Your audit sink being down shouldn't fail an LLM request.
What it doesn't do
Regex won't catch contextual PII — a person's name in free text, or a condition described in prose. Presidio's NLP models are genuinely better at that, and if you need it, use them. The trade is that this has no dependencies, no sidecar, and nothing to ship into an air-gapped environment.
Everything runs in-process with no sockets opened in the redaction path, so it works unchanged in an isolated cluster. You can verify that with strace -f -e trace=connect.
31 tests, no pytest needed: python3 tests/test_cloakwall.py
Repo: https://github.com/Rishikesh-glitch/Cloakwall-v0.1 — AGPL-3.0. Feedback welcome, particularly if you've hit detection edge cases I haven't.
Top comments (3)
The offset-shift bug is the detail that tells me you actually shipped this rather than wrote it up from a tutorial — "replace as you find" quietly corrupting your audit spans is exactly the kind of thing that only surfaces once someone in compliance cross-references a log entry against the original text. Collecting all spans first and resolving overlaps in one pass with longest-match-wins is the right call. Two things I'd be curious about: first, the modulus-11 and Luhn checks are great for structured identifiers, but names and free-text addresses are where in-process regex approaches usually give up ground to Presidio's NER — did you scope PII to structured entities deliberately, or are you handling names some other way? Second, the SHA-256 hash-chain for tamper-evidence is elegant, but a single appended log is still vulnerable to truncation from the end (drop the last N lines and the chain still verifies). Anchoring the head periodically somewhere external closes that gap. Really nice writeup — the "record which entity types, never the values" choice is the kind of thing a lot of homegrown redactors get backwards.
You were right, and it took about two minutes to reproduce: ten entries, drop the last four, verify() returns "chain intact." Fixed now — monotonic sequence numbers plus a head anchor (seq + hash) written after every append and mirrored to the SIEM every N entries. Truncation now reports anchor records 10 entries, log holds 6.
Worth being precise about what that buys, though: an anchor on the same filesystem only stops accidental truncation and a careless attacker. An anchor in a SIEM under separate access control is what stops someone who can write to the pod. verify(anchor=...) takes a SIEM-retrieved anchor so you can check against a copy the log can't reach. I've documented it that way rather than claiming more than it does.
On names and addresses — deliberate, yes. The whole positioning is zero-dependency and air-gapped, and NER means shipping a model into an isolated cluster, which is exactly the friction I was trying to remove. So structured identifiers only, and I say so in the limitations rather than pretending regex handles free text. The right answer is probably a pluggable detector interface so Presidio can be a backend for people who want the recall and can afford the containers. That's next if anyone asks for it.
Thanks — genuinely useful catch.
Is contextual PII a blocker for your use case?