I've been running LiteLLM as a gateway in front of a few models, and hit two things that turned out to be more annoying than expected.
First, PII redaction. LiteLLM supports it via Presidio, but that means deploying and maintaining two extra containers alongside the proxy, plus YAML config per entity type. For a small deployment that's more moving parts than the gateway itself.
Second, audit logs. If you need a record of what was redacted and when — which you do, the moment anyone in compliance asks — that's an Enterprise feature.
So I wrote a custom guardrail that does both in-process. No sidecars, no external calls, standard library only.
yaml
guardrails:
- guardrail_name: cloakwall litellm_params: guardrail: cloakwall.guardrail.Cloakwall mode: pre_call default_on: true redaction_mode: mask audit_path: /var/log/cloakwall/audit.log in: "Patient MRN: A1234567, email bob@clinic.org, card 4111111111111111" out: "Patient , email , card " The part that actually took the time
Writing regexes for email and credit cards is easy. Making them not fire constantly is the hard bit, and it's where I'd expect most naive implementations to fail.
Two examples.
NHS numbers look exactly like US phone numbers. Both are 10 digits in a 3-3-4 grouping. 555-867-5309 matched my NHS pattern perfectly, and every phone number in my test set came out labelled as a health identifier. The fix is the modulus-11 check digit that real NHS numbers carry: weight the first nine digits 10 down to 2, sum, take mod 11, subtract from 11. Now genuine NHS numbers get labelled NHS and phone numbers don't.
Every 16-digit number looks like a credit card. Order numbers, transaction IDs, tracking references. Without a Luhn check, 1234567890123456 gets redacted and your users lose trust in the redactor within about a day. With it, only real card numbers match.
There's a third one I only caught by testing: if you replace matches as you find them, the offsets shift under every subsequent detector and your audit record points at the wrong positions. Collect all spans first, sort by start position with longer matches winning ties, then resolve overlaps in one pass. That ordering is also what stops a card number being partly eaten by the phone pattern.
Audit logging
Each entry commits to the SHA-256 of the previous one. Edit or delete any line and verification tells you which:
chain intact
line 3: entry altered after writing
One deliberate choice: the log records which entity types were found and how many, never the values. An audit log that quotes the PII it just redacted is a second copy of the thing you were protecting. I have a test asserting the card number doesn't appear in the log file.
SIEM export (Splunk HEC, Datadog) runs on a worker thread. With a dead endpoint, 20 events took 214ms and never blocked. Your audit sink being down shouldn't fail an LLM request.
What it doesn't do
Regex won't catch contextual PII — a person's name in free text, or a condition described in prose. Presidio's NLP models are genuinely better at that, and if you need it, use them. The trade is that this has no dependencies, no sidecar, and nothing to ship into an air-gapped environment.
Everything runs in-process with no sockets opened in the redaction path, so it works unchanged in an isolated cluster. You can verify that with strace -f -e trace=connect.
31 tests, no pytest needed: python3 tests/test_cloakwall.py
Repo: https://github.com/Rishikesh-glitch/Cloakwall-v0.1 — AGPL-3.0. Feedback welcome, particularly if you've hit detection edge cases I haven't.
Top comments (0)