A support message can contain two different things: the task you want an AI model to perform, and details the model does not need to perform it.
“Draft a reply about a delayed delivery” is the task. A customer's email address in that same message may be unnecessary context.
I am building Sether around that boundary. It is an open-source library that replaces detected sensitive values with stable tokens before text goes to an LLM. Your application keeps the mapping and can restore recognized tokens in the response.
The useful question is not “did we add a privacy tool?” It is “what actually left the application?”
A practical evaluation has four parts:
- Start with fictional data. Create messages containing values your workflow encounters, plus ordinary text that should remain untouched.
- Inspect the redacted request before sending it anywhere. Count missed values and unnecessary replacements separately.
- Check task quality. Can the model still draft a useful response with the reduced context?
- Check restoration. If the response preserves a recognized token and the mapping is available, restore it. Test altered tokens and missing mappings as failure cases too.
Streaming adds another case: a value may be split across chunks. Include that in your tests rather than relying only on complete strings.
The token mapping also deserves attention. Decide which request or user owns it, who can read it, and how long it should exist. Redacting the outbound prompt is only one part of the application's data flow; logs, traces, attachments and downstream tools need their own review.
Sether is not a promise that every sensitive value will be detected, and installing it does not establish regulatory compliance. Detector selection, configuration and workflow evaluation matter.
The released library is MIT licensed. The hosted gateway is not part of this release.
Source and installation instructions: https://github.com/raeven-co/sether
If you build AI support or internal-assistant workflows, which test would you want a redaction library to pass before you put it between your app and a model?
Disclosure: I am Sether's founder. This article was prepared with AI assistance using the project's documentation.
Top comments (1)
the honesty in the disclaimer landed harder than the features. "not a promise that every sensitive value will be detected" is the line most library authors won't write. and the four-part evaluation is solid, especially testing the streaming chunk-split case. most redaction writeups stop at happy-path detection