DEV Community: Trevor

Redaction is Not Enough: When an LLM can still Infer the PII You Stripped Out

Trevor — Tue, 30 Jun 2026 16:52:20 +0000

A reader left a sharp question on my last post about redacting PII before sending prompts to an LLM. Paraphrased: If you redact "John from ACME" to "[NAME] from [COMPANY]", can the model still infer who it is?

Yes. And it is worth pulling that apart, because it points to a real limit that a lot of teams miss. Redaction handles the identifiers it can see. It does not stop a model from reasoning about everything you left in.

Two problems people treat as one
When people say "keep PII out of my prompts," they are usually worried about two different things:

Direct personal data sitting verbatim in a third party's logs. The literal email, phone number, SSN, or card number ends up in OpenAI's or Anthropic's systems, outside your control. This is the concrete GDPR and HIPAA exposure for most teams.
The model inferring or re-identifying someone from what is left, even after the obvious identifiers are gone.
Redaction solves the first problem completely. Swap the direct identifiers for placeholders, and they never appear verbatim in logs you do not own. For a large share of real compliance work, that is the whole job, and it is cheap to do.

The second problem is different and harder. It is not really a redaction problem at all.

How inference leakage actually happens
Three common ways the data you stripped comes back:

Quasi-identifiers. No single field identifies a person, but a combination does. Latanya Sweeney's well-known finding is that ZIP code, date of birth, and sex uniquely identify roughly 87 percent of Americans. Strip the name and email, leave "a 41-year-old cardiologist in Boise who joined in March," and you may have described exactly one person.

Context that gives away the blank. Placeholders do not help if the surrounding text answers the question for the model. "[NAME] runs the electric vehicle company in Austin" leaves very little to guess. The model fills in the blank because you told it how.

Rare details. An unusual job title, a specific dollar amount, a distinctive phrasing lifted from a support ticket. The more unique the detail, the more it points at one person.

What to actually do about it
This is an architecture decision, not a setting you flip. A few practical layers:

Minimize the context, not just the identifiers. Send the model only what the task needs. If you are summarizing a complaint, it probably does not need the customer's age, employer, and city. Data minimization is the highest-leverage move and the one most people skip.

Generalize the quasi-identifiers you do need. When the task allows, bucket the specifics. Exact age becomes an age range. A city becomes a region. "Cardiologist in Boise" becomes "a physician." You keep what the model needs to do the work and drop what re-identifies.

Match the protection to the threat. If your only goal is keeping direct PII out of logs you do not control, redaction is enough and you can stop there. If the content itself is sensitive enough that inference matters, that is a deliberate design choice, and you handle it above the redaction layer.

For the strictest cases, keep it off shared models. When inference is genuinely unacceptable, the clean answer is a self-hosted or local model. If the text never reaches a third party, there is nothing to infer in someone else's system.

Restore on your side and check the output. The model can reintroduce details or guess at them. Doing the restore step yourself, and reviewing what comes back before it reaches a user keeps that contained.

Where redaction fits
Redaction is the necessary first layer. It guarantees the explicit email, phone, SSN, card, and name never sit verbatim in logs you do not control, which is the concrete compliance win and the easiest to ship. Inference protection is a second layer you reach for based on how sensitive the content is. Most teams need the first. Regulated, high-sensitivity workloads need both.

*To be clear about my own tool: the API I built does the first layer. It detects and tokenizes direct identifiers fast and in-process, and restores them in the reply. It does not claim to stop inference, because that lives in how you design the prompt and what context you choose to send. Anyone who tells you a redaction step alone makes an LLM "private" is selling you something.
*

LLM Privacy Shield on RapidAPI if you want the first layer handled. The second layer is on you, and now you know what it is.

Wrapping up
Redaction is necessary, not sufficient. Strip the direct identifiers so they stay out of logs you do not own. Minimize the context so there is less to infer from. Generalize the quasi-identifiers you have to keep. And for the most sensitive work, keep it off shared models entirely. Redaction is the floor, not the ceiling.

If you missed the first post on the redact then restore pattern, it is here: How to redact PII before sending prompts to OpenAI, Claude, or Gemini.

Redaction is Not Enough: When an LLM can still Infer the PII You Stripped Out

Trevor — Tue, 30 Jun 2026 16:52:20 +0000

Two problems people treat as one
When people say "keep PII out of my prompts," they are usually worried about two different things:

The second problem is different and harder. It is not really a redaction problem at all.

How inference leakage actually happens
Three common ways the data you stripped comes back:

Rare details. An unusual job title, a specific dollar amount, a distinctive phrasing lifted from a support ticket. The more unique the detail, the more it points at one person.

What to actually do about it
This is an architecture decision, not a setting you flip. A few practical layers:

LLM Privacy Shield on RapidAPI if you want the first layer handled. The second layer is on you, and now you know what it is.

If you missed the first post on the redact then restore pattern, it is here: How to redact PII before sending prompts to OpenAI, Claude, or Gemini.

How to Redact PII before sending prompts to OpenAI, Claude, or Gemini

Trevor — Wed, 17 Jun 2026 19:21:56 +0000

If you send user text to an LLM, you are probably sending personal data with it without meaning to. A support message, a chat transcript, a pasted form. They carry names, emails, phone numbers, and sometimes card numbers, and all of it ends up in your prompt. Once that prompt leaves your server, the personal data is sitting in someone else's logs, which is a real problem under GDPR and HIPAA.

The fix is simpler than most people expect, and it does not mean giving up the model. You redact the personal data before the prompt goes out, send the safe version to the model, then put the real values back into the answer. This post walks through that pattern with working code.

The pattern in three steps

Redact. Take your raw text and swap each piece of personal data for a placeholder like [EMAIL_1]. Keep a small map on your side that records which placeholder stands for which real value.
Call the model with the redacted text. The model only ever sees placeholders, so the real data never lands in logs you do not control.
Restore. Take the model's reply and your map, and put the real values back so the final output is useful to your user.

That is the whole idea. The personal data takes a round trip through placeholders and never leaves your stack.

A real example

Say you run a support tool that uses GPT to summarize customer emails. A customer writes in with their email and phone number. You want the summary, but you do not want to ship their contact details to OpenAI.

Here is the flow end to end. I am using a small API I built for the redact and restore steps, but the shape is the same no matter how you handle those two calls.

// Real example: summarize a customer support email with GPT,
// without sending any personal data to OpenAI.
const SHIELD = "https://llm-privacy-shield.p.rapidapi.com";
const shieldHeaders = {
  "content-type": "application/json",
  "x-rapidapi-host": "llm-privacy-shield.p.rapidapi.com",
  "x-rapidapi-key": "YOUR-RAPIDAPI-KEY"
};

async function summarizeSafely(customerEmail) {
  // 1. Redact PII before the text leaves your server.
  const { redacted, token_map } = await fetch(SHIELD + "/api/redact", {
    method: "POST",
    headers: shieldHeaders,
    body: JSON.stringify({ text: customerEmail })
  }).then(res => res.json());

  // customerEmail:
  //   "Hi, my confirmation went to john@acme.com but nothing arrived.
  //    Please call me back on 415-555-0132."
  // redacted:
  //   "Hi, my confirmation went to [EMAIL_1] but nothing arrived.
  //    Please call me back on [PHONE_1]."

  // 2. Send only the safe version to your model. OpenAI is shown here.
  //    Anthropic and Gemini work the same way, just swap this call.
  const completion = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [
      { role: "system", content: "Summarize this support email in one sentence." },
      { role: "user", content: redacted }
    ]
  });
  const summary = completion.choices[0].message.content;

  // 3. Put the real contact details back for your support agent.
  const { restored } = await fetch(SHIELD + "/api/restore", {
    method: "POST",
    headers: shieldHeaders,
    body: JSON.stringify({ text: summary, token_map })
  }).then(res => res.json());

  return restored;
}

The customer's email and phone number never reach OpenAI. The model summarizes placeholder text, and you swap the real values back in at the end for your support agent to read.

The two calls, side by side

If you just want to see the redact and restore calls on their own:

// The two calls, at a glance
const HOST = "https://llm-privacy-shield.p.rapidapi.com";
const headers = {
  "content-type": "application/json",
  "x-rapidapi-host": "llm-privacy-shield.p.rapidapi.com",
  "x-rapidapi-key": "YOUR-RAPIDAPI-KEY"   // from your RapidAPI dashboard
};

const r = await fetch(HOST + "/api/redact", {
  method: "POST",
  headers,
  body: JSON.stringify({ text: "Email john@acme.com about invoice 4521" })
}).then(x => x.json());

// r.redacted   => "Email [EMAIL_1] about invoice 4521"
// r.token_map  => { "[EMAIL_1]": "john@acme.com" }

const back = await fetch(HOST + "/api/restore", {
  method: "POST",
  headers,
  body: JSON.stringify({ text: modelReply, token_map: r.token_map })
}).then(x => x.json());
// back.restored => the model reply with the real values put back

The redact call hands you back the safe text and the token_map. You hold the map. The restore call takes that map and the model's reply and rebuilds the real answer.

The three modes

Redaction is not one size fits all. There are three modes worth knowing:

Tokenize (reversible). Replaces each value with a placeholder you can reverse later. Use this when you need the real values back in the answer.
Mask. Replaces a value with a generic label like <EMAIL>. Good when you never need it back.
Remove. Deletes the value entirely.

Tokenize is the default in the examples above, because the restore step depends on it.

Do you have to use a hosted API?

No. If you would rather self-host, Microsoft Presidio is a solid open source option for detecting and anonymizing PII. The redact then restore pattern is the part that matters, and it works the same whether you run it yourself or call a service.

I built a hosted version because I wanted the redact and the restore in one place, running in-process so the protected data never goes to a third party, with a response time under a millisecond. It detects emails, phone numbers, SSNs, credit cards, IP addresses, and API keys, and there is a free tier if you want to try it without committing to anything paid:

LLM Privacy Shield on RapidAPI

Wrapping up

If your app sends user text to an LLM, run it through a redact step first. Keep the map, send placeholders to the model, restore at the end. Your users get the same useful output, and the personal data stays where it belongs.

What new app or existing pipeline can you heighten privacy using this API?