Zero-code PII sanitization for Fluentd and Fluent Bit in Kubernetes

TL;DR: send app logs through a PII-Shield sidecar before Fluentd or Fluent Bit read them, then collect only masked identifiers like [HIDDEN:a1b2c3] instead of maintaining regex filters that break the moment a new secret format shows up.

Fluentd and its lighter C-based sibling Fluent Bit are the most common unified logging layers in Kubernetes — CNCF graduated projects with a huge plugin ecosystem for routing logs to almost any backend (Elasticsearch, Loki, S3, Datadog, Splunk). That flexibility is exactly why they end up sitting directly in the path of every raw log line your applications write.

If an application logs a user's email, an API key, or a card number, Fluentd/Fluent Bit will happily parse, buffer, and ship that value to wherever you've configured — permanently, and usually to more than one destination at once.

The Standard Approach: Filter-Layer Masking

Both projects expose a way to scrub fields in-flight. Fluentd uses record_transformer with Ruby's gsub:

<filter app.**>
  @type record_transformer
  enable_ruby true
  <record>
    message ${record["message"].gsub(/[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+/, '[REDACTED]')}
  </record>
</filter>

Fluent Bit doesn't ship a built-in PII filter, so the common workaround is a lua filter calling a hand-written script:

[FILTER]
    Name    lua
    Match   *
    script  redact.lua
    call    redact_pii

function redact_pii(tag, timestamp, record)
    if record["message"] then
        record["message"] = string.gsub(record["message"],
            "[%w.+-]+@[%w-]+%.[%w.-]+", "[REDACTED]")
    end
    return 1, timestamp, record
end

Why this approach breaks down at scale:

1. The Regex Trap: Every secret shape — API keys, session tokens, internal ID formats — needs its own pattern. Miss one and it ships in cleartext with no warning, because nothing fails; it just silently doesn't match.
2. Per-Line Interpreter Overhead: Fluent Bit's Lua filter runs an embedded Lua interpreter for every single record. On high-throughput pods this is a well-known source of added latency and CPU pressure on the node running the DaemonSet — worse than Fluentd's native Ruby path, but both add real per-line cost that scales with log volume, not with how much PII is actually present.
3. Loss of Context: A static [REDACTED] destroys correlation. If ten log lines all become [REDACTED], you can no longer tell whether they're the same user hitting an error ten times or ten different users.

The Zero-Code Alternative: PII-Shield Sidecar

Instead of asking Fluentd's Ruby engine or Fluent Bit's Lua interpreter to parse and pattern-match every line, move sanitization to a dedicated, low-allocation Go sidecar that sits before either one ever sees the log: PII-Shield.

The Result:

// What your app generated:
{"level":"info", "message":"User authenticated", "email":"[e-mail]", "token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."}

// What Fluentd / Fluent Bit actually reads and ships:
{"level":"info", "message":"User authenticated", "email":"[HIDDEN:e9f1a2]", "token":"[HIDDEN:a1c7af]"}

How it works:

1. The App redirects its output to an ephemeral shared volume (like an emptyDir) instead of the main stdout.
2. PII-Shield (Sidecar) tails that file, scrubs it using entropy-based secret detection (no regex list to maintain for API keys and tokens), and prints the clean logs to its own stdout.
3. Fluentd or Fluent Bit — already running as a DaemonSet reading /var/log/containers/*.log — picks up the sidecar's stdout the same way it picks up any other container's, no config change required.

The record_transformer filter and the Lua script can both be deleted. Fluentd/Fluent Bit go back to doing what they do best — routing already-clean logs.

Kubernetes Implementation

This is the same sidecar pattern regardless of whether your DaemonSet runs Fluentd or Fluent Bit — neither needs to know PII-Shield exists.

apiVersion: v1
kind: Secret
metadata:
  name: pii-shield-secret
type: Opaque
stringData:
  pii-salt: "replace-with-a-long-random-value"
---
apiVersion: v1
kind: Pod
metadata:
  name: billing-service
  labels:
    app: billing
spec:
  containers:
    - name: billing-app
      image: billing-app:v2.1.0
      # The app writes its private output to a shared pipe/file
      command: ["/bin/sh", "-c"]
      args: ["./billing-binary > /var/run/logs/app.log 2>&1"]
      volumeMounts:
        - name: log-volume
          mountPath: /var/run/logs

    - name: pii-shield-sidecar
      image: thelisdeep/pii-shield:2.2.0
      env:
        - name: PII_SALT
          valueFrom:
            secretKeyRef:
              name: pii-shield-secret
              key: pii-salt
      # Scratch image: run the binary directly (no shell/tail). Reads, scrubs,
      # and outputs to stdout for Fluentd/Fluent Bit to pick up.
      command: ["/pii-shield"]
      args: ["--watch-file", "/var/run/logs/app.log"]
      volumeMounts:
        - name: log-volume
          mountPath: /var/run/logs

  volumes:
    - name: log-volume
      emptyDir: {}

Pro tip: The thelisdeep/pii-shield image is multi-arch (amd64/arm64).

Verify It's Actually Working

Before trusting the pipeline, confirm the sidecar's own output is already clean — don't just check what lands in your log backend, since a misconfigured mount can make Fluentd/Fluent Bit silently fall back to the app container's raw stream instead.

# Read the sidecar's own stdout directly — this is what Fluentd/Fluent Bit sees
kubectl logs billing-service -c pii-shield-sidecar --tail=50

# Confirm no raw emails/tokens survived
kubectl logs billing-service -c pii-shield-sidecar --tail=200 | grep -E "@|Bearer|sk-" | grep -v "\[HIDDEN:"

The second command should return nothing. If it does, that log line's format isn't being caught yet — file it as a scanner bug rather than assuming the pipeline is safe.

Honest Limitations

PII-Shield doesn't replace input validation or make it safe to log secrets on purpose — it's a last line of defense for what slips through anyway. It also doesn't retroactively clean logs already sitting in Elasticsearch, Loki, or wherever Fluentd already shipped them; it only sanitizes what passes through the sidecar from the moment it's deployed. And it's stream-based: for the same reason described above, always check the sidecar's own output during rollout instead of assuming the mount and file path line up on the first try.

---

Stop maintaining regex filters for secrets.

Check out the PII-Shield repository on GitHub and drop a star if this simplifies your Fluentd or Fluent Bit pipeline!