Masking PII in Kubernetes: How we solved 3 annoying sidecar edge cases (v2.0.0)

Building a mutating webhook for Kubernetes is easy in tutorials, but brutal in production. You immediately hit the reality of volume permissions, security contexts, and zombie sidecars.

I recently released v2.0.0 of the PII-Shield Operator. It’s a Go-based tool that injects a sidecar into your pods to mask sensitive data (PII) before the logs hit Datadog or ELK.

Getting the core Shannon entropy logic to work was step one. Making it bulletproof for strict SOC2 environments was step two. Here are the three K8s edge cases we solved for this release:

1. Dropping the Shell (Moving to Distroless)
   Security teams hate sidecars with shell access. In earlier versions, we used Alpine. Now, the agent is compiled with CGO_ENABLED=0 and deployed on gcr.io/distroless/static:nonroot. There is no /bin/sh and zero attack surface. It just tails the log files directly using native Go.

2. The "Immortal Sidecar" Problem
   If you have ever injected sidecars into a K8s Job, you know it breaks the lifecycle. The main container finishes its work, but the sidecar keeps tailing logs forever, so the Job never reaches the Completed state.
   To fix this, we moved to the new Native Sidecars feature (K8s 1.28+). The webhook now puts the agent inside the initContainers array with RestartPolicy: Always. Kubernetes finally understands how to kill the sidecar gracefully when the main app exits.

3. The emptyDir Permission Trap
   When the webhook mounts an emptyDir volume to share logs, you often get Permission Denied. If the user's main app runs as root with a strict umask 0077, the nonroot sidecar can't read the file.
   Instead of forcing users to rewrite their manifests, the Mutating Webhook now handles it silently. It checks the pod's SecurityContext and automatically injects fsGroup: 65532. The volume permissions match up, and the logs flow without errors.

I packaged the whole thing into a Helm chart so it takes about two minutes to test. You just label your pod with pii-shield.io/inject: "true" and the webhook handles the rest.

Comments

[Ilya Ploskovitov]

Source code and Mutating Webhook implementation here:
github.com/aragossa/pii-shield