Blur faces in video with MediaPipe and FFmpeg (without the flicker)

📦 Code: github.com/USER/video-redact (replace before publishing)

TL;DR

We'll turn per-frame face detections into padded, time-held blur regions and render them with a single FFmpeg filter graph. You get: no flicker on missed frames, audio copied untouched, one re-encode. Python 3.10+, mediapipe 0.10.x, FFmpeg 6+.

🎯 The problem

A face detector gives you one box per face per frame. Blur exactly those boxes and the blur flickers whenever the detector misses a frame, which is when someone turns their head. In a redaction pipeline, the flicker frame is the leak.

So we do three things to the raw detections:

1. Pad each box so the hairline and jaw are covered.
2. Hold each detection for N frames so a missed frame is bridged.
3. Merge overlapping boxes in space and time into one wider region.

Then we render with FFmpeg: crop, blur, overlay, enabled only during each region's time window.

⚠️ Note: The detection step below was not run on real faces by me (no face footage or model download in my test environment). The padding, hold, merge and FFmpeg rendering steps were tested on a synthetic clip with hand-written boxes. Measure recall on your own footage before trusting any of it.

1. Set up 🛠️

python3 -m venv .venv && source .venv/bin/activate
pip install mediapipe opencv-python
# download a face detector model (BlazeFace short range) per the MediaPipe docs
# and save it as blaze_face_short_range.tflite
ffmpeg -version | head -1

2. Detect faces, frame by frame

This follows the MediaPipe Tasks API in video mode. Check the current docs for exact option names in your version.

# detect.py
import json, sys
import cv2
import mediapipe as mp
from mediapipe.tasks import python as mp_python
from mediapipe.tasks.python import vision

path, model = sys.argv[1], "blaze_face_short_range.tflite"
opts = vision.FaceDetectorOptions(
    base_options=mp_python.BaseOptions(model_asset_path=model),
    running_mode=vision.RunningMode.VIDEO,
    min_detection_confidence=0.3,   # low on purpose: for redaction, recall beats precision
)
cap = cv2.VideoCapture(path)
fps = cap.get(cv2.CAP_PROP_FPS)
w, h = int(cap.get(3)), int(cap.get(4))
dets, i = {}, 0
with vision.FaceDetector.create_from_options(opts) as det:
    while True:
        ok, frame = cap.read()
        if not ok:
            break
        img = mp.Image(image_format=mp.ImageFormat.SRGB,
                       data=cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
        res = det.detect_for_video(img, int(i * 1000 / fps))
        boxes = [[b.bounding_box.origin_x, b.bounding_box.origin_y,
                  b.bounding_box.width, b.bounding_box.height]
                 for b in res.detections]
        if boxes:
            dets[i] = boxes
        i += 1
json.dump(dets, open("boxes.json", "w"))
print(f"fps={fps} size={w}x{h} frames_with_faces={len(dets)}/{i}")

Note the deliberately low confidence threshold. A false positive costs you a blurred poster. A false negative costs you a published face.

3. Pad, hold, merge ✅

This is the tested part. boxes.json maps frame index to a list of [x, y, w, h].

# redact.py
import json, sys

def hold_and_pad(dets, fps, w, h, pad=0.35, hold_frames=15):
    out = []
    for f, boxes in sorted(dets.items(), key=lambda kv: int(kv[0])):
        f = int(f)
        for x, y, bw, bh in boxes:
            px, py = bw * pad, bh * pad
            x0, y0 = max(0, int(x - px)), max(0, int(y - py))
            x1, y1 = min(w, int(x + bw + px)), min(h, int(y + bh + py))
            # each detection covers [f, f + hold_frames): bridges missed frames
            out.append((f / fps, (f + hold_frames) / fps, x0, y0, x1 - x0, y1 - y0))
    return out

def _overlap(a, b):
    ax0, ay0, ax1, ay1 = a[2], a[3], a[2] + a[4], a[3] + a[5]
    bx0, by0, bx1, by1 = b[2], b[3], b[2] + b[4], b[3] + b[5]
    return ax0 < bx1 and bx0 < ax1 and ay0 < by1 and by0 < ay1

def merge(regions):
    """Union boxes that overlap in space AND in time into one wider region."""
    merged = []
    for r in sorted(regions, key=lambda r: r[0]):
        for i, m in enumerate(merged):
            if r[0] <= m[1] and _overlap(m, r):
                x0, y0 = min(m[2], r[2]), min(m[3], r[3])
                x1 = max(m[2] + m[4], r[2] + r[4]); y1 = max(m[3] + m[5], r[3] + r[5])
                merged[i] = (min(m[0], r[0]), max(m[1], r[1]), x0, y0, x1 - x0, y1 - y0)
                break
        else:
            merged.append(r)
    return merged

def to_filter(regions):
    if not regions:
        return "null", "[0:v]"
    parts, cur = [], "[0:v]"
    for i, (t0, t1, x, y, bw, bh) in enumerate(regions):
        nxt = f"[v{i}]"
        en = f"between(t,{t0:.3f},{t1:.3f})"
        parts.append(f"{cur}split[a{i}][b{i}];"
                     f"[b{i}]crop={bw}:{bh}:{x}:{y},boxblur=luma_radius=min(w\\,h)/4:luma_power=3[bl{i}];"
                     f"[a{i}][bl{i}]overlay={x}:{y}:enable='{en}'{nxt}")
        cur = nxt
    return ";".join(parts), cur

if __name__ == "__main__":
    dets = json.load(open(sys.argv[1]))
    fps, w, h = float(sys.argv[2]), int(sys.argv[3]), int(sys.argv[4])
    g, last = to_filter(merge(hold_and_pad(dets, fps, w, h)))
    print(g)
    print(last, file=sys.stderr)

Try it with a hand-written file (a face drifting over three frames with one missed, then a second face later):

{"10": [[200,100,80,80]], "11": [[204,100,80,80]], "13": [[212,102,80,80]], "60": [[400,200,60,60]]}

python3 redact.py boxes.json 30 640 360 > graph.txt 2> last.txt
cat graph.txt

[0:v]split[a0][b0];[b0]crop=148:138:172:72,boxblur=luma_radius=min(w\,h)/4:luma_power=3[bl0];[a0][bl0]overlay=172:72:enable='between(t,0.333,0.933)'[v0];[v0]split[a1][b1];[b1]crop=102:102:379:179,boxblur=luma_radius=min(w\,h)/4:luma_power=3[bl1];[a1][bl1]overlay=379:179:enable='between(t,2.000,2.500)'[v1]

Four detections became two regions: frames 10 to 13 merged into one wider box (the missed frame 12 is covered), and the later face got its own window.

4. Render with FFmpeg 🎬

ffmpeg -hide_banner -y -i in.mp4 \
  -filter_complex "$(cat graph.txt)" \
  -map "$(cat last.txt)" -map 0:a \
  -c:v libx264 -crf 23 -c:a copy out.mp4

ffprobe -v error -show_entries stream=codec_name,nb_frames -of csv out.mp4

stream,h264,120
stream,aac,174

All 120 video frames survive, and the audio is a stream copy, not re-encoded.

💡 Tip: If your source can have no audio, use -map 0:a? so the missing stream isn't an error.

⚠️ Note: One split/overlay pair per region is fine for short clips. For long footage with hundreds of regions, render in chunks and concatenate, or you'll hit slow filter graph setup.

5. Quality control (the real work) 🔍

No metric tells you a face leaked. So:

• Sample frames from every output, weighted toward the second after each hold ends.
• Review scene cuts and entrances/exits.
• Measure recall yourself: take 30 seconds of your worst footage and count frames with a visible face and no box.

Blurring is also not anonymization. Voices, name badges, license plates and the room itself can identify people, and there may be legal obligations depending on where you operate. Ask counsel.

What's next

• Add a tracker (OpenCV trackers or IoU matching) to bridge longer misses than a half-second hold.
• Swap boxblur for pixelize if your reviewers prefer an obviously redacted look.
• Run detection every Nth frame to cut compute, then raise hold_frames to match.

If you tune the pad and hold values on real footage, share what worked in the comments. #python #ffmpeg #video #tutorial