Build a Clip-Join Review Sheet With FFmpeg

Two clips can look acceptable on their own and still make an awkward cut. The subject lands in a different place, a hand changes position, or the next shot begins halfway through another action.

For that review, I want the last few frames of the outgoing clip beside the first few of the incoming clip. A contact sheet makes small spatial changes easier to inspect without scrubbing back and forth.

Take actual frames from both sides

This script selects the last three decoded frames from the first clip and the first three from the second. Shorter clips contribute however many frames they have.

It gets timestamps from ffprobe instead of calculating a last-frame time from duration and average frame rate. FFmpeg then selects the corresponding zero-based frame indices.

Install Pillow and make sure ffmpeg and ffprobe are available on your PATH:

python -m pip install Pillow
ffmpeg -version
ffprobe -version

Save this as join_sheet.py:

import argparse
import json
import math
from pathlib import Path
import subprocess
import tempfile
from PIL import Image, ImageDraw, ImageOps


def probe(path):
    result = subprocess.run([
        "ffprobe", "-v", "error", "-select_streams", "v:0",
        "-show_frames", "-show_entries",
        "frame=best_effort_timestamp_time", "-of", "json", str(path),
    ], check=True, capture_output=True, text=True)
    frames = json.loads(result.stdout).get("frames", [])
    if not frames:
        raise ValueError(f"no video frames: {path}")
    times = [float(frame["best_effort_timestamp_time"]) for frame in frames]
    if not all(math.isfinite(t) for t in times):
        raise ValueError("non-finite frame timestamp")
    return times


def build(left, right, output):
    paths = [Path(left).resolve(), Path(right).resolve()]
    times = [probe(path) for path in paths]
    picks = [list(range(max(0, len(times[0]) - 3), len(times[0]))),
             list(range(min(3, len(times[1]))))]
    sheet = Image.new("RGB", (960, 448), "#151a1d")
    draw = ImageDraw.Draw(sheet)
    manifest = []
    with tempfile.TemporaryDirectory() as temporary:
        for row, (path, stamps, indices) in enumerate(zip(paths, times, picks)):
            selection = "+".join(f"eq(n\\,{index})" for index in indices)
            pattern = str(Path(temporary) / f"row{row}-%02d.png")
            subprocess.run([
                "ffmpeg", "-v", "error", "-i", str(path), "-map", "0:v:0",
                "-vf", f"select={selection}", "-fps_mode", "vfr",
                "-frames:v", str(len(indices)), pattern,
            ], check=True)
            for column, index in enumerate(indices):
                frame = Path(temporary) / f"row{row}-{column + 1:02d}.png"
                with Image.open(frame) as image:
                    thumb = ImageOps.contain(image.convert("RGB"), (320, 180))
                x = column * 320 + (320 - thumb.width) // 2
                y = row * 224
                sheet.paste(thumb, (x, y))
                side = "OUT" if row == 0 else "IN"
                draw.text((column * 320 + 8, y + 188),
                          f"{side} frame {index} | PTS {stamps[index]:.6f}s")
                manifest.append({"side": side, "file": str(path),
                                 "frame": index, "pts_seconds": stamps[index]})
    output = Path(output)
    output.mkdir(parents=True, exist_ok=False)
    sheet.save(output / "join.png")
    (output / "frames.json").write_text(json.dumps(manifest, indent=2) + "\n")
    return manifest


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    for name in ("left", "right", "output"):
        parser.add_argument(name)
    args = parser.parse_args()
    try:
        result = build(args.left, args.right, args.output)
    except (OSError, ValueError, KeyError, subprocess.CalledProcessError) as error:
        parser.exit(2, f"error: {error}\n")
    print(f"Saved {len(result)} frames to {args.output}/join.png")

Review the cut

python join_sheet.py outgoing.mp4 incoming.mp4 join-review

The new directory contains join.png and frames.json. The top row is OUT; the bottom row is IN. Both rows run chronologically from left to right. Each label gives the frame index and its source presentation timestamp in seconds, which need not start at zero.

I use three checks while looking at the sheet:

• Position and scale: does the subject jump without a motivated change of viewpoint?
• Action phase: does the incoming movement follow the outgoing movement, or restart it?
• Light and background: is there an accidental change that distracts from the cut?

The small synthetic check used two clips with a rectangle in the same position, then moved the incoming rectangle to the opposite side. The sheet exposes that displacement. One-frame and variable-frame-rate inputs were also checked.

This version decodes the clips rather than seeking, so I use it for short takes. It reviews the files' outer boundaries: export trimmed segments first if the intended edit falls inside either file.

Once a mismatch is visible, I can try a different in-point or take. I still play the cut at speed to decide whether the movement works; the sheet gives me a precise place to investigate.

References: ffprobe frame output and FFmpeg's select filter.