Catching broken vertical game clips before posting, with row-wise Laplacian sharpness

#gamedev #python #ffmpeg #showdev

We are a tiny studio, and we post a lot of 9:16 clips of our games. Last week one of them went out broken: the gameplay sat in the top half, the bottom third was an empty blurred background, a debug overlay was still visible, and the game's name was nowhere on screen.

Nobody "looked" because everybody assumed someone else had. So we wrote a check that a script runs before any vertical video can be posted. It has no ML model. It uses ffmpeg, numpy and one old trick: row-wise Laplacian sharpness.

The idea

Our vertical template puts the gameplay (landscape) in the middle of a 1080x1920 canvas. Above and below it we put a blurred copy of the same footage, plus two text bands: a hook line on top and the game name below.

Blurred background is soft. Real gameplay and text are sharp. So if you measure "how sharp is each row of pixels", you get a vertical profile of the frame. From that profile you can read where the real content sits.

Sharpness per row

For each sampled frame (1s, 25%, 50%, 75% of the clip), grab a PNG with ffmpeg:

ffmpeg -y -ss 12.5 -i clip.mp4 -frames:v 1 f2.png

Then a 4-neighbour Laplacian, averaged across each row:

import numpy as np
from PIL import Image

g = np.asarray(Image.open("f2.png").convert("L"), dtype=np.float32)
H, W = g.shape
lap = np.abs(4 * g[1:-1, 1:-1] - g[:-2, 1:-1] - g[2:, 1:-1]
             - g[1:-1, :-2] - g[1:-1, 2:])
row = np.zeros(H, dtype=np.float32)
row[1:-1] = lap.mean(axis=1)   # one number per row

On our footage, blurred background and flat colour rows land around 0.2–1.0. Gameplay rows are well above that. We call a row "sharp" at 2.0 or more.

What we check

Empty margins. Find the first and last sharp rows (ignoring runs shorter than 2 rows). Everything outside them is "blank". Our good template has about 27.6% blank at the bottom by design. The broken clip had 34–49%. The limit is 31%.
Is the gameplay centred? Mask out the two text bands, then join sharp runs with gaps of 60 rows or less (dark game scenes have soft rows inside them). Take the longest block of at least 100 rows. Its centre must be between y=760 and y=1040.
Is there text in the bands? In the hook band and the name band we check two things: at least 2% of pixels are bright (above 190), and the band's average sharpness is at least 3.0. White text on a blurred background passes. An empty band fails.

One trick that mattered: a single frame can be a dark scene with almost no sharp rows. So for the "centred" check we take the maximum of each row across all four frames and judge once. Margins and text bands are still judged per frame.

The gate

The check writes a contact sheet with the safe-zone lines drawn on it, and a JSON line with the file's sha256 and the measured values. Our posting command refuses a vertical mp4 unless:

the machine check passed for that exact sha256, and
a different person (not whoever made the clip) has looked at the sheet and signed off with a note.

The second rule is not technical, but it is the one that would have saved us last week.

What it doesn't catch

A debug overlay inside the gameplay area. We now crop it out when building the clip, but the check can't tell UI from gameplay.
A sharp but wrong frame (for example a menu screen). That's what the human sign-off is for.
Thresholds are tuned to our template. If your background isn't blurred, row sharpness won't separate content from margin.

It runs in a few seconds per clip, and on last week's broken clip it fails on the bottom margin straight away.

We make small games and apps in Seoul. The clip that started this was for Leather Atelier, a cozy bag-crafting game we're building in Godot.

Written by our in-house AI assistant from our actual code and measurements, and marked as AI-generated.

DEV Community