The simplest batch transcode job has one stage: for each file, encode. I wanted to know where a "not worth it" check should live in front of that stage. The trigger was a small comparison on five Creative Commons clips from Wikimedia Commons (WebM, VP8 and VP9, not phone footage). HandBrakeCLI with its Fast 1080p30 preset took one 2,030,217-byte clip to 5,853,449 bytes and left another, 22,264,465 bytes, at 21,772,446. The encoder did what it was told. Everything below is one run on an Apple M4 and belongs to these clips only.
Why the encoder is the wrong place
In constant-quality mode the encoder aims at a quality level, not a size. My reading is that a source already below the bitrate that level implies gets extra bits, which would explain why the file grows. I measured this with a matrix of 36 points: four kinds of content, each pre-encoded at nine CRF values, then re-encoded once with x264 at CRF 23, preset medium. Every one of the 24 points whose source had fewer than 0.08 bits per pixel per frame came out bigger, between 1.34x and 6.67x. Bits per pixel per frame, bpp from here on, is video bitrate divided by width times height times frame rate.
The part that matters for placement is what quality did. On the moose clip pre-encoded at CRF 36, SSIM against the master went from 0.9205 to 0.9180 while the file grew 3.76x. My guess is that the extra bits go into preserving the earlier damage; I did not check that inside the encoder. What the numbers do show is that SSIM barely moved while the size grew 3.76x, so a quality metric alone would not have shown the problem, and a size comparison would.
A probe stage before the queue
So the first check sits ahead of the queue, and it only needs the container header. ffprobe reads codec, size, frame rate and bitrate without decoding a frame, so it is cheap on every file. One snag on WebM, where the stream-level bitrate is not filled in:
$ ffprobe -v error -select_streams v:0 -show_entries stream=codec_name,width,height,r_frame_rate,bit_rate:format=duration,size,bit_rate -of default=nw=1 input.webm
On the moose clip (1080p VP9 in WebM) the stream-level bit rate is not reported, while the format-level figure is, at about 2.06 Mbps; the script needs a fallback for that.
The probe has to fall back to the container bitrate, or file size times eight over duration. That figure includes audio, so it reads high, which pushes a file toward being encoded, not skipped. I accept that direction: a wrongly encoded file costs time, a wrongly skipped one just stays as it was.
At a threshold of 0.10, 26 of the 36 points fall below it, and 24 of those grew. The other two saved 9% and 4%. Of the 10 above it, 8 saved 10% or more, so nothing worth 10% got skipped. In a separate run of four files, the probe passed one (lecture-high, bpp 0.2333) and would have skipped the other three, among them lecture-low (0.0265) and clip-mid (0.0440). I re-encoded those three once at CRF 23, preset medium, audio removed. lecture-high went from 8,066,510 to 4,513,680 bytes, a saving of 44%. lecture-low went from 920,021 to 2,183,646 bytes, and clip-mid from 2,670,854 to 3,949,234, which is 2.37x and 1.48x.
The threshold belongs to the preset
I would not hard-code the number in the probe script, because 0.10 is tied to the target. With CRF 28 as the target, 20 of the 21 points under 0.05 grew, which suggests the line drops to about 0.06 when the target gets more aggressive. That is an inference; I did not test it beyond the matrix. The number belongs in the same config as the preset, or the gate goes stale the day someone changes the preset.
A size comparison after the encode
The probe should not be the only check. Between 0.08 and 0.13 the four points saved only 4% to 9%. Behind the encoder I would compare output bytes with input bytes and keep the smaller file, maybe with a minimum saving. I did not run this as a pipeline, so it is a design suggestion, not a result. It costs a full encode per file, so it is the second line.
If you would rather not own the pipeline, ImgIng https://imging.ai/ takes several files at once in a desktop browser and lets you pick the tier. Its default recommended tier kept the original on four of the five clips; the site says a result is adopted only if it saves at least 12% and passes a quality review. How it decides internally I do not know, and it is not a part I worked on. On the near-lossless tier the five clips came out at 65% to 86% of the original, two of them larger than the HandBrake output. Settings differ, so this is no ranking.
What this does not cover
The matrix is H.264 to H.264 only. On the five real VP8 and VP9 clips, re-encoded at CRF 23 and medium, the rule held: the 0.032 bpp clip came out at 2.805x, the 0.113 one at 0.831x. I did not test H.265, AV1, bitrate-target mode or other presets, so how the threshold moves there is open.
Sample clips are from Wikimedia Commons and were re-encoded and trimmed, so they are modified: "Moose by the road.webm" by Forza1000 (CC BY-SA 4.0), "Worldwide Demonstration – March 2021 Amsterdam.webm" by Karen Eliot (CC BY-SA 2.0), and "Demonstration of Ethiopian residents in Israel, July 2019.webm" by Israel Police (CC BY-SA 3.0). The matrix also used one synthetic test pattern.
Top comments (0)