A teammate wanted to add a frame-differencing step to the job that turns our onboarding walkthroughs into GIFs, on the usual promise that storing only what changed roughly halves the file. Before agreeing to another build step I wanted a cheap per-frame number saying whether that step would have anything to remove: the bounding box of the changed pixels, and how much of that box didn't actually change. Getting it to measure the right thing took longer than writing it, and on noisy footage the box turned out to be set by a few stray pixels.
The samples are public, not customer screens: a line-art running horse from Wikimedia Commons (Janke, CC BY-SA 2.5, 307×230, 8 frames), a dim phone clip of a cat wagging its tail (Mateus2019, CC BY-SA 4.0, 320×240, 208 frames), and a camera pan I cut from the cat's first frame, a 240×180 window sliding 2 px right and 1 px down for 40 frames. Pillow 11.3.0 and numpy 2.0.2 on an M4 Mac.
Compare the frames a viewer sees
Seven of the horse's eight frames are stored as partial rectangles, so diffing raw frame data compares patches, not pictures. Pillow gives you the composited canvas when you seek and convert. I compare decoded RGB rather than palette indexes, because with per-frame local palettes one index can mean two colours. A pixel counts as changed when any channel moves by more than tol.
def change_boxes(path, tol=0):
gif = Image.open(path)
prev_rgb = None
for k in range(gif.n_frames):
gif.seek(k)
stored = tuple(gif.tile[0].extents) # read before convert(): loading empties .tile
rgb = np.asarray(gif.convert("RGB"), dtype=np.int16) # frame 0 is mode P, later RGB/RGBA
if prev_rgb is not None:
changed = (np.abs(rgb - prev_rgb) > tol).any(axis=2)
if not changed.any():
yield k, None, stored, 0.0, 0.0, 1.0
else:
rows, cols = np.nonzero(changed)
box = (int(cols.min()), int(rows.min()), int(cols.max()) + 1, int(rows.max()) + 1)
inside = changed[box[1]:box[3], box[0]:box[2]]
yield k, box, stored, inside.size / changed.size, changed.mean(), 1 - inside.mean()
prev_rgb = rgb
Two Pillow details broke my first version
Frame 0 decodes as mode P, a 2-D array of indexes, while later frames come back as RGB, or RGBA for the cat, so skipping the conversion died on frame 1 with a broadcast ValueError. The other trap: gif.tile holds the rectangle the file stores for the current frame, but convert() loads the frame and empties it, and reading it afterwards gives an IndexError that looks unrelated. The not changed.any() branch never fired here, but a recording paused on one page will hit it.
The original encoders already cropped as tight as possible
The first horse frame and the summaries at tol=0 (horse, cat, pan), trimmed from the real output:
f1 box=(5, 39, 282, 225) stored=(5, 39, 282, 225) area=73.0% moved=11.2% still_in_box=84.6%
7 diffs | area 73.6% | moved 12.8% | still_in_box 82.6% | full-canvas boxes 0 | no-change frames 0 | box==stored 7
207 diffs | area 100.0% | moved 91.4% | still_in_box 8.6% | full-canvas boxes 207 | no-change frames 0 | box==stored 207
39 diffs | area 100.0% | moved 82.0% | still_in_box 18.0% | full-canvas boxes 39 | no-change frames 0 | box==stored 39
The last column is the one I cared about: my box matched the rectangle stored in the file on every frame of all three samples. Whatever encoded them already cropped to the minimum, so a rectangle-only pass has nothing left to take.
One stray pixel keeps the box at full canvas
I raised the tolerance to get past the cat's sensor noise. At 16 only 11.3% of pixels differ from the previous frame, yet the box is the whole canvas in 177 of 207 comparisons, and the other 30 only shrank to 96.7%–99.6% of the frame. The pan, with 5.7% of pixels left, kept a full-canvas box every time. A bounding box stays maximal while anything changes near each edge. The horse's numbers didn't move between 0, 8 and 16; my guess is that 16 flat colours either match exactly or differ a lot. These tolerances compare against the real previous frame; an encoder reusing near-matches compares against what is already on screen, so its counts will differ.
The gate I kept
The box became a cheap skip check rather than a forecast. If it equals the stored rectangle on every frame or covers about the whole canvas, the lossless pass is skipped. Anything in between, like the horse's 73.6%, only earns a real encode, because the share of still pixels inside the box said nothing about bytes. 82.6% of the horse's box sat still, yet with one shared palette and my own encoder (weaker than whatever made the originals) the best lossless layout came to 97.5% of full frames. For files I'm not scripting, I use ImgIng (https://imging.ai/) in Chromium 149 on the same Mac; its animation tool runs only in desktop browsers, and Safari can't use it to compress an existing animated GIF. It switches between difference frames and full frames, and I don't know how it decides per frame, so I try both on the actual file. What I haven't settled is a tolerance that keeps our walkthroughs readable, since small UI text may break up well before 16.
Top comments (0)