DEV Community: hao jia

Three ffmpeg one-liners that broke a portrait video, and the check that caught each one

hao jia — Tue, 13 Oct 2026 08:05:21 +0000

A portrait clip plays upright in a <video> element. Then it goes through a transcode job, and what comes out is either a wide, flattened picture or a clip lying on its side. I rebuilt that failure this week with three ffmpeg one-liners. Each of them looks like a reasonable thing to paste into a job definition, each was wrong in a different way, and none of them printed an error.

First, the test file. Portrait video is often stored with the pixels lying sideways, plus a rotation tag: a note in the container that tells the player to turn the picture at playback. My test file was built that way on purpose, and it is not a phone original. I took a public wind turbine clip, drew a red block in the top-left corner and a green one in the top-right so the orientation is readable at a glance, then used ffmpeg to store the pixels sideways at 1920×1080 with a rotation tag. The result is five seconds of H.264 with no audio. ffmpeg 7.1 prints the tag as rotation of -90.00 degrees, and the sign surprised me, because this is the file that needs a clockwise quarter turn to look right. Chromium 149, Chrome 154, Firefox 151 and WebKit 26.5 (macOS) all played it upright and reported videoWidth × videoHeight as 1080×1920. Playback was not where it broke.

ffmpeg -i in.mp4 -vf scale=1280:720 -c:v libx264 -crf 26 -an a.mp4
ffmpeg -i in.mp4 -c copy -metadata:s:v rotate=0 b.mp4
ffmpeg -display_rotation 0 -i in.mp4 -c copy c.mp4

The first line is the usual "make it 720p". The output was 1280×720 with no tag, and the picture was upright but squashed, a tall frame pressed into a wide one. What I had missed is that ffmpeg applies the rotation before my filter runs. The same input re-encoded with no filter at all comes out as 1080×1920 with no tag, so the scale filter was handed an upright portrait frame and told to become 1280×720. Changing the filter to scale=-2:720 gave 406×720 and a normal picture.

Look at the red and green blocks on the left. They were squares in the source, and here they are flat bars, which is the quickest way to tell a squashed frame from a cropped one.

The second line was my attempt to clear the tag without re-encoding. It changed nothing: b.mp4 was still 1920×1080 and still reported -90.00. This one is harmless and misleading at once, because the job goes green and I walk away believing the tag is gone. I only ran ffmpeg 7.1, so I can't say what other versions do with that metadata key.

The third line does remove the tag, and it was the worst of the three. c.mp4 is 1920×1080 with no rotation line. Stream copy never touches pixels, though, so they are still lying sideways, and now nothing tells the player to turn them. All four browsers played it on its side and reported 1920×1080. A metadata check would pass this file, and every browser I tried showed it sideways.

The red block has moved from the top-left to the bottom-left. The top of the picture now points left, which is exactly how the pixels were stored all along.

All three got past me the same way: I judged them by exit code. Now I read the output file back after every command, with two lines.

ffmpeg -hide_banner -i out.mp4
ffmpeg -noautorotate -i out.mp4 -frames:v 1 stored.png

The first prints the stream size and, when a tag exists, the rotation line. For a tagged file that size is the storage size, 1920×1080 here, not what a viewer sees. The second writes one frame without applying the tag, so I get the stored pixels as they are. I keep both because dimensions alone miss cases. My 180-degree sample is stored at 1080×1920 upside down, and its size looks perfectly normal. This machine has no ffprobe, so I haven't compared its output.

To check a route that doesn't involve my own flags, I used ImgIng (https://imging.ai/) in Chromium 149 with the preset that favors smaller files, and the three tagged samples, at 90, 270 and 180 degrees, each came back as 608×1080, stored upright, with no rotation tag. That is one browser, one preset and five-second clips, so I read it as a result for these files only.

There are gaps. The tags were written by ffmpeg rather than by a phone, and I did not test phone originals, HEVC, iPhone Safari or Android. Within those limits, the command that came out right for a tagged portrait source was the plain re-encode with no rotation flags. When I need a smaller size, I fix one side and let ffmpeg compute the other. I look at both read-back lines before I call a job done. The turbine footage is by Grendelkhan on Wikimedia Commons, under CC BY-SA 4.0.

Mapping Word's srcRect crop to CSS and why center scaling drifts

hao jia — Mon, 12 Oct 2026 08:03:19 +0000

Our internal doc portal turns uploaded DOCX files into HTML, and I was checking what happens to cropped pictures before we let anyone use it. One test picture is a screenshot of an order admin page with the shipping block cropped off the bottom. After conversion with docx-preview 0.4.1 the visible area was correct, but the whole picture sat about 70 px too high and covered the heading above it, so only "Scree" was left. The bigger problem showed up when I looked at what the HTML actually carried. A note on the sample first: the DOCX is one I wrote by hand following the OOXML structure, with seven picture cases, so it is not a file Word saved. The admin screenshot comes from a demo page I built myself, and every name, phone number and address on it is a placeholder.

Three values describe a cropped picture

In word/document.xml a picture carries three things. wp:extent is the size of the frame on the page. a:srcRect holds l, t, r and b, each in thousandths of a percent, so b="35484" means 35.484 % trimmed from the bottom. a:xfrm rot is the rotation in 1/60000 of a degree. The file in word/media/ stays whole. In my sample that is by construction, so it proves nothing about Word itself. Microsoft's support page on reducing picture file size does describe a "Delete cropped areas of pictures" option in Word for Mac, which suggests the trimmed pixels stay unless you tick it. I have no Word on this machine and have not tried it.

Overflow plus negative margins works in all three engines

The mapping I trust is a fixed box at extent size with overflow: hidden, holding the full image scaled up and pulled left and up:

const pct = (v = 0) => v / 100000;
const imgW = boxW / (1 - pct(l) - pct(r)), imgH = boxH / (1 - pct(t) - pct(b));
const imgStyle = { width: imgW, height: imgH, marginLeft: -pct(l) * imgW, marginTop: -pct(t) * imgH };

I rendered the three cropped cases this way and compared each against a reference made by really cropping and resizing the image. Normalized correlation came out 0.967 / 0.998 / 0.958 in a Chromium 149 open-source build, 0.962 / 0.999 / 0.961 in Firefox 151 and 0.956 / 0.994 / 0.961 in WebKit 26.5, which is the engine build and not a Safari release. The shorter route, object-view-box: inset(...), only worked in Chromium. Firefox and WebKit returned false from CSS.supports and showed the whole picture.

Scaling from the center pushes the picture off its frame

The converters I looked at do it differently. Judging from the exported markup, the <img> is sized to extent, clip-path keeps the visible rectangle, and transform: scale() blows it back up. Since transform-origin defaults to the center, the visible part lands off target whenever left and right, or top and bottom, are trimmed by different amounts. The helper takes the four trims as fractions, so 35.484 % is 0.35484:

const drift = (L, T, R, B, w, h) => ({ dx: w * (L - R) / (2 * (1 - L - R)), dy: h * (T - B) / (2 * (1 - T - B)) });
// B -> dx 57.6, dy -19.2   D -> dx 0, dy -70.4   G (15 % each side) -> 0, 0

Measured offsets matched: (58, −19) for case B, −70.4 px for case D. For the converter side I also used ImgIng (https://imging.ai/) for DOCX to HTML, with the same seven-case sample DOCX in the Chromium 149 open-source build. It got the crop content and the scaling right and made 0 non-GET requests during conversion. It showed the same drift as docx-preview, and its interface gave no warning about crops or rotation. Cropping only the bottom, probably the most common way to trim a screenshot, is exactly the case that slides up over the text above it.

The photo in this figure is from Wikimedia Commons by Dietmar Rabich, CC BY-SA 4.0, and the figure is shared under the same license. On the left the picture spills past the red frame on the right and top edges. On the right, with all four sides trimmed equally, it sits flush.

CSS only hides the cropped part

None of this removes anything. The ImgIng export was 9,665,338 bytes, and all seven data URIs were the full originals, byte-identical to the media in the DOCX. Decoding the case D PNG gave 1800×1240 with the shipping details, phone number, address and payment ID all there. The mammoth output carried the same full images and simply displayed them. I once spent three months with legal reviewing what left our network, so this part stopped me more than the drift did.

Bake the crop before converting

What fixed both problems was cropping the media to match srcRect and then clearing srcRect, re-encoding JPEG at q92 and keeping PNG lossless. The DOCX went from 7,234,550 to 5,714,763 bytes, 21.0 % smaller. The ImgIng export dropped from 9.22 MB to 7.28 MB, drift for B and D went to (0, 0) and (0, −0.4), and the trimmed region could no longer be found on the page. My script does not handle one media file referenced with different crops, negative srcRect or floating images. Word, WPS, LibreOffice and pandoc are untested, and I only ran the ImgIng export in Chromium 149. In our pipeline the check now runs before conversion: grep document.xml for a non-zero srcRect, bake those pictures, then compare one cropped picture's position against its frame.

The preview used my font, but the PDF's embedded font list didn't

hao jia — Sun, 11 Oct 2026 08:03:04 +0000

The admin console my team owns exports statements as PDF. I've sat through a compliance review that went over every data flow we had, so I don't sign off on an export pipeline because a screenshot looks right. So when we started shipping fonts with the page, I wanted proof that the PDF actually used them. I wrote a placeholder statement page myself, with a made-up company and invented amounts. The body is set in Noto Sans SC and the title in ZCOOL KuaiLe, a rounded display font that's easy to spot. Both are OFL fonts that were already on my machine. Then I stopped judging the PDF by eye and read its font table.

Three font sources gave three font tables

I printed the same page with page.pdf() in an open-source Chromium 149 build on an M4 Mac, once per font source. With the system font, the PDF held 39 Type3 fonts and weighed 154,603 bytes, and the PingFang names only showed up inside the font descriptors. With the Google Fonts version, printed after document.fonts.ready, the page fetched 16 woff2 slices (412,984 bytes) and the PDF held 15 Type0 subsets, roughly one per slice, at 117,799 bytes. With both full TTFs shipped next to the page, Chromium loaded 13,880,960 bytes of font and wrote a PDF of 71,102 bytes.

Blue is what the page loaded and orange is the PDF. Look at the bottom pair. The biggest font input produced the smallest file, because Chromium subsets fonts when it prints. The six-letter prefix in AAAAAA+ZCOOLKuaiLe-Regular marks a subset, and the KuaiLe program inside was 48,088 bytes, down from 3,274,128. Two things almost fooled me here. Type3 fonts carry no usable BaseFont, so the real name has to come from FontName in the FontDescriptor. My first script skipped that and nearly told me the server PDF had no font names at all. And the Google Fonts Noto is a variable font, listed as NotoSansSCThin, even though the page renders at regular weight.

The server kept the subsets but not my full fonts

The setup I actually worry about is a generator I don't control. For that I used ImgIng (https://imging.ai/) HTML to PDF, with the same placeholder statement imported once as a single file and once as a whole folder. Import and preview run in the browser. On convert, it sends one POST with a script-free snapshot to a server-side Chromium 151 on Linux. I counted 0 non-GET requests before the click and exactly 1 after. That request body is the whole snapshot, so every font you inline travels with it. Worth knowing if someone reviews what leaves your network. The folder import inlined both TTFs as data URLs, and the snapshot came to 17.65 MB. The preview title was in KuaiLe. The PDF title was Noto Sans CJK SC, and the quality report said it had waited for document.fonts and the page images, with no font warning.

The top two rows had the full KuaiLe (3.27 MB) and the full Smiley Sans (2.64 MB), and both came out in the server's Noto Sans CJK SC. The bottom two had the 48 KB KuaiLe subset, and that same subset padded with a dummy table to the size of the full file. Both rendered in KuaiLe. The padded one is why I don't think upload size is the cause. A full 2.05 MB pixel font also worked. I can't see the reason from outside. The upload cost is plain, though. The two full-font runs sent about 18.5 MB each for fonts that never showed up, while a two-font subset sent 501,152 bytes and both fonts made it into a 44,084-byte PDF.

My export check reads the table first

This check now runs on every PDF we generate. It lists every font per type, pulls the name from the descriptor for Type3, and reports which CSS families never got embedded.

PREFIX = re.compile(r"^[A-Z]{6}\+")

def font_inventory(pdf_path):
    pdf, inventory = fitz.open(pdf_path), {}
    for page in pdf:
        for xref, _, ftype, basefont, *_ in page.get_fonts(full=True):
            name, size = basefont, 0
            if ftype == "Type3":  # Type3 keeps its real name in the descriptor
                kind, ref = pdf.xref_get_key(xref, "FontDescriptor")
                if kind == "xref":
                    name = pdf.xref_get_key(int(ref.split()[0]), "FontName")[1].lstrip("/")
            else:
                size = len(pdf.extract_font(xref)[3])
            family = PREFIX.sub("", name)
            hits, biggest = inventory.get((ftype, family), (0, 0))
            inventory[(ftype, family)] = (hits + 1, max(biggest, size))
    return inventory

def families_not_embedded(pdf_path, wanted):
    got = {fam.lower().replace("-", "") for (ftype, fam) in font_inventory(pdf_path) if ftype != "Type3"}
    return [w for w in wanted if not any(g.startswith(w.lower().replace(" ", "")) for g in got)]

full-fonts.pdf
  Type3 NotoColorEmoji           x2         0 B
  Type3 NotoSansCJKsc-Bold       x4         0 B
  Type3 NotoSansCJKsc-Regular    x41        0 B
  not embedded: ['ZCOOL KuaiLe', 'Noto Sans SC']
subsets.pdf
  Type0 NotoSansSC-Regular       x1    326272 B
  Type0 ZCOOLKuaiLe-Regular      x1     48088 B
  not embedded: []

Code notes: full-fonts.pdf is the folder-import server PDF and subsets.pdf the two-subset one, and I cut the Type3 lines from the second listing. Server system fonts come back as Type3, so I only count Type0 fonts as embedded web fonts. The table can't catch everything, though. On the server, 𪚥 (U+2A6A5) was drawn as a crossed box, but text extraction still returns 𪚥. So the check also renders page one to PNG for a person to look at.

In the bottom row, 𠀀 and 𪚥 are crossed boxes and the other five rare characters are fine. I haven't tested a Linux box with no CJK fonts at all, or WOFF2 fonts shipped with the page. Before you trust an HTML-to-PDF export, compare its embedded font names with your @font-face families, and subset your fonts to the characters on the page before you hand them to a generator you don't run.

Measure ink and lighting before you binarize an image for OCR

hao jia — Sat, 10 Oct 2026 08:03:34 +0000

My team is looking at client-side OCR for an internal tool where people attach screenshots and photos of printed notices. The first preprocessing snippet anyone pastes into that kind of pipeline is grayscale plus a fixed threshold at 128. It's one line of OpenCV, so before it got near our upload flow I spent an evening measuring what it does. It fails in two quiet ways. It deletes text lighter than the cutoff, and under uneven light a global threshold paints whole regions black. Neither one throws an error, the OCR call just returns fewer characters.

The samples were a web page I wrote myself (a made-up Chinese notice), screenshotted at 1x and 2x and cut into blocks, including an 11 px footer in #999 gray on #f5f5f5, plus two crops from a holiday-notice template I found online. For recognition I used ImgIng (https://imging.ai/) with its default Professional OCR tier, plus Fast OCR and Ultimate OCR on the same ten crops, on an Apple M4 Mac in an open-source Chromium 149 build. The control was tesseract.js 5 (default parameters, chi_sim+eng). Errors are character error rate. Recognition ran in a Worker on the machine with zero non-GET requests. After three months of legal going through our data flows during a compliance cleanup, that is the first thing I check. The first run downloads the model, a download and not an upload of anyone's screenshots.

The gray text was lighter than the cutoff

The footer's darkest pixel is 153. With the cutoff at 128, or at 100, every pixel of that text lands on the background side and the crop turns blank white. All three tiers at both pixel ratios returned 0 characters, and so did tesseract.js. The progress panel said no reliable text was found and suggested rotating, turning off scan enhancement, or a higher tier.

Raising the cutoff left only the stroke cores

At 160 the cutoff sits just above those darkest pixels, so only the center of each stroke survives. Professional OCR read the 1x crop as gibberish, 75.5% wrong. At 200 it was 3.1%, with Otsu (which picked 208) 5.1%, and the untouched original 0.0%. Across all ten crops Professional OCR went from 0.1% on originals to 25.8% at 128, while the three tiers on originals were less than one point apart. Colored backgrounds fail too. The template's white words sit on red tags, and at threshold 100 the tags turned largely into black slabs that cut into their white words. Professional OCR scored 15.5%, and the "返岗时间" (back-to-work date) tag vanished.

Uneven light breaks any global threshold

I don't have real phone photos yet, so I generated two fake ones with a script: an A4 notice warped about 2.5 degrees, lit brighter top-left than bottom-right, blurred and noised, and one with an extra shadow over the text. These show a mechanism on program-generated samples, not real-photo error rates. On the shadowed one Otsu picked 137 and the shadow went solid black. Professional OCR scored 31.0%, and the order-independent error was also 31.0%, so those characters were really gone. Local adaptive thresholding scored 19.0% with an order-independent error of 0.0%. Every character was there, and the rest came from the tilt reordering short lines. Straightening the page from its corners got all three tiers to 0.0%.

What I check before any threshold

So before a threshold I measure two things: how dark the ink really is, and how much paper brightness drifts across the image.

import cv2 as cv
import numpy as np

def threshold_audit(file, cutoff=128, trim=0.15):
    g = cv.imread(file, cv.IMREAD_GRAYSCALE)
    h, w = g.shape
    core = g[int(h * trim):int(h * (1 - trim)), int(w * trim):int(w * (1 - trim))]
    paper = cv.dilate(core, np.ones((41, 41), np.uint8))  # local "paper brightness"
    otsu, _ = cv.threshold(g, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)
    report = {"ink": int(core.min()), "paper_low": int(np.percentile(paper, 1)), "otsu": int(otsu)}
    if report["ink"] >= cutoff:
        report["verdict"] = f"fixed {cutoff} erases the text"
    elif report["paper_low"] < otsu:
        report["verdict"] = "global threshold blackens paper, go adaptive or skip"
    return report

The trim drops the desk border around the generated photos. On the gray footer it reports ink 153 and flags 128. On the shadowed sample the darkest paper is 100 against an Otsu of 137, so it flags the global threshold, while the unshadowed one reads 159 against 135 and passes. Two cases get through. A cutoff of 160 sits above the ink, so the stroke-core failure isn't flagged, and the red tags pass because the dilate picks up their white words. I haven't found a cheap check for that one yet.

ImgIng's own "Scan enhancement" toggle is described as "Gentle grayscale, contrast and sharpening; no forced binarization." It's off by default, and turning it on gave 0.2% against 0.1%, so I'd call it neutral. Not binarizing lines up with everything above. Our default will be to send the original and only fix geometry. Before you ship a threshold, print the darkest pixel of your lightest text and the spread of your paper brightness.

256-colour PNGs can keep soft shadows. tRNS shows whether yours did

hao jia — Fri, 09 Oct 2026 08:08:03 +0000

Before letting 256-colour PNGs into an image pipeline, I wanted to understand one failure. A sticker with a soft drop shadow goes through colour reduction, and on a white page the shadow comes back as a solid black band with a knife-cut edge. My test file is a pink ribbon sticker from Wikimedia Commons (by K B, CC BY 2.0), a third-party image, and the figures only show its top loop. 17.9% of its pixels are semi-transparent. 256 colours were not the problem. An indexed PNG can store soft alpha fine, and the result depends on how many distinct alpha values the encoder writes into one small chunk.

Colour type 3 keeps alpha in its own chunk

The original is colour type 6, four bytes per pixel with alpha stored next to RGB. A 256-colour PNG is colour type 3. Each pixel is an index into PLTE, which holds up to 256 RGB entries of three bytes. Transparency lives in the optional tRNS chunk, one alpha byte per palette entry in the same order, and entries past its end count as opaque. How many distinct values those bytes hold is up to the encoder. I wrote a small reader in Node with no image library:

function paletteAlpha(bytes) {
  const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
  const report = { colourType: null, paletteSize: 0, alphaBytes: 0 };
  let alpha = [];
  for (let at = 8; at < bytes.length; ) {
    const size = view.getUint32(at);
    const tag = String.fromCharCode(...bytes.subarray(at + 4, at + 8));
    const payload = bytes.subarray(at + 8, at + 8 + size);
    if (tag === 'IHDR') report.colourType = payload[9];
    if (tag === 'PLTE') report.paletteSize = size / 3;
    if (tag === 'tRNS') { report.alphaBytes = size; alpha = [...payload]; }
    at += size + 12;
  }
  while (alpha.length < report.paletteSize) alpha.push(255);
  const steps = [...new Set(alpha)].sort((a, b) => a - b);
  report.belowOpaque = alpha.filter((a) => a < 255).length;
  report.steps = steps.length;
  report.widestGap = Math.max(0, ...steps.slice(1).map((a, i) => a - steps[i]));
  return report;
}

It walks the chunks, pads the alpha list to palette size per the spec, and reports entries below 255, distinct alpha values and the widest gap between neighbouring values. Count distinct values, not entries. My 98-level file has 129 entries below 255 but only 98 distinct values, because several colours share one alpha. Output for six versions of the sticker:

ribbon.png               type=6 tRNS=0B below255=0 steps=0 gap=0
ribbon-2.png             type=3 tRNS=256B below255=1 steps=2 gap=255
ribbon-10.png            type=3 tRNS=256B below255=9 steps=10 gap=43
ribbon-34.png            type=3 tRNS=256B below255=33 steps=34 gap=18
ribbon-98.png            type=3 tRNS=256B below255=129 steps=98 gap=12
ribbon-png8-export.png   type=3 tRNS=256B below255=1 steps=2 gap=255

The two-step file is the black band. Its tRNS is a full 256 bytes, yet the only entry below 255 is the fully transparent colour, so every pixel at alpha 128 or above went opaque and the rest vanished. The 10, 34 and 98 level files come from a quick reducer I wrote for comparison, not from pngquant or any product. The last line is the PNG-8 export from ImgIng's image compressor at its defaults, run on 2026-09-30, and it has the same two steps. Its PNG compression page says transparent PNG-8 keeps multi-level alpha. My file did not, and I am reporting bytes, not guessing why.

Browsers read alpha per palette entry

A format feature only helps if decoders honour it. I decoded the 34-level file in the Chromium 149 open-source build, Firefox 151 and WebKit 26.5 (the engine, not a Safari release) and read the pixels back from a canvas. All three reported 34 alpha levels and matched Pillow's decode on semi-transparent pixel count and alpha sum.

This is a Noto Color Emoji outline (Apache 2.0) at 4x on blue. The original has 17 alpha levels and only 1,548 semi-transparent pixels. With two steps the anti-aliased rim becomes a staircase, and with 20 steps it looks like the original.

Fewer alpha steps show up as rings

A shadow is a smooth ramp, and every gap between neighbouring alpha values becomes a visible ring. Error here is the mean RGB difference (0 to 255) from the original over semi-transparent pixels, both composited on white. Two steps gave 44.17, 10 steps 8.23, 34 steps 3.48 and 98 steps 1.75. In the same encoder the 34-level file was 121,346 bytes against 111,178 for two steps, only 9.1% more. At 10 steps, with a widest gap of 43, the shadow shows clear contour lines. At 34 steps (gap 18) they thin out, and at 98 steps (gap 12) the shadow is close to the original.

Dithering adds grain but no alpha steps

With two steps, dithering never touches alpha. ImgIng's exports with dithering on and off had identical alpha and the same white error (44.07 and 44.06), but the dithered file was 1.59 times larger, 148,242 bytes against 93,317. On my 34-level file, Floyd-Steinberg diffusion over colour and alpha did break the rings into grain. White error still rose from 3.48 to 4.29 and the file grew 65%. Opaque pixels pushed to semi-transparent went from 573 to 1,850, seen as white specks on the pink edge.

What I check before shipping an indexed PNG

I run the reader first. A type 3 file with one tRNS entry below 255 has lost every soft pixel, however fine it looks on a dark page. Then I look on white and on a colour, never only on black, since the two-step ribbon scored 2.33 on black against 44.17 on white. For assets with large soft shadows I skip PNG-8. For the ribbon I used ImgIng (imging.ai) to convert to WebP at its default quality 84, and it went from 430.7 KB to 98.8 KB with alpha identical to the original pixel for pixel. The conversion ran in the browser and the export sent no non-GET requests.

This covers two samples. Coloured shadows, glows, Chrome stable, Safari, display pipelines and chat apps that recompress images are untested. Before switching a batch of PNGs to 256 colours, run a tRNS count over the exports and reject any file whose source had soft pixels but whose output has just two alpha steps.

Detecting open passwords and permission locks in PDFs with pdf.js

hao jia — Thu, 08 Oct 2026 08:03:03 +0000

A PDF that opens without asking for anything can still be encrypted. I ran into this while thinking about upload validation for document tools, so I generated seven two-page test PDFs filled with placeholder contract text and checked them in the browser. Five of them had only a permissions password. Chromium 149's open-source build and Firefox 151 displayed all five straight away, with no prompt and no warning. Yet every one of those files carried a full encryption dictionary. The user password was simply empty, so any reader could decrypt the content on its own. Only one sample, the one with a real open password, stopped at a password box. For a frontend that receives these files, that means "encrypted" is not a single state. It covers files nobody can read without a password and files anyone can read but that ask the viewer to block printing, copying, or editing.

Each viewer enforces permission limits differently

That second group matters because permission flags only work when the viewer chooses to honor them. On the same all-restricted sample, Chromium's built-in viewer blocked printing and copying but still let me draw on the page. Firefox with default settings blocked none of the three. After I turned on its permission enforcement setting, it blocked printing and editing but still did not block copying. Changing the cipher from AES-256 to RC4-128 or RC4-40 changed nothing in those results. Only the permission bits did. So whatever your app shows the user should come from what the file declares, not from what one browser happened to do with it.

Processing tools make their own call on top of that. To see what a real processing tool does, I used ImgIng's PDF compressor (imging.ai is its international site, and my run was on the Chinese deployment) with the permission-only and open-password PDFs I generated. It failed all four encrypted samples I imported with a message that translates to "this PDF is encrypted or permission-protected and needs the protection removed first", and it offered no password field. The page made 46 requests, all of them GET. It did not say why it refuses, and I would only be guessing at the reason.

Three pdf.js calls separate open and permission passwords

I used pdf.js 6.3.289 as a page library. An open password shows up through the onPassword callback. getMetadata() exposes info.EncryptFilterName, which was "Standard" for every permission-only sample and empty for the plain one. getPermissions() returns the granted flags, or null when the file declares none. This is the detection function I ran against all seven files:

async function classifyPdf(bytes) {
  const task = pdfjs.getDocument({ data: bytes });
  let askedForPassword = false;
  task.onPassword = () => { askedForPassword = true; task.destroy(); };
  try {
    const doc = await task.promise;
    const { info: meta } = await doc.getMetadata();
    const perms = await doc.getPermissions();
    const granted = perms == null ? null : new Set(perms);
    return { kind: meta.EncryptFilterName ? 'permission-only' : 'plain', granted };
  } catch (reason) {
    if (askedForPassword) return { kind: 'open-password' };
    throw reason;
  }
}

I destroy the task inside onPassword on purpose. The check never asks for a password, and it never holds one. The open-password file then rejects with a PasswordException, and the flag tells the catch block why. Checking PRINT, COPY, and MODIFY_CONTENTS against the granted set gave this, one line per sample:

S0 plain -
S1 permission-only PRINT,COPY,MODIFY_CONTENTS
S3 permission-only COPY
S4 permission-only PRINT
S5 open-password -

S2 and S6 matched S1. The new Set(perms) line is there because of a version difference that bit me. Firefox's built-in pdf.js 5.7.195 returns an array from getPermissions(), while 6.3.289 returns a Set. Code that calls .includes works on the old one and throws on the new one. Normalizing first means the same check runs on both.

This is a check page I built with the same three calls, fed English placeholder copies of three samples. The first card is the all-restricted file, where every row is red except accessibility extraction. The second only blocks copying, so that is the one red row. The third card needs an open password, and the page shows nothing for it except that fact. The page only reads the files and never modifies them.

Report the state and point to the owner

After detection, the frontend should explain what it found and leave the file alone. I have sat through a compliance review, and quietly stripping limits from a file someone else sent is the kind of thing it asks about. I would show "opens normally, but the author restricted printing and copying" for the middle case and "needs the open password" for the last. Then point people to the proper route. They can ask whoever sent the file for an unrestricted copy, or, if the file is theirs and they know the owner password, re-export it from the software that created it. I did not test Acrobat, macOS Preview, or Safari itself, and the WebKit 26.5 run covered only the pdf.js page library. Before adding a PDF upload step, run your own sample files through classifyPdf and write the message for each of the three states first.

A completed video seek did not always produce a new frame callback

hao jia — Wed, 07 Oct 2026 08:02:48 +0000

A paused video seek finished, but the next video-frame callback did not arrive within my observation window. Treating that as a failed capture would have thrown away a usable result: the target still belonged to the frame already on screen. This happened in a controlled WebKit test, and it made me separate seek completion from waiting for a newly submitted frame. Neither signal could stand in for the other.

I tested a generated H.264 clip with an independently readable frame number encoded in its pixels. It contained 180 frames at 30 fps, with no B-frames. Three versions used fixed GOP lengths of 15, 60, and 180 frames. The pixel number was important: I could inspect what had actually been drawn instead of reading back the same currentTime value I had just assigned. This was a test fixture, not footage of a production incident.

After loading and pausing the clip, I let the initial frame callback finish. I then registered a new callback and a separate seeked listener before assigning the next position. In the WebKit 26.5 test build, seeking from zero to 0.017 seconds produced seeked for all three GOP versions. The canvas still contained frame zero. No new frame callback arrived within 1.7 seconds. That interval was my observation limit, not a measurement of decoding work.

At 30 fps, the requested 0.017 seconds lies within the first frame’s interval. The unchanged pixels were therefore not automatically a stale result. Each version then continued through eleven other targets; WebKit produced a new callback for all of those. The next target, 0.049 seconds, drew frame one. Comparing a same-frame request with a different-frame request explained more than treating every missed callback as the same error.

A separate sea-footage check showed why I would not fix this by accepting any existing canvas immediately. Here, assigning 5.017 seconds and drawing at once captured a different wave shape from drawing after seeking completed. Both displayed time values could already say 5.017. In the numbered test fixture, 99 of 108 immediate drawings still contained the previously presented frame. The remaining nine were the same-frame requests, where the old pixels also happened to satisfy the target.

This independent reproduction uses a six-second, resized copy of public waves footage. It illustrates that a changed time property is not proof of updated pixels. It does not illustrate the 1.7-second callback timeout, which came from the numbered fixture. Keeping those two observations separate is essential: otherwise an example of stale content could be mistaken for proof that every unchanged frame is wrong.

I also ran a product-side check with ImgIng, using the Chinese video-matting Beta workbench. I imported the waves, moved to 2.117 seconds, waited for seeking, and requested its current-frame preview. It returned a no-obvious-subject message. That confirmed only the observed preview path and result. I did not inspect its internal waiting strategy or test a complete thumbnail export, so the application cannot serve as evidence for a callback fix.

For an implementation, I would retain the requested position, seek completion, available pixel evidence, and callback outcome as separate results. A timeout should end the wait and preserve those observations; it should not automatically label the canvas black. Deciding whether an existing frame satisfies the request needs a defined tolerance or another trustworthy reference. In this fixture I had an exact frame number. Arbitrary variable-frame-rate inputs need a different validation method, which I have not tested here.

The waves are Alexander Grebenkov’s Ocean waves at Lækjavik beach, Iceland, under CC BY 3.0, trimmed, resized, and transcoded. WebKit here was a test build, not released Safari. Before adopting a capture helper, I would run both a same-frame target and a different-frame target against it, then check cleanup after a timeout or cancellation. Those last implementation paths remain work to verify; the bounded experiment establishes why a new callback alone is an incomplete completion condition.

Testing MP4 delivery with identical media payloads and 200 or 206 responses

hao jia — Tue, 06 Oct 2026 08:02:42 +0000

I wanted a video-delivery test in which a changed picture could not explain a changed loading time. Re-encoding a clip at a lower bitrate would have moved too many things at once. Instead, I kept the encoded media payload identical and changed two other inputs: where the ordinary MP4 stored its movie metadata, and whether the HTTP server honored byte-range requests. That gave the playback results a narrower meaning.

The inputs were two public waves clips, each prepared as an eight-second H.264 MP4 at 960×540 without audio. The smaller comparison file was 1,433,114 bytes; the larger was 5,068,844 bytes. For each source, I made front-metadata and end-metadata versions by copying the encoded packets during remuxing. Their mdat payload hashes matched within each pair. The complete files did not need to match, because their container layout was the variable being tested.

I served the files from a local HTTP process with two modes. One honored a Range request and returned a partial response with status 206. The other ignored the requested range and returned the whole file with status 200. All active responses in a test shared a 256 KiB/s write budget, scheduled in 40 ms steps, and each request began with an 80 ms delay. Giving every simultaneous response its own full budget would have changed the total delivery capacity when a browser opened another request.

That setup still has limits. It throttles application writes rather than network packets, so socket buffering, scheduling, and decoding affect the observations. The 80 ms initial wait is not a complete model of an 80 ms round trip. I used fresh browser contexts and no-store responses, then repeated each source, engine, layout, and server-mode combination twice. Across three engines, the ordinary MP4 matrix contained 48 runs.

The smaller end-metadata file made the server-mode difference clear in Chromium. With Range supported, its first-frame times were 0.776 and 0.813 seconds. With Range ignored, they were 5.773 and 5.792 seconds. The media bytes were unchanged. In one supported run, requests moved from byte zero to byte 1,409,024 near the end, then back to byte 32,768. The browser had a way to obtain the late metadata before reading all intervening media.

This is a separate no-Range illustration of the larger file at 1,250 ms, not a screenshot of the smaller file’s measured runs. It comes from a native player in the local reproduction. The larger file’s two Chromium first-frame waits in the main matrix were 20.272 and 20.278 seconds. A screenshot can show the waiting state; it cannot replace the event record used to calculate that interval.

I also made real exports with ImgIng, operating the Chinese video compression workbench. Those outputs were fragmented MP4s from the complete originals, retaining audio and different durations. I kept them out of this paired ordinary-MP4 comparison. Otherwise a product export, a shortened silent derivative, and a changed container structure would all be competing as if only one variable had changed. Their playback checks are useful, but they answer a different question.

The illustrated source is דוד שי’s Water waves in Herzliya beach, under CC BY-SA 4.0; the comparison copy was trimmed, resized, stripped of audio, transcoded, and remuxed. The test engines were Chromium 149, Firefox 151, and a WebKit 26.5 test build rather than released Safari. One larger-file WebKit condition failed before producing a frame, so I did not convert every non-success into a slow-loading measurement.

For a repeatable check, I would keep the file-pair hashes, actual response mode, request ranges, and first-frame event together. Seeking should have its own result and target, rather than being inferred from startup. Only after that local comparison would I test the final hosted URL with its real cache and delivery behavior. These numbers isolate behavior in the test server; they do not certify a CDN configuration or predict public-network startup times.

Keep the original text when a PDF page falls back to OCR

hao jia — Mon, 05 Oct 2026 08:03:22 +0000

A PDF extraction failure becomes harder to investigate when the fallback result replaces the first output. You eventually have readable text, but cannot tell which page needed help or where the replacement came from. I tested a small recording approach using controlled PDF samples. The useful change was keeping each result attached to its page and source, including results that looked wrong. This was a local experiment, not a production deployment.

My main sample was a notice generated with ReportLab. I kept the source wording before producing the PDF, then made separate versions with a missing Unicode mapping and an incorrect mapping. Neither was a document supplied by a customer. Under the same MuPDF rendering conditions, both altered versions looked identical to the original. Their extracted text did not. That gave me a way to examine the extraction result without also changing the visible wording.

The missing mapping removed all 56 Chinese characters from the extracted output. With pypdf, the result still contained 152 characters, exactly the same total as the normal version. A nonempty string would have passed a basic import check. Counting replacement characters would not have helped either: this output contained no U+FFFD. I kept the actual returned string rather than deleting unusual characters to make the report easier to read.

Next I combined a normal page and a damaged page into one PDF. This made the problem with a document-level result obvious. The complete extraction contained Chinese text because the first page was fine. The second page still needed investigation. Once both outputs were flattened into one string, that distinction was harder to maintain. Saving separate page records preserved it without requiring an OCR scheduler or a new document-processing service.

Each record in my experiment carried the file name, page number, raw checks, font observations, and reference comparison. The reference was the original generated wording, not another extractor's output. A page without obvious character anomalies received no_signal, which deliberately did not mean correct. Pages matching their complete known reference were marked sample_checked. Pages without a complete reference remained unverified. These names describe evidence, rather than promising that every downstream use is safe.

The distinction mattered for a second failure. I changed one mapping entry so that two occurrences of a Chinese character were extracted as another valid character. The output still contained 56 Chinese characters and no replacement symbol. PDF.js, pypdf, and PyMuPDF all returned the same wrong substitutions. My basic character checks missed them. Comparing against the independent source text found the mismatch. Agreement between extractors would have hidden it if I had used a majority vote as ground truth.

I also needed a counterexample to avoid sending every missing mapping to recovery. A simple Helvetica document using WinAnsiEncoding contained an English invoice-style sentence and no ToUnicode mapping. All three extractors recovered that sentence correctly. Rejecting it merely because the mapping was absent would have introduced a false alarm. I therefore recorded missing mappings as an observation, while leaving the text evidence and the final decision separate.

Across the eight pages I checked, the records ended with three needing review, three checked against their known source, and two unverified. The last two were pages from an existing copy of RAG-Safety-Bench. Selected passages had been inspected, but I did not have a complete manually checked reference for either page. I did not count those pages as failures or passes. The controlled set also reused the notice, so these numbers are not an accuracy benchmark.

I then tested ImgIng in the browser on its Chinese site, imging.cn. The link to imging.ai is its overseas homepage; these trials did not retest that deployment. For the damaged text PDF, enabling and disabling image-text recognition inside PDF content extraction produced identical text. Converting that file to HTML finished but left the Chinese text damaged. Both outcomes belong in the record. A completed operation is useful evidence about execution; it does not tell us whether the resulting characters match the page.

There was a successful image-recognition path, with an important boundary. The clear PNG came from an external renderer. ImgIng's PDF-to-image attempt on this sample had failed with SVG render failed, so the PNG was not a successful output of that step. Feeding the existing clear image into the separate image OCR tool recovered the notice's five lines. Spaces inside the Chinese text differed from the original PDF. The interface count of 134 characters was not an accuracy measurement.

The screenshot shows the image OCR result with an English interface. The notice itself remains Chinese because that was the tested input. It documents a different extraction source, not a repair to the original PDF's mapping. If I kept only this readable result, the failed direct extraction and failed conversion attempt would disappear from the explanation.

For a fallback integration, I would retain the original page record and attach the OCR output as another candidate with its own source and comparison scope. That integration is a proposal; this experiment did not implement retries, queues, or automatic merging. Start by checking whether your current pipeline overwrites the first text result. Preserve it, keep the page number, and compare the replacement against a known passage before allowing the new output to stand in for the original.

Reading a canvas back doubles its memory: numbers from three engines

hao jia — Sun, 04 Oct 2026 08:02:46 +0000

Our product-image component crops, watermarks and samples a dominant color from every upload. All three steps call getImageData on the full canvas. For a long time our memory estimate for a large image was "width × height × 4". That number is right for the canvas itself, and it is exactly half of what the component actually holds once it reads the pixels back.

What a full-canvas getImageData really costs

The numbers below come from a round of measurements on an Apple M4 with 16 GB, using three engine builds: the Chromium 149 open-source build, a WebKit 26.5 build and a Firefox 151 build. None of them is the browser you download, so read nothing here as Chrome, Safari or release Firefox behavior. Memory was measured at the OS level by summing phys_footprint across every browser process of that launch, because nothing inside the page can see canvas memory.

The x axis is canvas pixels in millions, the y axis is MiB above the idle page. Each engine has a dark line for "after fill" and a light line for "after full getImageData". Look at the 268 tick, which is 16384×16384. Chromium's dark line sits at 1025 MiB and its light line at 2050 MiB, so the second copy is the whole ImageData, 1,073,741,824 bytes. Firefox lands almost on top of it (1025.3 and 2035.6 in the raw data), and its lines keep going to 537 because that build can draw up to 23168×23168. The light orange WebKit line is the outlier: 1206.4 MiB after the fill and 3265.4 MiB after the read-back at the same size. The WebKit build also carries tens of MiB of extra overhead on small canvases, 44.6 MiB for a filled 1024×1024 where the formula says 4 MiB. I quote WebKit figures per size and do not turn them into a ratio.

Two more details changed how I wrote the estimate. Creating a canvas and calling getContext('2d') without drawing added less than 1 MiB in all three engines, so the pixel buffer is allocated on first draw, not at creation. And setting width = height = 0 and dropping the reference did not bring the number down within 0.8 s: Chromium still showed +1025 MiB at 16384×16384. No GC was forced, so this is not evidence of a leak, only a reminder that "released" and "returned to the OS" are different moments.

I re-ran the three steps on one 8192×8192 canvas in Chromium 149 to see them in order. createElement plus getContext added +0.3 MiB. The fill brought it to +256.7 MiB, one frame of 256 MiB, and the read-back to +512.9 MiB. Meanwhile JSHeapUsedSize only moved from 949,212 to 1,115,836 bytes while half a gigabyte is allocated. That is why the numbers above come from the OS and not from the page.

The estimate I use now

The component computes a budget before anything is drawn:

function frameBudget(width, height, { readBack = false } = {}) {
  const pixels = width * height;
  const bytes = pixels * 4 * (readBack ? 2 : 1);
  return { pixels, mib: bytes / 2 ** 20 };
}

I ran it in Node for the sizes above. 16384×16384 gives 1024 MiB without read-back and 2048 MiB with it, within 1% of what Chromium and Firefox measured. 8192×8192 gives 256 and 512 MiB. The WebKit build measured 823.3 MiB after read-back at 8192×8192, well above the formula, so our ceiling is set lower than the formula would allow instead of adding a per-engine multiplier I cannot justify.

I also wanted to know whether image tools that work in the browser keep large files local, since our compliance review cares about that more than about memory. I used ImgIng (https://imging.ai/), operating the Chinese interface, to convert a 10000×10000 JPEG to WebP and JPG on the same M4 across the same three engine builds. Across 12 cases there were zero non-GET requests. By the formula, that 100-megapixel image needs about 381 MiB just to be drawn once, and about 763 MiB if anything reads it back.

What I still don't know

The measurements stopped at safety thresholds on a machine that was also running other work, so I have no crash point to report. GPU-backed canvases were not covered either, and I never repeated any of this in release browsers.

If your pipeline calls getImageData on the full canvas, count that second copy in your budget. Then draw your largest allowed size in the browsers you ship to and read one corner pixel back to confirm it actually rendered.

My cutout PNG looked fine in canvas 2D and glowed in WebGL

hao jia — Sat, 03 Oct 2026 08:02:50 +0000

I took one transparent PNG, drew it onto a black background with canvas 2D, and the edges looked right. I drew the same file with WebGL and a bright white band appeared under the dog's belly. The semi-transparent edge pixels averaged 54.9 in luminance with canvas 2D and 108.9 with WebGL. Same file, same browser. The only difference was two WebGL settings that disagreed with each other.

Part of the halo is already in the file

For the cutout I used ImgIng (https://imging.ai/) in its fast mode, ISNet INT8, on two white-background photos from Wikimedia Commons, then composited the exports onto black. The one I use here is a puppy photo by George Hodan, released under CC0. The model runs in the browser and I watched the network panel during export: zero non-GET requests. The refine panel says the default output uses the AI result pixel by pixel without changing alpha or edge colors, so whatever I measured came from the model output and my compositing, nothing in between.

The default export has 113,438 semi-transparent pixels. Their stored color averages 160.1 in luminance, while the opaque subject within 3 px of the edge averages 108.6. So the soft edge still carries some of the original white background. Composited correctly onto black, the edge averages 75.4 against an estimated 64.3 if it only held subject color. That gap of about 11 is baked into the file. Exporting WebP instead of PNG gave identical edge pixels, so the format is not the cause. Your compositing code cannot remove this part. What it can do is double it.

Straight alpha needs exactly one multiply

PNG stores straight alpha: color as is, alpha in its own channel. Compositing has to multiply color by alpha exactly once. Premultiplied data has already been multiplied, so it must not be multiplied again. Canvas 2D handles this for you. WebGL splits the decision across two independent settings: UNPACK_PREMULTIPLY_ALPHA_WEBGL at texture upload, and the source factor you pass to blendFunc.

I cropped a 480×480 px edge region with 43,740 semi-transparent pixels and ran all four combinations in Chromium 149, reading pixels back after drawing onto black. The correct answer for that crop is 54.9.

Canvas 2D: 54.9, max difference from the formula 0.
Straight upload + SRC_ALPHA: 54.9, mean difference 0.2.
Premultiplied upload + ONE: 54.9, mean difference 0.2.
Straight upload + ONE: 108.9. Color never gets multiplied, so the edge is twice as bright.
Premultiplied upload + SRC_ALPHA: 48.8. Color gets multiplied twice and the edge goes dark.

To make sure multiply count was the whole story, I computed rgb*a, rgb and rgb*a*a on the CPU over the same 43,740 pixels. The means came out 54.9, 108.9 and 48.8, matching what WebGL returned.

Look at the bottom edge of each panel. The left one is canvas 2D and fades cleanly into black. The middle one has a near-white stripe below the belly and every fur tip glows. The right one looks fine at first glance, but the edge is thinner and darker than on the left. That dark variant is the one I would miss in a screenshot review.

Tie both settings to one value

What I do now is never set the two separately:

const BLEND_FOR_UPLOAD = { straight: 'SRC_ALPHA', premultiplied: 'ONE' };

function uploadCutout(webgl, image, mode) {
  webgl.pixelStorei(webgl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, mode === 'premultiplied');
  webgl.texImage2D(webgl.TEXTURE_2D, 0, webgl.RGBA, webgl.RGBA, webgl.UNSIGNED_BYTE, image);
  webgl.blendFunc(webgl[BLEND_FOR_UPLOAD[mode]], webgl.ONE_MINUS_SRC_ALPHA);
}

I ran both modes on the same crop and compared edge luminance against the canvas 2D result on black. The mean difference was 0 in both cases. Anyone who changes the upload mode later changes the blend factor with it.

What is still open

Canvas 2D makes a good reference here: getImageData on a transparent canvas returned the file's RGB with zero error, including pixels with alpha below 32. But I only tested Chromium 149, where WebGL may be running on a software renderer. Firefox and Safari are untested.

The roughly 11 units baked into the file are a separate decision. ImgIng has a "remove background color fringe" toggle, off by default. On the puppy it brought the edge to 76.0 against an estimated 75.6, so the halo is basically gone. The cost is that it rewrites alpha: 76,236 pixels changed and a jagged patch appeared where the front leg meets the belly. I have left it off for now. Before shipping anything that draws cutouts with WebGL, I check that upload mode and blend factor come from the same variable, and compare one frame against canvas 2D.

Decoding a progressive JPEG took 14 MiB more memory in my test

hao jia — Fri, 02 Oct 2026 08:02:53 +0000

Our image worker runs in containers with a hard memory limit, and someone upstream wants to switch the product hero images to progressive JPEG. Before signing off I wanted one number I didn't have: how much more memory does a progressive file need while it's being decoded? The answer on my machine was about 14 MiB more for a 2736×1824 photo, and it lines up almost exactly with one buffer you can calculate on paper.

The setup is small. Two images I found online: a landscape photo from Wikimedia Commons scaled to 2736×1824, and a holiday notice template from a Chinese design-template site, 1242×2688. Each was encoded at quality 85 with Pillow 11.3 (libjpeg-turbo underneath), once as baseline and once as progressive. Same pixels in, same quality, only the scan layout differs. The machine is an Apple M4 with 16 GB, macOS 26.5.2.

To measure, I decode each file in a fresh Python process and look at how far the peak resident size moved:

CHILD = '''
import resource, sys
from PIL import Image
before = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
img = Image.open(sys.argv[1]); img.load()
after = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
print(before, after)
'''

A new process per file matters. Peak RSS never goes down, so decoding two files in one process would hide the second one. On macOS ru_maxrss is in bytes (on Linux it's kilobytes). I ran each file five times and took the median.

File	Baseline	Progressive	Extra
landscape 2736×1824	+20.6 MiB	+35.0 MiB	14.4 MiB
template 1242×2688	+14.3 MiB	+24.0 MiB	9.7 MiB

The baseline numbers are roughly the decoded RGB bitmap plus some overhead. 2736×1824×3 bytes is 14.3 MiB, and the process grew by 20.6.

The extra part is the interesting bit. A baseline JPEG is stored block by block. The decoder reads one block's 64 DCT coefficients, runs the inverse transform, writes pixels, and forgets the coefficients. It only needs a few rows of blocks in flight. A progressive file is split into several scans (Pillow's default script produces 10): first the DC value of every block, then bands of AC coefficients, then refinement passes that add the low bits. No block is complete until the last scan arrives, so the decoder has to keep every coefficient of the whole image around until the end.

You can size that buffer by hand. Both files use 4:2:0 chroma subsampling, and libjpeg stores each coefficient as 2 bytes. For the landscape image that's 2736×1824 luma coefficients plus two chroma planes of 1368×912, times 2 bytes: 14.3 MiB. For the template, with the width padded up to a multiple of 16, it comes to 9.6 MiB. The measured extras were 14.4 and 9.7. I didn't instrument libjpeg itself, so treat this as a match rather than proof, but it's close enough that I stopped looking for another explanation.

So for capacity planning the rule I wrote down is: a progressive JPEG costs roughly one more bitmap-sized buffer at decode time, on top of the output image. In our case the worker occasionally handles several large images in parallel, and that doubles the worst case. That's the part that would have hit the container limit, not the decode time (which also went up, about 2.3 to 2.6 times in Chromium in a separate test, still just milliseconds per image).

One thing that made this easier: files exported in the browser don't need this budget at all. I compress most of my front-end images with ImgIng (https://imging.ai/), and its JPG exports in the Chromium 149 open-source build I tested are baseline (SOF0), because it uses the browser's native encoder. In that build, canvas.toBlob also returns baseline JPEGs. Progressive files only enter our pipeline through server-side scripts.

I haven't measured browser memory for this, only Pillow. If you want to check your own worker, the snippet above plus a loop over five runs is all it takes. Run it on the largest image your service accepts, once as baseline and once as progressive, and compare the difference with width × height × 2 × 1.5 bytes for a 4:2:0 file.