DEV Community: AsyncMonk

OCR said done and returned half the page, so count lines before trusting it

AsyncMonk — Thu, 05 Nov 2026 08:06:20 +0000

I keep circling the idea of adding "upload a scanned PDF, get the text back" to one of my small tool sites, and the failure that worries me is not a crash. A crash shows up in the logs. What worries me is an OCR step that finishes cleanly and hands back text with a chunk of the page missing. On 2026-11-04 I ran a small test that produced exactly that: a scanned Chinese page tilted by 5° went through a browser-side OCR engine, roughly half of its 14 lines came back, and the status still said processing was complete. No warning, no error.

The setup was cheap on purpose. Two public-domain pages went in as image-only PDFs: the 1986 note to the Chinese simplified character list, a government document on the Internet Archive (557 characters), and page 8 of H. G. Wells' The Time Machine, 1895 edition, from Wikimedia Commons (1,002 characters). The engines were the text recognition built into macOS (the Vision framework, accurate mode), Tesseract 5.5.3 with English only, since that machine had no Chinese pack, and a browser-side PP-OCR engine: the PDF content extraction in ImgIng https://imging.ai/ , on a Mac with an Apple M4 in a desktop browser. Errors are character-level edit distance, so one extra, missing or wrong character counts as one. I also counted missing characters ignoring order. These are single runs on two clean single-column pages, with tilt and noise added by a script, so they say nothing about any engine's accuracy in general.

Straight pages were fine and tilted ones were not

On the straight Chinese page macOS made 5 errors and the browser engine on its default tier made 2. On English, macOS made 0, the browser engine 1 (the page number) and Tesseract 11. At 1° the Chinese page still came back with 1 error. From 3° the line order got scrambled and two Chinese lines were gone. At 5° the Chinese output was missing 275 of 557 characters and the English one 115 of 1,002. Over 0 to 5°, macOS stayed between 4 and 8 errors on Chinese. When I denoised, straightened and binarized the noisy 3° page before feeding it in, the browser engine was back to 2 errors on both pages. The engine handles a straight page well; nothing tells you when the page isn't straight.

Wrong output still looks like output

Half a page of clean text reads fine, so neither a user skimming it nor a search index has a reason to complain. Other failures in the test had the same shape. On its largest model tier the browser engine repeated a chunk inside 13 of the 14 Chinese lines (114 edits, 113 extra characters), and the default tier did better on this page. The English page at 5° with only a 3×3 median filter was later rerun with a second page-segmentation setting: Tesseract's default layout analysis had cut the page into 64 non-empty lines instead of 28 and scored 34.03%, while --psm 6 on the same input scored 96.51%. That was a layout split, not the filter. The macOS fast mode produced 526 edits on the 557-character Chinese page, my guess being that the mode doesn't handle Chinese. None of these raised an error.

Cheap checks before trusting the text

User uploads have no ground truth, so each check compares the output with something the page image or the request gives you. The first is a line count: count horizontal ink bands with a row projection and compare with non-empty output lines. Both bad cases are far off, about half of 14 in one and 64 against 28 in the other. I haven't built this, and it's the part I trust least, because tables, two columns and running headers will confuse a band count and both test pages were one column.

A rough characters-per-line figure for the language should catch that repetition, and so should looking for any run of eight or more characters that appears twice in one line. A script check is even cheaper: if the user picked Chinese and most of the output isn't CJK, the fast-mode result would get flagged. I haven't run either of these against the outputs yet.

One check runs before OCR. On the 3° page a projection-based skew estimate came back as 3.3°. The browser engine was fine at 1° and broke at 3°, so above 1° I'd rotate first. It fixes the problem instead of only flagging it, which makes it the one I'd ship first.

None of these counts sees order. On the English page at 3° almost every character survived, but each paragraph's first line landed after the line that should follow it. Comparing the output's first words with a quick OCR of the top band is the closest idea I have, and I don't know yet whether it's reliable enough to be worth the extra pass.

My matting pick won all 90 frames and still flickered more in 89 of 89

AsyncMonk — Wed, 04 Nov 2026 08:07:06 +0000

If a small client job asks for talking-head clips with the background cut out, I'd want to know before quoting whether a free FFmpeg chroma key is enough or whether I need an AI matting model. So on 2026-11-03 I built a small test, scored both, and got a result that changed how I read my own numbers: the AI matte had a lower error than the chroma key on every one of 90 frames, and it also flickered more on 89 of the 89 frame-to-frame steps.

The test I could afford

I don't have hand-pulled ground-truth mattes and won't pay anyone to make them, so I went cheap. I took a public-domain green-screen clip from Wikimedia Commons, kept the three seconds where the presenter waves (frames 90 to 179), and used a model's transparent export of those frames as the "known" alpha. Then I composited that foreground onto two generated backgrounds at 540×960: a flat green with a little per-frame sensor-style noise, and a busy pattern of shapes that slides left by one pixel per frame. Since I made the composites, I know the alpha every method should return.

That shortcut has a cost I can't measure away. The ground truth is a model's output, not a careful manual matte, so "correct" here only means "what that model said". It came from RVM ResNet50, and the AI method I scored is RVM MobileNetV3 from the same family, so the AI side may look better than it would against a hand-made matte. For the AI side I ran ImgIng https://imging.ai/ on its compatible tier (still Beta, run in a desktop browser). The chroma key was FFmpeg's chromakey with similarity 0.10 and blend 0.

Two scores that disagree

For per-frame error I used SAD inside an edge band, the trimap unknown region, which averaged about 15.8% of the frame. On the green background the AI matte averaged 2.89 there and the chroma key 3.88, and the AI matte was lower on all 90 frames. Gradient error, which tracks edge softness, was 0.53 against 1.04, since the chroma key at blend 0 outputs pure 0 or 1.

For flicker I took the pixels whose ground-truth alpha barely changed between two frames (under 0.5/255, about 93.3% of pixels) and measured how much the predicted alpha moved there:

still = np.abs(gt_cur - gt_last) < 0.5 / 255
jitter = np.abs(pred_cur - pred_last)[still].mean() * 1000
diff = (pred_cur - pred_last) - (gt_cur - gt_last)
dtssd = np.sqrt((diff ** 2).sum() / diff.size) * 100

Restricted to the edge band, the AI matte scored 2.205 and the chroma key 0.198, roughly 11 times lower, and the chroma key was steadier on 89 of 89 frame pairs. My guess: a hard 0/1 key with small noise rarely tips over its threshold, while soft model output drifts a little every frame. A control made of the ground truth plus random per-frame noise (σ=0.06 in the edge band) had the best per-frame SAD in the table, 1.53, and one of the worst flicker scores, 11.844. The usual dtSSD still ranked that noisy control best at 1.93, since it averages over the whole frame and mixes missed motion with jitter, so I keep the static-pixel number separate.

Sample: VOA Learning English, Wikimedia Commons, public domain; trimmed, composited, modified. What to look at: in the top chart the orange constructed-noise line sits lowest and the red AI green-background line stays under the blue chroma key line the whole way; in the bottom chart the order flips, the blue line hugs zero and the orange and green lines bounce between roughly 8 and 23.

What changed in how I pick

The busy background moved the numbers more than the choice of method did. The same AI matte went from SAD 2.89 to 7.44 and from flicker 2.205 to 11.984 when only the background changed, and its per-frame SAD swung from 4.90 to 10.41, peaking around frames 60 to 70. A spot check on one early frame and one between frames 60 and 70 would tell two different stories. So for client work the plan is to test on the background the client actually shoots against, not on a clean green I generated.

The other change is that I don't report one averaged score anymore. I keep two columns, per-frame error and static-pixel flicker, and next to each I write how many frames or frame pairs one method won, like 90/90 and 89/89. A mean of 2.89 versus 3.88 sounds mild; "90 of 90" says it's consistent, and "lost 89 of 89 on flicker" says there's a second question open. These numbers only hold for this clip and my code, not against paper tables.

What I still don't know is whether a flicker of 2.205 (an average change of about 0.002 in alpha per frame) is something a client would notice on a phone, or only something my script notices. Until I check, my working assumption is AI matting where there's soft hair and the chroma key for flat, even green where a hard edge is fine.

Source clip: "Filming with the green screen" by VOA Learning English, Wikimedia Commons, public domain, trimmed and composited by me. The two backgrounds are images I generated.

A skip rule for re-encode scripts, and why the bpp threshold moves with your CRF

AsyncMonk — Tue, 03 Nov 2026 08:07:49 +0000

A re-encode job in a side project has one quiet failure mode: it runs over a file that is already small and hands you back a bigger one. I wanted a skip rule for that, so I measured where the line is. The line turned out not to belong to the file. It moves with the CRF you encode to.

Start from bits per pixel

The script cannot compare raw Mbps, since the same number is generous at 720p and stingy at 1080p. So it works in bits per pixel per frame: bpp = bitrate / (width * height * fps). Across my content types, the point where re-encoding stopped growing files sat at about 0.09 to 0.13 bpp.

The setup: three real clips (a crowd at 720p, a moose at 1080p, a street at 1080p) plus one synthetic testsrc2 clip, all with audio stripped. Each was encoded once at nine CRFs from 12 to 48 with x264 medium, which gives 36 "already compressed" files. Each of those was then re-encoded at CRF 23 and again at CRF 28. Numbers below are this set, one run, on an M4 Mac.

At CRF 23 the line sits near 0.10

Of the 36 files, 24 had a source bpp under 0.08, and every one of them came out bigger, from 1.34× to 6.67×. Four files between 0.08 and 0.13 saved 4% to 9%.

With a cutoff of 0.10, 26 files fall below it. 24 of those grow, and the other two save only 9% and 4%. Nothing that would have saved 10% or more gets skipped.

The same file gets two verdicts

Take the 1080p moose clip pre-encoded at CRF 28. It is 2,670,854 bytes with a bpp of 0.0440. Re-encoded at CRF 23 it becomes 3,949,234 bytes, 1.48×. Re-encoded at CRF 28 it becomes 2,402,497 bytes, which is 10.05% saved. Same input, opposite answer.

My guess is that the cutoff is roughly the bpp your target settings would produce on that content, and below it the encoder has to add bits. I did not test that mechanism on its own. What the table below shows is that the cutoff for CRF 28 comes out lower, around 0.06.

Cutoff	Target	Files below	Of those, grew	Files above that grew
0.06	CRF 23	23	23	1
0.06	CRF 28	23	20	0
0.10	CRF 23	26	24	0
0.10	CRF 28	26	20	0

Use 0.06 while encoding to CRF 23 and one file that grows 1.34× slips through. Use 0.10 while encoding to CRF 28 and three files that would have saved 10% or more get skipped.

Tie the threshold to the target

I did not fit a formula, because I only have two points. The script keeps a lookup and refuses anything else. The part that matters:

THRESHOLD = {23: 0.10, 28: 0.06}
crf = int(sys.argv[1])
if crf not in THRESHOLD:
    sys.exit(f"no threshold measured for CRF {crf}")
# ... bpp from ffprobe, then:
print(f"{'SKIP' if v < THRESHOLD[crf] else 'ENCODE':6s} crf={crf} bpp={v:.4f} {p}")

Run against the moose clip above:

SKIP   crf=23 bpp=0.0440 clip-mid.mp4
SKIP   crf=28 bpp=0.0440 clip-mid.mp4
no threshold measured for CRF 35

Notice the second line. At CRF 28 the rule skips a file that would have saved 10.05%. I accept that. The rule is meant to fail on the cautious side, and I would rather leave 10% on the table than have the job quietly hand back bigger files.

Where the rule stops

Everything here is H.264 to H.264 with preset medium. I did not test H.265, AV1, other presets, or target-bitrate mode, so treat the numbers as a starting point for your own CRF rather than a constant. Five real VP8 and VP9 clips from Wikimedia Commons agreed with the 0.10 rule at CRF 23, but five is a small set. WebM also reports bit_rate=N/A on the video stream, so the script falls back to file size ÷ duration, which runs a little high when audio is inside.

For the five Commons clips I did not need a script. I ran them through the video compressor at ImgIng https://imging.ai/, which takes several files at once and lets you pick the quality profile. It runs in a desktop browser only, since phones and tablets get image compression there. Its default smart profile kept the original for four of the five, so choose a profile if you want every file processed.

Clip credits

Sample videos are from Wikimedia Commons, re-encoded, trimmed and otherwise altered.

Moose by the road.webm, Forza1000, CC BY-SA 4.0
A Demonstration of Disabled People June 2018 1.webm, DGW, CC BY-SA 3.0
Worldwide Demonstration – March 2021 Amsterdam.webm, Karen Eliot, CC BY-SA 2.0
Demonstration of Ethiopian residents in Israel, July 2019.webm, Israel Police, CC BY-SA 3.0
Demonstration in Al-Zabadani, 23 March 2012.webm, Allfor Syria, CC BY 3.0

What one 10-frame sample can and cannot tell you before you change your GIF export settings

AsyncMonk — Mon, 02 Nov 2026 08:07:29 +0000

Before changing the export settings of demo GIFs for a small site, I wanted to know two things: does turning dithering on cost bytes, and how far does a lower colour count go. All I had to look at was one sample, a 10-frame GIF, so this is a list of what it answered and what it left open.

The sample is File:MORPH.gif from Wikimedia Commons, author Kowloonese, CC BY-SA 3.0. It is 675×599, 10 frames at 100 ms each, 3,003,893 bytes, a photo-style gradient with text morphing across it. I ran it through the animation tool in ImgIng (图映) https://imging.ai/ in a desktop browser, then rebuilt the same variants with Pillow and gifsicle 1.98 to see whether the pattern survives a second pipeline. One file, one Apple M4, one run each, and every size below is exact bytes.

What the sample settled

Fewer colours means fewer bytes, and by a lot. In the tool, with dithering off, 256 colours gave 1,525,126 bytes and 32 colours gave 811,310. The 256-colour number is the baseline for everything else here.

Dithering costs bytes at every palette size I tried. With Floyd–Steinberg on, 32 colours went from 811,310 to 1,110,431 (+36.9%), 64 from 1,083,830 to 1,339,435 (+23.6%), and 128 from 1,337,247 to 1,475,692 (+10.4%). In my own rebuild the gap was wider, +32% to +50% across eight variants. At 32 colours the blurred copy of the source went from 572,768 to 861,321 bytes (+50.4%).

The lossy pass did not close that gap either. With gifsicle -O3 --lossy=N on the 256-colour rebuilds, the dithered file stayed 32.7% to 35.4% larger at lossy 30, 60 and 100, and 34.9% larger at 200. Lossy 200 saved about 22% on the plain 256-colour file, at the highest level I tried.

What the sample cannot settle

The first thing it cannot tell you is the size of the dither penalty. In the tool it ran from +36.9% at 32 colours down to +0.7% at 256 (1,525,126 against 1,535,115). My guess is that the source GIF is already at 256-colour level, so there was no error left for dithering to spread, but I did not test that.

The second is what "32 colours, dither off" costs, because that phrase named four different files in my runs: 811,310 from the tool, 625,102 and 572,768 from my two rebuilds, and 426,315 from gifsicle's own colour reduction on the original. They come from different palettes and different starting frames, so none of them can be ranked against another. The commands behind the last one:

$ gifsicle --info demo.gif | head -4
* demo.gif 10 images
  logical screen 675x599
  global color table [256]
  background 0
$ gifsicle -O3 --colors 32 --no-dither demo.gif -o demo_s.gif; stat -f%z demo_s.gif
426315
$ gifsicle -O3 --colors 32 -f demo.gif -o demo_s.gif; stat -f%z demo_s.gif
1005103

Dither on versus off gave +135.8% in this gifsicle run and +50.4% in the Pillow rebuild at the same 32 colours, which says more about the pipeline than about dithering.

The third is why. Counting LZW output codes on four of the rebuilt files, dithering raised the code count by 50.3% at 32 colours, almost the same as the +50.4% in bytes, and the share of pixels equal to their left neighbour fell from 85.8% to 55.8%. That much I measured. Reading it as "the noise breaks up repeated pixel runs" is my explanation, and I did not run anything to prove it.

Last, the clip itself is a poor stand-in for most GIFs. Nearly every pixel changes in every frame, so frame differencing had nothing to save, and -O3 only trimmed tens to a few thousand bytes. It is not a screen recording or a flat-colour UI loop. I would not carry any of these percentages over to those.

What I would run on my own files

Take three of your real GIFs, not one. Export each at 256, 64 and 32 colours with dithering off and on, write down the byte counts, and zoom one frame to 10× to see whether the dithered version shows single-pixel noise where the plain one shows flat patches. If the dither-on column is not clearly larger for your content, the switch is not your problem, and frame count or dimensions are the next things to try.

For the byte counts I used the tool in a desktop browser, because it lets me set colour count and dithering by hand. In my capture, no request carried the file during compression. Phones and tablets only offer image compression there, so this is a desktop job.

What one 4-page sample can and cannot tell you before you fix a preview DPI

AsyncMonk — Sun, 01 Nov 2026 08:05:33 +0000

I wanted to fix one number before building a preview pipeline for PDF manuals: the DPI. Pick too low and a preview may look soft on a dense screen, pick too high and every page costs more bytes to serve. The only thing I had to decide with was one 4-page brochure converted a handful of ways. This is a list of what that one file settled, and what it quietly left open.

The brochure is the first four pages of an eight-page Wikimedia Commons PDF from Wiki Education Foundation and others, CC BY-SA 3.0. It is 1,180,300 bytes, 504×720 pt per page, exported from InDesign, with a big title, body text, screenshots and page numbers. I ran it through ImgIng (图映) https://imging.ai/ in a desktop browser at its three DPI settings, and through three free online converters for comparison. Everything is one file, one Apple M4, one run each, and all sizes are exact bytes.

What the sample settled

The pixel formula held without surprises. Pixels equal points times DPI divided by 72, and because 504 and 720 are whole numbers, every setting came out exact: 1008×1440 at 144 DPI, 1512×2160 at 216, 2100×3000 at 300.

Bytes grow more slowly than pixels. The JPG output was 1,000,564 bytes at 144 DPI, 1,745,636 at 216 and 2,822,064 at 300. By my arithmetic that is 2.25 times the pixels from 144 to 216 for 1.74 times the bytes, then 4.34 times the pixels at 300 for 2.82 times the bytes. WebP and PNG moved in the same direction: 581,714 / 941,614 / 1,498,212 for WebP and 1,780,434 / 3,195,502 / 5,076,139 for PNG.

Transparency is easy to miss on a sample like this. The browser tool's PNG and WebP keep the page background transparent, with 83.3% of the pixels on page 2 at alpha 0 and 9.5% on page 1, while the JPG fills the same area with white. If a preview goes onto a dark card, that decides the format before DPI does.

On uploads, I watched the network requests. The three online converters each sent the PDF to their servers (Smallpdf to cloud storage, PDF24 and iLovePDF to their own hosts). Across 18 runs of the browser tool I saw only GET requests and nothing that carried the file. That is what these runs showed, nothing more.

What the sample could not settle

Rounding. My pages are whole-number points, so nobody had anything to round. An A4 page at 300 DPI is exactly 2480.32×3507.88 pixels, and the tools I checked disagree about the last pixel: pdftoppm and PyMuPDF give 2481×3508, a pdf.js script using floor gives 2480×3507, and ImgIng gives 2480×3508. A sample of 504×720 pt never shows you that.

The byte comparison between tools. The four JPG encoders were not set to the same quality. Reading the quantization tables suggests roughly 75 for Smallpdf, 85 for PDF24, 100 for iLovePDF and 90 to 95 for ImgIng depending on the DPI, which is an inference and not a vendor parameter. At 300 DPI the totals were 1,658,508 for Smallpdf, 2,231,511 for PDF24 and 2,822,064 for the JPG from the browser tool, and that spread is mostly the quality setting. Even inside that one tool the quality guess moved with the DPI, so my 2.82 times is DPI and quality changing together.

Clarity. I looked at one small block of page 2, a line of small print from a screenshot, at 300 DPI. Smallpdf, PDF24 and the browser tool were indistinguishable to my eye on that block, while iLovePDF's default output is 150 DPI, so the same block has fewer pixels to work with. That is one block, no PSNR or SSIM.

The free-tier limits. Smallpdf's support page says, as read on 2026-10-30, that free and unregistered users get "up to 2 conversions per day". I converted the same file 8 times without a login from one IP and the page never showed a limit message. I did not find out why, so I will not call the official number wrong. PDF24 says "No limits" on its tool page, and iLovePDF's pricing page says "Limited document processing" without a number.

The kinds of PDF I have no data on. No scans, no pages with transparency groups or unusual colour spaces, no timing, and no real huge page. The only huge pages were synthetic blank vector pages, where a canvas past 268,435,456 pixels in Chromium 149 returned a transparent blank without an error. Phones and tablets are out as well, since ImgIng only does image compression there, so PDF to image needs a desktop browser.

What I would still measure before fixing the DPI

Take the ugliest real file you have, not a clean brochure. Write down its page size in points, work out the three pixel sizes by hand, and decide how you round before any tool decides it for you. Then convert at 144, 216 and 300, and record bytes next to pixels per page, so that the 1.74 and 2.82 in this post get replaced by your own ratios. Look at a block of small text at the size your page will really show it, and only then pick the DPI.

What one portrait and one hair sample can and cannot tell you before you change a thumbnail pipeline

AsyncMonk — Sat, 31 Oct 2026 08:05:35 +0000

Before touching a thumbnail pipeline that shrinks cut-out PNGs, I wanted to know one thing: does it matter whether RGB is premultiplied by alpha before the resize? This post uses two of my three test cut-outs, the portrait and a white dog, and covers what they settled and what they left open.

The cut-outs came from ImgIng https://imging.ai/ in a desktop browser, using its AI cut-out tool and exporting PNG at the input size. The sources are two Wikimedia Commons photos, a portrait (Dmitry Makeev, CC BY-SA 4.0) and a Bichon Frise (TudorTulok, CC BY-SA 4.0), both scaled to a 2000 px long edge. I shrank each result by a factor F and enlarged it back, once per channel and once premultiplied. Everything below is these two images, one Apple M4, one run each.

What the two samples settled

The blur and the dirty edge are two separate problems. The alpha channel came out identical both ways, with a maximum difference of 0, so the softer edge comes from interpolation, which both methods share. Only the RGB under the edge changes. That is the one result I would trust across both images.

The portrait showed the damage clearly. After a 4x round trip, the mean error in the edge band against the premultiplied baseline was 2.991 levels on a 0 to 255 scale, the worst pixel was off by 46.9, and 13.83% of the edge pixels differed by more than 8 levels. At 8x it grew to 3.879, 53.5 and 19.27%. The premultiplied run sat at 0.025 on the same 4x test, but that number is flattering: the baseline is itself a premultiplied resize, so it only says how much extra colour the straight version smears in.

The dog behaved differently. At 4x the mean was 0.505, the maximum 15.1, and only 0.12% of edge pixels moved by more than 8 levels. At 8x it rose to 1.560, 26.9 and 4.98%. If I had tested only the dog, I would have shrugged and kept the old code.

The reason sits in what is stored under the nearly transparent pixels. Pixels with alpha 1 to 5 averaged about (221, 218, 219) in the portrait and (30, 31, 32) in the dog. One row of the portrait shows what that does when channels are averaged separately:

sharp default            (49, 43, 49, 41) (0, 0, 0, 0) (255, 255, 255, 11) (79, 82, 89, 74) (33, 34, 38, 191)
sharp, RGB split by hand (220, 218, 219, 41) (255, 255, 255, 0) (240, 241, 243, 11) (180, 181, 184, 74) (72, 74, 78, 191)

Same alpha in both rows, 41, 0, 11, 74 and 191. The RGB is not the same at all. A separate box-and-bilinear run on the same row shows the same thing on the dark clothing. The pixels that stored white at alpha 1 to 5 end up with red values of 195 to 244 when channels are scaled separately, against 76 to 113 when premultiplied.

What they could not settle

Direction. The pale fringe on the portrait is in the numbers above. A dark fringe on the dog is only my reading of its stored colours (30, 31, 32), and I did not confirm it with a second set of images. Two photos tell you the effect depends on the colour hidden under the transparent edge. They do not tell you your files look like either one.

Other scalers. I used box filtering to shrink and bilinear to enlarge. In the library comparison, sharp 0.35.5 on the default path sat close to the premultiplied result, with the mean brightness of semi-transparent pixels at 109.63 against 112.0 for premultiplied and 129.48 for straight. Splitting RGB and alpha by hand moved it to 125.1. In an eight-pixel test, canvas drawImage kept the transition pixel clean, while createImageBitmap with premultiplyAlpha set to none let the transparent blue leak into it. That was Chromium 149 only. I did not run Safari or Firefox, and a different version of either library may differ.

A trap in canvas too. A pixel written as (200, 100, 50) with alpha 2 reads back as (255, 128, 0). So if your pipeline round-trips through a canvas, the colour under faint alpha is already gone before you resize anything. I did not check whether ImgIng's export goes through a canvas.

Uploads. I looked at network requests while cutting out on ImgIng and saw only model file downloads, with no request that carried an image. That is what this run showed and nothing more.

What I would measure before changing anything

Take your worst real cut-out, not a nice one. Read the RGB of pixels with alpha between 1 and 5 straight from the PNG file, not through a canvas, and see whether they are near white, near black or something odd. Then do one 4x round trip both ways and count the edge pixels that differ by more than 8 levels. If that count is near zero on your files, as with the dog, leave the pipeline alone.

What 12 PNGs can and cannot tell you before you swap a paid API for local tools

AsyncMonk — Fri, 30 Oct 2026 08:06:21 +0000

If you pay per image for compression, sooner or later you look at the bill and wonder whether two open-source command-line tools could do the job for free. I ran that experiment on 2026-10-29 with 12 PNGs, and the numbers came out clean. This post is about what those numbers support, and what they do not.

What I ran

The 12 files are four kinds of PNG, three of each: screenshots, icons, illustrations and photos saved as PNG. All come from Wikimedia Commons under CC licenses, 4,454,411 bytes in total. The machine is an Apple M4, and the tools are pngquant 3.0.3 and oxipng 10.2.1. Every step ran once per file. Sizes below are bytes (1 KiB = 1024), and times are wall-clock for the three files of a class together.

For one icon (40,369 bytes) the whole pipeline is two commands. This is the real output for that file.

$ pngquant --version
3.0.3
$ oxipng --version
oxipng 10.2.1
$ pngquant --quality=65-80 --speed 4 --force --output Q.png IN.png; echo exit=$?
exit=0
$ cp Q.png Q_ox.png; oxipng -o 4 --strip safe Q_ox.png

Files processed: 1/1   
Input size: 7.42 KiB (7600 bytes)
Output size: 6.79 KiB (6954 bytes)
Total saved: 646 bytes (8.50%)
exit=0

What the numbers do say

Palette reduction does most of the work. pngquant at quality 65–80 cut each class by 66.3% to 79.8%, and across all 12 files the total fell from 4,454,411 to 1,411,877 bytes, a 68.3% saving. Running oxipng afterwards took it to 1,283,130, which is 71.2% overall. The extra pass was worth 1.6 to 4.0 percentage points per class and cost roughly 0.8 to 1.6 seconds per class (three files together).

Order matters. When I ran oxipng first and pngquant second, the result was almost the same as pngquant alone: 694,253 bytes against 694,735 for the photos. My reading is that pngquant re-encodes the file and throws away the earlier work, but I did not look inside to confirm that. The measured part is that the lossless step only paid off after the colour step.

zopflipng was not worth it in this setup. It added 0.0 to 0.5 points on top of pngquant and took 3.5 to 19.5 seconds per class. I ran it in its quick mode (-q) with -y --filters=01234mepb and did not try -m, so this says nothing about zopfli with more effort.

What they do not say

Twelve files is not a corpus. Each class has three, and the icons are small (23 KB to 120 KB), so one file moving by a few hundred bytes shifts a class percentage noticeably. The ranking of the extra gain, photos then icons then screenshots then illustrations, is a ranking of three files each. I do not know why the classes differ and I am not going to guess in print.

I also never compared quality. Reducing to a 65–80 palette is lossy, and I did not measure it or sit down and compare pairs by eye. If the paid service you use picks its own quality level, my byte counts say nothing about whether the output looks the same. I also ran nothing at volume, so I cannot tell you about memory or failures over thousands of files, and a single run per step means the timings are one sample.

The comparison I most wish I had is the one you actually need: your own files against your current service. The Wikimedia set is CC-licensed and credited, which says nothing about how closely it resembles your uploads.

Where a browser tool fits

If you only need a one-off batch and do not want to install anything, the same palette-then-lossless idea can be tried in a browser. I used the batch page of ImgIng (https://imging.ai/), which runs in a desktop browser and, in the network log I checked, sent no request carrying my files. Its "unified PNG" mode did not behave the way I assumed: 7 of the 12 outputs came out larger than the originals, and one was left untouched. The four that did shrink were palette PNGs, and running oxipng on them saved another 28.6% to 45.5%. That is four files, so treat it as a hint that browser output can still be squeezed, not as a rate.

What I would check before switching

Take 30 of your real files, not mine, across the categories you actually serve. Run the two commands above, keep the originals, and look at the results at the sizes your users see. Count how many files grew or came back unchanged, because a pipeline that only works on average is still a problem on the file that fails. Only then compare the total against what you pay per call, including the time you spend maintaining the script. If the 12-file numbers roughly hold on your 30, the swap is worth pursuing. If they do not, you have saved yourself an afternoon of migration.

Three things I saw when a PDF had fonts I may not embed

AsyncMonk — Thu, 29 Oct 2026 08:06:09 +0000

Putting a PDF brochure on a website is a common small-site job, and converting it to a web page is the easy part. The fonts are harder to reason about. If the PDF carries a font whose license says no embedding, what does the converter do with it? I tried a set of test files to find out, and the answer was not one thing but three.

What the license flag says

A font with an OS/2 table carries a 16-bit license field there, called fsType. The spec page for that table on learn.microsoft.com, which I read on 2026-10-28, puts the usage permission in bits 0–3 (valid values 0, 2, 4 and 8), a no-subsetting flag in bit 8 and a bitmap-only flag in bit 9.

I scanned the fonts on my Mac. Pacifico, Liberation Sans, Helvetica and Noto Serif SC report 0. Palatino and PingFang SC report 4. Arial and Georgia report 8. I found no installed font with 2, 256 or 512, so for those values I made test copies by editing a font file. Everything below is one machine, one run per case, so read the numbers as examples and not as rules.

Case one: the font turns into outlines

I printed a bilingual page to PDF with Chromium 149 using copies of Pacifico and Liberation Sans at each fsType value. Then I opened the PDFs and listed the font objects. With fsType 2 or 512, Chromium did not embed the font at all. The letters were written as Type 3 outlines instead.

I ran those PDFs through ImgIng (https://imging.ai/), a browser tool that is still labeled Beta. It works in a desktop browser and processes the file locally. For the fsType 2 and 512 files it inlined no font. Every glyph came out as a vector path, and the page still showed the Pacifico lettering. Text selection goes through a transparent layer set in plain Arial. I did not check how well that layer lines up with the lettering.

In these test files this was the quiet outcome: no font is embedded, and the page still looks right. I did not record whether the summary panel said anything about it.

Case two: the whole font ends up in the HTML

A font with fsType 256 means "do not subset". Chromium respected that by embedding the entire font in the PDF. The Liberation Sans PDF grew from 85,024 to 502,702 bytes.

The converter then did what it saw. In the HTML, each of the two fonts was inlined as a complete file: Liberation Sans at 211,588 bytes with 2,620 glyphs, and Pacifico at 144,432 bytes with 1,533 glyphs. With the normal subsets I got 94 glyphs for Liberation Sans. So one flag took the Liberation Sans glyph count from 94 to 2,620.

This is the case I would check by hand. A whole font file sitting inside a public web page is a different thing from a subset of the letters you used. If the license is the reason for the flag, that is the moment to read it.

Case three: the font is not usable, so you get a stand-in

This one has nothing to do with fsType. The 4-page sample I used for layout comparisons is a brochure ("Evaluating Wikipedia" by Wiki Education Foundation and others, CC BY-SA 3.0, first four pages) whose body text is set in a Type 1 font, Plantin. The summary panel says the font stream the PDF carries cannot be used directly by a browser. The converted page switched those 509 text elements to Arial, and the summary panel said so, with a note on pages 2 to 4 that the font was not embedded in the PDF and a nearby system font was used.

The other four fonts in that file were TrueType with fsType 4 or 8, and they were inlined as usual. So one page can mix all of this: real fonts, and a stand-in for the one that could not travel.

What I would check before publishing

These are my assumptions from the three cases, not tested rules. First, read the summary panel. It tells me which pages got substituted fonts, which is more useful than any guess about licenses. Second, look at the converted page next to the original PDF at the same zoom. Outlines look right, substitutes do not. Third, if a font in the PDF has the no-subsetting flag, check how big the HTML is and open the license before it goes live.

I did not test every converter, and I have not compared this with pdf2htmlEX. These three things happened in my files, on one machine, once each.

My transparent WebM is 2.4 MB. The editor wants 62 MB. Which format do I send?

AsyncMonk — Wed, 28 Oct 2026 08:34:32 +0000

Say an editor asks you for a transparent clip with the person already cut out. I had one: an 8-second, 540×960 WebM from ImgIng (https://imging.ai/), 2,394,232 bytes, audio included. Converting it to something an editing app accepts made the file 26 to 38 times bigger. I live on free tiers and I hate paying for egress, so I did the arithmetic before picking a format.

ImgIng's video matting is Beta, runs in desktop Chrome or Edge, processes locally, and does not work on phones. Transparent output is WebM only, which is why this question exists at all. The footage is a VOA Learning English clip from Wikimedia Commons (public domain), cut to its first 8 seconds. Everything below is that one clip, one run, FFmpeg 7.1 on an M4 Mac. MB here means 1,000,000 bytes. The ImgIng UI counts 1,048,576, so its numbers read smaller.

Three formats keep the alpha at different prices

I converted the same WebM to ProRes 4444, a PNG sequence and QuickTime Animation (qtrle), then read the alpha back from each file. All three matched the source exactly: mean alpha 46.762 on frame 60, range 0 to 255. The outputs have no audio, while the source WebM still carries about 0.3 MB of sound, so the true ratios are slightly worse than the ones below.

ProRes 4444 came out at 62,034,554 bytes, about 26 times the WebM. The PNG sequence, 240 files, was 63,857,616 bytes, about 26.7 times. qtrle was 92,176,076 bytes, about 38.5 times. Per second of footage that is roughly 7.8 MB for ProRes and 11.5 MB for qtrle, and about 16 ProRes clips of this length would fill a 1 GB budget. That is plain division on this clip. I would not scale it to a 10-minute video without converting a slice of it first.

ProRes and PNG frames cost the same and qtrle costs half again

An image sequence sounds like the heavy, safe option, and ProRes sounds like the compromise. On this clip they are 62.0 and 63.9 MB, three percent apart. The odd one out is qtrle, about 49 percent bigger than ProRes for the same alpha.

So size alone settles one thing: qtrle has to earn its place some other way, for example because the editor's tool only takes that codec. Between ProRes and PNG frames the bytes are a wash and the choice moves to handling. One .mov is one upload and one link. Two hundred and forty PNGs are a zip and a numbering convention to agree on, and every extra file is a chance to lose one on the way.

One flag decides whether the alpha survives

$ ffmpeg -hide_banner -c:v libvpx-vp9 -i input.webm -an -c:v qtrle -pix_fmt argb out_qtrle.mov
frame=  240 fps=0.0 q=-0.0 Lsize=   90016KiB time=00:00:08.00 bitrate=92176.1kbits/s speed=12.2x

The -c:v libvpx-vp9 before -i picks the decoder. The ProRes run is the same command with -c:v prores_ks -profile:v 4444 -pix_fmt yuva444p10le on the output side. Leave the decoder flag off and FFmpeg's built-in VP9 decoder reads every alpha value as 255. That gave me a 41,105,728-byte ProRes file, smaller and playable, with no error and the whole frame opaque. The cheaper file was the broken one.

What I send

ProRes 4444 as a single .mov by default, because it is the smallest of the three I verified. PNG frames if the editor asks for frames or wants to touch single ones. QuickTime Animation only when someone names it. I also keep the 2.4 MB WebM, since that is the copy worth archiving.

One thing stays open. I have no source on how a given editing app treats a transparent WebM, so I cannot tell you which editors would take the small file directly and skip all of this. If yours does, the 2.4 MB file saves you about 60 MB. Test one short clip first.

Our upload cap is 10 MB: how much margin does a bitrate-targeted encode need?

AsyncMonk — Tue, 27 Oct 2026 08:36:30 +0000

A hard 10 MB upload cap is easy to trip over, and I wanted to see whether a bitrate calculation could fit a demo clip under it without re-encoding until it works. The textbook formula is target size × 8 ÷ duration. I ran it on two clips on an M4 Mac with FFmpeg 7.1 (x264 medium, AAC 128 kbps). Every number below is one run, on these two clips only.

Clip A is a 60-second 1080p street-performance clip by Isak Zvegelj (CC BY 3.0, Wikimedia Commons), which I re-encoded to a phone-like bitrate beforehand. Clip B is a generated screen recording, much quieter than a real capture would be.

First, what does 10 MB mean to your server

A cap written as 10 MB is either 10,000,000 bytes or 10 MiB, which is 10,485,760 bytes. I use decimal MB for everything below, so check which one your limit is before you pick a margin. The gap is 4.9%, which is more than any margin I ended up needing on clip A.

One example of how this bites: the smallest-size preset in ImgIng (https://imging.ai/), a browser tool I ran in desktop Chromium 149, gave 10,487,546 bytes on clip A at 1080×608, 24 fps. That is 10.49 MB decimal, and also 1,786 bytes over 10 MiB. ImgIng has presets, not a target-size option, so landing near a number is luck there, not a setting.

The plain formula overshoots by 9 to 11 percent on clip A

Dividing 10,000,000 × 8 by 60 seconds gives 1333 kbps for video. I fed that to a single pass and a two-pass encode.

Clip A, 10 MB target	Bytes	vs. target
Single pass, no audio deducted	10,917,084	+9.17%
Two pass, no audio deducted	11,124,442	+11.24%

Most of the excess is the audio track. AAC 128 kbps over 60 seconds is about 0.97 MB, which is 9.7% of a 10 MB budget. The MP4 container added roughly 66 KB. In the single-pass run the video stream itself came out a little under budget on A, so the audio was the main thing the formula left out. Two-pass put the video about 86 KB over.

Deduct the audio and the error shrinks, but not evenly

Subtracting 128 kbps first is one line of arithmetic. With a 2% margin written in, it looks like this:

target, secs, margin = 10_000_000, 60, 0.02
audio = 128_000 // 8 * secs
print(round((target * (1 - margin) - audio) * 8 / secs / 1000))

It prints 1179 kbps. I did not re-encode with the margin included. The runs below used 1205 kbps, which is the same formula with margin 0.

10 MB target, audio deducted	Clip A	Clip B (synthetic)
Single pass	9,965,325 (−0.35%)	9,134,627 (−8.65%)
Two pass	10,174,131 (+1.74%)	11,195,510 (+12.0%)
CRF bisection	9,854,805 (−1.45%)	9,838,907 (−1.6%)

Two-pass was the only one that went over on A. I had assumed it would be the most accurate, and on these clips it was not. On clip B it overshot by 1.2 MB, because a mostly static picture gives the rate controller very little to bite on, and it spent bits on the scroll jumps. That reason is my guess from the per-second bitrate curve. I did not isolate it.

CRF bisection never went over, but each round re-encodes the whole file. Seven rounds took 157 seconds on A and 59 seconds on B, against about 23 seconds for a single pass.

What margin I would leave

On clip A, 2% was enough to cover the worst case, which was two-pass at +1.74%. That is arithmetic, not a rerun. On clip B, 2% would not have saved the two-pass run, which was 12% over. So the margin depends on how predictable your footage is.

For a hard cap, I would take 3% off the target for busy camera-style footage like clip A, run single pass, and check the file size after the encode. If it is over, drop the bitrate by the overage and encode again. For screen recordings I would avoid two-pass until I had checked a real one: a single synthetic clip is a thin sample, but the miss was large.

Do the MiB check too. A 10,174,131-byte file fails a 10,000,000-byte cap and passes a 10 MiB one, so knowing which cap you have can matter more than the encoder choice.

Keeping 18-digit IDs intact between OCR text and a spreadsheet

AsyncMonk — Mon, 26 Oct 2026 08:13:53 +0000

The OCR text of my sample score sheet had every 18-digit ID right. The .xlsx I built from it had 7 of the 8 IDs wrong, and all 8 student codes had lost their leading zero. Nothing was misread on the way. I wrote the IDs into the file as numbers, and that one choice did the damage.

The OCR text was already fine

The sample is a made-up score sheet: 8 students, 7 columns, a 7-digit code that starts with 0, an 18-digit ID number, and one ID that ends in X. I ran it through ImgIng (https://imging.ai/) in a desktop browser with the default Professional model, and worked from the text it copies out. It only runs on a desktop, it doesn't read Korean, and the numbers still need checking by eye.

The copied text separates columns with tabs, but not all of them. In my 8 data rows the English score and the total are joined by a space, and in four of them the name and the ID share a cell too, so I get 5 or 6 cells where the table has 7. I haven't pasted that text into a spreadsheet, so I can't say what Excel does with it. Because positions looked shaky, I pulled the IDs out by shape instead of by column. \d{17}[0-9X] found 8 IDs and 0\d{6} found 8 codes, and both lists matched my ground-truth CSV exactly.

The same 16 cells, written four ways

I wrote the code and ID columns (8 rows, 16 cells) with openpyxl 3.1.5 on Python 3.9.6, then read the file back and compared with the source strings.

How the cell was written	Cells that differ after reading back
Python `int`	15 of 16
Python `float`	15 of 16
`int`, then number format set to text (`@`)	15 of 16
`str`, with or without `@`	0 of 16

The 15 are the 8 codes (0230101 came back as 230101) and 7 IDs that came back as floats. The ID ending in X survived only because it was never a number.

Passing an int didn't help. Inside sheet1.xml, the first ID was stored as <v>4.401062010041836e+17</v>, so the file never held the 18 digits at all. Setting the text format afterwards just labels a value that was already rounded. The order is the whole story: decide the type before the write.

# what I did first
for code, pid in rows:
    ws.append([int(code), int(pid) if pid.isdigit() else pid])

A double can't hold 18 digits

int(float("440106201004183613")) gives 440106201004183616. Across the 7 all-digit IDs, 7 came back different. A 64-bit double keeps roughly 15 to 17 significant digits, and 18 doesn't fit.

JavaScript has the same trap before any spreadsheet is involved. In node, JSON.parse on an array of the same 7 IDs written without quotes changed 7 of 7, and 440106201004183613 came back as 440106201004183600. If the OCR result goes through an API as JSON, the ID has to be a string there too.

What Excel adds on top

Microsoft's page on keeping leading zeros and large numbers says Excel stores 15 significant digits and turns anything after the 15th into 0, and that ID-like columns should be text. By that rule my first ID would show 440106201004183000 in a number cell. That is my reading of the docs. I didn't open any of these files in Excel, so I can't tell you what the screen shows.

What I changed

IDs, codes and barcodes stay str from the moment the OCR text is split until the file is written. Being .isdigit() isn't a reason to call int(). Then a check reads the file back and fails the job if any ID cell isn't a string of the right shape:

ok = isinstance(v, str) and re.fullmatch(r"\d{17}[\dX]", v)

On the int and float files this flagged 7 cells each. On the str file it flagged 0.

One question I haven't closed: when a plain CSV of these IDs is double-clicked open in Excel, does the ID column turn into numbers by itself? I haven't tested it. Until I have, the checklist is two lines: write strings, read them back.

What these numbers cover

One table, 8 rows, two columns. The IDs are random 18-digit strings and not valid ID numbers, and the sheet is clean and English-and-digits only. Everything above is about what openpyxl and node store, not about what any spreadsheet app displays.

Pulling assets out of a client's brochure PDF: where raw extraction stops being enough

AsyncMonk — Sun, 25 Oct 2026 08:06:33 +0000

A typical freelance message: "Here's our brochure as a PDF, can you put these pictures on the site?" Pulling out the images stored in the file is the cheapest answer, and it hands back original pixels instead of a screenshot. Before sending anyone a hero image I wanted to know when that is enough and when the extracted file isn't the picture the brochure shows. I have no client file to show, so I used a public brochure (Creative Commons Indonesia's "Brosur Creative Commons Indonesia", CC BY 4.0, via Wikimedia Commons) and a one-page PDF I wrote byte by byte with four images whose answer I know. Desktop Chromium 149 on an M4 Mac, 1 KB = 1024 bytes, one sample each, default settings.

The brochure was the easy case

The brochure is two A4 landscape pages. I opened it in ImgIng (https://imging.ai/) in the desktop browser, which reported 8 results, 419 KB, with cards marked page 1 and page 2, and the saved ZIP was 430,633 bytes. The cover came out at 624×882. A Retina screenshot at 100% zoom would be 399×565 (simulated, not a real screenshot).

Catches I'd tell a client up front: it takes one PDF per run (I dragged in two files and it read the first, with no message about the second); a page with no images gives 0 results; and ZIP files are numbered image-001 and so on, so page numbers exist only on the on-screen cards.

The brochure couldn't tell me if anything was cropped

PyMuPDF lists 9 image placements and 20 clip entries across the brochure's two pages. The list doesn't say which clip sits on which image and I never matched them, so I can't say whether any of the 8 pictures is partly hidden. I built a file where I know.

Four images, three agreeing tools

The hand-built page has four lossless images. Each has coloured corners and a big letter so I can tell its direction. The page draws A through a clip rectangle that keeps the left half, B rotated 90° counterclockwise, C mirrored, and D as a 16-colour palette image with a /Decode [15 0] entry. I ran pdfimages 26.08.0, PyMuPDF 1.26.5 and the decoded bitmap from pdf.js 4.10.38 (it has no public extract call, so I took the bitmap in Chromium).

Compare "Visible on the page" with "Extracted": A shows its left half but comes back whole at 240×160, B is portrait on the page and landscape in the file, and C is mirrored on the page but not in the file. The last column says "yes" because pdfimages, PyMuPDF and pdf.js returned identical pixel hashes, and on D all three also matched the page. ImgIng is not in this picture.

I ran ImgIng on the same file. On A, B and C it returned the stored image like the other three: whole, not rotated, not mirrored. On D it did not match the page: it saved the palette colours as stored, so the red bar sat on the opposite side, where the other three had applied the /Decode. I only tried this 4-bit [15 0] case, not an 8-bit palette.

What I look at before I hand over an extracted file

The pixels can't say how the page uses them, but PyMuPDF gives each image's transform.

import fitz
pdf = fitz.open("four_images.pdf")
for tag, im in zip("ABCD", pdf[0].get_image_info(xrefs=True)):
    a, b, c, d = im["transform"][:4]
    print(tag, "rotated" if b or c else "-", "mirrored" if a < 0 else "-")

A - -
B rotated -
C - mirrored
D - -

On this file that is enough: B has (0, -180, 120, 0) and C has a negative first number. That reading comes from the two transforms in this file, so the rule is a starting point.

Clipping is trickier. The bbox for A is 180 pt wide, the whole drawn picture, though the page shows 90 pt. Two ways out: render the page and crop, or read the clip rectangle and crop the extracted file. On this file the second matches a 96 dpi page render exactly (average error 0.00 on a 0 to 255 scale), because the image is stored at 96 pixels per inch. At other render sizes the error is 0.98 to 2.51, from resampling, and a 600 dpi render only upscales the stored 240 pixels.

What I haven't tested

A mirror combined with a clip, rotations that aren't multiples of 90° and three or more nested clips are untested. I also don't know how the brochure's InDesign output compares with a hand-built PDF; I'd guess it lays images out the same way, but that is a guess. My routine for a client file is now: extract, run the transform check on every image, then put the extracted cover next to a render of its page before I rename anything.