I had the fan-out command typed and my finger on the key. Forty-three agents, 1,700 images, a batch size chosen by a model I’d derived twenty minutes earlier that fitted my measurements exactly.
The only reason I didn’t run it is that one more batch happened to finish first, and its number was impossible... well... okay... "improbable".
What I was doing
1,785 hero images scraped off someone’s content-mill blog, filenames like 112.png and 1_HYBQIscoOfGr5jso8iX9Mg.jpg. The job was mechanical: show each to a cheap vision model, get back a description and keywords, write a JSON metadata sidecar for each.
One decision was actually interesting. How many images should each agent process before I start a fresh one?
Why that isn’t obvious**
An agent looping over images doesn’t pay a flat per-image cost. It pays three things:
- Startup: its system prompt and tool definitions, loaded before it sees anything.
- The work: the image, plus the JSON it writes.
- Resend: every turn re-sends the conversation so far. Image 40 drags 39 images of history behind it.
Term 1 wants big batches. Term 3 wants small ones. Somewhere between them, a sweet spot.
Wrong #1: “smaller is cheaper”
My reasoning: resend grows with batch size, so keep batches small. I wrote that down, with numbers, confidently.
Then I measured a batch of 5. 6,468 tokens per image. A batch of 25 had come in at 2,600.
Smaller wasn’t cheaper. Smaller was two and a half times worse. Startup turned out to cost roughly 30,000 tokens spread across 5 images. That’s 6,000 each of pure overhead before a single pixel gets looked at; across 25 it’s 1,200.
I had confidently optimised the smaller of the two terms.
Wrong #2: the beautiful curve
Chastened by failure, I did it “properly.” Three points (5, 10, 25) and a two-term model: fixed cost over batch size, plus a resend term growing with it. Differentiate, set to zero.
n ≈ 24.7. The agent predicted 2,600 tokens/image at n=25. Measured: 2,600.
An exact match. That’s the moment I typed the fan-out command.
Then batch 30 landed: 2,045 tokens per image, against a predicted 2,646. And its total was 61,362 tokens versus n=25’s 65,010.
Thirty images cost less in absolute terms than twenty-five. Total cost cannot fall as you add work. That’s not a process needing refinement, we must be measuring something other than what I thought.
My “exact match” was never validation. I’d derived the curve from three points, then congratulated it for passing through those three points.
The number that settled it
I stopped theorising and just ran the job: 41 production batches, all at n=40, 1,640 images.
| Statistic | Tokens per Image |
|---|---|
| Mean | 2,212 |
| Median | 2,177 |
| Minimum | 1,820 |
| Maximum | 2,717 |
| Standard deviation | 198 |
The spread across identical-size batches is 40.5% of the mean.
Sit with that. The gap I’d built an entire model on (n=25’s 2,600 against n=30’s 2,045) is smaller than the variation between batches of the same size. It's the same trap agent teams fall into when they trust a model's own confidence signal instead of integrating structural checks (see The 'Yes Reflex': Why Your AI Agent's Biggest Risk Isn't Hallucination) for the agent-governance version of this problem. Every conclusion I’d drawn about the shape of that curve was inside my own noise floor. The curve wasn’t wrong so much as it was never measurable with what I had.
Images per Run (n) |
5 | 10 | 15 | 25 | 30 | 40 (41 runs) | 74 |
|---|---|---|---|---|---|---|---|
| Tokens per Image | 6,468 | 3,678 | 3,002 | 2,600 | 2,045 | 2,212 avg. | 2,246 |
Cost falls hard from 5 to 15, then flattens. Past about 25, every measurement is within noise of every other measurement. No knife-edge optimum: a cliff, then a plain.
The shaded band contains the argument in one picture. It shows the observed run-to-run variation across 41 identical-size runs at n=40. Every result from n=25 onward falls inside that range, suggesting that the apparent differences among larger batch sizes are smaller than the system’s normal variability.
What this does not prove
The part I’d have skipped before, and the part that matters most.
The batches weren’t randomised. Each took the next 40 files alphabetically. On this blog, alphabetical order correlates with era and source, which correlates with image dimensions. So my batch-size comparisons ran across systematically different image populations, not equivalent samples.
That’s almost certainly why n=30 looked cheaper than n=25: different files, not a better batch size. I don’t know the magnitude, because I never recorded per-batch pixel totals, and I can’t reconstruct it now.
So the honest claim is narrow: per-image cost falls steeply below ~15 and is flat from ~25 to at least 74, for this image population, unrandomised. “The plateau is caused by batch size” is not something this experiment isolated. To establish that you’d randomise file assignment across sizes, record total pixels per batch, repeat each size several times, and normalise by image tokens.
I didn’t do that. I ran a job and watched the meter.
Wrong #3: the keywords are too generic
Different domain, same failure mode.
A few hundred images in, the keyword distribution worried me. technology was attached to roughly half the library, business to a fifth. A tag matching 900 of 1,785 images can’t narrow anything. It means this is one of the images.
I was ready to rewrite the prompt and re-run everything. Then I measured it, and the measurement said don’t.
The right question isn’t “how common is the commonest keyword.” It’s does each image have at least one keyword that’s rare? Retrieval doesn’t care that an image is tagged technology as long as it’s also tagged conveyor belt. So for every image I found its rarest keyword and asked how rare that one is:
| Rarest Keyword on the Image | Images | Share |
|---|---|---|
| Unique across the library | 867 | 48.6% |
| Rare (2–5 uses) | 535 | 30.0% |
| Uncommon (6–20 uses) | 315 | 17.6% |
| Common (21–100 uses) | 67 | 3.8% |
| Generic only (>100 uses) | 1 | 0.1% |
| Total | 1,785 | 100.1%* |
Nearly half the images contain a keyword found nowhere else in the library, and another 30% contain one used only two to five times. At the other extreme, just one image relies entirely on generic keywords used more than 100 times. The metadata is not merely descriptive; it gives nearly every image a highly selective retrieval key.
One image out of 1,785 had no distinguishing term, a laptop on a desk that genuinely looks like every other laptop on a desk. The model wasn’t wrong. There’s nothing else there.
The generic tags were never the problem, because they were never all it produced. It spent the head of each list on the obvious and the tail on the specific. 2,680 unique keywords across 14,690 slots, 58% of that vocabulary appearing exactly once. That long tail is the entire retrieval system.
Third time, measurement beat reasoning.
The audit I got for free
The sidecars answered questions I hadn’t thought to ask.
Duplicate images fall out of duplicate descriptions. Eight descriptions appeared twice, covering 16 files, and every pair was a genuine duplicate saved under two names. Two identical sentences from a model that never saw both files together is a strong signal. Hashing afterwards found nine more. Several of those pairs had completely unrelated filenames, because the source reused one image across different articles. Only content comparison finds those.
has_text stratifies hard by style:
| Image Style | Images | Contain Text |
|---|---|---|
| Screenshot | 42 | 100% |
| Photo collage | 86 | 95% |
| Flat vector illustration | 182 | 72% |
| Stock photo | 503 | 61% |
| 3D render | 209 | 51% |
An image with a headline burned into it can’t be re-headlined, so has_text: false is the real reusability filter. Separately, the site’s own logo showed up in text_content on 357 images, a fifth of the library, watermarked and useless to anyone else. I deleted all of them on the strength of that one field, and it was wrong exactly once in 559 images.
None of this was designed. It’s what you get from structured fields instead of a paragraph.
The two levers that actually mattered
While I was busy measuring batch sizes, the two decisions with real money attached sat untouched.
Don’t send full-resolution images. Median here is 1,043×720, billing around 1,280 vision tokens each. At 512px on the long edge: 218. Across 1,785 images that’s 1.9M tokens (46% of my entire run) for a categorisation task where 512px is plainly enough to see a whiteboard or a flowchart. I ran on the originals. That one default cost more than every batch-size decision in this post combined. Automation that perfects the wrong variable is a pattern worth naming on its own, The Efficiency Illusion covers it from the process-automation side.
im = Image.open(src).convert("RGB")
im.thumbnail((512, 512), Image.LANCZOS)
im.save(dst, format="JPEG", quality=82)
Use the cheap model. The whole job ran on a small vision model. It read branch labels off flowcharts, identified a fulfilment centre from the conveyor and logo, and picked “kubernetes” out of a BGP diagram from topology alone. I never A/B’d it against a frontier model, so I can’t show you a quality delta. What I can say is that nothing in 1,785 captions made me wish I’d paid more.
Both are thirty-second decisions made before any code runs, and together they dominate everything I spent hours measuring.
The bug that actually cost me something
I obsessed over token curves worth a couple of dollars. Then, tidying sidecar filenames, I wrote this:
if os.path.exists(target):
os.remove(current) # must be a duplicate
Windows filenames are case-insensitive. When a sidecar’s correct name differed only in capitalisation (e.g. interview.json needing to become Interview.json) os.path.exists() reported a collision that wasn’t there, and I deleted the only copy.
It destroyed 24 captions. Then I under-reported the damage as “six,” because I measured the blast radius using the same case-folding assumption that caused it.
The root cause was a design error, not a typo: I’d assumed the mapping from image name to sidecar name was reversible. It isn’t, once Interview.png and interview.jpg are both real files needing distinct sidecars on a filesystem that can’t tell Interview.json from interview.json. The fix was to stop inferring. Every sidecar is now named .json and carries an authoritative file field, and the rename ran in two phases through temp names so nothing could collide mid-flight.
Nothing was unrecoverable. But the contrast is the joke: hours of careful attention on a curve made of noise, and a data-destroying bug shipped in three lines I didn’t think twice about.
What changed
1,785 images catalogued, then trimmed to 1,417 unique and unbranded. Roughly 4.1M tokens, about $7.44 all in, including every experiment and both rounds of recovery. Downscaling first would have made it $5.50 for identical output.
The $7 was never the point.
I don’t trust a difference now until I’ve run the same thing twice and looked at the spread. Three identical batches up front would have cost me twelve minutes and shown a 900-token range and I’d have known instantly that a 555-token gap meant nothing, before I built a model on it, and before I nearly launched forty-three agents on the strength of a curve fitted to noise.
That’s the habit I came away with. Measure your variance before you interpret your differences. Everything else in this post is a footnote to it.
Epilogue: the call that came the next day
Less than 24 hours after I’d finished writing this up, a colleague called about something completely unrelated. A client had gotten four sales-demo recordings from four competing vendors and wanted to know, concretely, how the demos differed: who led with which feature, where the 80% overlap was, where the 20% divergence lived. Video files, not images. A different client, a different problem, a different person asking.
I started describing the pieces almost before I’d finished registering what he was asking. Transcripts are trivial... that part’s solved in our Conopsys.ai we're using every day on our project teams. The harder half is turning a recording into the right screenshots: detect where the screen actually changes, grab that frame, and now you have a stack of stills instead of an hour of video. Which is exactly the object I’d spent the last three weeks learning to handle cheaply.
Cheap model. Downscaled frames with ffmpeg’s scene-change detector does the boring work of deciding which moments matter, so you’re not paying to look at 30fps of a static slide. And the same discipline of forcing structured output instead of a paragraph, so each frame gets a timestamp and a description rather than a caption you’d have to re-read to search.
The batch-size lesson came along too, and this is the part worth being precise about, because I almost drew the boundary in the wrong place. The number (25, 30, 40) is not a universal constant; it was bound to this dataset’s image sizes and this model’s context handling. What transfers is the method: run a few batches at the same size before you trust any difference between sizes, because the plateau is wide and the noise inside it is louder than most differences you’ll be tempted to act on. Stills pulled from a screen recording are still just images going through the same agent loop. The math that says “don’t bother going below 25” doesn’t care whether the images came from a blog scrape or a demo recording.
None of this was planned. I built a tool to solve “I have 1,785 pictures and no way to search them,” and it turned out I’d actually built “a cheap way to turn a pile of unstructured visual anything into something queryable.” The images were the instance. The demo videos are the second instance, and they showed up before I’d finished writing about the first one.
That pipeline: scene detection, timestamp correlation against a transcript, and a cross-video diff of what each vendor emphasized and when... Well, that's its own story. It’s next.
Top comments (0)