DEV Community

Cover image for I filmed a Short about running 32 conversions at once — here's what that actually costs
InApp
InApp

Posted on Originally published at imapp.blogspot.com

I filmed a Short about running 32 conversions at once — here's what that actually costs

I just published a 60-second walkthrough showing my document converter chewing through 32 files in a single batch instead of one at a time. The video is here: https://www.youtube.com/shorts/7EnCZ9Vt1Dw

The reason it exists: I kept getting asked "can it handle a batch?" and answering in text was useless. People don't believe a queue until they see 32 progress bars move at once. So I recorded the real thing — one POST with 32 document URLs, watch the dashboard, then download the combined result.

The concrete detail that made the cut: the worker pool caps at 8 concurrent conversions regardless of batch size. Anything above that waits in a FIFO queue. I show this in the video because it's the part everyone gets wrong — "32 at once" in the title means 32 accepted at once, not 32 rendering simultaneously. The download endpoint serves a single zip assembled from the results table, so the operator never has to poll 32 individual jobs.

What it costs the operator is the honest part: peak memory. Eight parallel PDF parses at once pushed my small VPS from 400MB to about 1.6GB resident. I had to add a circuit breaker that rejects batches over 32 items when free memory drops below 500MB. So the feature exists, but it taught me that batching is a resource negotiation, not a free lunch.

The Short compresses all of that into a minute. Sometimes the demo is the documentation.

Top comments (0)