The bill, up front
Last week my news pipeline rewrote a full weekly batch of 85 articles in the cloud, across all four of its writer personas. The rewriting bill was $4.31, which is $0.051 per article. Even counting the blind two-judge panel that then scored every one of those articles against the old local output, the whole exercise metered $7.47.
For a year that same work ran on a Mac Mini under my desk: a 72-billion-parameter model on Ollama, FLUX generating images on the GPU, launchd waking everything on Thursdays. I'd already measured the economics of that setup and published the break-even: at my scale, hosted APIs with batch pricing were roughly 8 to 12 times cheaper than the fully-loaded cost of the local hardware. Then I kept running it locally anyway, because the marginal cost of a machine you already own is nearly zero and the data said the money didn't matter much either way.
So why move? Because money was never the real bill. The real bill was operational: a single point of failure with a 40-minute cold load, model files on a USB SSD, and a fragility class all of its own. When the machine sneezed, the product missed its week. You can't put a price on that in a cost model, which is exactly how it hides.
What actually moved
Three things were local; everything else already lived in the cloud.
The LLM calls went from local Qwen (72b for writing, 14b for classifying) to hosted Claude, tiered the same way: a cheap fast model for clustering, tagging and novelty checks, a quality model for the rewrites. Every pipeline call now goes through the Batches API, which halves the token price in exchange for latency I don't care about at 3am.
The images went from FLUX running in-process on the GPU to the same FLUX model behind fal.ai's API. Same model, so the product's visual language didn't change. That one mattered to me: a migration should be invisible to readers.
The orchestration went from launchd to Azure Container Apps Jobs: a container on a cron, scale-to-zero, secrets in Key Vault, and a 24-hour task timeout that lets a batch submit, wait and collect inside a single invocation.
One codebase runs both worlds. Each stage picks its backend from an environment variable, so cutover is a config change and rollback is the same change backwards.
What the plan got wrong
The migration brief was written against a clean mental model. Reality filed corrections.
There were 21 LLM call sites, not the 13 I'd counted. Thirteen named helpers, plus eight inline HTTP calls hiding in scripts, most with their own hand-rolled streaming loop and hardcoded URL. The first real move wasn't "swap the provider", it was "make there be one client". If you take nothing else from this piece: count your call sites before you estimate your migration, then consolidate before you
swap.
There were no retries anywhere. A local Ollama never rate-limits you, so the code had never needed them. Hosted APIs do. Without a retry layer, every transient 429 becomes a permanently failed article.
Resumable state turned out to be dangerous. The old design persisted an in-flight batch id so a crashed run could reconnect. In a container that restarts from a clean disk, the run re-derives its candidate list, and here's the trap: the new list isn't guaranteed to match the old one. Reconnect to the old batch and you
can publish content under the wrong article. We made recovery fail closed instead: abandon the orphaned batch, eat the few dollars, re-submit fresh. Correctness beats cost recovery every time the two collide.
The local model's scars were still in the prompts. A 4k context cap and a 3,000-character source-content limit existed purely because of VRAM. Lifting them was a deliberate decision with a measurable cost delta, not a cleanup. Same for the banned-phrase filter that force-rewrote model output: the local model needed it; the hosted one triggered it exactly once in 85 articles. It now runs as a detector first and an enforcer second, because you should measure a guard before you let it edit your prose.
Did quality survive?
Better than survived. A blind two-model judge panel, both judges covering all 85 articles from both backends, scored persona adherence at cloud 4.73 out of 5 against local 2.19, a win on every one of the four personas. The mechanical checks said the same thing from a different angle: every cloud article passed the structure checks; 78% of the local ones did. The panel design (and the bug it caught in its own gate) is the next piece; the headline here is that the migration wasn't a cost-neutral quality trade. The bigger source window alone, un-cramped from its VRAM-era 3,000 characters, gives the writer model more to work with.
What it costs now
About $10 a month of pipeline compute delta: rewrites at $0.051 an article with batch pricing and prompt caching, images at fractions of a cent, the container mostly asleep. Set against roughly £50 a year of electricity for the Mini, this migration will never pay for itself in cash. It pays for itself the first Thursday the pipeline runs while the Mini is switched off, in a hotel, or in a skip.
The other currency
Cost models compare money because money is easy to put in a column. The column that never appears is time, and locally the pipeline spent it in three ways.
Machine time. A 72-billion-parameter model on consumer hardware writes one article at a time, because a second one would blow the memory. Before it writes anything at all, cold-loading the model from an external SSD could take forty minutes. On the measured run, the local pass spent about four and a half hours rewriting the 85 articles (an upper bound from file timestamps, since the machine was juggling other stages in the gaps). The cloud pass wrote the same 85 in 48 minutes. The Mini spent its hours as a pipeline appliance and nothing else.
My time. This is the expensive one. The local stack had a fragility class the cost model never saw: model reloads, broken Python relinks, a launchd job that needed its owner nearby. Every "cheap" local run carried an invisible surcharge of owner-hours, billed at whatever your evening is worth.
The exchange rate you choose. The cloud makes the time-money trade explicit and lets you pick a side per workload. Synchronous calls cost full price and answer in seconds. The Batches API charges half price for answers within a day. A pipeline that runs while you sleep should always sell its latency, because overnight latency is worthless to you and the API pays you 50% for it. That's the quiet insight batch pricing encodes: time and money are convertible, and the smart move is to sell the one you're not using.
Cloud inference isn't just cheaper per token at my scale. It converts hours of machine-sitting into minutes of nobody-sitting, and it hands the 3am problem to someone whose job is 3am problems.
Measure your own crossover, in both currencies. And when the money column says "either way", read the two columns the spreadsheet doesn't have: whose hours are being spent, and what breaks at 3am with only you to fix it.
Top comments (0)