When Mythos went dark in June 2026, the reaction across the tech ecosystem wasn’t just disappointment. It was a collective chill down the spine of ...
For further actions, you may consider blocking this person and/or reporting abuse
The sovereignty argument is real, but the economics here are best read as a workload-specific scenario—not a general 6–15× rule. The break-even depends on the model and serving setup behind that 1,000-token/s figure, plus prompt length, batching, latency targets, utilization, and the cost of keeping capacity available. Input-token processing also needs to be measured separately from output generation.
I’d compare cost per successful outcome, not just cost per token: include quality, retries, engineering and operations, and the cost of failures. Fine-tuning can help a model follow schemas and use tools reliably, but it doesn’t make outputs hallucination-free or replace retrieval for changing facts. The useful question isn’t simply “own or rent?”—it’s which workloads justify owning, and where managed APIs remain the better tradeoff.
Thanks, this is a fair correction, and I agree with most of it.
The 6–15× comes from one worked example (50,000 agent tasks a day, 25k input + 5k output tokens each, list API prices, a dual-H100 node sized at roughly 1,000 output tokens/s). It is not a general rule. Change the model, prompt length, batching or latency target and the node count moves; at low or spiky volume the API usually wins, because otherwise you pay for idle capacity. You're also right that prefill and decode should be measured separately: the example sizes nodes by output throughput and treats the 50% duty-cycle case as headroom for input processing, which is an approximation.
Cost per successful outcome is the better metric. Retries, failed runs, the engineers who operate the stack and the cost of a wrong answer all belong in it, and they can flip the result either way.
Agreed on fine-tuning too: it makes a model better at the procedure (schemas, tool use, checking its own work), but it doesn't make it hallucination-free, and retrieval is still how you handle facts that change.
So the question we'd actually ask a team is the one you put: which workloads are large, stable and sensitive enough to justify owning, and which should stay on a managed API.
This is a useful correction. The 6–15× figure only means something alongside the assumptions—and where they stop holding. Workload shape, idle capacity, retries, and the people needed to run the stack can all change the answer.
Agreed that cost per successful outcome is the real comparison, not cost per token. If you were helping a team decide what to self-host, what would you have them measure first: workload stability, success rate, or fully loaded operating cost?
Workload stability first, because it decides whether the other two are worth measuring. In practice that means two to four weeks of request logs: daily volume, peak-to-average ratio, and prompt and output lengths. If traffic is small or spiky, a dedicated node sits idle most of the day and the API wins before anything else matters.
If the workload is large and steady, the next step is success rate on the team's own tasks: run a few hundred real cases through the API model and the candidate open model (base or fine-tuned) with the same checker. Cost per successful outcome depends on that number directly, and it is the one teams usually guess instead of measure.
Fully loaded operating cost comes last. It matters, but it is the easiest of the three to estimate: GPUs, on-call, upgrades, and ongoing evaluation. The usual outcome is a split: the high-volume, stable, well-defined workflows move in-house, and the long tail stays on an API.
That ordering makes sense: first establish whether the workload is steady enough to justify the comparison, then measure cost per successful outcome—not just cost per token.
The “same checker” part seems crucial. When teams run those few hundred cases, what’s usually hardest: building a representative test set, trusting the checker, or getting comparable cost and impact data across API and self-hosted options?
In our experience it's usually the test set. Teams start from the cases they remember, which skew toward the easy and the dramatic. A better start is a few hundred items sampled from real logs, stratified by task type, with the long tail included and an answer someone has actually checked.
Trusting the checker comes second, and it's more tractable: before using it to compare anything, have a person grade 50 or so cases independently and look at every disagreement. Most checker bugs show up in that first pass.
Cost is the easiest to measure but the easiest to get wrong, because the self-hosted number depends on utilization and on who runs the stack. Both are assumptions, so write them down next to the result rather than burying them in a single cost-per-outcome figure.
Do not follow any external links! DEV.to uses Sloan for automated messages, this is likely phishing.