I built an AI agent to help me spend less on AI. His name is Tycho, and he runs on Opus 5.5. Four days after I made him our FinOps agent, he found a way to take the projected cost of one scoring job from $6,541 to about $600.
Tycho was evaluating a job projected at 6.53 million calls. Haiku had been our original choice for this task. He tested a DeepSeek route, where each call sent roughly 2,800 tokens of the same prompting instructions along with the item being scored. Paying to process that static prefix millions of times would dominate the bill.
The first probe
Tycho sent four requests with an identical prefix. All four reported zero cached tokens. The requests worked, but we were paying the ordinary price each time. Fireworks' prompt cache is replica-local. Repeating the same instructions does not reuse a cache entry if the next request lands on a different replica.
He changed the routing so subsequent requests returned to the same replica and ran the probe again. Across five warm calls, roughly 2,812 of 2,960 input tokens per call were cached. He then checked an eight-request batch: 19,688 of 23,677 input tokens were cached. The batch result was consistent with one cold prefix and seven warm requests. It also answered the question that mattered for this job: cache reuse survived the batch route in the small test.
What that did to the projected bill
| Same DeepSeek scoring job | Projected cost |
|---|---|
| Ordinary calls, no cache | $6,541 |
| Batch, no cache | $3,271 |
| Ordinary calls, cache working | $1,105 |
| Batch and cache working | About $600 |
The model and the 6.53-million-call workload stayed the same across these four estimates. Tycho changed how the calls were served. The Haiku line in the illustration, about $2,800, comes from an earlier setup with a longer rubric. It is a reference point, rather than a controlled four-way comparison.
This is the check I would run before scaling a repeated-prompt workload: send identical prefixes, inspect the cached-token count, repeat with session affinity, then test the batch route separately. A price card tells you the nominal token rate. These probes changed our full-job projection under each routing choice.
Tycho runs on Opus 5.5. I spent a few expensive calls investigating 6.53 million cheaper ones. Anthropic and OpenAI are offering frontier intelligence. But that very intelligence can now be used to figure out how to spend less money on the frontier labs.
Scope: The dollar figures are full-job projections extrapolated from small probes, not a completed 6.53-million-call invoice. The full story, including the original Haiku comparison, is in Using AI to Save on AI.
Top comments (0)