Cutting LLM Bills in Half: Cache Reads, Batch Endpoints and Model Routing

If your LLM bill keeps growing while traffic stays flat, the fix is usually not a cheaper model — it's using the discounts you already pay for.

Three levers most teams ignore

1. Cache reads. Long system prompts and retrieval context get re-sent on every call. Providers that expose cache pricing charge a fraction of the base input rate for repeated prefixes. If your gateway doesn't pass cache savings through, you're paying full price for identical tokens.

2. Batch endpoints. Background jobs — summaries, backfills, classification sweeps — don't need sub-second latency. Batch APIs typically cost about half of interactive pricing.

3. Model routing. Not every request needs a frontier model. Route classification and extraction to small models, drafting to mid-tier, and reserve the expensive tier for genuine reasoning. Teams routinely cut 40-70% with no measurable quality loss.

Measuring the effect

Track cost per endpoint, not cost per app. Compute your effective rate (actual spend ÷ actual tokens) rather than trusting list price. Then verify that a routing or caching change actually moved the number.

Where the numbers live

We publish the full per-token table — input, output and cache rates for every model — in one place, so you can model these levers before shipping them: KeyoAPI [OI] rates.

The endpoint is [OI]-compatible, so testing a routing change is a model-id swap, not a rewrite. Compare [OI] API pricing on KeyoAPI against your current effective rate — the gap is usually bigger than you expect.