A cheap model that handles 70% of your requests can cut the bill by 65% — or raise it by 5%

A popular way to cut an LLM bill is to send every request to a small, cheap model first and hand it to the big one only when the small one gets it wrong. I priced that setup on five pairs of models from the same vendor, with the small model handling 7 requests out of 10. On one pair it cuts the bill by 65%. On another it makes the bill 5% bigger. Same success rate, opposite outcome.

The failed attempt is billed too

You pay per token, a chunk of text about three-quarters of an English word, at one price for input (what you send) and a higher one for output (what the model writes back). The per-token prices vendors publish are called list prices and are quoted per million tokens.

The setup above is called a cascade: try the cheap model, check its answer, and escalate to the strong model only if the check fails. The detail that's easy to miss is that the cheap attempt is billed whether it passes or not. Every request pays for the small call, and the failed ones pay for the big call on top.

Call the share of requests that get through the check the pass rate, p. Call the small model's cost per request divided by the big model's the price ratio, r. Then the cascade costs r + (1 − p) of what the strong model alone would cost, and your saving is p − r. The cascade breaks even when the pass rate equals the price ratio, and loses money below it.

Five pairs, one support ticket

To get real ratios I took one task: turn a customer email into a JSON ticket with six fields. The instructions and the email come to 346 input tokens and the reply to 65, counted with tiktoken, OpenAI's open-source token counter, using o200k_base, the encoding of current GPT models. List prices come from the catalog of the LLM gateway I work on, snapshot of 19 October 2026. I didn't measure pass rates: 70% in the last column is a round number for illustration.

Small → strong	List price in / out, $ per 1M	Break-even pass rate	Bill change at 70% pass
claude-haiku-5.5 → claude-sonnet-5.5	0.10 / 0.50 → 2 / 10	5%	−65%
gpt-5.6-mini → gpt-5.6	0.20 / 1.20 → 2 / 12	10%	−60%
gemini-3.8-flash → gemini-3.1-pro	0.75 / 3.75 → 2 / 12	34%	−36%
claude-sonnet-5.5 → claude-opus-5.5	2 / 10 → 4 / 20	50%	−20%
gemini-3.5-flash → gemini-3.1-pro	1.50 / 9 → 2 / 12	75%	+5%

The model's name tells you nothing about the ratio. Two Gemini "flash" models in front of the same 3.1 Pro land at 34% and 75%. Haiku in front of Sonnet pays off above a 5% pass rate, while Sonnet in front of Opus has to handle half the work just to break even. The 3.8 Flash row is the only one where the task shape matters: its input costs 37.5% of Pro's and its output 31%, so the ratio slides between those two numbers.

In dollars, at 100,000 tickets a month, gpt-5.6 alone costs $147 and the mini-first cascade $59. The Gemini 3.5 Flash cascade costs $155 against $147 for 3.1 Pro alone. You pay extra for an extra moving part.

Measure two numbers before you build it

Use what your calls actually cost, not list prices. The ratio in the table assumes both models write the same number of tokens. If the small model writes five times more, gpt-5.6-mini's ratio climbs from 10% to 31%. If it's a "thinking" model that spends 1,000 reasoning tokens, hidden notes billed as output, on this short ticket while the big one answers straight away, the ratio hits 92% and the cascade almost never pays. So run 50 real requests through the small model, check every reply, and plug in the average token counts the API returned:

# Averages from your own calls, and the prices you pay ($ per 1M tokens)
SMALL = {"in": 346, "out": 65, "price_in": 0.20, "price_out": 1.20}   # gpt-5.6-mini
BIG = {"in": 346, "out": 65, "price_in": 2.00, "price_out": 12.00}    # gpt-5.6
PASSED, TRIED = 35, 50   # small-model replies that passed your check

def cost(m):
    return (m["in"] * m["price_in"] + m["out"] * m["price_out"]) / 1e6

r = cost(SMALL) / cost(BIG)
p = PASSED / TRIED
print(f"break-even pass rate: {r:.0%}, yours: {p:.0%}")
print(f"cascade vs strong model only: {r + 1 - p:.0%} of the bill")

It prints break-even pass rate: 10%, yours: 70%, then cascade vs strong model only: 40% of the bill.

The check has to be automatic, or there's nothing to escalate on. For structured output it costs nothing: parse the JSON and compare the keys. Return which model answered and log it. In production, that log is your pass rate.

import json

KEYS = {"order_id", "product", "issue", "wants_refund", "urgency", "summary"}

def passes(reply):
    try:
        data = json.loads(reply)
    except ValueError:
        return False
    return isinstance(data, dict) and data.keys() == KEYS

def cascade(prompt, ask_small, ask_big):
    reply = ask_small(prompt)
    if passes(reply):
        return reply, "small"
    return ask_big(prompt), "big"

small = lambda p: 'Sure! Here is the JSON: {"order_id": "A-48213"}'
big = lambda p: json.dumps(dict.fromkeys(KEYS))
print(cascade("...", small, big)[1])

The two lambdas stand in for real API calls, so the snippet runs as is. The fake small model chats before its JSON, the parse fails, and it prints big.

What this doesn't solve

Failed requests wait for two calls, so at a 70% pass rate three tickets in ten get slower, not faster. A wrong answer that passes the check is never escalated: a key check catches broken JSON, not a wrong order number. Free-text answers have no cheap check, and grading them with a third model adds its cost to every request. tiktoken is OpenAI's tokenizer, so Claude and Gemini counts will differ.

Prices below list don't fix this and can make it worse. The gateway I mentioned sells all eight models under list, but not at the same discount: Gemini 3.1 Pro gets a deeper cut than 3.5 Flash, so at our prices that pair's break-even moves from 75% to 85%. Recompute the ratio at the prices you actually pay.

The takeaway

A cheap first try is never free. Divide what the two calls cost, measure how often the small model passes, and build the cascade only if the pass rate sits well above that ratio.