DEV Community: chenyu

The cheapest model per million tokens was the most expensive per request

chenyu — Sat, 24 Oct 2026 08:13:30 +0000

I measured nine model-task combinations on the same endpoint. Per-million pricing predicted almost none of it.

Every "which model is cheapest" post compares price per million tokens. That number is easy to find and almost useless for deciding what to run.

So I measured the thing that actually shows up on a bill: what one request costs, on three task shapes, across three model families. Everything below comes from the usage fields the API returned, not from a price list.

Method

Same endpoint, same key, same three prompts, three runs each, reporting the median.

short — "In one sentence, what is a prompt cache?" (a chat reply you'd get from any assistant)
code — "Write a Python function that validates an email with a regex. Only code." (a small, well-scoped generation)
long — a ~400-token input asking for a three-sentence summary (input-heavy, output-light)

Cost per request = input tokens × input rate + output tokens × output rate.

The numbers

Model	Task	Output tokens (median)	Median cost/request	$10 buys
deepseek-flash	short	78	$0.00024	~41,000
deepseek-flash	code	509	$0.00143	~7,000
deepseek-flash	long	160	$0.00071	~14,000
glm-5.3	short	44	$0.00056	~17,800
glm-5.3	code	68	$0.00084	~11,900
glm-5.3	long	70	$0.00209	~4,800
kimi-k2.6	short	37	$0.00036	~28,000
kimi-k2.6	code	61	$0.00059	~17,000
kimi-k2.6	long	64	$0.00074	~13,500

Finding 1: the "cheap" model cost 70% more

On paper deepseek-flash is $0.68 in / $2.75 out per million. glm-5.3 is $3.50 / $11.00 — roughly five times more expensive on every line of the price table.

On the code task, the expensive model was half the price per request.

The reason is in the output column: 509 tokens versus 68 for the same prompt, producing a function of similar length. The cheap-per-token model simply wrote more — explanation, alternatives, a docstring you didn't ask for. You pay for all of it at the output rate, which is 4× the input rate.

Per-million pricing assumes every model emits the same number of tokens for the same request. They don't.

Finding 2: output is the bill, not input

Take the two extremes:

deepseek-flash, code task: 45 input tokens, 509 output. 97% of the cost was output.
glm-5.3, long task: 378 input tokens, 70 output. 37% of the cost was output.

So "input is cheap, right?" depends entirely on what the model does after reading it. A 400-token input is almost free; a model that answers it in 700 tokens is not.

This is also why "summarise this document" and "write me a function" have completely different economics even though both are one API call.

Finding 3: latency follows output too

Median wall time on the short task: 1.0s (deepseek-flash), 2.2s (glm-5.3), 1.3s (kimi-k2.6). On the code task, 1.5–2.9s.

The slow requests were always the ones emitting a lot of tokens, never the ones with a long input. If a call feels like it's hanging, look at the output length before you blame the network.

What I'd actually do with this

Measure your own three shapes. Take three prompts that represent the bulk of your traffic — a short one, a generation one, an input-heavy one — and log usage for each. Ten minutes of work, and it replaces every "cheapest model" list you've read.
Route by task, not by model ranking. A short classification call and a 2,000-token refactor are different purchases. One model winning the first says nothing about the second.
Watch output length, not just rate. If a prompt starts producing noticeably longer answers after a model update, your bill moves even though the price per million didn't.
Check variance before you trust a number. On one model, the code task varied 2.5× across consecutive runs of the identical prompt. A single measurement is an anecdote, which is why the table above is medians of three.

Reproduce it

Any OpenAI-compatible endpoint returns the fields you need:

curl https://api.switchllm.net/v1/chat/completions \
  -H "Authorization: Bearer $SWITCHLLM_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-flash",
    "messages": [{"role": "user", "content": "In one sentence, what is a prompt cache?"}]
  }'

The usage object in the response has prompt_tokens and completion_tokens. Multiply by your provider's rates and you have the real cost of that request. Log those two numbers for a day and you'll know more about your bill than any comparison table can tell you.

Caveats

One run per cell, three repetitions, medians. Variance is real — see point 4 above.
Rates are the ones on the endpoint I used, at the time of writing. Your provider's rates will differ; the shape of the finding is what generalises.
The prompts are synthetic and small. Real application prompts are longer and messier, and a single number per task hides a wide distribution.
Provider-side caching, if enabled, changes input costs substantially and is not reflected here.

Disclosure

I run SwitchLLM — the OpenAI-compatible endpoint these measurements were taken on. One key for DeepSeek, GLM and Kimi, prepaid from $9.9, sign-up with an email address.

Everything above is the advice I'd give you on someone else's platform: the point is that per-million pricing doesn't tell you what a request costs, and the only way to know is to measure yours.

Getting a DeepSeek, GLM or Kimi API key when your phone number and your card are the problem

chenyu — Fri, 23 Oct 2026 10:49:21 +0000

Every few weeks someone emails the same thing: the model is fine, I just can't buy it.

The models stopped being the hard part a while ago. Signing up is the hard part, and it fails in three separate places:

A phone number you don't have. Some platforms only accept a mobile number registered in one country. Virtual numbers get flagged within minutes, and then you're locked out of the account you were trying to create.
An identity check before billing. Some platforms won't let you add credit until you've uploaded an ID document. Reasonable if you're in that market, a dead end if you're not.
A card that gets declined. Even where signup works, AI and API merchants are routinely declined on foreign cards — especially on smaller banks and on prepaid or virtual cards.

Any one of those is enough to stop you. They tend to arrive together.

The four routes that actually exist

Route	What you pay with	Works when	The catch
Buy direct from the provider	Whatever method they support	That method works for your country and bank	Phone checks, ID checks, declines
An international cloud that hosts the models	Card, invoice	You already buy cloud there	Model versions lag, rates differ
A gateway with one OpenAI-compatible key	Card, prepaid	Your card clears a normal online merchant	You pay a margin over raw token cost
Self-hosting an open model	Your own hardware	You need full control	It isn't the same model

People jump straight to the last row, which is the most expensive way to get a worse answer. Work down the table instead.

Six things to check before you prepay anyone

This applies to us as much as to anyone else.

What's the minimum top-up? If the cheapest way to evaluate a provider is $20, keep looking. You should be able to test the latency from your own city for pocket change.
Prepaid, or subscription? Subscriptions fail at renewal for reasons that have nothing to do with you: an expired card, a re-authentication rule, a merchant flagged as high-risk. A prepaid balance cannot fail at renewal.
Is it genuinely OpenAI-compatible? "Compatible" should mean you change base_url and the key, and nothing else. Ask for the exact endpoint before you pay.
What happens if the credit doesn't land? The honest answer is "email us and we check the payment record against your account". The suspicious answer is "it's fully automatic, don't worry".
Does the balance expire? Look for the exact wording, not the marketing version of it.
Can you spend it down and leave? No minimum commitment, no auto-renew, no cancellation maze. If you can't leave, you're not a customer, you're a subscriber.

The integration is the boring part, which is good

If the endpoint is OpenAI-compatible, migration is two lines:

curl https://api.switchllm.net/v1/chat/completions \
  -H "Authorization: Bearer $SWITCHLLM_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-flash",
    "messages": [{"role": "user", "content": "hello"}]
  }'

Or with the standard SDK:

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.switchllm.net/v1",
    api_key=os.environ["SWITCHLLM_KEY"],
)

print(client.chat.completions.create(
    model="deepseek-flash",
    messages=[{"role": "user", "content": "hello"}],
).choices[0].message.content)

Everything downstream keeps working, because they all speak this format: Cursor, Cline, LangChain, your own loop.

The cost math, without the marketing

Credit packs are one-time purchases from $9.9. Nothing renews.

A project making 300 calls a day costs about $0.36 a month on the cheapest model listed, about $10 a month on deepseek-flash, and about $45 a month on the largest model on the site. Those figures are just the published per-million rates multiplied out — 9,000 calls a month times the cost per request. You can redo the arithmetic from the pricing page instead of trusting me.

The point isn't that it's cheap. It's that the bill is small enough to predict, and you can stop by simply not topping up.

Disclosure

I run SwitchLLM — the third row of that table. One key for DeepSeek, GLM and Kimi behind an OpenAI-compatible endpoint, prepaid from $9.9, sign-up with an email address. New accounts get a small trial balance to test the integration before paying anything.

If a card works for you and the providers you want don't put a phone check and an ID check in your way, buying direct is cheaper and you should do that. This post is for the case where they do.

If you want your own numbers

Send me the shape of your sessions — turn counts, or a sample of request logs with the text stripped out — and I'll run the same three variants against your distribution and send back the table.

It costs me an afternoon and a few cents of tokens, and it's the only way I get to see sessions that aren't mine. support@switchllm.net

Your prompt-cache fix is worth 0% if your users only send one message

chenyu — Mon, 05 Oct 2026 11:58:25 +0000

A 30-turn measurement of how the saving amortises — and why the "one-line fix" quietly decays as conversations get longer.

Last week I published a benchmark showing that moving a roughly 30-token volatile header out of the top of a system prompt cut steady-state inference cost by 96%. The setup, the code and the raw API data are here: https://github.com/chenyu520-ai/rp-cache-lab

The most useful response came from Max Quimby, who runs scheduled multi-agent jobs and pushed back on the framing:

The amortization depends heavily on session length: if most of your sessions are only 2-3 turns, the cold turn-1 miss dominates and the win shrinks fast. Did you look at the hit-rate curve as a function of conversation length? That distribution is what decides whether this is a 95% win or a rounding error for a given app.

He was right, and I hadn't measured it. The first run was 8 turns, which is too short to answer the question. So I reran it at 30 turns.

This post is the answer, plus something I did not expect to find.

What was measured

Three ways of building the same request. The stable content — character card, world book, long-term memory, about 20,000 tokens — is byte-identical in all three. Only a small volatile header moves:

Session context: local time <ISO timestamp>, turn <n>, session <uuid>, mood index <0.00>.

Variant	Where the volatile header goes
A — naive	Front of the `system` message
B — minimal fix	End of the `system` message
C — optimized	End of the newest `user` turn; the system message never changes

30 turns per variant, deepseek-flash, thinking disabled, 90 requests total. Every number below comes from the API's own usage fields. The whole run cost $0.096.

The curve

The turn-1 miss is unavoidable — there is no cache to hit yet. That cost gets spread across the rest of the session, so short sessions carry it disproportionately.

Session length	A avg/turn	B avg/turn	C avg/turn	C vs A
1 turn	$0.002573	$0.002574	$0.002575	~0%
2 turns	$0.002584	$0.001339	$0.001342	−48.0%
3 turns	$0.002590	$0.000930	$0.000932	−64.0%
5 turns	$0.002602	$0.000609	$0.000599	−77.0%
10 turns	$0.002625	$0.000383	$0.000350	−86.7%
20 turns	$0.002673	$0.000301	$0.000229	−91.4%
30 turns	$0.002723	$0.000301	$0.000189	−93.1%

Read the first row carefully. On a one-turn session there is no saving at all — C is fractionally more expensive, because its prompt is very slightly longer (the header rides in the user turn). There is no history to amortise the cold miss over.

An app where users routinely send one or two messages and leave gets roughly half the headline number. An app with 10+ turn sessions gets the whole thing.

Which means the honest answer to "does this save 96%?" is: not until you know your session-length distribution.

The part I didn't expect

The first run made B and C look equivalent. Over 8 turns their aggregate hit rates were 85.8% and 86.8% — a 1 percentage point gap, well inside noise.

Over 30 turns they are not equivalent at all:

	Turn 2	Turn 30	Change
B cache hit rate	98.9%	90.5%	decaying
C cache hit rate	98.7%	98.7%	flat
B cost per turn	$0.000104	$0.000333	3.2×
C cost per turn	$0.000110	$0.000110	flat

B starts out nearly as good as C and then gets worse every turn. C does not move.

The mechanism is visible in the raw cache-hit token counts, and it is the most useful single table in this post:

Turn	B hit tokens	C hit tokens
2	16,896	16,896
5	16,896	17,280
10	16,896	17,664
20	16,896	18,688
30	16,896	19,584

B's hit count never changes. 16,896 is 264 × 64, and 64 appears to be the cache granularity. Because B's system message changes every turn, only the stable block ahead of the header can ever be reused — conversation history is permanently locked out of the cache. As the history grows, the uncached portion grows with it.

C's hit count climbs, because its system message is frozen and its history is append-only, so each turn's cached prefix includes everything before it.

So the "one-line fix" is real but incomplete. It buys you the stable block and nothing else. If your sessions are short, that is most of the win. If they are long, the gap between B and C becomes the thing that matters.

What this means in practice

Sessions of 1–2 turns: don't restructure for this. There is nothing to amortise and you will see no meaningful change.
Sessions of 3–10 turns: real, in the 64–87% range. Worth doing. Don't quote 96%.
Sessions of 10+ turns: the full effect, ~90%+. This is where it pays for itself.
Regardless of session length: prefer the append-only architecture (C) over the header move (B). B's advantage decays, and the extra work to freeze the system message is a one-time cost.

Caveats

I'd rather the numbers hold up than look good:

One provider. These are DeepSeek's numbers. The mechanism should generalise to any prefix-based cache, but I have not measured Anthropic's, which uses explicit breakpoints plus a TTL rather than pure prefix matching. That is a real gap — if you have data there, I would like to see it.
Best-effort caching. No provider guarantees a 100% hit rate, and repeated runs will vary.
History compression changes the picture. A real app summarises or truncates history instead of letting it grow. That would cap the divergence between B and C, so the 30-turn numbers are the pessimistic case for C's advantage, not the optimistic one.
The stable block is synthetic, though its structure and scale are realistic. This measures cache behaviour, not model quality.
Off-peak pricing throughout. Peak is twice off-peak.

Reproducing it

The benchmark is small and self-contained:

node bench.js --self-test --dry-run   # proves the three variants differ, costs nothing
node bench.js --turns 30 --budget 1.00

--self-test diffs the serialised request bodies of turn 1 and turn 2 and reports how many bytes are identical. For variant A it is 75 bytes out of 84,665. That is the whole story in one number.

Full per-turn data for both runs, and the session-length analysis, are in the repository:
https://github.com/chenyu520-ai/rp-cache-lab

Thanks to Max Quimby, Tom Veber, IndianInfraNotes and Rulestack for the comments that turned this into a second measurement rather than a single headline. If your sessions are shorter or longer than the ones here, I would like to know what you see.

Your system prompt is silently killing your prompt cache

chenyu — Sun, 04 Oct 2026 08:33:37 +0000

A benchmark on DeepSeek. Moving roughly 30 tokens from the top of a system message to the bottom cut steady-state inference cost by 96%.

If you run a chat or roleplay app, your system message is probably the largest thing you send to the model. A character card, a lorebook, a memory summary — tens of thousands of tokens, resent on every single turn.

DeepSeek's context caching is supposed to make that cheap. It's on by default, it requires no code changes, and cached input costs 50x less than uncached input ($0.003 vs $0.15 per million tokens, off-peak).

In practice, most apps get almost none of it. Not because caching is broken, but because of one line of code near the top of the system prompt.

I built a benchmark to measure exactly how much that line costs. Here's what it found.

The setup

I took a realistic roleplay prompt — character card, world book with 20 lorebook entries, long-term memory, relationship state — and built a stable block of roughly 20,000 tokens.

Then I ran three variants. The stable content is byte-identical in all three. The only thing that changes is where a small volatile header goes:

Session context: local time <ISO timestamp>, turn <n>, session <uuid>, mood index <0.00>.

That's about 30 tokens. Almost nothing. Here's where each variant puts it:

Variant	Where the volatile header goes	Typical of
A — naive	Front of the `system` message	Most first implementations
B — minimal fix	End of the `system` message	A one-line change from A
C — optimized	End of the newest `user` turn; the system message never changes	Append-only architecture

Each variant ran 8 turns of a real conversation against deepseek-flash, with thinking mode disabled. Every number below comes from the API's own usage fields — prompt_cache_hit_tokens and prompt_cache_miss_tokens. Nothing is modelled.

Results

Variant	Cache hit rate	Total input tokens	Cost for 8 turns	Cost per turn	vs A
A	0.0%	138,237	$0.021006	$0.002626	—
B	85.8%	137,788	$0.003465	$0.000433	−83.5%
C	86.8%	138,987	$0.003326	$0.000416	−84.2%

Turn 1 is always a cache miss in every variant — the cache is cold, there's nothing to match against yet. If you exclude that cold start and look at steady state:

Variant	Cost per turn	vs A
A	$0.002633	—
B	$0.000127	−95.2%
C	$0.000107	−95.9%

Variant A never had a single cache hit. Not one turn out of eight. The entire 17,000-token prefix was reprocessed at full price, every time, because the first ~30 tokens of the request were different.

Two details worth looking at

1. The one-line fix gets you almost everything.

B and C land within 0.7 percentage points of each other. You do not need to redesign your architecture to capture most of this. You need to move one string.

2. But C pulls ahead as the conversation grows.

Look at the absolute cache hit tokens per turn:

Turn	B hit tokens	C hit tokens
2	16,896	16,896
4	16,896	17,152
6	16,896	17,280
8	16,896	17,536

B flatlines at exactly 16,896 tokens — which is 264 × 64, and 64 tokens appears to be the cache granularity. Because B's system message changes every turn, only the stable block ahead of the header can ever be reused. The conversation history is dead weight that gets recomputed forever.

C keeps climbing, because its system message is frozen and its history is append-only, so each turn's cached prefix includes everything before it.

Over an 8-turn conversation that difference is small. Over a 50-turn session it isn't.

Why this happens

DeepSeek documents the rule clearly, and it's easy to miss:

A cache hit requires that the corresponding prefix has already been persisted... A subsequent request can only hit the cache if it fully matches a cache prefix unit.

Fully is the operative word. Prefix caching is all-or-nothing at the point of divergence. Put a timestamp at position zero and everything after it is new content as far as the cache is concerned — even if 99.99% of the bytes are identical to the last request.

I also verified this without spending anything, by diffing the raw request bodies of turn 1 and turn 2:

Variant	Request bytes	Identical prefix	Share
A	84,665	75	0.1%
B	84,665	84,493	99.8%
C	84,665	84,664	100.0%

75 bytes out of 84,665. That's the whole story.

What it costs at scale

Extrapolating the measured steady-state per-turn cost to an app with 1,000 daily active users averaging 50 turns each:

Variant	Monthly cost	Monthly saving vs A
A	$3,948.78	—
B	$190.78	$3,758.00
C	$161.12	$3,787.67

That's a 96% reduction from moving a header. The ratio is the finding; the absolute dollars depend on your prompt size and volume.

The other free win

If you're on this model family, check one more thing: thinking mode is enabled by default, with effort set to high, and reasoning tokens are billed as output.

Roleplay and chat apps don't need a chain of thought. Users want a reply in character, not an internal monologue. If your integration never explicitly disables it, you may be paying for reasoning on every message.

{ "thinking": { "type": "disabled" } }

Caveats

I'd rather you trust the parts that hold up than be surprised later:

Caching is best-effort. DeepSeek makes no guarantee of a 100% hit rate, and repeated runs will vary.
Cache entries expire automatically, typically within hours to days. A quiet app caches less than a busy one.
The first request of any conversation is always a miss.
A real app compresses history rather than letting it grow, so absolute figures will differ from the projection above.
Peak pricing is twice off-peak. This run was entirely off-peak.
This measured one provider. The specific numbers are DeepSeek's; the principle — volatile content at the front destroys prefix caching — applies to any provider with prefix-based caching.

What to check in your own app

You don't need a benchmark harness for the first pass. Just look at your request construction and ask:

Does anything in my system message change between turns? Timestamps, turn counters, user state, random IDs, "current mood" — anything.
Do I rebuild the message array each turn, or append to it?
Is my history being re-summarised in a way that changes earlier tokens?

If the answer to the first one is yes, you have a one-line fix and it's probably worth more than any model swap you could make this quarter.

Measured on deepseek-flash with thinking disabled, 8 turns per variant, ~20k-token stable block. Raw per-turn usage data available on request.

If you run an app like this, I'd genuinely like to know whether these numbers match your real bill — especially if your prompts are larger than 20k tokens, where I'd expect the gap to widen.

Benchmark and raw data: https://github.com/chenyu520-ai/rp-cache-lab