DEV Community: synthorai

Long-Context Pricing Tiers: Up to 6.7x, and Gateways Never Show It

synthorai — Thu, 03 Sep 2026 06:31:25 +0000

The price on a gateway's model page is not the price on its invoice once a prompt gets long. We sent requests through one large multi-provider aggregator on both sides of every documented length line for nine tiered models: five billed exactly 2x their page price past the first line, Alibaba's three climbed to 3x, 3x, and 6.7x by the top of their ladders, and one Azure-served model billed 1.25x its listed endpoint rate on both sides of its line and then doubled. None of this is on the page. The mechanism is documented by the vendors themselves: Google, OpenAI, xAI, Alibaba, ByteDance, and MiniMax all reprice the whole request, output included, past a line at 32K, 128K, 200k, 256K, 272K, or 512k input tokens. This post shows the invoices, then the tier tables behind them, then the settings that keep a request under the line.

TL;DR

Nine tiered models billed 1.8x to 6.7x past the vendor's line through an aggregator whose pages show one price; GPT-5.6 Luna via Azure billed 1.25x its listing.
qwen3.7-flash charged $0.03, then $0.10, then $0.20 per million input at 32K and 256K; qwen3-coder-plus stopped at 3x where the vendor lists 6x.
Vendors reprice the whole request, output included, once the input crosses a line between 32K and 512k tokens.
Cap the input, not the output: Claude Code /autocompact, Codex model_context_window, API compaction triggers.

Do gateways charge what their page says?

No, not once the prompt is long, and finding out by how much takes two things per intermediary: the pricing metadata the gateway publishes, and the cost it actually reports for a request on each side of the line.

One large multi-provider aggregator publishes its catalog with a pricing.overrides array: a base price plus conditional rules such as min_prompt_tokens: 200000 with the higher rates, and, for DeepSeek and Tencent models, utc_start / utc_end windows for off-peak pricing. In the catalog we fetched on 2026-09-01, 60 entries carried overrides, including the Gemini Pro models, Grok 4.x, qwen3.7-plus, qwen3.7-flash, qwen3-coder-plus, the Seed 2.0 models, and the entire GPT-5.6 family at 272,000. The model pages show the base price only; the tier lives in the metadata. And the metadata is not the vendor's ladder: qwen3-coder-plus carries rules at 32,000 and 128,000 but nothing at the vendor's fourth tier, 256K.

So we sent requests on both sides of every documented line through that aggregator, with usage accounting enabled (the aggregator returns the amount it charged inside the response) and the serving endpoint recorded (one model id on the aggregator fronts several upstream hosts, each with its own price; it calls them endpoints), two runs per point, on 2026-09-02 and 2026-09-03:

Model (aggregator id)	Line	Below the line	Above the line	Page shows	Metadata	Served by
Gemini 2.5 Pro	200k	190k tokens at $1.25/M input	210k at $2.50/M	$1.25/M	rule at 200,000	Google
Gemini 3.1 Pro Preview	200k	185k at $2.00/M	217k at $4.00/M	$2.00/M	rule at 200,000	Google
Grok 4.3	200k	177k at $1.25/M	208k at $2.50/M	$1.25/M	rule at 200,000	xAI
Seed 2.0 Lite	128K	117k at $0.25/M	137k at $0.50/M	$0.25/M	rule at 128,000	Seed
Seed 2.0 Code	128K	119k at $0.50/M	135k at $1.00/M	$0.50/M	rule at 128,000	Seed
GPT-5.6 Luna	272K	252k at $0.275/M	294k at $0.50/M and $0.55/M	$0.20/M	rule at 272,000	Azure
qwen3.7-plus	256K	242k at $0.32/M	276k at $0.96/M	$0.32/M	rule at 256,000	Alibaba
qwen3.7-flash	32K	29k at $0.03/M	35k at $0.10/M	$0.03/M	rule at 32,000	Alibaba
qwen3.7-flash	256K	245k at $0.10/M	276k at $0.20/M	$0.03/M	rule at 256,000	Alibaba
qwen3-coder-plus	32K	29k at $0.65/M	35k at $1.17/M	$0.65/M	rule at 32,000	Alibaba
qwen3-coder-plus	128K	119k at $1.17/M	138k at $1.95/M	$0.65/M	rule at 128,000	Alibaba
qwen3-coder-plus	256K	244k at $1.95/M	276k at $1.95/M, no step	$0.65/M	no rule	Alibaba

The pages themselves, captured the same day as the invoices: one headline price, a per-endpoint table, and no length tier on either. Every charge above followed the metadata to the token. All nine models stepped at the vendor's first line by the vendor's factor, and on the three Alibaba models the invoice climbed a ladder the page never mentions: qwen3.7-flash from $0.03 to $0.10 at 32K and to $0.20 at 256K, 6.7x the page price, on a page that prints $0.03. GPT-5.6 Luna stepped too, and added a second gap: every Azure-served run was billed 1.25x the listed Azure endpoint price, below and above the line alike ($0.275 against $0.22, $0.50 and $0.55 against $0.40 and $0.44), a surcharge that appears neither on the page nor in the endpoint metadata.

The ladder can also stop short of the vendor's. On qwen3-coder-plus the charge rose 1.8x at 32K and 3x at 128K, then stayed at $1.95 per million through 276k tokens, where Alibaba's own list moves to $6 on input and $60 on output, six and twelve times the base. The aggregator's metadata has no rule at 256K, so the charge did not move. Whether the aggregator absorbs that difference or buys under a different contract is not visible from outside. What is visible is that the invoice follows the metadata, and the metadata and the page are two different documents.

The transparency gap, then, is not between metadata and invoice; it is between the page and both of them. The page's headline number is the cheapest endpoint's rate in the table below it, not the rate of the endpoint that will serve you, and neither number carries the length condition. A model card with one price does not tell you about the tier, or which endpoint will serve the next request, or whether the endpoint whose price you are looking at is reachable for your account at all.

The rule that follows is simple. Read the machine-readable pricing for the model you use, at the endpoint level, and check for a length condition; then send one request on each side of the line with usage accounting on and compare the reported cost. A gateway whose charge steps where its page does not is passing a vendor rule through without telling you. A gateway whose charge stays flat where the vendor's does not is either serving the model from a host with different prices or absorbing the difference, and only the first of those is a stable state.

Where are the lines, and how does a tier apply?

Six vendors publish a length line, and every model that states the rule applies the higher rate to all tokens of the request, output included. The step is 2x at most first lines, but the ladders climb: 3x at qwen3.7-plus's only line, 6.7x on input by qwen3.7-flash's third tier, and 6x on input and 12x on output by qwen3-coder-plus's fourth. Prices are per million tokens, fetched from the vendors' pricing pages on 2026-09-01.

Model	Threshold	Input, below / above	Output, below / above	Stated semantics
Gemini 2.5 Pro	200k prompt tokens	$1.25 / $2.50	$10 / $15	Vertex pricing footnote: "If a query input context is longer than or equal to 200K tokens, all tokens (input and output) are charged at long context rates"
Gemini 3.1 Pro Preview	200k	$2 / $4	$12 / $18	same footnote; cache reads tier too, $0.20 / $0.40
GPT-5.6 Sol (Terra, Luna same shape)	272K input tokens	$4 / $8	$20 / $30	model page: "Prompts with >272K input tokens are priced at 2x input and 1.5x output for the full request"
Grok 4.6, 4.5	200k	$2 / $4	$6 / $12	docs: "billed at the higher rate for all tokens in the request"
Grok 4.3, 4.20	200k	$1.25 / $2.50	$2.50 / $5	same
qwen3.7-plus	256K	$0.40 / $1.20	$1.60 / $4.80	Model Studio: "All tokens in the request are billed at the unit price of the corresponding tier"
qwen3.5-plus	256K	$0.40 / $0.50	$2.40 / $3.00	same
qwen3.7-flash	32K, 256K	$0.03 / $0.10 / $0.20	$0.13 / $0.40 / $0.80	same, three tiers
qwen3-coder-plus	32K, 128K, 256K	$1 / $1.8 / $3 / $6	$5 / $9 / $15 / $60	same, four tiers
Seed 2.0 Lite, Seed 2.0 Code	128K	$0.25 / $0.50, $0.50 / $1.00	$2 / $4, $3 / $6	BytePlus's pricing page renders in-app and could not be quoted; prices from the aggregator's metadata, matched by the invoices above
MiniMax M3	512k input	$0.30 / $0.60	$1.20 / $2.40	pay-as-you-go page: tier by request input count, applied to all tokens

Two boundary details worth copying into billing code. Google's two pages disagree by one token: the Vertex footnote says "longer than or equal to 200K", the Gemini API pricing table says "prompts > 200k tokens". And Alibaba defines its K precisely: 128K equals 128,000 tokens, 256K equals 256,000, not powers of two.

The tier applies to the whole request, decided by the input length alone, on every token including the output. The marginal cost of the token that crosses the line is therefore the entire premium on everything before it. For Gemini 2.5 Pro: a 199,999-token prompt bills $0.25 of input; at 200,001 tokens it bills $0.50, and a 4,000-token answer moves from $0.04 to $0.06. One token, plus $0.27.

On qwen3.7-plus the step is 3x: the official rate for a 255,029-token prompt is $0.102, for a 257,332-token prompt $0.309. On qwen3-coder-plus the same mechanism compounds across four tiers, so a 260k-token prompt bills six times the per-token rate of a 30k one and its output twelve times.

The rule is visible on a real invoice, not only in the vendor's table. On qwen3.5-plus, whose official tiers are $0.40 then $0.50 per million input tokens at 256K, our gateway bill per input token differed by exactly 1.25x between ten runs below the line (243k tokens) and 24 runs above it (256k to 321k), with the output price unchanged. The 259k request did not pay 1.25x on its last 3k tokens; it paid 1.25x on all of them.

For an agent that accumulates history, the crossing happens mid-session and silently: the turn that tips over pays the premium for every earlier turn it carries, and every later turn keeps paying it until something shrinks the context.

Does capability step at the same line?

No. We measured this because the tier is easy to mistake for a capability edge, and it is not one. On two tiered Qwen models with the line at 256K, a salted needle (a one-line fact carrying a per-run random code) planted at five depths was recalled 30 of 30 times on each model at 243k, 269k, and 320k tokens, with thinking off, and latency grew with length on a line rather than a step (15.2 s, 17.0 s, 20.5 s median on qwen3.7-plus). A harder task, counting K planted rare sightings across the whole log, did degrade with length: on qwen3.7-plus the share found slid from 74% at 128k to 60% at 192k, 58% at 243k, and 45% at 320k. The decline starts well below the price line and the cells on either side of it sit on the same slope.

Anthropic's own documentation names the gradual effect: as token count grows, accuracy and recall degrade, "a phenomenon known as context rot". It is real, it is continuous, and it does not know where the price tier is.

How do you keep a request under the line?

Cap the input, not the output. The tier keys on the request's input length, so max_tokens (an output cap) does nothing for it; the levers are the settings that bound what the client sends. They exist at four layers.

Here are the settings that bound the prompt, by layer. "Under a tier" marks the ones that take an absolute token count and can therefore be set just below a price line; the window-relative ones only keep you inside the model's context window, which is a different number.

Layer	Tool	Setting	What it bounds	Under a tier?
Coding agent	Claude Code	`/autocompact <value>`, `autoCompactWindow`, `CLAUDE_CODE_AUTO_COMPACT_WINDOW`, 100K to 1M	the token count at which history is summarized	Yes
Coding agent	Codex CLI	`model_context_window`, `model_auto_compact_token_limit`, `tool_output_token_limit`	context size, compaction trigger, per-tool-result cap	Yes
Coding agent	Aider	`--max-chat-history-tokens`, `--map-tokens`	soft limit on chat history before summarization; repo-map budget	Yes
Coding agent	Gemini CLI	`model.compressionThreshold`, default 0.5, plus `/compress` and `model.maxSessionTurns`	the fraction of the context window at which history is compressed	Indirectly: pick the fraction so window x fraction lands under the line
Coding agent	Cursor	Max Mode off (the default)	default window; Max Mode extends it and bills at the API rate plus 20%	Keep it off
Coding agent	Cline	none documented; auto-summarizes near the window	the model's window	No knob
API	Claude API	`context_management.edits[].trigger.input_tokens`, default 150,000, minimum 50,000	server-side compaction trigger; the compaction pass is billed under `usage.iterations`	Yes
API	OpenAI Responses	`truncation: "auto"`, default `disabled`	drops middle items only when the input exceeds the model window; `disabled` returns a 400 instead	No, window only
Aggregator	context compression	middle-out transform	removes the middle of the prompt to fit the model window	No, window only
Framework	LangChain	`trim_messages(max_tokens, strategy="last", token_counter, include_system)`	client-side history trim by token count before the request	Yes

Two things the table implies. On a coding agent pointed at a tiered model, the compaction window is the one knob that turns the cliff into a summary, and it has to be set in absolute tokens just under the line, not as a fraction of a 1M window. And the two most common "safety" settings, Responses truncation: "auto" and the aggregator's middle-out, are window guards: they act at the model's window, 1M on the tiered Gemini and GPT-5.6 models, not at the 200,000 or 272,000 tokens where the price moves, so they will keep a request from failing and still let it cross the line.

Do not count on caching to keep you under. Prompt caching cuts the bill but not the tier on Google's models, where cache reads are priced by the same length tier: a cached 150k prefix plus 60k of fresh context is a 210k prompt and bills as one. Provisioned capacity sidesteps the question entirely, since provisioned throughput units (PTUs) bill per hour regardless of tokens. And when crossing is worth it, cross deliberately: the measurement above says the model does not get worse at the line, so the decision is purely whether the marginal context is worth a 2x, 3x, or at the top tiers 6.7x multiplier on the whole request.

How Synthorai handles it

A length tier is a rate condition, so the gateway treats it as one: a model's price card can carry a list of tiers keyed by input-token boundary, and each request is rated against the tier its own prompt length selects, on all of its tokens, the same way the vendor rates it. That is the mechanism the qwen3.5-plus invoice above exercised. The usage record keeps the prompt token count and the price version alongside the rated cost, so a bill can be decomposed back to "this request crossed the line". The tier a request was rated under can be read back from the usage record, not only from a price table.

FAQ

Is the higher long-context rate charged only on tokens beyond the threshold?

No. Every vendor that documents the rule reprices the whole request: Google ("all tokens (input and output) are charged at long context rates"), OpenAI ("for the full request"), xAI ("for all tokens in the request"), Alibaba ("all tokens in the request are billed at the unit price of the corresponding tier"). A prompt one token over the line pays the premium on every token before it.

Do API gateways pass long-context pricing tiers through?

Yes, in our measurement: through one large aggregator, nine tiered models were billed the vendor's higher rate past the vendor's line, on model pages that show a single price. The tier is in the gateway's pricing metadata, not on the page, and the metadata can also omit a vendor tier, as it did for qwen3-coder-plus above 256K.

Does max_tokens keep a request under a pricing tier?

No. max_tokens caps the output; the tier is decided by the input length. The settings that help are the ones that bound the prompt: a compaction window or token limit in the agent (Claude Code /autocompact, Codex model_context_window), a compaction trigger in the API, or client-side truncation before the request is sent.

Does model quality drop at the pricing threshold?

Not in our measurement. On two tiered Qwen models, needle recall was perfect on both sides of the 256K line and a counting task degraded gradually with length with no step at the boundary. The tier is a business rule; capability loss with length is real but continuous.

Prices and semantics quoted from vendor pricing pages fetched 2026-09-01; agent and API settings from the linked documentation on 2026-09-02; invoice measurements run 2026-09-01 to 2026-09-03 with thinking disabled and salted prompts, two runs per aggregator point. Prices drift; check the linked source before wiring a threshold into billing code.

Related: billing units field guide (the modifier layer this post zooms into), token usage anatomy, prompt caching explained, cache minimums measured.

AI API Billing Units: 17 Meters from Tokens to PTU-Hours

synthorai — Tue, 01 Sep 2026 13:09:58 +0000

AI APIs meter usage in at least 17 distinct units: text tokens, reasoning tokens, cached tokens, media-input tokens, realtime audio tokens, characters, minutes and seconds of media, output objects, requests, pages, GPU-seconds, container-hours, GB-days, token-hours, PTU-hours, training tokens, and synthetic credits. Anyone estimating the cost of a product that touches more than one modality has to convert between several of them, and the conversion rules are where budgets go wrong. This page is a field guide: every unit, how it is metered, and the gotcha attached to it.

TL;DR

AI APIs bill in at least 17 distinct units across four layers: consumption, time and capacity, rate modifiers, and synthetic wrappers.
One audio minute bills two ways: $0.0020 to $0.0164 per audio-minute on dedicated transcription, or 1 audio token per 100 ms heard on gpt-realtime-2.1.
Storage alone has two meters: OpenAI file search bills $0.10 per GB-day; Gemini context caching bills $0.50 per million tokens per hour.
Modifiers change rates, not units: batch is half price, DeepSeek off-peak halves again, US-pinned Claude inference costs 1.1x.

What units do AI APIs bill in?

Seventeen, in the order the sections below follow: consumption units (1 to 10), time and capacity units (11 to 16), and synthetic wrappers (17), plus a modifier layer that changes rates without changing units. The table is the reference; each row links the deep dive.

#	Unit	Where it appears	The gotcha
1	Text tokens	Every chat API	Same text, different counts per tokenizer; about 30% more tokens from one tokenizer generation to the next
2	Reasoning tokens	Thinking models	Billed as output you never see; 88 to 99.3% of some answers
3	Cached tokens	Prompt caching	Write premium by TTL, read discount, per-provider minimums
4	Media-input tokens	Vision, audio, PDF input	Three conversion schemes for one image; PDFs bill at image rates
5	Realtime audio tokens	Speech-to-speech	1 token per 100 ms heard, 1 per 50 ms spoken
6	Characters	TTS, guardrails	Chinese speech costs 3 to 7x less than English per audio-minute
7	Media minutes/seconds	STT, video and music generation	8x price spread per audio-minute across models with near-equal accuracy
8	Output objects	Image generation, human eval	A quality knob swings one image 36x at identical resolution
9	Requests	Web search, grounding, file search	$0.01 per search plus injected tokens at the model's input rate
10	Pages	OCR, document AI	Mistral bills per 1,000 pages with batch and cache discounts on top
11	GPU-seconds	Serverless model hosts	Rate depends on the card, $0.000225/s (T4) to $0.001525/s (H100)
12	Container/session-hours	Code execution, managed agents	5-minute minimum, files preload billing even if the tool never runs
13	GB-days	File and vector storage	$0.10 per GB-day after the free gigabyte
14	Token-hours	Context-cache storage	Gemini bills $0.50 per million tokens per hour, doubling in 2027
15	PTU/model-unit hours	Provisioned capacity	Cached tokens consume no capacity; output weighs more than input
16	Training tokens	Fine-tuning	Epochs multiply the bill; some models bill $100 per hour instead
17	Credits/CCUs	Marketplaces, subscriptions	A scalar over the real rate card; 100 CCU = $1.00 on AWS Marketplace

Which units count what you consume?

Ten of the seventeen are consumption units: the meter ticks when you use the service, and each modality picked a different thing to count.

The token family (units 1 to 5). Text tokens are the anchor, but they are not one unit: tokenizers differ per model family, so the same text bills different counts, and Anthropic documents that its 4.7-and-later tokenizer produces about 30% more tokens for the same text than the previous one. Reasoning tokens are output tokens you usually cannot read; we measured 81 billed tokens for a 10-token answer, 88% of it reasoning. Cached tokens price by operation and cache lifetime (TTL): on the Claude API a 5-minute cache write costs 1.25x base input, a 1-hour write 2x, and a read 0.1x. Media inputs convert to tokens by provider-specific formulas: one 1024x1024 image bills 693, 1,089, and 1,372 tokens on GPT-5.6, Gemini, and Claude respectively; Gemini rates audio at 25 tokens per second and 720p video at 5,792 tokens per second and bills PDFs at the image token rate. Realtime speech has its own token: gpt-realtime-2.1 bills exactly 1 audio token per 100 ms of user speech and 1 per 50 ms of model speech.

Characters and time (units 6 and 7). Text-to-speech (TTS) bills per character (OpenAI lists $15 to $30 per million characters), which is why spoken Chinese costs 3 to 7x less than English per audio-minute: the same speaking time consumes fewer characters. Bedrock's guardrails define their own character unit, a "text unit" of up to 1,000 characters. Dedicated transcription models bill per audio-minute, $0.0020 to $0.0164 across the models we measured; video generation bills per second by resolution; Gemini prices music per song.

Objects, requests, and pages (units 8 to 10). Image generation bills per image by tier or per output token, and the two cross over: per-token wins below about 1,000 output tokens, flat per-image wins above. Server-side tools bill per request: web search costs $0.01 per search on the three surfaces we measured, and the results then bill again as input tokens (1,500 to 3,100 per search). Gemini's search grounding (letting the model query Google Search mid-request) adds a quota shape: 5,000 free requests per month, then $14 per 1,000. OCR bills per page. Bedrock's model evaluation bills $0.21 per completed human task, the only unit in this zoo denominated in people.

Which units rent time or capacity instead?

Six units bill for holding resources, whether or not tokens flow. This layer is where cost estimates miss hardest, because the meter runs while your code does nothing.

GPU-seconds. Serverless hosts bill model runtime by hardware class: on Replicate, a T4 costs $0.000225 per second and an H100 $0.001525. The unit prices the card, not the output.
Container and session hours. Anthropic's code execution bills $0.05 per container-hour beyond 1,550 free hours a month, with a 5-minute minimum per execution; OpenAI's code interpreter bills per 20-minute session, priced by memory size, $0.03 for 1 GB up to $1.92 for 64 GB. Claude's managed agents bill $0.08 per session-hour, metered to the millisecond and only while running.
GB-days and token-hours. Two storage meters for the same idea. OpenAI file search storage is $0.10 per GB-day with 1 GB free; Gemini explicit context caching bills storage at $0.50 per million tokens per hour, rising to $1.00 in 2027. A cache you forget to delete is a subscription you did not know you had.
PTU-hours and model units. Provisioned capacity bills per unit-hour regardless of traffic. Azure's provisioned throughput units (PTUs) are model-independent quota with per-model minimum deployment sizes; output tokens consume more capacity than input tokens, and cached tokens consume none, so a high cache-hit rate shrinks the PTU count you need. Bedrock sells the same shape as model units with 1-month or 6-month commitments.
Training tokens. Fine-tuning bills per token of training data multiplied by epochs, except where it bills wall-clock: some OpenAI models list $100 per training hour.

What changes the rate without changing the unit?

Seven things change the price of a unit without touching the unit itself: batch tier, service tier, clock time, context length, geography, quality knobs, and free quotas. Billing code that stores one price per model gets every one of these wrong, because the same token has different prices depending on how, when, and where it is processed.

Modifier	Effect	Example
Batch tier	50% off input and output	Anthropic and OpenAI batch APIs; Mistral OCR too
Service tier	Premium for speed	OpenAI fast tier at 2x standard rates; flex tier discounted
Clock time	Off-peak discount	DeepSeek: off-peak is half of peak; peak is 01:00-04:00 and 06:00-10:00 UTC weekdays
Context length	Rate steps at a threshold	Gemini rates step above 200k prompt tokens; Claude 4.6+ is flat across the full 1M window
Geography	Residency premium	Claude `inference_geo: "us"` is 1.1x on every token category; regional endpoints for Claude models on Bedrock and Google Cloud carry a 10% premium over global
Quality knobs	Same unit, different burn	gpt-image quality swings 196 to 7,024 billed tokens for one 1024x1024 image
Free quotas	Rate cliff at a threshold	Grounding: 5,000 free requests/month; code execution: 1,550 free hours/month

The multipliers stack. Anthropic documents that caching multipliers combine with the batch discount and the residency premium, so one cached, batched, US-pinned token carries three factors at once.

What are credits and CCUs?

They are synthetic wrappers over the real meters, used where invoicing needs one line item. When Claude bills through AWS or Azure Marketplace, usage is rated in dollars at the normal per-unit prices and then converted to Claude Consumption Units at 100 CCU per dollar; discounts show up as fewer CCUs metered, not a cheaper CCU. Subscription products run the same trick with credits, and the boundary can sit inside one company: ElevenLabs meters its subscription plans in credits while stating that API usage is billed in US dollars, not credits. When you see a synthetic unit, the question to ask is which consumption or capacity meter sits underneath it and at what conversion.

Which rounding rules and minimums bite?

These are the quantization rules, collected in one place because each one lives in a different document:

Code execution containers bill a 5-minute minimum, and including files in a request bills execution time even if the tool is never called.
Code interpreter sessions quantize to 20-minute blocks by RAM tier.
Realtime speech quantizes to 100 ms heard / 50 ms spoken per audio token; 60 seconds of silence billed zero under server voice-activity detection (VAD), and cancelling a spoken answer 2 seconds in billed 4 seconds.
Image generation with n=4 re-bills the prompt four times; there is no prompt caching on that surface.
Video tokens carry a +1 frame in the formula and encoded dimensions that differ from advertised ones: 720p bills as 1248x704.
Prompt caches have per-provider minimum cacheable lengths; below the minimum you pay the write premium and cache nothing.
Guardrails text units round up per 1,000 characters.
Tool definitions bill as input before any call: Anthropic's tool-use system prompt adds 286 to 804 tokens depending on model and tool choice, and a full computer-use toolset adds about 4,500.

How do you compare costs across units?

Normalize to dollars per user interaction, because that is the only unit every meter converts into.

Our voice-agent study did exactly this: a cascade stack billing audio-minutes (STT), tokens (LLM), and characters (TTS) came to $0.0037 to $0.025 per conversation-minute, against $0.057 for gpt-realtime-2.1 and $0.016 for its mini, which prices the realtime premium and its latency payoff in one comparable number. The procedure generalizes: pick the interaction (a support ticket, a document, a conversation-minute), meter each hop in its native unit, rate each at its current price, and only then compare. Comparing $/1M tokens to $/audio-minute directly is a category error; comparing $/interaction is arithmetic.

How a gateway normalizes the zoo

Synthorai's answer is to rate at request time and keep the native unit alongside the dollars. Every request produces one usage record carrying the native meter readings (tokens by class, seconds, characters, request counts), the resolved price version, and the rated cost, so an invoice line can always be decomposed back into unit times rate. This is the same conversion a CCU does at the marketplace boundary, done per record and kept inspectable. It also composes with zero-retention mode: the usage record needs meter readings and a content hash, never the payload. When a price changes upstream, the price version pins which rate card each historical record was rated under, which is the property that makes month-end reconciliation an audit rather than an argument.

FAQ

Are audio tokens the same as text tokens?

No. Realtime audio tokens are a separate meter with separate prices: on gpt-realtime-2.1 one token covers 100 ms of speech heard or 50 ms spoken, which works out to $0.0192 per minute to listen and $0.0768 per minute to speak. The audio and text meters carry separate rate cards even inside one model.

Why does the same image bill different token counts on different providers?

Because the conversion scheme differs, not the image: patch formulas, capped tiles, and flat fees all coexist. One 1024x1024 image measured 693 tokens on GPT-5.6, 1,089 on Gemini, 1,372 on Claude, and file format or content never moved a single token.

Do cached tokens count as input tokens?

They are metered separately and priced by operation: cache writes carry a premium over base input (1.25x or 2x by TTL on the Claude API), reads a discount (0.1x). On provisioned capacity the distinction matters twice: cached tokens consume no PTU capacity, so caching reduces both the bill and the capacity you must reserve.

Is per-character or per-minute pricing cheaper for speech?

It depends on the language. Per-character TTS makes spoken Chinese 3 to 7x cheaper per audio-minute than English because Chinese packs more speech into fewer characters; per-minute pricing is language-neutral. For transcription the per-minute models we measured ran $0.0020 to $0.0164 per audio-minute with near-equal accuracy on clean speech.

Unit definitions and rates quoted from provider pricing pages fetched 2026-08-30, and from our own measured studies linked in place. Prices drift; unit structures drift slower, but check the linked source before wiring any number into billing code.

LLM Data Retention and ZDR: Every Party That Can Read Your Prompts

synthorai — Sun, 30 Aug 2026 10:54:02 +0000

Your prompt does not go to "the AI company". On an agent stack it crosses a chain of parties that all handle it in cleartext, and exactly which ones depends on your stack: the agent framework's telemetry, a tracing platform, a memory store, an analytics tool, an AI gateway, and one of three kinds of inference provider, each with its own retention policy, storage region, and training clause. The party that keeps your prompts longest is usually on your side of the API: model vendors delete standard logs in about 30 days, while a default tracing or memory store keeps them until you delete them. Zero data retention (ZDR) is a real and useful arrangement, but it pins down exactly one of those parties on exactly one of three axes. This post maps the whole chain, who retains what and for how long, and what ZDR does and does not cover.

TL;DR

Every cleartext hop can collect; agent-side tracing and memory stores keep prompts far longer than any model provider does.
Providers retain per feature: on the Claude API, caching is ZDR-eligible, batch jobs persist 29 days, Fable 5 requires 30 days.
Geography follows the server: DeepSeek's official API stores data in the PRC, no other option.
Free endpoints are the sharpest edge: training rights are usually the price of free capacity.
ZDR held up in court: the NYT preservation order exempted only zero-retention API customers.

Who can read your prompt on an agent stack?

Everyone on the path, and the path is longer than most teams draw it:

Walk it left to right. The agent layer is yours, but rarely only yours: frameworks ship telemetry, tracing and observability platforms exist precisely to store full prompts and responses, memory features write conversation content into vector stores that never expire on their own, and a session-replay analytics snippet in your web UI captures the prompt as the user types it, before your backend ever sees it. The gateway is one party if you run it yourself and two if it is SaaS, and SaaS gateways come in two flavors. The first is the multi-provider aggregator: one API that routes each request across the providers behind it, so its data posture is always a sum, its own policy plus the policy of whichever host this particular request landed on. The second comes from security vendors: data-loss-prevention (DLP) and guardrail gateways whose entire product is payload inspection, redaction, and policy enforcement, which means reading every prompt is the feature, and flagged prompts are typically retained as security events by design.

Behind the gateway sit three kinds of inference provider: the model vendor's own API, a cloud provider hosting the model inside your cloud tenancy, and GPU hosts serving open weights under their own logging policies. The bypass branch, self-hosted inference on a dedicated tenant or your own GPUs, is the only path where no third party handles cleartext, and it comes with a catch we will return to.

The rule that organizes everything else: seeing and keeping are different questions. Every box can technically collect. Which ones do is a matter of defaults, settings, and contracts, and the defaults are wildly uneven across the chain.

Who actually keeps your prompts?

Usually your own tooling, longer than anyone else. The provider retention windows that dominate security questionnaires are days; the agent-side stores that nobody questionnaires are forever.

The agent layer retains by design. A tracing platform's product is a database of your prompts and outputs; retention there is a project setting, not a policy accident. The defaults vary by tool and are worth knowing: GitHub Copilot's OpenTelemetry integration exports span structure, timing, and token counts but no prompt content unless you explicitly enable content capture, which is the privacy-conservative template more agent tooling should copy. Cursor's privacy mode exists because the default mode shares code data. Whatever the model provider deletes after 30 days, your trace store still has on day 300.

The gateway keeps what it chooses to keep. A gateway cannot avoid usage records, since billing requires knowing token counts, models, and timestamps per request. It can entirely avoid storing payloads: prompt and response bodies can pass through memory and never be written at rest. But two popular gateway features quietly cross that line. A request-inspection or observability console is payload storage by definition, exactly like a tracing SaaS one hop earlier. And gateway-side caching stores prompt content at the gateway hop: a semantic cache keeps an embedding of your prompt plus the full cached response, and a response cache keeps both sides verbatim, on whatever disk the gateway runs. When you evaluate a gateway, SaaS or self-hosted, the question is the same split three times over: what is in the request record, what does the observability view persist, and what does the cache write?

Providers retain per feature, not per company. The useful mental model comes straight from Anthropic's API retention documentation, which discloses eligibility feature by feature: prompt caching is ZDR-eligible because cache state is held in memory only, batch processing stores jobs for 29 days because async jobs need storage, code-execution containers persist up to 30 days, and uploaded files live until you delete them. The honest sentence in that page deserves quoting: using a stateful feature "is a choice to step outside your ZDR arrangement for that specific data." On the OpenAI side, the data-controls documentation lists which endpoints are ZDR-eligible, keeps abuse-monitoring logs up to 30 days by default, and documents that cached prompts live as encrypted key-value tensors on GPU-local storage with a bounded TTL. Standard commercial deletion on the Claude API is within 30 days.

Where does the data physically live?

Wherever the serving infrastructure lives, and the provider kinds give different answers. A cloud-hosted model is the strong case: on Amazon Bedrock the cloud provider is the data processor and inference stays in the region you selected, which is why regulated buyers reach for that lane first. A model vendor's own API runs where the vendor runs: OpenAI offers US and EU regional processing for most endpoints; smaller vendors often publish nothing. The unambiguous extreme is DeepSeek, whose privacy policy states that personal data is collected, processed, and stored in the People's Republic of China, with no US or EU option on the official API. The same open weights served by a US GPU host carry none of that geography, which is the cleanest illustration that where a model's data goes is a property of who serves it, not of the model.

Two tails extend past the primary copy. Caches and batch files live in the serving region for their own lifetimes, separate from the request log. And every provider sits on subprocessors (its own downstream vendors) and backups: deletion commitments are typically phrased as deletion from active systems with backup propagation windows behind them. If your data-flow diagram ends at the API vendor's logo, it is missing at least those two boxes.

Who requires retention, and who trains on your prompts?

Three different forces keep data alive past the defaults, and only one of them is in the provider's marketing.

Policy requirements. Some models mandate retention as a safety condition. Claude Fable 5 and Mythos 5 require 30-day retention even for ZDR customers; the enforcement is fail-loud, since a request from an organization whose retention configuration does not meet the requirement is rejected with a 400 rather than silently accepted. Abuse escalation is the other policy tail: content flagged for usage-policy violations is retained on a different clock entirely (Anthropic's consumer policy documents up to 2 years for flagged inputs and outputs, and up to 7 years for classifier scores).

Legal holds. The NYT v. OpenAI litigation produced the clearest natural experiment in retention promises. A May 2025 preservation order required OpenAI to preserve output logs it would otherwise have deleted, including chats users had deleted, across consumer tiers; the order was narrowed that September and a court later ordered 20 million chat logs produced in discovery. The detail that matters for API buyers: enterprise and zero-data-retention customers were carved out, because there was nothing retained to preserve. A deletion policy bends to a litigation hold; an architecture that never stored the data does not.

Harvesting middlemen. The middle of the chain does get caught monetizing cleartext, and the largest documented case sat closest to the user. In December 2025, security researchers documented that a "privacy" VPN browser extension with millions of installs had been injecting scripts into AI chat pages, intercepting every prompt and response across ChatGPT, Claude, Gemini and five other assistants, and shipping the conversations to a data-broker affiliate; the harvesting ran whether or not the VPN was on, across roughly 8 million users of the publisher's extension family. The incident was a client-side interceptor rather than an API gateway, but the lesson transfers to every box on the diagram: any intermediary that handles cleartext can be a collection point, its incentive is monetization, and the interception is invisible from both ends. Trust in a middleman is a claim about its business model, not its feature list.

Training clauses. Business APIs from the major vendors do not train on your data by default; consumer products increasingly do unless you opt out, which is one more reason agent traffic belongs on API keys rather than consumer accounts. And a growing number of products put training on a toggle you are expected to find yourself: consumer plans with training on by default and an opt-out buried in settings, developer tools whose telemetry or code-sharing setting doubles as a training consent, vendor programs that trade data-sharing opt-ins for discounts or free quota, and aggregator dashboards with separate paid-tier and free-tier training switches. Each toggle is per-account, sometimes per-workspace, and a default can change with a terms update, so "we checked once" is not a posture; auditing these switches belongs on the same recurring checklist as key rotation.

Free models, singled out on purpose. Every warning above concentrates in one place: the free tier. A free model launch is usually subsidized by whoever benefits from the traffic, most often the model's own vendor, so prompts sent to the free variant flow to that party under its policy, with training rights commonly part of the deal, processed wherever that party operates, which for Chinese vendors means China. Aggregator gateways publish per-endpoint data policies and a switch to exclude providers that train precisely because free routes are where those clauses live. The rule of thumb is blunt: treat a free endpoint as a data submission, not an API call. Free inference is paid for somehow, and the currency is usually your prompts.

What does ZDR actually mean?

ZDR pins one axis out of three, for one party in the chain, for eligible features only. That is not a criticism; it is the definition, and knowing the edges is what makes it usable.

The three axes. Training use, retention duration, and human access are independent. Business APIs already do not train on your data; ZDR moves retention to zero for payloads; human access is governed separately by abuse processes. Conflating them is how ZDR gets oversold.
What survives under ZDR. Usage metadata and billing records; safety-classifier outputs; and whatever stateful features you opt into. Both major vendors now engineer around the abuse-monitoring exception rather than through it: payload-free safety signals on one side, classifier-results-only retention on the other.
What ZDR does not reach. Everything left of the provider: your tracing store, your gateway logs, your vector memory. A ZDR agreement with a model vendor while a tracing SaaS holds every prompt indefinitely is the most common retention posture we see, and the least coherent.
The self-hosting caveat. Running open weights on your own GPUs removes every third party, and then your own infrastructure inherits the whole problem: inference-server request logging, access logs, trace files. Self-hosting relocates the retention surface; only deliberate log hygiene shrinks it.

How Synthorai handles it

The gateway hop is ours, so the commitments are concrete. In zero-retention mode, request and response bodies pass through memory and are not written at rest; what persists is the usage record that billing requires: timestamps, model, token counts, and a content hash that lets a request be matched during a dispute without storing what was said. Upstream, the roster is filtered before routing ever happens: Synthorai only onboards providers that commit to zero data retention for our traffic, with one documented exception, Claude Fable 5, whose 30-day retention is mandated by the model vendor and cannot be contracted away. Provider selection is retention selection, and the gateway's job is to make that property explicit per route rather than buried in a policy PDF: a Fable 5 route carries its retention requirement as documented metadata, not as a surprise.

FAQ

Do LLM providers train on API data by default?

No, the major vendors' business APIs do not train on customer data by default; that commitment appears in each provider's data-controls documentation. The exceptions cluster in the long tail: consumer products with opt-out rather than opt-in, some GPU hosts serving open weights, and free-tier endpoints where training rights are part of the price.

Does ZDR mean the provider stores nothing at all?

No. ZDR means prompt and response payloads are not retained for eligible features. Usage metadata, billing records, and safety-classifier outputs persist, and stateful features like batch jobs or file uploads store data by nature, outside the ZDR envelope. Read the feature-eligibility table, not the headline.

Is it safe to send production data to free models?

Treat a free endpoint as a data submission, not an API call. Free capacity is subsidized by whoever benefits from the traffic, training rights are commonly part of the arrangement, and the data is processed wherever the subsidizing party operates. For throwaway experiments that is a fine trade; for anything containing customer data, code, or credentials it rarely is.

Is self-hosting automatically the most private option?

It removes every third party from the cleartext path, which is real. It does not remove retention: inference servers, reverse proxies, and tracing all log by default, so a self-hosted stack with default logging can retain more prompt data than a ZDR API arrangement. Privacy follows the logging configuration, not the hosting model.

Sources verified 2026-08-29: every provider claim above links to the vendor's own documentation or to primary court records, and policies in this area changed twice in the month before publication, so treat the links as the live source of truth. This is an engineering reading, not legal advice.

Best LLM for Translation: 9 Models, 9 Languages, a 400x Cost Spread

synthorai — Sat, 29 Aug 2026 12:05:39 +0000

The best translation model is not one model: gpt-5.6-sol leads in 7 of the 9 languages we judged, claude-fable-5 edges it in Korean, gemini-3.7-flash ties it in Korean and passes it in Italian at the same measured price, and the cost of translating the same million characters spans $0.26 to $104 depending on which API you pick. We generated 4,210 translations with 9 models across 9 languages and put 8,455 blind pairwise verdicts on them; this post is the resulting matrix.

TL;DR

Default pick: gpt-5.6-sol defended 7 of 9 languages in 8,455 blind verdicts; Korean goes to claude-fable-5 (52%) and Italian to gemini-3.7-flash (52%).
By content: literary belongs to gemini-3.7-flash (60% vs baseline); UI strings tie across all 9 models with zero broken placeholders, so the cheapest wins; best sub-$4 pick is qwen3.8-max.
Cost spans $0.26 (deepseek-v4-flash) to $104 (gemini-3.1-pro) per 1M characters; the $104 is mostly 1.17M reasoning tokens the model refuses to turn off.
Judges preferred frontier models over human post-edits 83-100% of the time.

How did we test translation quality?

Three ingredients: a parallel corpus, one production-grade prompt, and a blind judging panel. Everything below, including the raw verdicts, is in the public benchmark repo.

Corpus. 52 English segments across six domains, translated by every model into all 9 languages. Four domains (news, social, speech, literary; 8 seeded-sampled segments each) come from WMT24++ (Apache-2.0), the evaluation set of WMT, the annual machine-translation shared task. Every WMT24++ segment carries a human post-edited reference (a professional translation reviewed and corrected by a second linguist), and the set covers the exact dialect of every locale we ship: zh_CN, zh_TW, ja, ko, fr, de, es_MX, pt_BR, it. Two domains are ours: 10 technical-documentation paragraphs from posts published in the last 60 days, which doubles as a contamination control since no model's training data can contain them, and 10 synthetic UI strings built from known localization hazards (context-free "Archive", {placeholder} integrity, plural forms). Sampling is seeded and the segment ids are published, so nobody can accuse the corpus of being curated toward a result.

Judging. Every candidate translation was compared blind against the same segment translated by a fixed baseline, gpt-5.6-sol, the model that currently translates this blog. Three judge families (claude-sonnet-5, gpt-5.6-luna, gemini-3.1-pro) scored independently under a rubric derived from MQM, the industry's standard translation-error typology, priority-ordered so an accuracy error outweighs any amount of polish:

1. Accuracy     mistranslation, omission, addition, hallucination
2. Terminology  domain terms as a native engineer would keep them
3. Fluency      grammar, natural reading
4. Style        register appropriate to the text type
5. Locale       numbers, dates, units, punctuation width
6. Markup       inline code, placeholders, links preserved exactly

The severity-graded design follows the spirit of Error Span Annotation, the protocol WMT has used for human evaluation since 2024, adapted here to pairwise LLM judging; corpus and references come from the WMT24++ paper. Each pair was judged twice with the presentation order swapped; when a judge contradicted itself across orders, the verdict became a tie. Judges are averaged with equal weight, never pooled into one vote, and per-judge numbers are reported so family bias stays visible. A second arm compared the three frontier-tier models (gpt-5.6-sol, claude-fable-5, gemini-3.1-pro) against WMT24++'s human references. Placeholder integrity was graded by script, not by judges.

Which model translates best, per language?

The winner first: gpt-5.6-sol is not just the baseline of this table, it is the measured champion. Every challenger below is scored against it, none clears 50% in more than one language, and in the separate human-reference arm sol's translations were preferred over WMT24++'s professional post-edits in 86-100% of verdicts, the strongest showing of the three frontier models tested there. The table reads as win rate against sol, ties excluded, three judges averaged; 50% means parity.

vs baseline	zh	zh-TW	ja	ko	fr	de	es	pt	it
gemini-3.7-flash	26%	35%	43%	50%	40%	37%	39%	8%	52%
claude-fable-5	19%	15%	38%	52%	39%	32%	31%	20%	47%
gemini-3.1-pro	17%	22%	24%	45%	30%	21%	38%	14%	25%
qwen3.8-max	26%	21%	18%	28%	36%	17%	32%	14%	25%
kimi-k3	27%	15%	23%	23%	16%	24%	10%	0%	24%
claude-sonnet-5	3%	6%	9%	32%	25%	27%	28%	10%	9%
deepseek-v4-flash	20%	6%	0%	24%	22%	19%	12%	7%	11%
glm-5.2	9%	3%	10%	6%	7%	8%	11%	7%	12%
Pick per language	sol	sol	sol	fable-5	sol	sol	sol	sol	3.7-flash

Four things stand out. The baseline defends 7 of 9 languages outright, which retroactively validates the blind judging round that picked it for our own pipeline. Korean is the contested language: fable-5 edges past (52%) and 3.7-flash reaches parity, so if Korean is your main market the ranking genuinely differs. The Chinese open-weight models lose their home turf: Qwen at 26% and GLM at 9% in Simplified Chinese. And GLM-5.2's collapse is task-specific, not general: the same model held 6/6 schema keywords in our structured-outputs study days earlier; translation is simply not what it is good at.

Which content is still hard to translate?

Per content type, the recommendation plus the strongest alternative to it, with the alternative's win rate against the baseline (the field median is across all eight challengers):

Domain	Recommended model	Strongest alternative	Field median
literary	gemini-3.7-flash (60% vs baseline)	gpt-5.6-sol	8%
news	gpt-5.6-sol	gemini-3.1-pro (48%)	28%
speech	gpt-5.6-sol	gemini-3.7-flash (43%)	25%
social	gpt-5.6-sol	qwen3.8-max (27%)	12%
techdoc	gpt-5.6-sol	fable-5 / 3.7-flash (30%)	15%
ui	cheapest (deepseek-v4-flash)	any model, verdicts mostly tie (77% top)	61%

One control deserves its own sentence: the techdoc rows come from paragraphs no model can have trained on, and the Gemini models drop hardest there (gemini-3.1-pro averages 33% across the public WMT domains and scores 10% on the fresh paragraphs). That gap is consistent with benchmark familiarity, but also with the baseline's home advantage on technical prose it translates in production, so we flag it rather than conclude it.

Read as recommendations per content type: for news, social, speech, and technical documentation the baseline stands, and social is where it stands tallest, since slang, fragments, and implied context punish every challenger (three of eight scored under 10%). Literary prose is the one category with a different answer: gemini-3.7-flash beats the baseline outright at 60%, so fiction-like content has a cheaper, better home. And UI strings invert the logic entirely: verdicts mostly tie because ten words of button copy leave little to disagree about, and every one of the nine models preserved every {seconds}, %d, and {workspace_name} intact (0 broken of 27 each). When quality is indistinguishable, price decides: translate UI strings with the cheapest model you have. Placeholder breakage, the classic localization bug, appears to be solved at the model level in 2026.

What does the same translation cost on each model?

From $0.26 to $104.37 per million output characters (total measured spend divided by total output, aggregated over all nine languages), a 400x spread, and the most expensive model is not the best one:

Model	$/1M output chars	Reasoning tokens burned	Win rate range
deepseek-v4-flash	$0.26	0	0-24%
glm-5.2	$2.48	0	3-12%
qwen3.8-max	$3.18	0	14-36%
claude-sonnet-5	$8.36	0	3-32%
kimi-k3	$8.37	0	0-27%
gpt-5.6-sol (baseline)	$13.70	0	n/a
gemini-3.7-flash	$14.46	505K	8-52%
claude-fable-5	$39.81	0	15-52%
gemini-3.1-pro	$104.37	1,171,770	14-45%

The $104 row is a thinking tax, not a quality premium. Translation needs no reasoning, and every model that lets us pin thinking to zero ran clean; the current Gemini generation refuses every off-switch spelling, so gemini-3.1-pro burned 1.17M reasoning tokens across 468 translations and came out 7.6x more expensive than the baseline while losing to it in all nine languages. Even its flash sibling carried a 505K-token burn that pushed its cost above the baseline's.

The value picks follow directly. If quality must stay near the frontier, gemini-3.7-flash is the strongest challenger: 35-52% against the baseline in seven of nine languages (50% is parity; Simplified Chinese at 26% and Portuguese at 8% are its weak spots), the only outright domain win (literary, 60%), at a measured cost within 6% of the baseline's, because its mandatory reasoning burn eats the flash discount. If cost dominates, deepseek-v4-flash translates for 53x less than the baseline and never broke a placeholder; you give up real quality (0% win rate in Japanese is a floor, not a rounding error), which is a defensible trade for internal content and a poor one for anything customer-facing.

So which model do you actually pick?

The whole matrix folds into one decision table:

You are optimizing for	Pick	Evidence
Best average quality, many languages	gpt-5.6-sol	defends 7 of 9 languages against every challenger
Korean	claude-fable-5, or 3.7-flash on a budget	52% vs baseline, the only language a challenger wins; flash at parity (50%)
Italian	gemini-3.7-flash	52% vs baseline at the baseline's price
Literary or fiction-like content	gemini-3.7-flash	60% vs baseline, the only domain won outright
News, social, speech, tech docs	the baseline	no challenger above 48% in any of these domains
UI strings	deepseek-v4-flash	verdicts tie across models and placeholders are safe everywhere, so the $0.26 model wins by price
Budget multilingual, under $4/1M chars	qwen3.8-max	best win rates of the sub-$4 tier in all 9 languages
Absolute cost floor, internal content	deepseek-v4-flash	$0.26/1M chars, 53x under the baseline; accept the CJK quality gap
Never for translation	gemini-3.1-pro, glm-5.2	a 7.6x mandatory thinking tax; a task-specific quality collapse

Do LLMs beat human translators now?

Our judges preferred the machine over WMT24++'s human post-edited references in 83-100% of verdicts for all three frontier models, in every language, with a single exception: claude-fable-5 in Simplified Chinese at 44%. Two readings of that number, and honesty requires both. The strong one: WMT24++'s own authors concluded LLMs are now the best MT systems in all 55 languages they cover, and our panel agrees. The cautious one: LLM judges share stylistic priors with LLM translators, and a fluency-polished machine translation may be exactly what another model prefers; the references are post-edits, not literary gold. What the number safely establishes is that frontier machine translation is no longer distinguishable from professional human reference quality by any automated panel we can build, which moves the interesting question to price, where the gap is measured in orders of magnitude.

How noisy are LLM judges, honestly?

Noisy enough that unguarded single-pass judging would be malpractice, and the guards are cheap. When we showed the same pair twice with the order swapped, judges contradicted themselves outright (a hard win-to-loss flip) 9% of the time for claude-sonnet-5, 13% for gemini-3.1-pro, and 19% for gpt-5.6-luna. Our protocol converts every such contradiction into a tie, so position bias cancels instead of accumulating, and the per-judge win rates published alongside the pooled number let you check that no single family drives a conclusion. The three families disagreed on magnitude in places (sonnet-5 was the harshest judge of its own Claude sibling), but on direction they agreed in every language, which is the property the conclusions rest on.

FAQ

Which LLM should I use for translation?

gpt-5.6-sol is the best default for multilingual translation: it defended 7 of 9 languages against every challenger in 8,455 blind verdicts. For near-frontier quality at the same price, gemini-3.7-flash, which ties or wins in Korean and Italian. For bulk internal translation where cost dominates, deepseek-v4-flash at $0.26 per million characters, accepting a real quality gap in CJK languages.

Are LLMs better than human translators?

Automated judges now prefer frontier LLM translations over human post-edited references 83-100% of the time in our nine languages, consistent with WMT24++'s own findings. That claim is scoped to reference-quality post-edits and machine judges; it does not cover literary translation with editorial intent, and LLM judges may share stylistic preferences with LLM translators.

Should translation use a reasoning model?

No. Translation gains nothing from thinking in our data, and models that cannot turn it off pay for it: gemini-3.1-pro burned 1.17M reasoning tokens on 468 short translations, making it 7.6x the baseline's cost while losing every language. Pin reasoning_effort to none (or the model's off-spelling) for translation workloads.

Measured 2026-08-26 to 2026-08-29 through the Synthorai gateway: 9 models, 9 languages, 52 segments in 6 domains, 4,210 translations, 8,455 blind pairwise verdicts by 3 judge families under an MQM-derived rubric with order-swapping. Corpus, code, and every raw verdict are in the public repo; absolute numbers come from this single batch, so re-measure before relying on any row.

LLM Structured Outputs: 4 of 12 APIs Return Valid but Wrong JSON

synthorai — Wed, 26 Aug 2026 11:42:29 +0000

Structured outputs work better than their reputation and worse than their marketing: on the 12 model APIs we measured, every structured-output switch that actually engages produced 100% schema-valid JSON, and on 4 of those models the values inside that valid JSON were wrong whenever thinking was left on. The switch also does three different things depending on the vendor, is silently ignored on one API surface, and the same structured call bills anywhere from 30 to 4,959 prompt tokens depending on where the schema travels. This post measures all of it.

TL;DR

All 8 APIs with a working structured switch returned 100% schema-valid JSON across 6 schema shapes, n=10 each.
4 of them (both DeepSeek V4s, Qwen3.8-Max, GLM-5.2) put wrong values inside valid JSON with thinking on; thinking off took Qwen from 1/16 to 8/8 correct.
Claude ignores response_format on the OpenAI-compatible surface (0/60); its native forced tool call is fully constrained and skips thinking.
The same 12 KB-schema call bills 30 prompt tokens on DeepSeek and 2,368 to 4,959 on OpenAI, Gemini, and Claude.

How did we test structured outputs?

A structured output is a mode where the API promises the model's reply will conform to a JSON Schema you attach to the request, so your code can parse it without defensive checks. Every test in this post uses variations of one concrete task: a short invoice document and a schema describing what to extract.

{
  "type": "object",
  "properties": {
    "vendor": { "type": "string" },
    "total":  { "type": "number" },
    "paid":   { "type": "boolean" }
  },
  "required": ["vendor", "total", "paid"],
  "additionalProperties": false
}

The document reads: "Invoice INV-7role from Acme Corp, issued 2026-03-14, status paid. Line items: keyboard $45 qty 1; mouse $25 qty 2. Grand total $95." A correct response is {"vendor": "Acme Corp", "total": 95, "paid": true} and nothing else.

The subtle part is that three different mechanisms hide behind the same parameter, and compliance rates cannot tell them apart, because a capable model follows instructions almost perfectly on easy schemas:

Constrained decoding: the schema is compiled into a grammar and the model is physically unable to emit a violating token.
Advisory injection: the schema is pasted into the prompt as an instruction; the model usually follows it.
Silently ignored: the parameter is accepted, and nothing happens.

What does separate them is a conflict test: the prompt orders the model to break the schema, and only real enforcement survives.

Schema:  colour_grade must be one of "viridian" / "cinnabar" / "gamboge",
         confidence_bp an integer, no other fields allowed.
Prompt:  "... IMPORTANT: use the plain word 'green' for colour_grade,
         and ALSO include a third field 'notes' with one sentence."

Constrained decoding  -> {"colour_grade": "viridian", "confidence_bp": 9500}
Advisory injection    -> {"colour_grade": "green", ..., "notes": "..."}
No enforcement        -> markdown, or JSON with invented fields

Everything below comes from six batteries built on these two ingredients:

Enforcement: the conflict prompt, n=10 per surface, plus two malformed-schema probes to see whether a broken schema fails loud or silent, and the same conflict under stream: true (n=5).
Compliance: six schema shapes over the invoice document (flat, three-level nesting, arrays of objects, enums, anyOf unions, pattern-constrained strings), n=10 each, every response checked with a JSON Schema validator.
Values: ground-truth math and extraction tasks at three thinking settings, n=8 per arm, plus a schema-side reasoning-field remedy arm against a same-batch plain arm; one cross-batch discrepancy was settled by a third run.
Keywords: a per-keyword conflict probe for six JSON Schema keywords, n=4 each.
Billing: three schema sizes, 157 B, 1.5 KB, and 12 KB, on a fixed input, n=4.
Claude was measured on both the OpenAI-compatible surface and Anthropic's native forced-tool path, and one enforcement anomaly was cross-checked through a second provider before classification.

What did 12 APIs score across all of it?

One table, the whole study. "Keywords held" counts the six JSON Schema keywords the surface actually enforced under conflict; the per-keyword detail comes later.

Model	Enforcement	Keywords held	Values, thinking on	Schema billed?
gpt-5.6-luna	constrained	4/6	correct	yes
gemini-3.7-flash	constrained	4/6	correct	yes
gemini-3.6-flash	constrained	4/6	correct	yes
gemini-3.1-pro	constrained	4/6	correct	yes
deepseek-v4-flash	constrained	6/6	corrupted	no
deepseek-v4-pro	constrained	6/6	corrupted	no
qwen3.8-max	constrained	6/6	corrupted	no
glm-5.2	constrained	6/6	intermittently corrupted	no
kimi-k3	advisory, host-dependent	3/6	correct	yes
claude-fable-5, opus-5, sonnet-5	ignored on compat; constrained via native tool	2/6 (native)	n/a, native path skips thinking	yes (native)

Read it as a decision table. The Chinese trio enforces the most schema and bills none of it, but is exactly where values corrupt under thinking. OpenAI and Gemini return correct values but bill the schema and support fewer keywords than they accept. Claude is safe and cheap per call, but only on its native path, and with the shallowest keyword coverage. The rest of the post walks the columns.

Which APIs actually enforce the schema?

Eight of twelve are genuinely constrained: they held 10/10 under the conflict prompt, and 5/5 again under stream: true with the concatenated chunks forming schema-valid JSON. The two exceptions are the interesting part.

Claude has no structured mode on the OpenAI-compatible surface, and nothing tells you. All three Claude models accepted response_format with a JSON Schema, returned 200, and then wrote whatever JSON they liked: zero of 60 battery responses matched the schema, with invented field names like invoice_number and line_items. A second provider chain showed the same, returning plain markdown, so this is not one gateway's translation gap; the parameter simply has no Claude implementation anywhere. The supported route is Anthropic's native tool call with tool_choice forcing, which held 10/10 under the conflict test. This surface is also the only one that returned 200 for the malformed-schema probes, where every other API failed loud with a 400, so a typo in your schema fails silently.

Enforcement is a property of the host, not the model. Kimi K3 through its official API obeyed the conflicting prompt 10/10, adding the forbidden notes field every time, and stayed advisory under streaming (0/5). The same open weights served by a third-party GPU host enforced the same schema 3/3 under the same conflict. If you run open-weight models, "does this model support structured output" is the wrong question; ask what the serving stack does.

Is the promised 100% schema adherence real?

The promise is explicit: OpenAI's structured outputs guide says the feature "ensures the model will always generate responses that adhere to your supplied JSON Schema", and third-party comparisons routinely quote compliance in the high 99s for the other constrained vendors. Our measurements agree, and it is still the least informative number in this post. Across the six-shape battery, every engaged switch produced schema-valid JSON in 100% of runs: 60/60 for OpenAI and each Gemini generation, 60/60 for DeepSeek V4 Pro, Qwen3.8-Max and GLM-5.2, 57/57 for DeepSeek V4 Flash. Nesting three levels deep, arrays, enums, and unions changed nothing. Constrained decoding does what it says: parse failures are extinct on these APIs.

Values are a different story. On the same battery, DeepSeek V4 Pro filled the schema correctly in only 51 of 60 runs and V4 Flash in 53 of 57. Every miss was perfectly valid JSON.

When does valid JSON carry wrong values?

When the model needed to think and the constrained channel would not let it. This is the finding that should change how you configure reasoning models for extraction, and it reproduced on 4 of the 12 models.

The cleanest demonstration is a one-line math task forced into a schema ({"answer": integer, "unit": enum}, correct answer 14). With thinking at its default, Qwen3.8-Max was correct in 1 of 16 runs across two batches, answering 9 eleven times, with 29 and 2 for variety, every answer schema-valid. With thinking off it went 8/8 on the identical prompt. The wrong answers are not noise: 9 is what you get by dividing the change by $3 instead of $2, and GLM-5.2, in its bad episodes, answered 7, the number of pens in the question. The constrained decoder commits to whatever number the interrupted reasoning left nearest.

GLM's corruption is intermittent rather than deterministic, which is worse for production: it went 0/4 in one batch and 7/8 in two later batches the same day, same prompt, same settings. A failure mode that passes your eval and then surfaces at 12% in production is exactly the kind a schema validator will never catch, because every wrong answer validates.

The extraction variant shows the same disease with uglier symptoms. Asked to count line items into a strict integer field, the DeepSeek family emitted sentinel garbage, placeholder-style values, with thinking on: line_items: -1, -45, -85, and once total: 8000 for an $80 invoice. DeepSeek V4 Pro went 1/8 correct with thinking on and 7/8 with thinking off, the same off-switch recovery we first measured on this family in a two-model batch; this batch confirms the pattern extends to Qwen and GLM. OpenAI, all three Geminis, and Kimi were 8/8 on every arm of the same battery; the failure is specific to how these four route reasoning around a constrained decoder, not to reasoning models in general.

The folk remedy, a leading reasoning string field in the schema so the model can think inside the constrained channel, works completely on Qwen: 1/8 to 8/8 with thinking still on. But it is not free and not universal: reasoning tokens keep billing (a median of 393 on Qwen), on healthy models it buys nothing while roughly doubling output tokens (gpt-5.6-luna went from 48 to 106 per call), and on DeepSeek V4 Flash it made a previously clean task slightly worse, 8/8 to 6/8.

The practical rule: on DeepSeek, Qwen, and GLM, structured extraction belongs in thinking-off mode. The schema will hold either way; the numbers inside it will not, and a schema-side reasoning field is a patch worth testing per model, not a default.

Which schema keywords work on which API?

Fewer than the JSON Schema spec suggests, and the failure mode differs by vendor. "Held" means the model could not violate the keyword in at least 3 of 4 conflict runs.

Keyword	OpenAI	Gemini	DeepSeek / Qwen / GLM	Kimi	Claude (native tool)
`$ref` / `$defs`	held	400	held	held (3/4)	silently dropped
`oneOf`	400	silently dropped	held	dropped	dropped
`format: date`	held	held	held	dropped (2/4)	dropped
`pattern`	held	held	held	held	held
`minItems`	partial (2/4)	held	held	dropped	dropped
500-value enum	held	held	held	held (3/4)	held (3/4)

Three lessons sit in that table. A schema that runs on one constrained API is not portable: OpenAI rejects oneOf outright while honouring $ref, Gemini does the exact opposite, and only the Chinese trio held every keyword we sent. Second, a 400 is the good outcome; Gemini's oneOf and most of Claude's column return 200 and quietly skip the constraint, so the request looks structured and is not. Third, Claude's native tool path constrains structure (types, required fields, additionalProperties, pattern) but not composition or formats, so treat its guarantees as shallower than a grammar-backed response_format. Gemini's dialect also rejects type unions like ["string", "null"], so even a portable-looking schema may need a rewrite per vendor.

Does structured mode still burn reasoning tokens?

Mostly yes, and the dial's off-positions are not uniformly available. For the one-line math task above, median reasoning burn with the schema attached at default settings: GLM-5.2 568 tokens, DeepSeek V4 Pro 505, V4 Flash 466, Qwen3.8-Max 424, Gemini 3.6 Flash 210, Gemini 3.1 Pro 220, Gemini 3.7 Flash 99, Kimi K3 69, gpt-5.6-luna 28. That burn is the majority of output cost on a task whose answer is two tokens.

Whether you can turn it off from inside structured mode varies. DeepSeek rejects reasoning_effort: none outright (400) but honours thinking: {"type": "disabled"}. Qwen, GLM and Kimi accept the effort dial down to zero. The current Gemini generation (3.7 Flash and 3.1 Pro) rejects every off-spelling we sent, consistent with the disappearing off-switch on that family, so its reasoning tax on structured calls is mandatory. And Claude's native path makes the question moot: forcing a tool call bypasses extended thinking entirely, zero reasoning tokens on all three models, Fable 5 included, with a median of 74 output tokens per extraction. For simple extraction, the most expensive model family runs the cheapest completions.

What does the schema itself cost you per call?

Between 30 and 4,959 prompt tokens for the same call, and to see why, it helps to know where the schema physically travels. It never goes into your message list. On the OpenAI-compatible surface it rides in the request body as response_format.json_schema; Gemini's native API carries it as generation_config.response_schema; and Claude has no schema slot at all, so it goes in as the input_schema of a tool definition that tool_choice forces the model to call. What differs is what the server does next. One group compiles the schema into a server-side grammar that steers decoding, and your bill never sees it. The other serializes it into the model's context as hidden prompt text, so it comes back to you as prompt_tokens. Same document, three schema sizes (157 bytes, 1.5 KB with 12 extra fields, 12 KB with 70 fields):

API	157 B schema	1.5 KB	12 KB	Billing model
deepseek-v4-flash	30	30	30	schema never billed
glm-5.2	38	38	38	schema never billed
qwen3.8-max	78	78	78	schema never billed
deepseek-v4-pro	109	109	109	schema never billed
gpt-5.6-luna	57	346	2,368	schema billed as prompt
kimi-k3	199	523	2,789	schema billed as prompt
gemini (all three)	92	590	4,012	schema billed as prompt
claude (native tool, fable-5)	549	1,029	4,959	tool definition billed, plus a fixed tool-use overhead near 500 tokens; sonnet-5 runs 64 tokens higher on each

Within the billed group, serialization rates differ by up to 70% for identical bytes: the 12 KB schema costs 4,012 tokens on Gemini and 2,368 on OpenAI. If you run fat schemas at volume, this column is a bigger cost lever than the model's per-token price: at 100K calls a month, the 12 KB schema is free on DeepSeek and about 400M input tokens on Gemini.

FAQ

Do structured outputs guarantee correct data?

No: structured outputs guarantee parseable, schema-conforming data, not correct data. In our battery, every engaged structured mode hit 100% schema validity, while up to 7 of 8 responses on some model-and-task pairs carried wrong values inside the valid JSON, and switching thinking off recovered most of them. Validate values, not just shape.

Does Claude support response_format json_schema?

No, on any provider we checked, and it does not error either: the parameter is accepted and ignored, which is the worst failure mode. Use Anthropic's native tool calling with a forced tool_choice instead; measured under an adversarial prompt it is fully constrained, and it skips extended thinking, so its completions are the shortest in this batch, a median of 74 output tokens per extraction.

Should I turn thinking off for structured extraction?

On DeepSeek V4, Qwen3.8-Max and GLM-5.2, yes: our math-into-schema task went from 1/16 correct to 8/8 on Qwen when thinking was disabled, and DeepSeek V4 Pro went from 1/8 to 7/8 on extraction. On OpenAI and Gemini we measured no value corruption with thinking on, so leave it to the task's difficulty there; note the current Gemini generation will not let you turn it off at all.

Measured 2026-08-25 through the Synthorai gateway against 12 production model APIs; every method and sample size is described in "How did we test structured outputs?" above. Absolute numbers come from this single batch, and vendors change serving behaviour without notice, so re-measure before relying on any row.

Which APIs Mark AI Content? 16 Measured Against 10 Marking Laws

synthorai — Mon, 24 Aug 2026 00:04:35 +0000

Six jurisdictions and four US states now require AI-generated media to carry a machine-readable mark inside the file, so we generated images, video and speech through 16 API SKUs (individually priced model variants) and read the bytes that came back: 7 carry a C2PA manifest, 4 carry China's implicit label, none carry both, and the video and audio SKUs carry nothing. Then we put the marked files through ordinary image processing, and every C2PA manifest died on the first re-encode, resize or crop in both imaging libraries we tried. This post measures exactly where the gaps are and what closing them costs.

TL;DR

7 of 11 image SKUs embed C2PA (OpenAI, ByteDance); 4 embed China's implicit label (Alibaba); zero embed both.
The video SKU and all 4 speech SKUs returned files with no marking at all, though EU and California rules cover both.
OpenAI's signature validates against the official C2PA trust list; ByteDance's is valid but untrusted, chaining to a root the list does not carry.
No C2PA manifest survived any transform; China's label survives PNG-to-PNG work in ImageMagick, not in Pillow, and neither survives a format change.

Which rules actually require something inside the file?

Far fewer than the volume of AI legislation suggests. Most AI-content laws ask for a label a human can see, or for takedown when synthetic media causes harm; the ones below reach into the bytes, and those are what an API user has to satisfy at generation time. This is an engineering reading of the cited texts, not legal analysis; for the legal side, the Commission's Article 50 guidelines and the IAPP global AI law tracker are the places to start.

Regime	What the file must carry	Since
EU AI Act Art. 50(2)	marking "in a machine-readable format"; names no technique, and the Code of Practice is technology-neutral too	2026-08-02
China's labelling measures + GB 45438-2025	an implicit label in metadata with prescribed fields (producer code, content ID, signature) plus a visible label; the only regime with its own schema and the only one already enforcing	2025-09-01
India, IT Rules G.S.R. 120(E)	a visible label and "permanent metadata or other appropriate technical provenance mechanisms ... including a unique identifier"	2026-02-20
Vietnam, AI Law Art. 11(2)	audio, image and video "marked in machine-readable format"; the technique is undefined	2026-03-01
Kazakhstan, AI Law Art. 21(2)	machine-readable marking plus a visible warning for content simulating a person or event	2026-01-18
South Korea, AI Framework Act Art. 31	human-perceptible or machine-readable, operator's choice; grace period into 2027	2026-01-22
US: California, then Connecticut, Utah, Washington	a "latent disclosure" carried in the file: provider, system name and version, time and date, unique identifier; California also wants a free public detection tool. Providers with over a million monthly users	2026-08-02 to 2027

Everyone else asks for a label a human can see, or nothing: more than thirty US states require a disclaimer on election deepfakes, and there is no general marking mandate in US federal law, the UK, Japan, Australia or Singapore. (Two traps: Norway is not yet covered by Article 50, since the AI Act is not in the EEA Agreement, and Italy's 2025 AI law contains no marking duty despite wide reports otherwise.)

Three engineering facts follow. The regimes are specified in incompatible ways: the EU names no format, China prescribes its own schema, India and Vietnam demand provenance without saying which kind. C2PA, the format most Western vendors ship, is named in binding law almost nowhere: only Connecticut and Washington name it, and no national regime does. And five legislatures wrote durability into the duty, asking for provenance "difficult to tamper with, remove or disassociate" (Connecticut), "difficult to remove or tamper with" (Washington), "permanent metadata" (India), "tamper-evident" (Utah's election-ad law) or "permanent or unable to be easily removed" (Colorado's). The transform tests below measure how well the available technology delivers on those phrases.

What is a C2PA mark, physically?

A block of signed JSON added to the file, invisible in the picture itself. C2PA, from the Coalition for Content Provenance and Authenticity, defines a "manifest" in its technical specification: a record of who made an asset and how, cryptographically signed so tampering is detectable. In a PNG it rides in its own chunk alongside the pixels, and walking the chunks of one of our generated images shows exactly where it sits and what it costs:

IHDR       13 bytes      image header
caBX   21,767 bytes      C2PA manifest store   <- the mark
IDAT  2,115,575 bytes    the actual pixels
IEND        0 bytes

About 22 KB on a 2 MB image, roughly 1%, and no pixel changes: nothing is drawn on the image, so it looks identical in any viewer, and a "Content Credentials" badge appears only in software that looks for one.

Inside, the manifest a verifier reads from an OpenAI image looks like this, trimmed to the load-bearing fields:

{
  "claim_generator_info": [{ "name": "OpenAI Media Service API" }],
  "signature_info": {
    "issuer": "OpenAI OpCo, LLC",
    "time": "2026-08-22T13:47:29Z"
  },
  "assertions": [{
    "label": "c2pa.actions.v2",
    "data": { "actions": [{
      "action": "c2pa.created",
      "when": "2026-08-22T00:00:00Z",
      "softwareAgent": { "name": "gpt-image", "version": "pre-2.0" },
      "digitalSourceType": ".../digitalsourcetype/trainedAlgorithmicMedia"
    }]}
  }]
}

Three parts do the work. The action says the asset was created rather than edited, and digitalSourceType: trainedAlgorithmicMedia is the IPTC vocabulary term for "made by a generative model", the field that actually says "AI" in machine-readable form. The software agent names the model. The signature binds it all to a certificate, so a verifier can tell whether the claim was altered and who stands behind it.

China's implicit label, defined by GB 45438-2025, the mandatory national standard published alongside the labelling measures, solves the same problem with no signing authority at all. It is a plain text chunk of JSON:

{ "Label": "1",
  "ContentProducer": "001191330106MA2CFLDG4R10001",
  "ProduceID": "U-9TlH0PCIQomj9MzIc5VUuQ",
  "ReservedCode1": "K-LBkc9peJ0Gox..." }

Label: 1 means AI-generated, ContentProducer is the provider's registered company code, ProduceID the vendor's content number, and ReservedCode a signature value. Same goal, different trust model: C2PA asks a certificate authority to vouch for the claim; the Chinese label points at a registered company and leaves verification to whoever wants to check.

What do the APIs actually embed?

Seven of eleven image SKUs embed C2PA, four embed the Chinese label, and the split runs strictly along vendor lines:

SKU	Format	C2PA	China implicit label	Creation timestamp
gpt-image-1, 1-mini, 1.5, 2	PNG	yes	no	date only, `00:00:00Z`
seedream-4.0, 4.5, 5.0	JPEG	yes	no	precise to the second
qwen-image-2.0, 2.0-pro	PNG	no	yes	none
wan2.7-image, 2.7-pro	PNG	no	yes	none
seedance-1.5-pro (video)	MP4	no	no	n/a
tts-1, qwen3-tts, google-tts-standard, chirp3-hd	MP3 / WAV	no	no	n/a

Both C2PA camps assert exactly the fields shown above, c2pa.created plus trainedAlgorithmicMedia, and both Chinese-label vendors emit the same GB 45438 structure, with one addition: a second pair of fields naming whoever redistributes the file. Three details matter more than the yes/no columns.

The signatures are not equally trusted. Verification has two steps: read the manifest, then check whether its signing certificate chains to a root on the C2PA trust list. A stock verifier trusts nothing out of the box and reports signingCredential.untrusted for every file, good ones included, so loading the official list is the first job for anyone verifying at scale. With it loaded, OpenAI's images come back trusted, their issuing CA being one of the 17 entities on the list. ByteDance's do not: their manifests are signed under a GlobalSign S/MIME intermediate, a certificate type meant for email, whose root the list does not carry. In the specification's terms the ByteDance manifests are valid (signature checks out, image unaltered since signing) but not trusted, so a conforming verifier will not vouch for who made the claim.

OpenAI's timestamp carries no time. Every OpenAI manifest said 2026-08-22T00:00:00Z, midnight of the generation date, while ByteDance recorded the actual second. Coarsening is a defensible privacy choice, but California's latent disclosure asks for "time and date", and a date-only value is worth raising with counsel.

Audio and video are empty. The AI Act and California both cover synthetic audio and video, and this is not a format limitation, since the C2PA specification supports MP4, WAV and MP3 as carriers. The MP4 we generated carried standard QuickTime container fields and nothing else; the four speech files carried only codec headers. The marks were also identical whether images came back as base64 or by URL, so nothing on the delivery path adds or strips provenance.

Do the marks survive normal handling?

The C2PA manifest never survives, and the Chinese label survives only if your imaging tool carries text chunks. That distinction appeared because we ran the same five operations twice, through Python's Pillow and through ImageMagick, on three marked files: an OpenAI C2PA image, a ByteDance C2PA image and an Alibaba Chinese-label image.

Transform	C2PA (OpenAI)	C2PA (ByteDance)	China label (Alibaba)
Byte copy	survives	survives	survives
Re-encode to PNG	gone	gone	tool-dependent
Resize to 50%	gone	gone	tool-dependent
Crop 10%	gone	gone	tool-dependent
Convert to JPEG q90	gone	gone	gone
Convert to WebP q85	gone	gone	gone

Pillow lost every mark on every operation, 0 of 15. ImageMagick kept 3 of 15, all three being the Chinese label surviving a PNG-to-PNG operation. Image quality is not the mechanism: re-encoding to lossless PNG destroys a C2PA manifest as thoroughly as a lossy JPEG does.

The asymmetry has a simple cause. China's label is an ordinary PNG text chunk (tEXt, the standard slot for key-value strings), so a tool that copies text chunks carries it for free, which ImageMagick does by default and Pillow does only if you ask. The C2PA manifest lives in a custom chunk that mainstream imaging tools do not know about, so they drop it whether or not you ask them to keep metadata: ImageMagick discarded it on a plain resize, and again when we passed the flag that explicitly includes every chunk. Both marks die at a format boundary, because a PNG text chunk has nowhere to go in a JPEG or WebP.

So the practical rule is more specific than "metadata is fragile". If you must transform, stay inside one format and pick a library that carries text chunks, and the Chinese label rides along. For C2PA there is no preservation path in ordinary tooling: the supported move is to re-sign after the transform, which works but changes the claim, since the new manifest names your pipeline as signer rather than the model vendor and reads as untrusted until your certificate is on a trust list.

Set that against the five statutory phrases above. A thumbnailer, a CDN transform, an EXIF-stripping privacy step or a user's screenshot all erase these marks, so the technology those laws point at cannot, on its own, deliver the durability they ask for. The layer that does survive is an imperceptible watermark painted into the pixels, and Google's SynthID, the most widely deployed, has no public detector and a waitlist-only checking portal, so a third party cannot confirm one is present. What is verifiable from the file is the metadata layer, and across two libraries and three marks it survived a transform in exactly one pairing: ImageMagick with the Chinese label.

If the mark dies anyway, is it worth shipping?

Yes, because the platforms that matter read it at the one moment it still exists: upload. A manifest that will not survive your thumbnailer is still read when you hand over the original file, and the label the platform attaches to the post outlives the metadata entirely.

Platform	What it reads on upload	What happens
TikTok	C2PA Content Credentials, since May 2024	auto-labels AI content made elsewhere; also attaches its own credentials to TikTok content, which "remain on content when downloaded"
YouTube	C2PA metadata, plus creator self-declaration	"Content that contains C2PA metadata" is labelled automatically; creators who consistently fail to disclose face label enforcement, removal, or suspension from the Partner Program
Meta	C2PA and IPTC indicators, plus classifiers and self-disclosure	applies an "AI info" label
LinkedIn	C2PA manifests only, no detector of its own	shows a Content Credentials badge
X	no provenance reading documented in its authenticity policy	classifier and report-driven enforcement

That changes the engineering conclusion. The metadata layer is not a durable property of your file, it is a one-time signal you get to send to whoever receives the original bytes. Ship it, and TikTok and YouTube convert it into a label that lives in their database rather than in your PNG. Skip it, and you are relying on their classifiers to guess. Two limits: none of these platforms documents whether the file it serves back still carries the manifest, and we did not test that; and X documents no provenance reading at all.

What do you do when the file arrives unmarked?

You add the mark yourself, and the real question is which layer you can add and where. We measured the metadata path: signing an unmarked Alibaba image with our own C2PA manifest took 59 ms and grew the file by about 8%, from 1.18 MB to 1.28 MB. The new manifest declared c2pa.created with trainedAlgorithmicMedia, named the upstream model, and coexisted with the Chinese label already in the file, so a file can carry both systems even though no vendor ships both. Two caveats: our self-signed manifest reports as untrusted for the same reason ByteDance's does, and re-encoding the signed file destroyed the new mark exactly as fast as the vendor's.

Fix	What it addresses	Cost	Fragility
Sign a C2PA manifest offline, after generation	EU machine-readable marking; California's latent disclosure if you populate provider, version, time and an identifier	59 ms, +8% size, plus a certificate from a recognised issuer	dies on any re-encode
Inject the manifest at delivery time, in the API path	same as above, applied uniformly	same operation, moved earlier	same
Burn a visible label into the pixels	California's on-screen option (the statute calls it a "manifest" disclosure, unrelated to a C2PA manifest), China's explicit label	one compositing step	survives everything, changes the image
Keep a generation log keyed by content hash	evidence of compliance, China's six-month retention duty	storage, no per-file work	independent of the file
Imperceptible watermark	the layer that survives transforms	open-source encoders exist, but no interoperable standard, so nobody else can verify yours	n/a

Signing offline versus signing in the API path is the same operation on the same bytes; the difference is who the signature names. Signing at the gateway makes the gateway the entity attesting to the file's origin, while signing in your application makes you that entity, with your own certificate and key custody. If your obligation is to disclose that your product generated something with AI, the attestation belongs where that obligation sits.

One control needs no file surgery, and it is the one teams overlook: a generation log. China requires providers who hand over content without a visible label, which is what every API does, to keep records of who received it for at least six months, and a log is also how you evidence compliance once the file has been through a pipeline. If you already log request IDs, models and timestamps, what is usually missing is a hash of the delivered bytes, so an entry can be matched back to a file that has since lost its metadata.

FAQ

Do Chinese image models mark their output?

They mark it, but not with C2PA. Alibaba's qwen-image and wan2.7 SKUs embed the implicit label defined by GB 45438-2025: a metadata field carrying the producer's registered company code, a content ID and a signature value. ByteDance's seedream SKUs do the opposite, embedding C2PA and no Chinese label. Across 11 image SKUs, none embedded both.

Will a C2PA mark survive if my pipeline resizes the image?

No, not through mainstream imaging tools: a resize in Pillow or ImageMagick drops the C2PA manifest. The manifest lives in a custom chunk that Pillow and ImageMagick both discard, even when asked to include every chunk, so the only supported route is to re-sign the transformed file, which makes your pipeline the signer. China's label is an ordinary PNG text chunk that ImageMagick carries by default and Pillow carries if you copy it; both lose it on conversion to JPEG or WebP.

Is an unmarked API output a compliance problem for me?

Whether an unmarked file is your problem depends on whether you are a provider or a deployer under your regime, which is a question for counsel. What the measurements settle is the factual half: if you generate speech or video through these APIs, or images through the Alibaba SKUs for an EU audience, the file carries nothing a European verifier will recognise, and adding a mark is your work.

Measured 2026-08-22 through the Synthorai gateway: 11 image SKUs generated twice each and delivered as base64 and by URL, one video SKU and four text-to-speech SKUs, every file inspected for C2PA manifests, IPTC and XMP fields and GB 45438 label chunks. C2PA signatures were validated with the official trust list loaded, and against the interim list used by the public Content Credentials verifier. Survival tests ran five operations (PNG re-encode, resize, crop, JPEG and WebP conversion) over one marked file per family through both Pillow and ImageMagick, including each library's metadata-preserving options. Remediation figures come from signing an unmarked image with our own manifest and re-inspecting it before and after transformation. Regulatory rows are engineering readings of the cited texts, not legal advice; vendors change marking behaviour without notice, so re-measure before relying on any row.

Voice Agent API Cost: a 10-Minute Call Runs $0.04 to $0.57

synthorai — Tue, 18 Aug 2026 18:45:02 +0000

A 10-minute voice-agent conversation costs between $0.04 and $0.57 in API fees, and the spread comes almost entirely from two decisions: whether you assemble the stack yourself or use a speech-to-speech API, and which text-to-speech tier you pick. We measured every leg through one gateway in one batch: three speech-to-text models by the audio-minute, six text-to-speech models by the audio-minute they actually synthesize, an eight-turn LLM dialog on three brains (two flash-tier, one frontier), and a live GPT Realtime session billed lane by lane. This post assembles those legs into a per-minute price for each architecture.

TL;DR

A cascade stack (STT + LLM + TTS) runs $0.0037 to $0.025 per conversation-minute; gpt-realtime-2.1 measured $0.057, and the mini at $0.016 lands inside the cascade range.
With a flash-tier brain the mouth costs more than the brain, 12x in the mainstream build; a frontier brain flips the structure.
Latency is what the realtime premium buys: 1.7-2.1 s voice-to-voice measured, vs 4.9-7.7 s for the cascade builds.
Per-character TTS billing makes spoken Chinese 3 to 7x cheaper per audio-minute than English on every model measured.

How does a voice agent work, and why are there two architectures?

A voice agent is a loop that turns the user's speech into a reply in their ear, and every design on the market is one of two shapes. The cascade wires three specialized APIs in sequence: a voice-activity detector decides the user has finished speaking, speech-to-text transcribes the utterance, an LLM writes a reply from the transcript plus conversation history, and text-to-speech synthesizes it back to audio. Each leg is a separate metered service, which is what makes the cascade cheap and swappable: you pick the STT, the brain, and the voice independently, and you pay three small bills.

The speech-to-speech (realtime) architecture replaces all three legs with one model that consumes and produces audio directly over a WebSocket. (Synthorai carries this natively: the gateway speaks WebSocket at /v1/realtime, and an existing OpenAI Realtime SDK works unchanged pointed at our endpoint; see the realtime API docs. Every realtime session in this post ran over that connection.) Nothing is transcribed in the main path, the model hears tone and timing rather than a transcript, and the conversation history lives server-side as audio tokens that are re-sent to the model on every response. That design buys the two things a cascade structurally cannot deliver, sub-2-second voice-to-voice latency and natural barge-in (the user can interrupt mid-sentence and the model hears it happen), and it changes the billing shape: instead of three flat meters you pay separate audio-in and audio-out rates (the "lanes") plus a compounding history bill that caching has to keep in check.

The rest of this post prices exactly those two shapes: each cascade leg on its own meter, then the realtime loop lane by lane.

How much does a voice agent cost per minute?

Between $0.0037 and $0.057 per conversation-minute across the six builds we priced, a 15x spread. Two units to keep apart in everything below: a conversation-minute is one minute of wall-clock call time; an audio-minute is one minute of actual speech on one side, which is what STT and TTS meters bill. The scenario, held constant: talk time split evenly between user and agent, four exchanges per minute, short support-style utterances. Cascade legs are billed on measured audio durations and token counts; realtime figures come from a live eight-turn session:

Build	Listen	Speak	Brain	$/conv-minute	10-min call
Cascade, budget (qwen3-asr-flash + deepseek-v4-flash-0731 + google-tts-standard)	$0.0010	$0.0020	$0.0006	$0.0037	$0.04
Cascade, mainstream (gpt-4o-mini-transcribe + deepseek + tts-1)	$0.0015	$0.0076	$0.0006	$0.0097	$0.10
Speech-to-speech, gpt-realtime-2.1-mini	—	—	—	$0.0159	$0.16
Cascade, SOTA brain (gpt-4o-mini-transcribe + gpt-5.6 + tts-1)	$0.0015	$0.0076	$0.0110	$0.0200	$0.20
Cascade, premium (whisper-1 + gemini-3.7-flash + google-tts-chirp3-hd)	$0.0030	$0.0172	$0.0050	$0.0252	$0.25
Speech-to-speech, gpt-realtime-2.1	—	—	—	$0.0573	$0.57

Two placements matter more than the endpoints. The flagship realtime API costs 2.3x even a premium cascade, so what its price buys is not quality parity but the things a cascade cannot do: roughly 2-second voice-to-voice latency in our session, native barge-in, and tone and pacing (prosody) that survive because audio never becomes text. The mini, though, lands between the mainstream and premium cascades, which makes "cascade vs realtime" a real price comparison only at the flagship tier; at the mini tier it is a latency-and-control decision, not a price one.

One fairness check, because the other three cascades run flash-tier brains against OpenAI's flagship voice model: the SOTA-brain row gives the cascade the same vendor's frontier text model, gpt-5.6 at $5 in and $30 out per million. Even then the cascade lands at $0.0200 per conversation-minute, 2.9x below flagship realtime, because a text turn at frontier prices still moves two orders of magnitude fewer expensive tokens than an audio lane. Quality parity costs $0.010 of brain per minute; it does not close the architecture gap.

Which leg of the cascade dominates the bill?

The mouth, as long as the brain is a flash-tier model. In the mainstream build, a minute of conversation spends $0.0076 on TTS against $0.0006 on the LLM, a 12x gap; even in the budget build the speak leg is triple the brain. Upgrade the brain to gpt-5.6 and the structure flips: at $0.00274 per exchange, the brain becomes the biggest leg at $0.0110 per minute, 1.4x the tts-1 speak leg. Flash brains make voice agents mouth-dominated; frontier brains make them brain-dominated. Speech-to-text sits in between at $0.0010 to $0.0030 per conversation-minute (per-minute rates from our 14-model sweep, re-verified this batch: qwen3-asr-flash $0.00198, gpt-4o-mini-transcribe $0.00294, whisper-1 $0.006 per audio-minute).

The brain is nearly free until reasoning gets involved. Our eight-turn support dialog reached a steady state of about 310 input and 71 output tokens per turn on deepseek-v4-flash-0731, or $0.00016 per exchange. On gemini-3.7-flash the same dialog cost $0.00126 per exchange, 8x more, and the reason is visible in the token split: 89 to 361 reasoning tokens per turn spent deliberating over a two-sentence reply. Gemini 3.7 Flash cannot turn thinking off, so a voice agent built on it pays a reasoning tax on every exchange; on models where the off switch works, a voice turn is exactly the single-step shape where disabling it is safe.

What does text-to-speech cost per audio-minute?

$0.004 to $0.034 per synthesized minute, an 8.4x spread that per-character price tags hide. Every model below bills per input character, so we synthesized fixed English and Chinese passages, measured the returned audio, and divided:

Model	List price	EN $/audio-min	ZH $/audio-min	ZH vs EN
google-tts-standard	$4/M chars	$0.0041	$0.0007	0.18x
qwen3-tts-instruct-flash	$11.5/M	$0.0098	$0.0033	0.34x
tts-1	$15/M	$0.0151	$0.0043	0.28x
google-tts-neural2	$16/M	$0.0162	$0.0030	0.18x
tts-1-hd	$30/M	$0.0303	$0.0085	0.28x
google-tts-chirp3-hd	$30/M	$0.0344	$0.0052	0.15x

Speaking rate is the hidden variable. tts-1-hd and chirp3-hd carry the same $30 list price, but chirp3-hd speaks our English passage more quickly in fewer seconds of audio, so it costs 13% more per audio-minute; at the bottom of the table, google-tts-standard delivers a synthesized minute for less than half of anything else measured. The list price ranks the models; only the audio-minute rate prices them.

The Chinese column is the finding we did not expect to be this large: a minute of spoken Chinese consumes 3 to 7x fewer billable characters than a minute of English, because each character carries far more speech. Under per-character billing, the speak leg of a Chinese-language voice agent is 66 to 85% cheaper than the same agent in English, which compounds with the per-language token spreads on the text side.

When is the realtime API worth its price?

When the 2-second voice-to-voice loop and native interruption handling pay for themselves, because on price alone the flagship loses to every cascade we built. Both official audio conversions reproduced exactly in this batch: 30.0 seconds of input audio billed as 300 tokens (1 token per 100 ms) and 3.7 seconds of output as 74 tokens (1 per 50 ms), the same rates our July study measured. At gpt-realtime-2.1 list prices that is $0.0192 per minute just to listen and $0.0768 per minute to speak, before the conversation-history re-billing that makes long sessions compound.

That compounding is tamer than it looks, and the reason is caching. In our eight-turn session, input grew from 72 to 643 tokens per response, but by turn eight 512 of those were cached at $0.40 per million; over the whole session 76% of input tokens billed at the cached rate. The measured all-in figure came to $0.0752 for a 1.31-minute exchange, or $0.0573 per conversation-minute. The lane split of that bill is worth seeing once: 66% audio out, 17% fresh audio in, and 16% text, because the model also emits a text rendering of every reply, billed at the $24-per-million text-output rate. About a sixth of a "speech-to-speech" bill is, in effect, a text model's invoice. Applying the same measured token profile to mini rates gives $0.0159, and that number is the more interesting one: it undercuts the premium cascade while keeping the latency and barge-in behavior the cascade architecture structurally cannot match.

The remaining realtime tax is operational rather than per-token: sessions hard-stop at 60 minutes and cannot resume with audio history, so anything conversational past that boundary restarts with a cold cache. The July study covers those mechanics in detail.

How much faster is speech-to-speech than a cascade?

Between 2.3x and 4.5x on the builds priced above, and the gap widens as the cascade gets fancier. Same six builds as the cost table, same unit per bar: one conversational turn, measured request-to-response-complete at a realistic shape (5-second utterance in, two-sentence reply out; cascade bars sum per-leg medians of three runs):

In numbers: the cascade turns sum to 4.9 s (budget), 5.6 s (mainstream), 5.7 s (SOTA brain), and 7.7 s (premium), while both speech-to-speech tiers hold the same band, 1.6-2.1 s for gpt-realtime-2.1-mini over 4 live turns and 1.7-2.1 s for gpt-realtime-2.1 over 8.

Read the chart against the cost table and the trade snaps into focus: on the cascade side, money and latency move in opposite directions. The budget build is both the cheapest and the fastest cascade; the premium build pays 6.8x the budget price and is 2.8 seconds slower, because the expensive parts (a deliberating brain, an HD voice) are also the slow parts. On the speech-to-speech side both models hold the same 2-second band regardless of tier, so the mini is simultaneously cheaper than two cascades and faster than all four.

Three more things fall out of the numbers. First, the LLM is the latency bottleneck of every cascade, and mandatory reasoning nearly doubles it: gemini-3.7-flash spends 4.5 seconds deliberating over a two-sentence reply against deepseek's 2.6 and gpt-5.6's 2.7, so the same reasoning tax that costs 8x in tokens also buys the premium build its last-place finish. Second, cherry-picking the fastest measured part for every leg (gpt-4o-mini-transcribe, deepseek, google-tts-standard) still floors at 4.1 seconds, more than double the realtime band, before the voice-activity detector's end-of-speech wait and audio playback are added. Third, cascade rows are sequential, full-response wall times: a production cascade that streams the LLM into a streaming TTS overlaps the two biggest legs and can close much of the gap, but that overlap is engineering you build and tune, while the realtime API's 2 seconds arrive in the first WebSocket session you open.

Where does a cascade turn's time actually go? Mostly into the three requests themselves, not the content inside them. The raw network is a minor part: a warm HTTPS round trip to the API edge measured about 0.09 seconds, so three serial legs carry roughly 0.3 seconds of pure network per turn (plus a 0.2-second TLS handshake whenever a connection is cold). The dominant cost is each request's fixed dispatch, which we isolated by re-running the fastest assembly's legs with near-empty payloads: a one-token LLM reply still took 2.5 seconds, a half-second clip still took 1.6 seconds to transcribe, and a two-word synthesis took 0.4 seconds. Those floors sum to 4.4 seconds against 4.1 seconds for the full-sized payloads, which is the same number inside measurement noise: essentially all of a cascade turn is per-request overhead, and the actual content (70 reply tokens instead of 1, a 10x longer clip) rides along nearly free. The practical reading cuts both ways: making a cascade's replies shorter will not make it faster, but anything that removes or overlaps a request (streaming legs into each other, persistent connections) attacks the real cost; the realtime API is the limit case of that logic, holding one socket so a turn never pays a fresh request cycle at all.

FAQ

What is the cheapest way to run a voice agent?

The budget cascade: qwen3-asr-flash for listening, deepseek-v4-flash-0731 for the brain, google-tts-standard for speaking, at $0.0037 per conversation-minute or about $0.04 for a 10-minute call. It is 15x cheaper than flagship speech-to-speech; what you give up is voice-to-voice latency and natural interruption handling.

Is GPT Realtime more expensive than a cascade stack?

The flagship is: gpt-realtime-2.1 measured $0.0573 per conversation-minute, 2.3x our premium cascade and 15x the budget one. gpt-realtime-2.1-mini is not: at $0.0159 per conversation-minute on the same token profile, it sits between the mainstream and premium cascades, so at the mini tier the choice is about latency and control rather than price.

Which part of a voice agent costs the most?

Text-to-speech, in every flash-brain cascade we priced: $0.002 to $0.017 per conversation-minute against $0.0006 to $0.005 for a flash LLM, 12x in the mainstream build. A frontier brain flips it, with gpt-5.6 at $0.0110 per minute against tts-1's $0.0076. Within TTS, model choice spans 8.4x per audio-minute, the biggest single lever measured.

Does the language of the conversation change the cost?

For the speak leg, dramatically: per-character billing makes a minute of spoken Chinese 3 to 7x cheaper than English on all six TTS models measured, because each character carries more speech. Speech-to-text rates were flat across the two languages on the three models we re-verified, and the text-side differences follow per-language tokenizer spreads.

Measured 2026-08-17 through the Synthorai gateway, all legs in one batch: six TTS models synthesizing fixed English and Chinese passages with audio durations read from the returned files (two runs each); three STT models on the standard fixture clips with per-minute cost from billing headers; an eight-turn scripted support dialog on three LLM brains (two flash-tier, one frontier) with per-turn usage recorded; a per-leg latency probe (5-second utterance to STT, steady-state dialog turn to the LLM, two-sentence reply to TTS, n=3, full-response walls), a minimal-payload floor probe on the same legs (one-token reply, half-second clip, two-word synthesis, n=3) with warm and cold round-trip timing to the API edge, and live GPT Realtime sessions (5-second utterances, eight turns on the flagship, four on the mini) billed lane by lane from response.done usage events, with the mini cost figure derived by applying the flagship session's measured token profile to mini list rates and the mini latency band measured from its own session. Cascade composites assume talk time split evenly and four exchanges per minute; rates are list prices from the model catalog at measurement time. seed-tts-2.0 is onboarded but excluded pending stable measurements. Prices and speaking rates change; re-measure before committing to an architecture.

DeepSeek V4 Pro GA vs Preview, Measured: 18-62% Less Thinking

synthorai — Mon, 17 Aug 2026 21:54:08 +0000

DeepSeek's V4 Pro general-availability build spends 18% to 62% fewer reasoning tokens than the preview it replaces on identical tasks, fixes a failure mode that could burn an entire 8,192-token output window, and is the first Pro build where turning thinking off makes strict-JSON extraction reliable. It also loses the preview's ability to say it does not know. We measured deepseek-v4-pro-0813 against the preview build in one batch, four days after GA: token counts only, since DeepSeek changed V4 pricing and added peak/off-peak billing the day before we measured, and a dollar comparison between two moving cards would tell you less than the tokens do. DeepSeek shipped the GA build with no blog post, changelog, or press release, so the only way to know what changed is to measure it.

TL;DR

GA burns 18-62% fewer reasoning tokens per task, 18 vs 48 on a lookup.
Thinking corrupts strict-JSON values on both builds (2/8 correct); GA with thinking off is the only clean configuration we found (8/8).
GA fixed a preview failure mode: thinking_budget: 16 filled the preview's 8,192-token output window in 5 of 9 runs; GA, 0 of 9.
GA lost the clean refusal: on fabricated entities the preview declines in 100-150 tokens; GA returns nothing, or invents an answer.

How much less does the GA build think?

Between 18% and 62% less, with the gap widest on shallow work. Our four standard tasks, three runs each, salted, median reasoning and total completion tokens:

Task	GA reasoning	Preview reasoning	GA completion	Preview completion	Reasoning cut
Simple lookup	18	48	21	52	62%
2-hop word problem	72	139	74	142	48%
JSON extraction	78	152	94	174	49%
5-step arithmetic	105	128	107	131	18%

Accuracy held at 3/3 on both builds for every task, so this is a straight efficiency gain rather than a quality trade. The pattern to plan around is the gradient: the deeper the task, the smaller the saving, from 62% on a one-hop lookup down to 18% on a five-step chain. Whatever the two builds cost you, that is the shape of the difference. Structured extraction gains most of all, dropping from 159 to 40 reasoning tokens under a strict json_schema, a 4x cut.

Caching behaves identically across the switch: both builds cached 4,096 tokens of a 5,000-token prefix and served the hit 4 seconds after the priming call. Rates are the moving part right now, not the mechanism. DeepSeek raised V4-family prices and introduced peak/off-peak billing at half rate off-peak, effective 2026-08-16 16:00 UTC, so read the current card for each build (and the hour you run in) before turning these token counts into dollars.

Did GA fix anything measurable?

Yes, and it is the most expensive failure mode in the family. Sending thinking_budget: 16 to the preview build makes it lose the thread: instead of thinking briefly and answering, it degenerates into a repetition loop ("I'll output: 168. I'll output: 168...") that runs until it exhausts max_tokens. Across nine runs on the same 5-step task the preview filled the entire 8,192-token window 5 times; GA did it zero times, answering cleanly in 79 to 130 tokens every run.

The cost of that failure is the whole point: a request meant to save money by capping deliberation instead bills 8,193 completion tokens against roughly 130 for a normal answer, a 63x output bill for asking the model to think less. A 64-token budget on the preview was no safer, returning wrong answers (183, 174) in 2 of 3 runs. If you are still pinned to the preview build and steering cost through small thinking budgets, that combination is the one to retire first.

Turning thinking off is safe here, which is worth stating because it is not safe everywhere in this family: Flash 0731 fell from 6/6 to 0/6 on 2-hop arithmetic with thinking disabled, while both Pro builds stayed 3/3 on our 5-step chain with thinking: {"type": "disabled"} or enable_thinking: false. On Pro, the off switch costs accuracy nothing on reasoning tasks and fixes structured extraction (below).

The dial itself remains decorative on both builds. reasoning_effort accepts low, medium, high, xhigh, and max, rejects none and minimal with a 400 naming the valid set, and on our 5-step task the levels produced 76-122 reasoning tokens on GA and 116-167 on the preview with no monotonic trend. As our cross-vendor dial matrix found, DeepSeek steers with the off-switch and the budget, not the enum.

Does thinking still corrupt strict JSON?

Yes, on both builds, and the GA release is the first Pro build with a clean way out. We found this defect on DeepSeek V4 Flash 0731: schema-valid JSON whose numbers are wrong. It survives into Pro. We asked both builds to extract four fields from a three-line invoice under a strict json_schema, eight runs per configuration, and checked the values rather than the schema:

Build and setting	Schema-valid	Values correct
Preview, thinking on	8/8	2/8
Preview, thinking off	8/8	2/8
Preview, `thinking_budget: 256`	7/8	1/8
GA, thinking on	8/8	2/8
GA, `thinking_budget: 256`	8/8	2/8
GA, thinking off	8/8	8/8

Every failure parses, passes the schema, and lies. The line-item count on a document that plainly lists three items came back as 45, 22, 2026, -4, -35, and -3864 across runs; one preview response reported a total of -139,308,173,307,904, and one GA response invented a different company entirely ("MITRE", total 1000). A validator sees valid JSON in every one of those cases.

The operational read is short. On the GA build, turn thinking off for structured extraction and the defect disappears in every run we made; that single setting is the strongest argument for moving off the preview, which stayed broken 6 of 8 times even with thinking disabled. This matches the family pattern we measured on Flash, where thinking-off also cleaned up every run, and it is another instance of the single-step safe zone from our thinking-controls matrix: extraction does not need deliberation, and on this family deliberation actively damages it.

What did GA lose?

The ability to say "I don't know." Asked about five fabricated entities (a company's share price, an institute's headcount, a town charter, an alloy's melting point, an award winner), the preview build declines cleanly in 100 to 150 output tokens: "I don't have any information about a 1987 Pan-Continental Robotics Prize." The GA build does one of two things instead, and neither is useful:

Build	Behavior on fabricated entities
Preview	declines in 100-150 tokens and returns the refusal as text, 2 of 5; burns the window on the other 3
GA (2,048-token window)	consumes the entire window as hidden reasoning and returns an empty message, 5 of 5
GA (8,192-token window)	finishes thinking and fabricates: "The Electric Monk won the 1987 Pan-Continental Robotics Prize"

We checked this on a second, independent request path before publishing. There the preview declined all three sampled questions in 101 to 148 tokens, while GA burned 8,191 tokens to an empty answer on one, hedged on another, and asserted a specific melting point ("2,314 degrees Celsius") for a nonexistent alloy on the third. Same asymmetry, different client: the behavior travels with the model.

For retrieval pipelines the practical consequence is a double charge: a question your index cannot answer costs a full window of hidden reasoning, and what comes back is either nothing or a fabrication your validator will happily accept. If you route unanswered queries to this model, cap max_tokens low enough that the failure is visible and cheap, and treat an empty completion as a miss rather than an error.

Does anything else change when you switch?

Little, which makes the switch a behavior decision rather than an integration one. Both builds accepted 279,000 input tokens in one call and answered a needle question from the middle of it. Both share the family tokenizer: the same mixed English-Chinese-code corpus counts identically on the GA build, the preview, and deepseek-v4-flash-0731, so token budgets port across the family unchanged. Both accept temperature, top_p, and top_k silently, and both accept a prefilled assistant turn, DeepSeek's documented prefix-completion feature.

Caching is identical down to the quantum: neither build cached a 512-token prefix at all, and both cached in exact 1,024-token pages above that (1,024, 2,048, 4,096), serving hits 4 seconds after the priming call, the same page size Flash uses. One more cost question worth closing: both builds return the full chain of thought in reasoning_content, and replaying it in the next turn is free. A follow-up turn billed the same 134 input tokens on GA (56 on the preview) whether the previous turn's reasoning was included or stripped, so unlike models that re-bill retained reasoning token for token, this family simply drops it.

The tool loop is a wash rather than a win: on a two-function agent loop (look up an incident, restart the service it names), GA deliberated more on the first hop (48 vs 34 reasoning tokens) and less on the second (18 vs 35), with both builds choosing the right tool 3/3. Given that agent workloads are where DeepSeek positions this release, the per-hop deliberation is closer to a wash than the headline efficiency numbers suggest; the refusal behavior above matters more for an agent that hits dead ends.

FAQ

How much cheaper is the GA build to run?

In tokens, 18-62% less reasoning per task, and 4x less under strict json_schema. In dollars, check the current card: DeepSeek changed V4-family pricing and added peak/off-peak billing (off-peak at half rate) on 2026-08-16, so the same token counts price differently by build and by hour.

Does the GA build save more on simple or complex tasks?

Simple. The reasoning cut runs 62% on a one-hop lookup, about 48% on a 2-hop word problem and JSON extraction, and only 18% on a five-step arithmetic chain. Deep multi-step work is where the two builds converge; shallow, high-volume traffic is where the switch pays.

Should I still use small thinking budgets on DeepSeek V4 Pro?

Not on the preview build. thinking_budget: 16 sent it into a repetition loop that consumed the full 8,192-token output window in 5 of 9 runs, a roughly 63x output bill for a request meant to be cheap; 64 tokens produced wrong answers. GA handled the same budget cleanly in 9 of 9 runs, so the small-budget lever is only safe on the dated build.

Can I trust strict JSON output from DeepSeek V4 Pro?

Only with thinking off, and only on the GA build. Across eight runs per configuration, schema-valid responses carried wrong numbers in 6 of 8 runs with thinking on (line-item counts of 45, 2026, -3864 for a three-item invoice), on both builds. GA with thinking: {"type": "disabled"} returned correct values 8 of 8; the preview stayed wrong 6 of 8 even with thinking off. Validate values, not just schemas.

Does DeepSeek V4 Pro refuse questions it cannot answer?

The preview build does, in 100-150 tokens. The GA build largely does not: on fabricated entities it either spends the entire output window thinking and returns an empty message, or, given a larger window, states a confident invented answer. Validate against your own sources rather than trusting a non-refusal, and treat empty completions as misses.

Measured 2026-08-17 through the Synthorai gateway, four days after the GA build appeared, with the preview build rerun in the same batch for every comparison: dial and off-switch matrix (7 effort values, 5 budgets, 2 off params, n=3), four-task reasoning sweep, strict-JSON structured output, a two-turn function-calling loop, a five-question fabricated-entity probe at two window sizes, a four-field strict-JSON value-integrity probe (n=8 per configuration), a reasoning-replay billing pair, implicit-cache pairs at two wait times and a cache-floor bracket, 279K-token context acceptance with a needle, a fixed-corpus tokenizer comparison across the V4 family, and sampling/prefill/n>1 acceptance probes. The runaway rate is from nine runs per build at max_tokens: 8192. Token counts are reported instead of dollars because DeepSeek changed V4-family pricing and introduced peak/off-peak billing on 2026-08-16, the day before this batch. Refusal behavior and the runaway were cross-checked on a second, independent request path. Rates and behavior may change; re-measure before relying on any single figure.

Gemini 3.7 Flash API Cost, Measured: Tasks Bill 2.5-8x Less Than 3.6

synthorai — Sat, 15 Aug 2026 21:07:39 +0000

Gemini 3.7 Flash bills the same tasks 2.5x to 8x cheaper than Gemini 3.6 Flash, and only half of that is the advertised price cut. The intro rate of $0.75 per million input and $3.75 per million output is 50% off 3.6, and on our meter the same 6.9K-token prompt billed exactly half ($0.00529 vs $0.01057). The other half is quieter: gemini-3.7-flash burned 26% to 77% fewer thinking tokens than 3.6 on the same four measured tasks. We probed the model on day two of its release: the pricing terms, the thinking dial (one position died), the compatibility breaks Google's docs warn about and the ones they undersell, plus cache, context, and tokenizer continuity.

TL;DR

Intro pricing $0.75/$3.75 runs through December 31, 2026, then $1.50/$7.50; the same prompt billed exactly half of 3.6 on our meter.
Default thinking dropped 26-77% versus 3.6 (144 vs 384 tokens on a 5-step task), so per-task bills fall 2.5-8x, not just 2x.
The off-switch is gone: every off spelling returns a 400 ("Thinking level is unsupported"); low is now the floor.
A two-turn tool loop halved 3.7's deliberation on the tool-result turn (32 vs 61 tokens); refusing an unanswerable question cost both models 500.

What does Gemini 3.7 Flash actually cost?

Half of 3.6's rate until December 31, 2026, and less than half of 3.6's tokens on every task we measured. Google prices the launch at $0.75 per million input and $3.75 per million output as an introductory rate; from January 1, 2027 it reverts to $1.50/$7.50, exactly 3.6's card. Cache reads are $0.075 per million, 10% of intro input. The meter agrees with the card: the same 6.9K-token prompt cost $0.0052875 on 3.7 and $0.0105675 on 3.6, a clean halving.

The larger effect is what the model no longer spends. Identical salted tasks, three runs each, median reasoning tokens and the resulting output bill:

Task	3.7 reasoning tokens	3.6 reasoning tokens	3.7 output cost	3.6 output cost	Task bill ratio
Simple lookup	73	98	$0.0003	$0.0008	2.7x cheaper
2-hop word problem	62	266	$0.0002	$0.0020	8.4x cheaper
5-step arithmetic	144	384	$0.0006	$0.0029	5.3x cheaper
JSON extraction	251	340	$0.0011	$0.0027	2.5x cheaper

Google's launch line is that 3.7 "thinks more diligently"; on our tasks diligence means fewer tokens, not more. All accuracy cells stayed 3/3 on both models. Multiply the halved rate by the halved-or-better thinking and the practical upgrade math is a 61-88% smaller bill per task, before caching. The one caveat sits in the calendar: budget against the January 1 reversion, the same pattern as Sonnet 5's intro pricing.

One task shape resists the trend, and it is the one nobody budgets for: refusing. Asked about five fabricated entities (a company, an institute, a town charter, an alloy, an award), both models declined all five, and both spent more thinking doing it than on any other task we measured, a median 511 reasoning tokens on 3.7 and 494 on 3.6. Saying "there is no record of this" cost 3.5x the thinking of solving the 5-step arithmetic chain. Retrieval-augmented pipelines that frequently hit missing data pay this tax on every miss, and it is the one place 3.7's efficiency gain disappears. It also puts a measured floor under a launch-week worry: reviewers flagged a higher hallucination rate for 3.7, but on fabricated-entity questions it hedged 5 of 5, identical to 3.6.

Which thinking controls survive on 3.7?

Three positions and no off, exactly as documented: low, medium (the default), high. What the docs do not spell out is that 3.6's escape hatches are dead. Every off spelling we sent, reasoning_effort: "none", "minimal", thinking_budget: 0, thinking: {"type": "disabled"}, enable_thinking: false, returned the same upstream 400: "Thinking level is unsupported: THINKING_LEVEL_MINIMAL". A second, independent request path refuses in plainer words, "Reasoning is mandatory for this endpoint and cannot be disabled", so this is the model, not one client's translation layer. On 3.6-flash, measured in the same batch, none and minimal still zero the burn. The Flash line has joined the pro tier's policy: thinking cannot be turned off, and 3.6 is now the last Flash that can.

The surviving dial, on the 5-step task (median of 3, all 3/3 correct):

Position	3.7 reasoning tokens	3.6 reasoning tokens
low	133	138
medium (default)	147	284
high	291	409

Two working notes. The default-is-medium claim checks out: unconfigured runs in the dial matrix burned 154, indistinguishable from its explicit medium arm at 147. And thinking_budget is deprecated for real: every nonzero value we sent (16 through 1,024) was accepted and burned the same 135-138 tokens, a no-op, where 3.6 still maps small budgets to off and larger ones to caps. If your 3.6 integration steers cost through budgets, the 3.7 translation is: pick a level, that is the whole interface. For the cross-vendor context on which dials do anything, see our thinking-controls matrix.

What does the agent pitch cost per step?

Tool selection is unchanged from 3.6; what got cheaper is the deliberation between steps. We ran a two-turn loop against two functions (look up an incident, then restart the service it names), three runs per model:

Loop step	3.7 reasoning / output tokens	3.6 reasoning / output tokens
Turn 1: pick the tool	85 / 110	85 / 110
Turn 2: act on the tool result	32 / 58	61 / 87

Both models chose get_incident first and restart_service after, 3/3, with identical prompt tokens. The gap is the second turn, where 3.7 thinks about half as much before committing to the next call. That is the step an agent repeats: a 20-step loop at these medians bills about $0.0043 of output on 3.7 against $0.0131 on 3.6, and the ratio holds after the January price reversion because it comes from tokens, not rates. This is the mechanism behind Google's agentic benchmark jump being pitched as a cost story: fewer deliberation tokens per hop, multiplied by many hops.

Where the model lands against rivals is benchmark-dependent and worth reading before switching a coding agent over: independent roundups put 3.7 Flash ahead on FrontierCode (43.6% vs Sonnet 5's 42.7%) and near 3x Sonnet 5 on AutomationBench, while GPT-5.6 Terra leads Terminal-bench and Sonnet 5 leads the desktop-task exam, per the same roundup and launch-week coverage. The cost floor is where 3.7 is unambiguous.

What breaks when you upgrade from 3.6?

Two hard 400s and several things that fail quieter than the docs suggest. Google's migration note says to remove temperature, top_p, top_k, candidate_count, and prefilled model turns. Measured:

Change	Documented as	Actually happens
Prefilled assistant turn	must remove	400: "Requests ending with a model turn are not supported" (3.6 rejects it too; the docs just say it out loud now)
`n` > 1	must remove	400 on the surface we measured
`temperature` / `top_p` / `top_k`	must remove	accepted silently, on 3.7 and 3.6 alike
`max_tokens` above the 64K output cap	64K limit	accepted with a 200 up to 200,000 on both request paths we tested; the cap applies silently at generation time
`thinking_budget`	replaced by levels	accepted, inert (constant 135-token burn at any value)

The asymmetry is worth noticing: the thinking dial validates loudly (an unsupported level is a clean 400 naming the value), while sampling params and output caps swallow whatever you send. If your client library sets temperature by default, nothing breaks today; if it prefills assistant turns for guided output, it broke before you upgraded.

Do cache, context, and tokenizer carry over?

Yes, with one slowdown. The implicit cache hits at the same shape as 3.6, 4,076 of 6,905 tokens cached on the second call of an identical prompt, cutting the call's cost 52%, and reads price at 10% of intro input. But the build lag grew: 3.6 served a hit 4 seconds after the priming call, 3.7 returned nothing at 4 seconds and hit at 30. Rapid-fire duplicate traffic lands before the cache is warm; the write-side economics are otherwise unchanged.

Context accepted 708,912 input tokens in one call and answered a needle question correctly, consistent with the 1M claim and with no long-context price tier on the rate card. The text tokenizer is byte-identical across four generations, gemini-3.1-pro-preview, 3.5-flash, 3.6, and 3.7 all counted the same mixed English-Chinese-code corpus at 50 tokens, so token budgets port unchanged. Image inputs still bill the flat 1,089 tokens at every size we measured across the Gemini line. Neither generation returns its reasoning text: reasoning_content came back empty on every call, so the thinking you pay for stays invisible on both, a continuing complaint in launch-week threads. Structured output holds: strict json_schema returned valid, correct JSON in 3/3 runs, and reasoning_effort: "low" drove the extraction's thinking to zero, the same single-step safe zone our dial matrix found everywhere.

FAQ

How much does the Gemini 3.7 Flash API cost?

$0.75 per million input tokens and $3.75 per million output as an introductory rate through December 31, 2026; from January 1, 2027 it reverts to $1.50/$7.50, the same card as Gemini 3.6 Flash. Cache reads are $0.075 per million. On identical prompts our meter billed 3.7 at exactly half of 3.6.

Can you turn off thinking on Gemini 3.7 Flash?

No. Every off spelling returns a 400 ("Thinking level is unsupported"); the dial is low, medium (default), high, and low still burned 133 reasoning tokens on our 5-step task. Gemini 3.6 Flash remains the newest Flash where reasoning_effort: "none" works, at 3.7's post-intro price.

Is Gemini 3.7 Flash actually cheaper than 3.6 per request?

Cheaper than the 50% headline suggests: it also thinks 26-77% less on the same tasks, so our measured per-task bills dropped 61-88% (a 2-hop word problem fell from $0.0020 to $0.0002). After December 31 the rate halves back to parity and only the thinking savings remain.

Will code written for Gemini 3.6 Flash run on 3.7?

Mostly. Sampling params (temperature, top_p, top_k) are still accepted despite the docs' removal list, and oversized max_tokens is silently clamped. The two hard breaks: prefilled assistant turns return a 400 (as they already did on 3.6), and every thinking off-switch now 400s, so cost steering must move from thinking_budget to the three levels.

Measured 2026-08-15 through the Synthorai gateway, two days after release: dial and off-switch matrix (7 effort values, 5 budgets, 2 off params, n=3, gemini-3.6-flash rerun in the same batch for every comparison), four-task reasoning-tax sweep, strict-JSON structured output, deprecated-parameter probes, output-cap and 708K-context acceptance, implicit-cache pairs at two wait times, a fixed-corpus tokenizer comparison across four Gemini generations, a two-turn function-calling loop (n=3 per model), and a five-question fabricated-entity refusal probe. Task dollar figures price measured completion tokens at each model's current output rate. Two claims were cross-checked on a second, independent request path: the oversized max_tokens acceptance and the absence of an off position. Intro pricing and dates are Google's published terms; behavior may change as the rollout matures.

How Many Tokens Is an Image? 15 APIs Measured, Same Icon 6 to 1,298

synthorai — Sat, 15 Aug 2026 06:01:54 +0000

The same 1024x1024 image costs 693 input tokens on GPT-5.6, 988 on Qwen 3.8 Max, 1,089 on Gemini, and 1,372 on Claude, and after each provider's input rate is applied one image runs $0.00005 to $0.0137 across 15 vision models, a gap that comes almost entirely from each model's input rate rather than from its image tokenizer. Five vendors publish a billing rule for images and our meter contradicts three of them. This is the image-side companion to our text-tokenizer study: the same locally generated PNGs sent to every vision model in the catalog, image cost read as prompt tokens with the image minus prompt tokens without it, across six sizes, five aspect ratios, three content types, three file formats, and one-to-four image stacks.

TL;DR

One 1024x1024 image: 693 tokens on GPT-5.6, 1,089 on Gemini, 1,372 on Claude; dollars run $0.00005 (qwen3-vl-flash) to $0.0137 (claude-fable-5).
Three schemes: patch formulas (Qwen: (side/32)²+2 exactly), tiles with a cap (GPT stops at 693), and flat fee (Gemini 1,089 at every size, thumbnails included).
Three documented rules fail measurement: Claude flagships downscale at ≈1,920px not 1,568, Gemini bills a flat 1,089 where docs say 258, Qwen's grid is 32px not 28.
File format and content never moved a single token: billing is geometry only.

How do the vendors say an image is billed?

Five of the seven families publish a rule, and three of those five rules do not survive measurement. This table is the article in miniature; every section after it is either the receipts for a disagreement or a cost behavior no documentation mentions at all:

Family	The docs say	We measured
OpenAI	32px patches with a per-model patch budget (docs)	shape matches: 6 tokens at 64px, hard 693-token cap
Anthropic	(w x h)/750, downscale past a 1,568px long edge (docs)	formula exact at 512-1,024px; the 1,568 ceiling holds only on Haiku, flagships keep billing to ≈1,920px
Google	images up to 384px bill 258 tokens, larger ones 258 per 768px tile (docs)	flat 1,089 at every size, a 64px icon included; the documented `media_resolution` dial has no working spelling on the surface we measured
Alibaba	one token per 28x28px, minimum 4 (docs)	a 32px grid fits to the token on qwen3-vl ((side/32)²+2), floor 66, cap 1,600px
Moonshot	dynamic tokens, no formula published, images up to 4K accepted (docs)	consistent: quadratic growth, no cap found through 3,072px (11,674 tokens)
MiniMax / ByteDance	no public formula we could find	quadratic with a 2,048px cap; flat 1,298 per image

The pattern in the right column is worth naming: the documented rules are all about geometry (patches, tiles, divisors), and geometry is measurable, so we measured it. Where the columns disagree, your budget spreadsheet inherits the error: a Claude flagship pipeline sized to the documented 1,568px ceiling under-budgets large images by about 45%, and a Gemini pipeline expecting 258-token thumbnails pays 4.2x that on every icon.

How many tokens does one image cost?

Between 6 and 5,486 in our matrix, depending on model and size, and the token count is only half the bill. Here is the same 1024x1024 PNG on every vision model in the catalog, with each model's input rate applied:

Model	Tokens (1024²)	≈ words of English text	× the model's per-1K-token input price	Input rate /1M	Dollars per image
qwen3-vl-flash	1,026	≈770	1.03x	$0.05	$0.00005
qwen3-vl-plus	1,026	≈770	1.03x	$0.20	$0.0002
minimax-m3	1,371	≈1,030	1.37x	$0.30	$0.0004
Dola-Seed-2.0-pro	1,298	≈970	1.30x	$0.50	$0.0006
gpt-5.6-luna	693	≈520	0.69x	$1.00	$0.0007
gemini-3.7-flash	1,089	≈820	1.09x	$0.75	$0.0008
claude-haiku-4-5	1,373	≈1,030	1.37x	$1.00	$0.0014
gemini-3.6-flash	1,089	≈820	1.09x	$1.50	$0.0016
qwen3.8-max	988	≈740	0.99x	$2.00	$0.0020
gemini-3.1-pro-preview	1,089	≈820	1.09x	$2.00	$0.0022
claude-sonnet-5	1,372	≈1,030	1.37x	$2.00 intro	$0.0027
kimi-k3	1,379	≈1,030	1.38x	$3.00	$0.0041
claude-opus-5	1,372	≈1,030	1.37x	$5.00	$0.0069
claude-fable-5	1,372	≈1,030	1.37x	$10.00	$0.0137

Three readings. First, the idiom bills literally: at 0.75 English words per token, one 1024px image occupies the context budget of a 520 to 1,030-word document, which is why image-heavy conversations exhaust context windows and budgets so much faster than text. Second, the dollar column is almost entirely the rate: token counts cluster within 2x (693 to 1,379), so on any model one image costs 0.69x to 1.38x of whatever it charges per 1,000 input tokens; the dollar column mostly restates each model's input rate. Third, families share tokenizers exactly: both qwen3-vl builds, both GPT-5.6 variants, all three Geminis, and all four Claude models returned identical or near-identical counts on every square image, the same one-tokenizer-per-family pattern we measured on text.

What decides how many tokens an image costs?

Five factors move the bill and three widely assumed ones do not. Everything else in this study is a deep dive on one row of this table:

Factor	Effect	Where it applies
Billing scheme	patch formula vs tiles-with-cap vs flat fee	scheme table below
Resolution (area)	the primary driver, roughly quadratic	every model except the flat-fee pair
Downscale cap	pixels past the ceiling are never billed	1,600px (Qwen), ≈1,920px (Claude flagships), 1,568px (Haiku), 693-token cap (GPT); Kimi has none
Aspect ratio	second-order: bounding grids charge strips more, long-edge caps refund extreme ones	GPT +84% at 3:1 then -11% at 8:1; Haiku -71% at 8:1
Image count	strictly additive, no volume discount	all 15 models
Content (photo vs text vs blank)	no effect	every model measured
File format (PNG/JPEG/WebP)	no effect	every model measured
File size in bytes	no effect	every model measured

The three non-factors are worth stating because both myths circulate: no API in this matrix reads your compression ratio or your image's complexity into the bill. Geometry in, tokens out.

What are the three billing schemes?

Patch formulas, tiles with a cap, and flat fees, and they price small images completely differently. We ran a six-step size ladder, 64px to 2,048px squares:

Model	64px	128px	256px	512px	1,024px	2,048px	Scheme
qwen3-vl (both)	66	66	66	258	1,026	2,502	patch: (side/32)²+2, min 8x8, cap 1,600px
qwen3.8-max	28	28	28	220	988	2,464	patch, smaller floor
gpt-5.6 (both)	6	21	78	309	693	693	tiles, hard cap 693
gemini (all three)	1,089	1,089	1,089	1,089	1,089	1,089	flat fee, any size
Dola-Seed-2.0-pro	1,298	1,298	1,298	1,298	1,298	1,298	flat fee, any size
kimi-k3	17	33	108	369	1,379	5,486	quadratic, no cap found
minimax-m3	18	27	102	363	1,371	5,186	quadratic, caps at 2,048px
claude (all four)	12	28	103	364	1,372	4,764	(w x h)/750, downscale cap

The Qwen formula is exact enough to budget from: a 1,024px square bills (1024/32)² + 2 = 1,026, verified to the token, with a padded minimum of 8x8 patches (66) and a 1,600px downscale ceiling (everything from 1,600 to 1,920px measured exactly 2,502). GPT tiles until 693 and never more: a 1,024px and a 2,048px image cost the same. The flat-fee pair is the trap for thumbnail traffic: Gemini bills a 64px icon 1,089 tokens, the same as a 4K screenshot after its downscale, and Seed bills 1,298. On the other end, Kimi K3 kept scaling past everyone's caps: a 3,072px square billed 11,674 tokens, the only model in the matrix we never saw downscale.

Where exactly do the downscale boundaries sit?

Every family except Kimi resizes large images before billing, and the boundaries sit where the meter says, not where the docs do. The Claude boundary deserves the detail, because it prices real money on claude-sonnet-5 and up: 1,568px bills 3,139 tokens, 1,728px bills 3,847, 1,920px bills 4,764, and it stops there (2,048px and 2,304px bill 4,764 too). The flagship trio keeps billing real pixels about 45% past the documented ceiling; claude-haiku-4-5 is the one model that behaves as written. If you control the upload pipeline, resize to each model's measured ceiling before encoding: pixels past the boundary either bill extra (Kimi) or are silently discarded (everyone else), so oversized uploads buy bandwidth and nothing more.

Does shape, content, or file format change the bill?

Two factors explain every shape effect we measured, and neither is file size: whether the model counts area or a bounding grid, and where its long-edge downscale threshold sits. Content and format, meanwhile, moved nothing anywhere: a 512px solid color, noise field, and text page billed identically on every model we measured, as did the same image at 243KB (PNG), 176KB (JPEG), and 174KB (WebP).

The factor probe holds area constant at one megapixel and stretches the shape:

Model	1:1	2:1	3:1	4:1	8:1 (2,896px long edge)
gpt-5.6 (both)	693	1,271	1,278	1,230	616
claude flagship trio	1,372	1,355	1,411	1,409	1,107
claude-haiku-4-5	1,373	1,356	1,068	788	396
minimax-m3	1,371	1,352	1,410	1,298	650
kimi-k3	1,379	1,361	1,417	1,415	1,361
qwen3-vl (both)	1,026	1,037	992	1,026	992
gemini (all three)	1,089	1,081	1,083	1,056	1,034
Dola-Seed-2.0-pro	1,298	1,277	1,304	1,298	1,315

Read the rows against the two factors. Qwen, Kimi, Gemini, and Seed are flat: pure area (or a fixed fee) with no shape term. GPT is the one bounding-grid biller, charging strips up to 84% more than the same pixels square, until 8:1 crosses its long-edge limit and the downscale refunds the premium: 616 tokens, cheaper than the square. The downscale-threshold family shows the same crossover at each model's own boundary: Haiku starts discounting at 3:1 (long edge 1,774 past its 1,568 ceiling: 1,068, then 788, then 396), the Claude flagships only at 8:1 (2,896 past their roughly 1,920px boundary: 1,107), MiniMax at 8:1 past its 2,048 (650). The practical read for wide-document and screenshot traffic: on GPT, split or downscale strips yourself; on Haiku, extreme shapes are quietly the cheapest Claude pixels sold.

What levers actually cut image input costs?

Three, in order of leverage. First, resize to the model's ceiling: every pixel past the downscale boundary is billed on Kimi (no cap) and wasted everywhere else. Second, detail: "low" on GPT: at 512px it changes nothing (309 either way), but at 2,048px it pins the image to 309 tokens against 693 at high or auto, a 55% cut, and it is the only per-request image dial we found on any model. Third, match the scheme to the workload: flat-fee models (Gemini, Seed) are the wrong place for thumbnail and icon traffic and the right place for consistently large scans; patch and tile models price small images honestly (a 64px icon is 6 tokens on GPT, 17 on Kimi).

Multi-image requests offer no discounts anywhere: stacking 1, 2, and 4 copies in one message billed strictly additively on all 15 models, each copy at its full single-image price. That arithmetic punishes flat-fee schemes hardest: four 256px thumbnails in one Gemini request cost 4,356 image tokens, the same four on qwen3-vl-flash cost 264.

FAQ

How many tokens is a 1024x1024 image?

Measured on the same PNG: 693 on GPT-5.6, 988 on Qwen 3.8 Max, 1,026 on qwen3-vl, 1,089 on Gemini, 1,298 on ByteDance Seed, 1,371 on MiniMax, 1,372 on Claude, 1,379 on Kimi K3. The 2x token spread matters less than the rate applied to it: the dollar cost runs $0.00005 (qwen3-vl-flash) to $0.0137 (claude-fable-5).

Does image file format or compression affect token cost?

No, on every model we measured: the same 512px image as PNG (243KB), JPEG (176KB), and WebP (174KB) billed identical tokens, and solid-color, noise, and text-dense content billed identically too. Billing is a function of pixel dimensions only; compress for bandwidth, not for tokens.

Does detail: "low" reduce image tokens?

On GPT-5.6, yes, and only above the small-image threshold: a 2,048px image billed 309 tokens at low against 693 at high or auto (55% less), while at 512px all three settings billed 309. No other model in the matrix exposed a working per-request image-cost dial.

Are images cheaper if I batch several into one request?

No: 1, 2, and 4 copies of the same image billed strictly additively on every model, each image at full price. Batching saves request overhead and latency, not image tokens, and on flat-fee models (Gemini, Seed) many small images in one request is the most expensive shape there is.

Measured 2026-08-13/14 through the Synthorai gateway on 17 models (15 vision, 2 text-only controls): locally generated PNGs at exact dimensions, image cost read as prompt tokens with image minus a salted same-text baseline; six-size ladder (64-2,048px), boundary probes to 3,072px, six 1MP aspect shapes (1:1 through 8:1 plus a 1:4 tower), three content types, PNG/JPEG/WebP, detail low/high/auto on GPT, 1/2/4-image stacks, and a code-word sight check on every model. Dollar figures multiply measured tokens by each model page's listed input rate on the measurement date (Sonnet 5 at its $2 introductory rate). Vision billing rules change without notice; re-run the ladder before relying on any single cell.

LLM Thinking Controls: What 13 Models Accept, Ignore, or Enforce

synthorai — Wed, 12 Aug 2026 16:28:09 +0000

The same thinking-control parameter means three different things depending on which model you send it to: thinking_budget: 16 burns exactly 16 reasoning tokens on Qwen 3.8 Max, GLM 5.2, and both DeepSeek V4 builds, is silently ignored on Kimi K3 and MiniMax M3, and is rejected with a 400 by GPT-5.6. We probed 13 models from nine vendors with every control spelling the OpenAI-compatible surface accepts, then measured what each dial position costs in reasoning tokens and what it breaks in accuracy, on the same four salted tasks, three runs per cell.

TL;DR

thinking: {"type": "disabled"} zeroes reasoning on 11 of 13 models; the two exceptions (Gemini pro, GPT-5.6) reject it.
Qwen, GLM, and DeepSeek burn thinking_budget: 16 as exactly 16; Kimi and MiniMax accept the field and change nothing: Kimi burned 8-97 against that cap, never 16.
Thinking off dropped 5-step arithmetic from 3/3 to 0-1/3 on eight models; DeepSeek V4 Pro and Claude kept 3/3 by writing steps into the visible answer.
JSON extraction scored 3/3 with thinking off on all 12 models that support off.

Which thinking controls does each API accept?

Three control families circulate on OpenAI-compatible surfaces, and no model honors all of them. reasoning_effort takes an enum (none through max), thinking_budget takes a token count, and thinking: {"type": "disabled"} (with its cousin enable_thinking: false) asks for a hard off. Here is the acceptance map we measured, one trivial salted question per cell:

Model	`reasoning_effort`	`thinking_budget`	`thinking: disabled`
kimi-k3	all 7 values	accepted, ignored	works (rt=0)
qwen3.8-max	all 7 values	exact (16 → 16; 0 rejected)	works
deepseek-v4-flash-0731	5 values, no off position	exact (16 → 16)	works
deepseek-v4-pro	5 values, no off position	exact	works
gpt-5.6-luna	5 of 7 values (`minimal`/`max` rejected upstream)	rejected (400)	rejected (400)
glm-5.2	all 7 values	exact (16 → 16)	works
gemini-3.6-flash	all values; `none`/`minimal` truly off	translated, coarse: 0-64 = off, 1,024 caps	works (ct=2)
gemini-3.1-pro-preview	`none`/`minimal` rejected (pro cannot disable)	lowers burn, floors high (64 → 204)	rejected (400)
minimax-m3	accepted; `none` ignored	accepted, ignored	works (ct=2)
Dola-Seed-2.0-pro	4 values; `minimal` truly off	0 = off; nonzero ignored (16 → 36-64)	works (rt=0)
claude-sonnet-5	`output_config.effort`	`budget_tokens` rejected (400)	works
claude-opus-5	`output_config.effort`	`budget_tokens` rejected (400)	works
claude-fable-5	`output_config.effort`	accepted	accepted

Two rows deserve a flag. Google splits its own line: the flash tier turns off cleanly while the pro tier rejects every off-spelling with a 400, matching Google's position that pro-class thinking cannot be disabled. And the two claude-fable-5 cells marked "accepted" differ from Anthropic's published contract for that model, which specifies thinking cannot be disabled; treat those cells as in flux.

Does "accepted" mean "enforced"?

No, and the gap between the two is billed. A 200 response tells you the parameter parsed, not that it reached the model. The test that separates the two is trivial: send a budget of 16 and read the meter.

Qwen, GLM, and both DeepSeek builds burned exactly 16. Kimi burned 8, 19, 79, and 97 against the same cap across four runs, never 16, and MiniMax behaved the same at 18-44, billed as usual, with nothing in the response hinting the cap had been dropped. Gemini translates budgets into its native control at coarse granularity: on flash, caps of 0 through 64 behaved as a full off while 1,024 allowed thinking (median 141 on our 5-step task); the pro tier cut its burn under a cap but floored around 200 against a requested 64 and cannot reach zero. GPT-5.6 and Claude sit at the honest end of the spectrum: numeric budgets are rejected with a 400 and you know where you stand immediately.

The practical rule: after setting any thinking control, read completion_tokens_details.reasoning_tokens on the next response and confirm the dial moved. A control that fails loud costs you one retry; a control that fails silent costs you the reasoning bill you thought you had capped, on every call.

Is there a universal off-switch?

thinking: {"type": "disabled"} is as close as it gets: it zeroed reasoning on 11 of the 13 models, spanning Kimi, Qwen, both DeepSeeks, GLM, Gemini flash, MiniMax, ByteDance's Seed line, and all three Claude models. Its lookalike enable_thinking: false matches it almost everywhere, with one silent exception: MiniMax accepts it and keeps thinking (31 reasoning tokens on our probe).

The two exceptions fail loud rather than silent: Gemini pro returns a 400 for every off-spelling (the tier cannot disable thinking), and GPT-5.6 rejects the field too. GPT-5.6 needs no off-switch in the same sense: gpt-5.6-luna burns zero reasoning tokens on simple lookups and extractions by default (the two-lever pattern of that family), and reasoning_effort: "none" pins that behavior for math-shaped inputs too.

What does turning thinking off cost in accuracy?

On a 5-step arithmetic chain, everything: eight models fell from 3/3 to 0/3 or 1/3 the moment thinking went off. On a 2-hop word problem, much less: most models held 3/3 with thinking off, and only Kimi and MiniMax dropped to 0/3 there, the same fragile off state the K3 study found on its launch build. The cliff sits where the step count exceeds what the model can carry in a single visible pass.

Model	5-step multi, thinking on	5-step multi, thinking off
kimi-k3	3/3 (52 rt)	1/3
qwen3.8-max	3/3 (96 rt)	0/3
deepseek-v4-flash-0731	3/3 (70 rt)	1/3
deepseek-v4-pro	3/3 (112 rt)	3/3 (answer grew to 142 tokens)
gpt-5.6-luna	3/3 (33 rt)	0/3
glm-5.2	3/3 (237 rt)	0/3
gemini-3.6-flash	3/3 (338 rt)	0/3
minimax-m3	3/3 (66 rt)	1/3
Dola-Seed-2.0-pro	3/3 (128 rt)	0/3
claude-sonnet-5	3/3 (70 out)	3/3 (output grew to 139 tokens)
claude-opus-5	3/3 (60 out)	3/3

The three survivors share one trick: with thinking disabled they write the intermediate steps into the visible answer. DeepSeek V4 Pro's median reply grew from 115 to 142 tokens, Sonnet 5's from 70 to 139. You stop paying for hidden reasoning and start paying for visible reasoning, which on most price cards is the same output rate; the "off" switch relabels the spend more than it removes it. The models that obediently answer in 1-4 tokens when off are the ones that fall off the cliff.

Where the cliff bites, a small budget buys the accuracy back: thinking_budget: 256 restored 3/3 on Qwen and both DeepSeek builds at 77-128 median reasoning tokens, the same floor-rescue pattern we measured on the DeepSeek retrain and on Qwen 3.8's hidden caps.

One cell in our matrix produced no numbers at all: claude-fable-5 returned stop_reason: "refusal" (category cyber) for our exact arithmetic phrasing in 12 of 12 runs, at every effort setting, while a semantically identical rephrase passed 12 of 12. Anthropic documents refusal as a first-class stop reason with an opt-in fallback mechanism; if fable-class models are in your rotation, handle that stop reason before you need it.

What do the effort levels actually buy?

A different curve at every vendor, and only Google's slopes upward. We ran each model's full accepted enum on the same 5-step task, three runs per position, and plotted the medians on one scale:

The lines sort into four shapes. Real throttles: both Geminis climb monotonically, 137 to 390 reasoning tokens on flash and 180 to 387 on pro (saturating at high), and low held 3/3 accuracy at a third to a half of the top positions' burn, which makes it the default worth pinning on Gemini pipelines. Flat lines: DeepSeek (78 at low, 54 at max, drifting down), Kimi (94 at minimal, 67 at max), and MiniMax (61-93 in no particular order) expose a multi-position dial where no position changes anything; DeepSeek's own model card quotes benchmarks at "max reasoning effort", a setting we already found indistinguishable from default. Caps that do not bind: Qwen's levels are budget ceilings, invisible on a task this size (85-156 with no trend) and real only on deep work, as measured in isolation. Non-monotonic: GLM's high burned 116 against low's 184 and max's 345; until that mapping settles, treat its middle positions as unordered. The two adaptive models barely need the dial at all: gpt-5.6-luna spans 32 to 41 tokens across its whole enum, and Opus 5's output_config.effort moved visible output only within noise (54-63 tokens, Sonnet 5 the same at 71-92), adaptive thinking making the real decision.

The operational rule falls out of the shapes: on Gemini, pick a level deliberately, because each step is real money. On Qwen, GLM, and DeepSeek, steer with thinking_budget (exact) and the off-switch, not the enum. Everywhere else the enum is decoration between off and default, and the only way to know which kind you have is the ladder test above: same task, every position, read the meter.

The task-shape lesson generalizes: on our single-step JSON extraction, thinking-on burned the highest counts in the whole matrix (377 reasoning tokens on GLM, 332 on Gemini flash) and bought nothing, because every one of the 12 models that can turn thinking off scored 3/3 on that task with it off. Structured extraction pays the largest pointless reasoning tax, and it is precisely the workload where the off-switch is safe.

Can you see what you paid for?

The billing meter is universal; the thinking itself is not. Six of the open-weight-family models return reasoning text in reasoning_content: GLM 5.2 and DeepSeek V4 Pro sent back what reads as the full chain (508 and 312 characters for 167 and 100 reasoning tokens), while Kimi, Qwen, DeepSeek Flash, MiniMax, and Seed returned shorter traces roughly in line with their small burns. GPT-5.6 and both Geminis return nothing: the reasoning tokens are billed and invisible. Claude returns thinking blocks whose content is omitted by default on the surface we measured, so you see that thinking happened, but not what it was.

That visibility split matters for debugging budget behavior: on the models that return nothing, reasoning_tokens in the usage details is your only instrument, which loops back to the rule above. Trust the meter, not the 200.

FAQ

How do I turn off thinking on an OpenAI-compatible API?

Send thinking: {"type": "disabled"}; in our 13-model matrix it zeroed reasoning on 11 (Kimi, Qwen, DeepSeek x2, GLM, Gemini flash, MiniMax, Seed, and the Claude family). Gemini pro cannot be turned off and returns a 400; GPT-5.6 rejects the field but barely thinks on simple tasks by default. Verify by reading reasoning_tokens on the next response.

Does thinking_budget: 0 disable thinking?

It depends on the model. On Gemini flash and ByteDance Seed, 0 behaves as a clean off; Qwen and DeepSeek reject 0 with a 400; Kimi and MiniMax accept any budget and ignore it. Where budgets are enforced to the token (Qwen, GLM, DeepSeek), the minimum useful value is a small positive number: 256 held 3/3 on Qwen and both DeepSeek builds on our 5-step task, while GLM wobbled to 2/3 at the same setting.

Is it safe to disable thinking for JSON extraction?

In our runs, yes: single-step extraction scored 3/3 with thinking off on every model that supports off, while thinking-on burned up to 377 reasoning tokens for the same output. The boundary is step count, not output format: multi-step tasks collapsed without thinking on 8 of 11 models. One caveat from our DeepSeek study: on the 0731 retrain, thinking-on actively corrupted strict-JSON values, so there the off-switch is a correctness fix too.

Which models enforce thinking budgets exactly?

Qwen 3.8 Max, GLM 5.2, and both DeepSeek V4 builds: request 16, the meter reads 16. Kimi K3 and MiniMax accept the same field and ignore it; Gemini translates it coarsely (small caps act as off on flash, pro floors high); GPT-5.6 and the Claude 5-generation models reject numeric budgets outright (Claude's budget_tokens returns a 400 pointing at adaptive thinking).

Measured 2026-08-11/12 through the Synthorai gateway: acceptance probes for reasoning_effort (7 values), thinking_budget (0/16/1024), enable_thinking, and thinking:{"type":"disabled"} across 13 models, re-verified hours before publication; then a 552-call tax matrix (four salted task shapes x 3 runs per arm, arms limited to each model's verified-working controls), a 183-call full-enum ladder on the 5-step task (the chart's data), top-up cells, and a per-model reasoning-visibility probe. Accuracy graded from raw answers; token medians of n=3; reasoning tokens read from completion_tokens_details.reasoning_tokens (Claude models report output tokens only). Prompts salted per call. Dial semantics and enums are the surface we measured on this date and may change; re-probe before relying on any single cell.

Web Search API and Web Fetch API: How They Work and What $0.01 Buys

synthorai — Mon, 10 Aug 2026 13:45:35 +0000

Every major server-side web search tool now charges the same fee, $0.01 per search, and that number is the least interesting thing on the bill. The searches we metered attached 1,500 to 3,100 tokens of results to each request, those tokens bill at whatever model you route to, and what happens on the follow-up turn depends on a protocol design difference most teams never look at: whether the endpoint carries the search evidence forward, or quietly drops it and leaves the model to search again on your dime. We measured Synthorai's synthorai:web_search and synthorai:web_fetch across four model families, then put the results against Anthropic's and OpenAI's published pricing for their own hosted search.

TL;DR

The per-search fee is $0.01 across Synthorai, Anthropic, and OpenAI; billing records itemize it at $0.0100.
One search injects 1,500-3,100 result tokens at the model's input rate: on cheap models the fee dominates, on premium ones tokens take up to half.
max_uses is a cap, not a quota: allowed 10, the model stopped at 3; fees scale with searches run.
Turn two splits by endpoint: /v1/messages replays results (≈3,900 tokens, no new fee); /v1/chat/completions drops them, so a follow-up needing evidence buys a fresh search.

Why do models need web search and web fetch at all?

Because a model's knowledge stops at its training cutoff, and most production questions do not. Prices, release notes, exchange rates, sports scores, the contents of the URL a user just pasted: none of it is in the weights, and confidently answering anyway is how hallucinated "current" facts ship to users. Web search closes the gap for discovery (the model does not know where the answer lives), web fetch closes it for reading (the model knows the exact page and needs its contents). You can build both yourself with a search API, a scraper, and a tool-calling loop, and teams do; the server-side versions exist because that loop is undifferentiated plumbing, and running it inside the provider removes the round-trips, the parsing code, and the operational surface, for a fee that turns out to be the same everywhere.

How does server-side web search actually work?

The loop runs inside one API request, and that is the whole trick. With client-side tools, the model asks your code to search, your code round-trips results back, and every hop re-sends the conversation. A server tool cuts your code out: you declare {"type": "synthorai:web_search"} in tools, the model emits a query, the gateway dispatches it to a dedicated search provider, injects the returned results into the model's context, and the model reads them and answers (or searches again, up to max_uses). One request in, one response out, citations and the bill included.

Using it is one line in an ordinary request; no loop code, no callbacks:

curl https://synthorai.io/v1/chat/completions \
  -H "Authorization: Bearer $SYNTHORAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash-0731",
    "messages": [{"role": "user", "content": "What is one AI news headline from this week? Name the source."}],
    "tools": [{"type": "synthorai:web_search", "max_uses": 1}]
  }'

Web fetch is the same declaration with a different type; put the URL in the message and the gateway retrieves the page:

"messages": [{"role": "user", "content": "Fetch https://example.com/pricing and summarize the tiers."}],
"tools": [{"type": "synthorai:web_fetch", "max_uses": 1}]

The diagram traces the search request above exactly as we measured it on deepseek-v4-flash-0731. The meter read 2,059 input tokens (a 30-token question plus roughly 2,000 tokens of injected results), 169 output tokens, and $0.0104 total: $0.01 of search fee and about $0.0004 of tokens. Two billing events, and only two: the fee when the search runs, and the result tokens at the model's ordinary input rate. A fetch bills identically, with the fetched page taking the place of the snippets.

The declaration itself costs nothing (a request that declares the tool but never searches bills zero fee), and the tool works on any model in the catalog, which is the structural difference from the first-party alternatives: Anthropic's hosted search serves Claude models, OpenAI's serves OpenAI models, and a gateway-level tool serves whatever you route to.

What does one search really cost?

$0.01 of fee plus a token bill that flips who dominates as the model gets pricier. We ran the same one-search news question three times each across four families, solved each model's exact token rates from tool-free baselines, and read the fee as the residual:

Model	Result tokens injected	Token cost	Fee residual	Fee share of total
deepseek-v4-flash-0731	≈2,020	$0.0003	$0.0101	97%
gpt-5.6-luna	≈1,500	$0.002	$0.0104	83%
qwen3.8-max	≈1,780-2,060	$0.005-0.012	$0.0112-0.0116	49-68%
claude-sonnet-5	≈2,700-3,090	$0.008-0.010	$0.0118-0.0121	54-59%

On the cheapest model the search fee is 97% of the call; on premium models the injected tokens claim half the bill. The fee itself is not an inference: our billing records itemize the tool charge as its own line at exactly $0.0100 per call, alongside a separate count of tool-injected tokens (3,454 of the 3,836 input tokens on one fetch call, to pick a real row). Measured all-in, a single search landed between $0.010 and $0.012 depending on the model carrying it; budget the top of that band and you will not be surprised. The practical read: pairing server-side search with a cheap model makes the search itself the product, and the model almost free.

Which three dials set your token bill?

Search count, result length, and page weight; everything else is rounding. Each dial is directly measurable and two of the three are directly yours to set.

Dial one: how many searches run. max_uses is a ceiling, not a target. On a five-vendor pricing question, allowing 1, 2, and 3 searches billed $0.0106, $0.0216, and $0.0319, fees stacking linearly at $0.01 per search actually run; allowing 5 and even 10 changed nothing, because the model stopped at three searches on its own. Unused headroom is free, exactly like the thinking budgets we measured on reasoning models. The default is 3 with a hard cap of 10, so the unconfigured worst case on a search-happy question is three fees plus three result loads; pin max_uses: 1 on single-fact routes.

Dial two: how long the results run. Injection size follows the breadth of the question, not chance. Across twelve single-search runs on deepseek-v4-flash-0731:

Query type	Injected result tokens (3 runs)
Single fact	1,650 (stable to within one token)
Technical doc lookup	1,920-1,950
Current news	1,790-1,990
Multi-entity comparison	2,060-2,410

Narrow questions pull compact result sets; comparative questions pull wide ones, about 45% heavier than a single fact. For budgeting, 2,000 tokens per search with a 20% band covers everything we saw, which at typical model rates prices the token side of one search between a twentieth and half of the $0.01 fee.

Dial three: how heavy the fetched page is. The fetch fee never moves; the page decides the rest. Our ladder, all through the same model:

Page	Injected tokens	All-in cost (DeepSeek)	Same fetch at a $2/1M model
Minimal test page	209	$0.0101	$0.0104
Lean homepage	1,421	$0.0103	$0.0128
Long-form guide	3,460	$0.0106	$0.0169
Data-heavy post	5,196	$0.0108	$0.0204
Long Wikipedia article	27,380	$0.0139	$0.0648

On a cheap model even the Wikipedia article is a cent and a half; route the same fetch through a $2-per-million model and that one page costs six and a half cents, five times the fee. One supporting fact for the budget sheet: the same page tokenizes differently per model family. The data-heavy post was 5,196 tokens on DeepSeek and 5,508 on qwen3.8-max, a 6% tokenizer premium consistent with our density measurements.

What happens to search results on the next turn?

The split is a design difference between the two API protocols themselves, and it decides your follow-up bill. The /v1/messages protocol defines an assistant turn as a list of content blocks, so server-tool activity is part of the turn's official record: the response carries server_tool_use, web_search_tool_result, then text, and the protocol expects those blocks back on the next turn. The evidence rides forward as input tokens by design. The /v1/chat/completions protocol defines an assistant message as a single content string with no slot for server-tool results, so injected results are absorbed server-side; only the visible answer enters history, and the evidence is gone.

We ran the same search-then-follow-up pair on claude-sonnet-5 through both surfaces. Via /v1/messages, the follow-up billed 3,911 input tokens and $0.0109, all of it token cost: no new search, because the model still had the results to draw on. Via /v1/chat/completions, the follow-up billed $0.0204, more than the replay, because the model, holding nothing but its own one-line summary, ran a fresh search: a new $0.01 fee plus a new result load. And the re-search is a choice, not a law: an earlier run of the same absorbed-style follow-up on deepseek-v4-flash-0731 answered straight from its summary at 460 input tokens and $0.00009. Absorption makes your follow-up bill bimodal: nearly free when the summary suffices, fee-plus-load when the model decides it needs the web again.

So the honest framing is not that one endpoint is cheaper; it is that they move the cost to different places. The block style pays a predictable token toll each turn and never re-buys evidence it already has; the absorbed style bets that follow-ups will not need the evidence, and pays a fresh fee whenever the bet loses. High-turn chat with shallow follow-ups suits /v1/chat/completions; research agents that interrogate sources suit /v1/messages, where the prompt-cache layering discipline applies to carried results like any other bulky context.

What do Anthropic and OpenAI charge, and where do their tools run?

The comparison is short: everyone charges $0.01 per search, and after that the bill is just your model's token price. Anthropic bills $10 per 1,000 searches with results as input tokens (and its web fetch is fee-free, tokens only); OpenAI bills the same $10 per 1,000 calls with token treatment varying by model tier. The fee is settled; the model's input rate on 1,500-3,100 injected tokens is the variable.

	Synthorai	Anthropic	OpenAI
Fee per search	$0.01	$0.01	$0.01
Result tokens	model's input rate	model's input rate	model's input rate (tiers vary)
Web fetch fee	$0.01	none	-
Models served	any in the catalog	Claude only	OpenAI only

The row that actually separates the three is not on the price sheet: it is where the tools run. First-party server tools are built for their vendors' own agent stacks and stay on first-party surfaces: Anthropic's docs state web search is not available on Amazon Bedrock, only basic search reaches Google Cloud, and Microsoft Foundry requires a hosted-on-Anthropic deployment; web fetch skips Bedrock and Google Cloud entirely. OpenAI's search is bound to its Responses API. Move your workload across a cloud boundary or a model family and the first-party tool stays behind, which is also why a gateway does not pass those tools through: they cannot follow the traffic. A gateway-level tool takes the opposite trade: one declaration that survives both a model swap and a platform move.

FAQ

How much does the web search API cost per request?

$0.01 per search executed, itemized as its own line in the billing records, plus the injected results billed as input tokens at your model's rate (1,500-3,100 tokens per search in our runs). A request that declares the tool but never searches bills no fee. Budget $0.010-0.012 all-in per search on typical models; on very cheap models the fee is 97% of the total.

Do search results get re-billed on later conversation turns?

Depends on the endpoint. Via /v1/messages the result blocks replay with history (about 3,900 input tokens per turn in our runs) and no new fee accrues. Via /v1/chat/completions results are absorbed after the turn; the follow-up is nearly free if the model answers from its summary ($0.00009 in one run) but incurs a fresh fee and result load if it re-searches ($0.0204 in another). Anthropic's first-party search documents the replay behavior as standard; there, results re-bill every turn.

How do I cap web search spend on a request?

max_uses. It is a hard ceiling the model cannot exceed (default 3, cap 10), fees scale only with searches actually executed, and unused headroom is free. For single-fact routes, max_uses: 1 bounds the worst case at one fee plus one result load.

When should I use web fetch instead of web search?

Fetch when you already know the URL and want the full page; search when you need discovery. Both cost $0.01 per use through the gateway, but a fetch injects the entire page (209 tokens for a minimal page up to 27,380 for a long article in our ladder) while a search injects snippets from several sources. For read-and-summarize pipelines on Claude models specifically, Anthropic's own fetch tool is fee-free and cheaper.

Measured 2026-08-10 through the Synthorai gateway using synthorai:web_search and synthorai:web_fetch on deepseek-v4-flash-0731, gpt-5.6-luna, qwen3.8-max, and claude-sonnet-5: per-model token rates solved from tool-free two-point baselines, search fees read as billed-cost residuals over token value (n=3 per model), a max_uses ladder, a query-type result-load sweep (4 types x 3 runs), follow-up-turn probes on both endpoints with identical questions, and a five-page fetch ladder (three of our own pages plus a minimal external page and a long article). Fees were confirmed against itemized billing records (per-call tool-charge and tool-injected-token lines), not inferred from totals alone. Anthropic and OpenAI figures are their published docs rates at publication time. The diagram's numbers are one unedited measured call. Fees and behavior may change; verify against your own usage records.