DEV Community: synthorai

Claude Opus 5.5 vs Opus 5: Same Answers, Half the Output Tokens

synthorai — Wed, 23 Sep 2026 16:13:33 +0000

Claude Opus 5.5 lists 20% below Claude Opus 5, at $4 per million input tokens and $20 per million output against $5 and $25. That 20% arrives whatever the model does, so the only interesting question is how much of a saving is left after you take it out. On 13 single-shot tasks at each model's default setting, Opus 5.5 billed 65% less, and at identical prices it is still 56% cheaper. In a multi-hop tool loop the subtraction is far less kind: 37% off the invoice, 22% once the rate card is held constant.

TL;DR

Across 468 graded calls, Opus 5.5 billed $0.0072 per task against Opus 5's $0.0204, both correct on every task.
Priced identically, Opus 5.5 still ran 56% cheaper on those tasks: it wrote 341 output tokens on average against 799.
In a four-question tool loop the gap shrinks to 22% at equal prices, because input tokens dominate and both models read the same files.
At max effort Opus 5.5 cost 3.3x its own default in that loop and solved nothing extra.
tool_choice set to any or a named tool now returns HTTP 400; Opus 5 accepts both.

Anthropic released Opus 5.5 on 2026-09-22 claiming it "costs 40% less to run than Opus 5" on typical workloads. Only the second half of that claim, fewer tokens per task, depends on what the model does, so that is the half worth measuring.

What actually changed in Claude Opus 5.5?

The price drop is the smaller change. The bigger one is that the thinking switch is gone. Adaptive thinking means the model decides how long to reason before it answers, and those reasoning tokens are billed as output whether or not the API shows them to you. On Opus 5.5 that mode is always on, and the only control left is effort, a request parameter with five levels from low to max that sets how much thinking the model is willing to spend.

Opus 5 accepted thinking: {"type": "disabled"}, which was the single setting that brought its bill to parity with Opus 4.8 in our Opus 5 measurements. That lever no longer exists.

	Opus 5	Opus 5.5
List price (in / out per MTok)	$5 / $25	$4 / $20
Thinking	adaptive, can be disabled at effort `high` or below	adaptive, always on
Default effort	`high`	`medium`
Forced tool use	accepted	400 error (measured)
Context window / max output	1M / 128K	1M / 128K
Knowledge cutoff	May 2026	June 2026
Released	2026-07-24	2026-09-22
Text between tool calls	`text` blocks	`thinking` blocks, empty at the default display setting
Safeguard categories	cybersecurity	cybersecurity plus biology, plus a reasoning-extraction decline

Two rows matter beyond the table. The default effort moved down a notch, from high to medium, so a request that sends no effort field is not running the same depth it ran on Opus 5. And the progress-update change is silent: short notes the model used to write between tool calls now arrive as thinking blocks whose text is empty unless you ask for it, so a user interface that streams those notes goes quiet with no error to catch.

What do the release benchmarks say?

Anthropic published Opus 5.5 ahead of Fable 5.1 on every coding and knowledge-work benchmark in its release table, and ahead of GPT-6 Astra on most of them. The numbers below are Anthropic's, run with adaptive thinking at max effort, with the Terminal-Bench rows at xhigh.

Benchmark	Opus 5.5	Fable 5.1	Opus 5	GPT-6 Astra
Terminal-Bench 4.0 (agentic coding)	66.4%	55.8%	52.3%	57.9%
FrontierCode v1.1	54.4%	50.3%	48.0%	53.3%
CursorBench 4.0	57.8%	51.8%	46.6%	not reported
GDPval-AA v2.1 (knowledge work, Elo)	1846	1735	1708	1542
AutomationBench (business workflows)	40.0%	31.4%	26.9%	41.4%
Terminal-Bench-Science 0.1	58.7%	52.6%	29.0%	64.6%
OSWorld 2.0 (computer use)	81.8%	80.7%	74.0%	not reported

Anthropic attaches an unusual caveat to its own table: at this level "benchmark margins have become a less reliable guide to real-world differences", and the gap to Fable 5.1 is narrower in practice than the scores suggest. Two footnotes matter before anyone quotes these numbers. The AutomationBench run was done by Zapier without fallback models, so every safeguard intervention counted as a failure, and the whole table ran with production safeguards on, with cybersecurity tasks handed to Opus 4.8 and biology tasks to Opus 5 when a classifier intervened.

None of that tells you what a task costs, so we measured that.

Does the cheaper claim hold on single-shot work?

Yes, and most of it is real efficiency rather than the rate card. Single-shot means one request, one answer, no tools: arithmetic and counting puzzles such as a 200-step iteration rule or a path count on a grid with blocked cells. We ran 13 of them, 3 repeats each, at all five effort levels plus the default, on Opus 5.5 and Opus 5, which is 468 graded calls. Every answer key was brute-forced in Python before the run, and every prompt carried a unique random string so that no layer between us and the model could answer from a cached duplicate. Cost per task is computed from Anthropic list prices over the tokens each call billed, thinking included, rather than read back from the response.

Effort	Opus 5.5 median output	Opus 5.5 $/task	Opus 5 median output	Opus 5 $/task
default	193	0.0072	448	0.0204
low	185	0.0060	439	0.0201
medium	218	0.0085	538	0.0200
high	223	0.0094	542	0.0206
xhigh	233	0.0111	525	0.0197
max	762	0.0240	532	0.0220

Accuracy was 100% for both models at default, and 97% or better at every other level, with the three misses scattered across different tasks and levels rather than concentrated at the bottom of the ladder. On this task set there is no accuracy cliff to find, so the entire difference is spend.

The ladder behaves differently on the two models. Opus 5's spend is flat, $0.0197 to $0.0220 from low to max, a 12% spread. Opus 5.5 spans 4x, from $0.0060 to $0.0240. Effort is a real dial on Opus 5.5 and close to a no-op on Opus 5, so a migration that copies the setting across can land somewhere quite different from where it started.

On the five hardest tasks the gap widens. Opus 5.5 at default cost $0.0113 per task against Opus 5's $0.0375, and the median output was 585 tokens against 1,145.

What happens in a tool loop, where agent bills accumulate?

The savings survive into a loop, at a smaller multiple. A tool loop is the honest test of Opus 5.5 against Opus 5. One turn is one request: the model asks for a tool, your code runs it, and you send the whole transcript back, so a conversation of five turns pays for the growing transcript five times. Turn count, not price per token, drives that bill. We gave both models three tools (list files, read file, search) over a small synthetic service and four questions whose answers require following a call chain across three or four files, then ran 12 runs per arm on the same surface with the same tools and prompt.

Arm	Solved	Median turns	Median tool calls	Median output tokens	$/run
Opus 5.5, default	12/12	4	5	427	0.0326
Opus 5.5, `low`	12/12	4.5	5	430	0.0330
Opus 5.5, `max`	12/12	5	12	3,124	0.1087
Opus 5, default	11/12	5	7	666	0.0519

Opus 5.5 at default cost 37% less per run than Opus 5 at default, with 12% fewer turns and 30% fewer output tokens. Per question solved the gap is wider, 43%, because Opus 5 missed one run. That is close to the vendor's "40% less to run" figure, and it is the number your invoice shows.

It is also the workload where the rate card does most of the work, for a reason worth understanding: a loop re-sends the transcript every turn, both models read the same files, and total tokens fell only 19% while output tokens fell 30%. Writing less helps least exactly where you spend most. The next section puts a number on that.

Dropping Opus 5.5 to low in the loop saved nothing: the model asked for one more turn on average, and the extra re-sent transcript ate the token savings. The effort dial that works cleanly on single-shot tasks stops paying in a loop, because turns, not depth, set the bill.

How much of the saving is just the price cut?

From a seventh of it to two fifths, depending on the shape of the work. The middle column below reprices Opus 5.5's measured tokens at Opus 5's rates, so it shows what Opus 5.5 saves by writing less, with the rate card held constant against Opus 5.

Workload	Billed difference	At identical prices	Output tokens
13 single-shot tasks, default	-65%	-56%	-57%
five hardest of those	-70%	-62%	-63%
multi-hop tool loop, default	-37%	-22%	-30%

The single-shot rows are a genuine efficiency gain: the price cut supplies only 14% of the saving, and the rest is the model writing less. The tool-loop row is the one to plan around, because it is the shape most agent traffic takes, and there the price cut supplies 41% of the headline.

Is a higher effort setting ever worth it?

Not on this workload, and the penalty is steep. Opus 5.5 at max cost $0.1087 per run in the tool loop, 3.3x its own default, and solved exactly the same 12 questions. It spent the extra budget on tool calls: a median of 12 against 5 at default, along with 3,124 output tokens against 427. On single-shot tasks max was the only level where it cost more than Opus 5, $0.0240 against $0.0220.

Reasoning is where that budget goes. Thinking was 98.4% of Opus 5.5's output tokens at default and 99.6% at max on these single-answer tasks. You are billed for all of it at the output rate, and at the default display setting you never see it. Anthropic's own guidance says to reserve max for frontier problems. The measured version: on tasks a model already solves, every step up the ladder is pure cost.

What breaks when you change the model ID?

Two request shapes that work on Opus 5 return 400 on Opus 5.5. Both are easy to hit in existing code.

Forced tool use is rejected. Any client that pins a tool call, which is the common way to get structured JSON out of a chat model, gets this:

tool_choice: type "tool" and "any" are not supported for this model.

That error arrives through an OpenAI-compatible client as well, where the same request is written as tool_choice: "required" or as a named function. auto and none still work, so the migration is to describe in the prompt when the tool applies, and to use strict tool schemas or structured outputs for schema-valid JSON.

Thinking cannot be turned off. thinking: {"type": "disabled"} and a manual budget_tokens both fail, and on an OpenAI-compatible surface the equivalent, reasoning_effort: "none", is refused too. The replacement is the effort parameter:

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=4096,
    messages=[{"role": "user", "content": "Summarize this incident report in five bullets."}],
    output_config={"effort": "low"},   # low | medium | high | xhigh | max; medium is the default
)

# Thinking is billed as output whether or not you can read it.
print(response.usage.input_tokens, response.usage.output_tokens)

low is the closest thing to the old thinking-off behaviour, and on our single-shot set it was both the cheapest level and accurate. It is not free, though: the model still thinks, and the thinking is still billed as output.

Do prompt budgets carry over?

Token budgets carry over exactly. The same four inputs counted the same on both models: 1,277 tokens against 1,275 for an English passage, 583 against 581 for Python, 482 against 480 for a JSON tool-argument blob, and 500 against 498 for Chinese prose. The constant two-token offset is request framing, not text, so a context budget or a chunking threshold tuned on Opus 5 needs no re-baselining on Opus 5.5.

When should you switch?

For cost, move to Opus 5.5 as soon as the two request shapes above are fixed. Even with the price cut taken out, equal accuracy at 56% less on single-shot work and 22% less in a tool loop is not a marginal difference, and default against default is the comparison most deployments will actually experience.

Treat effort as the thing to re-tune rather than carry over. The default moved from high to medium, the dial has four times the range it had on Opus 5, and the winning level depends on shape: low was cheapest and accurate on single-shot tasks, while in the tool loop the default beat both low and max.

FAQ

Is Claude Opus 5.5 really 40% cheaper than Opus 5?
On the invoice, close to it: Opus 5.5 cost 37% less per run than Opus 5 in our tool loop and 65% less per single-shot task. Twenty of those percentage points are the lower rate card, which arrives no matter how the model behaves. The efficiency left after that is 22% in the loop and 56% on single-shot tasks.

Can I still disable thinking on Opus 5.5?
No. thinking: {"type": "disabled"} and manual token budgets both return 400 on Opus 5.5. Use output_config.effort instead, where low is the cheapest setting. Thinking tokens are billed as output at every level.

What replaces forced tool use?
Keep tool_choice: {"type": "auto"} and make the tool's trigger explicit in the prompt, then use strict tool schemas or structured outputs when you need schema-valid JSON. On Opus 5.5 the any and named-tool forms are rejected with a 400 rather than downgraded silently, so this shows up as a failed request rather than a wrong answer.

Do I need to re-measure my prompt sizes?
No. Identical text billed identical token counts on Opus 5 and Opus 5.5 across prose, code, JSON, and Chinese, so context budgets carry over unchanged.

Is Opus 5.5 a replacement for Fable 5.1?
On Anthropic's own benchmark table, Opus 5.5 scores above Fable 5.1 on every listed benchmark at 40% of its token price. Anthropic still positions Fable 5.1 for demanding reasoning and long-horizon agentic work, and says the real-world gap is narrower than the scores suggest, so the honest answer is to re-run your own evals rather than read the table.

Measured 2026-09-23, one day after release, through a gateway to the Claude API and an OpenAI-compatible surface. 468 graded single-shot calls (13 tasks with locally brute-forced answer keys, 3 repeats, 6 effort settings, 2 models) plus 48 tool-loop runs (4 multi-hop questions, 3 repeats, 4 arms). Prompts salted; each model driven through the surface that carried its effort parameter; costs computed from Anthropic list prices rather than read from responses.

Jev vs Flash LLMs: 7x Cheaper on Workflows, Same Cost on One Label

synthorai — Tue, 22 Sep 2026 14:49:29 +0000

Whether TypeSafe's Jev is cheaper than a flash LLM depends on the shape of the job. Asking 12 questions about each support transcript, Jev cost 7 times less than GPT-5.6 Luna with thinking off and took 0.12 seconds after network time against 1.66. On a single 77-way label, it cost the same as Qwen3.8 Flash and GLM 5.3 Flash. Jev is a decision model: instead of writing text, it returns a choice, a score or a probability for each question you define. We ran it against five flash-tier chat models, at their cheapest and at their default thinking settings, on four tests built from public datasets. It was the fastest model on every test and the most accurate on none.

TL;DR

With 12 questions per case, flash LLMs cost 4.8 to 37 times more than Jev and took 11 to 27 times longer, at similar accuracy.
On one label per message, Jev cost the same as Qwen3.8 Flash and GLM 5.3 Flash with thinking off.
At default thinking, one label cost 1.5 to 26 times more on flash models than on Jev.
GPT-5.6 Luna was more accurate than Jev on every test except prompt injection.
Jev's answers at probability 0.99 or above were 98% correct.

What is Jev, and how is it different from an LLM?

Jev is TypeSafe's "System One" model: it reads a piece of text or JSON (the state) and answers typed questions about it, with no generated text to parse. There are three question types:

Question type	What you ask	What comes back
Noul (TypeSafe's name for yes/no)	a yes/no question	a probability from 0 to 1
Choice	pick one of up to 255 options you define	the option, every option's probability, a confidence
Score	rate on 2 to 10 ordered levels you define	a probability-weighted score, a confidence

A request asks its questions by name and gets one answer back per name. This one, in TypeSafe's documented format (API reference), asks three yes/no questions about a short support chat:

curl https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jev-latest",
    "state": {"transcript": "Customer: My card was declined twice today.\nAgent: One moment.\nCustomer: Also, how do I change my PIN?"},
    "questions": {
      "declined_card": {"type": "noul", "instructions": "In `transcript`, does the customer report a declined card payment?"},
      "change_pin":    {"type": "noul", "instructions": "In `transcript`, does the customer want to change a PIN?"},
      "lost_card":     {"type": "noul", "instructions": "In `transcript`, does the customer report a lost or stolen card?"}
    }
  }'

What came back when we ran it:

{
  "model": "jev-1.13.0",
  "answers": {
    "declined_card": {"type": "noul", "noul": 0.99},
    "change_pin":    {"type": "noul", "noul": 0.98},
    "lost_card":     {"type": "noul", "noul": 0.02}
  },
  "usage": {"input_tokens": 359, "output_tokens": 56}
}

Your code reads answers.<question>.noul and compares it with a threshold, the way it would compare any number. The model field names the version that answered; jev-latest moves when TypeSafe ships a new version, so pin jev-1.13.0 if you tune thresholds against it.

TypeSafe evaluates the questions in parallel. Jev 1.13 costs $0.042 per million input tokens and output is free (models), which is 3.6 times less per input token than Qwen3.8 Flash, 4.8 times less than GPT-5.6 Luna and 18 times less than Gemini 3.8 Flash. Input includes TypeSafe's own template, about 300 tokens a call, plus every question and option you define: the call above billed 359 input tokens, $0.000015, and its 56 output tokens were free. TypeSafe lists what Jev is weak at in its jaggedness notes: counting, arithmetic, date comparison, multi-step questions and long states full of irrelevant text.

How did we test it?

Four tests, each run once on 2026-09-21 and 2026-09-22 through Jev and through five flash chat models: GPT-5.6 Luna, Qwen3.8 Flash, GLM 5.3 Flash, DeepSeek V4.1 Flash and Gemini 3.8 Flash. Each chat model ran at its cheapest working thinking setting (off, or low where off is not accepted) and, on the three classification tests, again at the vendor default.

12 questions per case: 150 customer support transcripts, each assembled from one to six messages of the Banking77 dataset (PolyAI, CC BY 4.0), so the right answers are known. Every model answers 12 yes/no questions per transcript ("does the customer report a lost or stolen card?") with a probability each: Jev as 12 Noul questions in one call, the chat models as one JSON object. We reran 60 of them with a 3,000-token and a 12,000-token reference document (Wikipedia articles on banking) added in front of the transcript.
One label: 308 Banking77 messages, 4 from each of 77 intents. Jev gets one Choice with 77 options; the chat models get the same list and reply with the intent.
28 labels: 200 GoEmotions comments (Google, Apache 2.0), each with 28 possible emotions.
Prompt injection: the 116-row deepset prompt-injections test set (Apache 2.0), one yes/no question each.

Cost is each vendor's list price times the tokens billed. Latency comes from a separate run of 40 calls per model on a kept-open connection, with the network round trip subtracted; the speed section explains how. We also ran GPT-5.6 Terra and Seed 2.0 Mini on the three classification tests; they are in the tables but not the chart.

When is Jev cheaper than a flash LLM?

When one text gets many questions, when the text is long, or when the chat model would otherwise think. On a single short label against a cheap model with thinking off, Jev costs the same.

Three things move the multiple. First, the number of questions: a chat model has to write an answer for every question, and output is its expensive side (GPT-5.6 Luna charges $1.20 per million output tokens against $0.20 for input), while Jev's answers are free and each extra question adds about 17 input tokens. Second, thinking: at vendor defaults the chat models wrote a median of up to 161 output tokens per label where thinking off wrote 3 to 8, and the multiple against Jev rose with it. Third, input length: with a 12,000-token reference, the fixed template and the answers stop mattering, and the multiple settles near the ratio of input prices, 3.5 times for GLM 5.3 Flash up to 18.6 times for Gemini 3.8 Flash; DeepSeek V4.1 Flash sits at 7.1 times.

The one-label test is the case where Jev saves nothing against the cheapest models: 77 options add about 1,400 tokens to every call, so a one-line message bills about 1,690 input tokens on Jev against 440 on Qwen3.8 Flash.

Test, cost per 1,000 cases	Jev	Luna	Qwen3.8 Flash	GLM 5.3 Flash	DeepSeek V4.1 Flash	Gemini 3.8 Flash
12 questions per case	$0.027	$0.19	$0.13	$0.14	$0.27	$1.02
12 questions, 12,000-token reference	$0.43	$2.02	$1.57	$1.53	$3.06	$8.06
One label, cheapest setting	$0.071	$0.098	$0.069	$0.068	$0.16	$0.99
One label, vendor default	$0.071	$0.11	$0.19	$0.13	$0.61	$1.83

How much faster is Jev?

About 7 to 28 times once the network is taken out: Jev's median was 0.11 to 0.12 seconds and the chat models took 0.79 to 3.25 seconds.

We timed 40 calls per model and task, one at a time on a connection kept open, and subtracted the round trip measured on that same connection with a request that does no model work. What is left is the model's own time plus the path from our proxy to its provider, the same route for every model.

Seconds after network, median (95th percentile)	Jev	GPT-5.6 Luna	Qwen3.8 Flash	GLM 5.3 Flash	DeepSeek V4.1 Flash	Gemini 3.8 Flash
One label, cheapest setting	0.11 (0.21)	1.33 (1.80)	0.96 (1.68)	1.22 (3.06)	0.79 (1.35)	1.95 (3.76)
One label, vendor default	0.11 (0.21)	1.45 (5.29)	3.25 (21.5)	2.97 (7.32)	1.19 (10.4)	2.18 (8.22)
12 questions per case	0.12 (0.20)	1.66 (2.56)	3.21 (5.21)	3.13 (6.20)	1.33 (1.70)	2.14 (2.89)

The 95th percentile is where thinking hurts: at vendor defaults, one Qwen3.8 Flash call in twenty took over 21 seconds for a single label. Jev's slowest calls stayed near 0.2 seconds.

Adding questions did not slow Jev down: going from 1 to 60 yes/no questions about one support ticket moved its median by less than 0.1 seconds, and the 60-question call billed 1,371 input tokens, $0.000058. Every answer on the 20-question calls was right in all five runs.

How do these results compare with TypeSafe's own claims?

They agree on the direction and are smaller in size, because the comparisons differ. TypeSafe's launch post quotes up to 444.6 times cheaper and 193.6 times faster, measured against frontier models with reasoning on multi-step workflows, and calls those numbers "the higher end of real world gains" (launch post).

Its published workflow evals include GPT-5.6 Luna: Jev at 67.8% accuracy, $0.0004 and 0.4 seconds per case against Luna at 66.8%, $0.0033 and 12.9 seconds, about 8 times cheaper and 32 times faster. Our 12-question test measured Luna with thinking off at 7 times Jev's cost and 14 times its time after network, the same order of magnitude. Against the cheapest flash models on a single label, the cost gap closes to nothing.

Is Jev as accurate as a flash LLM?

Close on the workflow test, behind on the classification tests, and never first. On 12 questions per case, Jev's micro F1 (every yes and no across all questions, where 1.0 is perfect) sat inside the range of the chat models, and its probabilities ranked right from wrong answers about as well as the best of them.

12 questions per case (150 transcripts)	Micro F1 at 0.5	All 12 answers right	AUC
Jev 1.13	0.909	61.3%	0.990
GPT-5.6 Luna, thinking off	0.933	71.3%	0.973
GPT-5.6 Luna, default	0.931	71.3%	0.992
Gemini 3.8 Flash, low	0.919	66.7%	0.974
Qwen3.8 Flash, thinking off	0.913	62.7%	0.975
GLM 5.3 Flash, low	0.906	64.7%	0.975
DeepSeek V4.1 Flash, thinking off	0.896	60.0%	0.958

AUC measures how well the probabilities rank: 1.0 means every true "yes" got a higher probability than every "no". With the 12,000-token reference in front, Jev held its accuracy (F1 0.914 against 0.922 on the same 60 transcripts without it).

On the classification tests the chat models pulled ahead, most of all at their default thinking settings:

Model and setting	One label (Banking77)	28 labels (GoEmotions micro F1)	Prompt injection
Jev 1.13	80.2%	0.228	74.1%
GPT-5.6 Luna, thinking off / default	85.1% / 87.3%	0.301 / 0.342	69.6% / 72.2%
GPT-5.6 Terra, thinking off / default	83.4% / 85.4%	0.273 / 0.324	80.9% / 79.1%
Gemini 3.8 Flash, low / default	84.4% / 83.8%	0.373 / 0.372	81.9% / 82.8%
Qwen3.8 Flash, off / default	77.6% / 81.5%	0.286 / 0.338	81.9% / 80.2%
GLM 5.3 Flash, low / default	77.9% / 79.9%	0.255 / 0.310	81.9% / 81.9%
DeepSeek V4.1 Flash, off / default	77.3% / 81.8%	0.289 / 0.365	82.8% / 86.2%
Seed 2.0 Mini, thinking off	70.8%	0.245	66.4%

GPT-5.6 Luna and Terra answered 115 of the 116 injection texts and 199 of the 200 comments; one request each returned an error, and accuracy is over the answers returned. A reply that was not a yes or no counts as wrong. On emotions Jev said yes too often: an emotion it gave 0.80 to 0.95 was in the human label only 15% of the time. The emotion labels are sparse (annotators picked one or two per comment), which lowers every model's score, but every model got the same instruction. On prompt injection, Jev's 0.5 default was the wrong cut-off, which the next section covers.

Can you trust Jev's confidence?

Its probabilities separate right from wrong answers well, but they are not calibrated in the plain sense, so each task needs its own threshold. Calibrated means a 0.8 is right 80% of the time; on Banking77 Jev was right 98% of the time at 0.99 and above, and about half the time below 0.8.

Nearly half of the Banking77 messages (48%) got a top probability of 0.99 or more, so a pipeline that accepts those and sends the rest to a larger model keeps 98% accuracy on that share of its traffic. Prompt injection shows the other side. At the default 0.5 cut-off Jev flagged only half the attacks: the other half got probabilities between 0.03 and 0.5, low in absolute terms but mostly above the benign texts, whose median was 0.02. Ranked by probability, it separated attacks from benign text at an AUC of 0.984.

Threshold on the injection set	Jev accuracy	Attacks caught	False alarms (of 56 benign)
0.5 (default)	74.1%	50.0%	0
0.1	88.8%	80.0%	1
0.05	93.1%	91.7%	3

Those thresholds were chosen on the same 116 texts, so treat them as an upper bound. A chat model can give a probability too when its API returns token log-probabilities, the model's own probability for each token it writes; among the chat models here, Qwen3.8 Flash and Seed 2.0 Mini returned them in our runs, and Qwen3.8 Flash's probability of "yes" ranked the injection texts at an AUC of 0.977.

Should you use Jev or a flash LLM?

If your job is	Use	Why, from these tests
many yes/no or pick-one questions about the same text	Jev	4.8 to 37 times cheaper, 11 to 27 times faster after network, similar accuracy
a decision inside a loop that must answer in under a second	Jev	0.11 to 0.12 s after network
long documents with a few judgments each	Jev	cost settles near the input price ratio, 3.5 to 18.6 times less
one label per short message, at the lowest cost	Jev, Qwen3.8 Flash or GLM 5.3 Flash with thinking off	the same cost; the LLMs return no probability unless you read log-probabilities
the most accurate label	GPT-5.6 Luna at its default	87.3% on Banking77 at $0.11 per 1,000
numbers, dates, counting or generated text	a chat model	TypeSafe lists these as Jev's weak spots

The pattern that got the most out of Jev in our runs: decompose the decision into many narrow questions about the same text, act on answers above a threshold tuned on your own labeled data, and send the rest to a chat model.

FAQ

What is TypeSafe Jev?
Jev is a decision model from TypeSafe that answers typed questions (yes/no, pick one, rate on a scale) about a piece of text and returns probabilities instead of generated text. Jev 1.13 is the current version, served under the jev-latest alias, at $0.042 per million input tokens with free output.

Is Jev cheaper than an LLM?
It depends on the job. Asking 12 questions about each support transcript, GPT-5.6 Luna with thinking off cost 7 times more than Jev and Gemini 3.8 Flash 37 times more; on a single 77-way label, Qwen3.8 Flash and GLM 5.3 Flash with thinking off cost the same as Jev.

Is Jev more accurate than GPT-5.6 Luna?
Not in our tests: GPT-5.6 Luna with thinking off scored a micro F1 of 0.933 against Jev's 0.909 on 12 questions per case, and 85.1% against 80.2% on Banking77. Jev caught more prompt injections than Luna with thinking off at its default 0.5 cut-off.

Why does TypeSafe say Jev is 444 times cheaper?
That figure compares Jev with frontier models running reasoning on multi-step workflows, and TypeSafe calls it the high end. Against flash models we measured 4.8 to 37 times cheaper on a 12-question workflow and about the same cost on a single label.

Are Jev's probabilities calibrated?
Not in the plain sense. On Banking77 its answers were right 98% of the time at a probability of 0.99 or more but only about half the time below 0.8, and on prompt injections it gave half the attacks a probability under 0.5. Tune a threshold per task on your own labeled data.

GLM 5.3 API Cost: Always Thinking, 2.5x Cheaper per Answer Than 5.2

synthorai — Tue, 15 Sep 2026 15:22:14 +0000

GLM 5.3 costs the same $1.40 per million input tokens and $4.40 per million output as GLM 5.2, and it no longer lets you turn thinking off: every way of disabling it returns a 400 error, and the default effort is max. On 11 tasks with a checkable answer, run three times each, max was the only setting that got all 33 right, at $0.00468 per correct answer, 2.5x less than GLM 5.2 at max because GLM 5.3 reasons about half as much. low costs 63% less per correct answer and misses 5 of 33. GLM 5.3 Flash reached 31 of 33 at a tenth of the price. reasoning_effort, the parameter that sets how long a model thinks before answering, now decides accuracy as well as cost; we measured both builds against GLM 5.2 and DeepSeek V4.1 Flash in one batch.

TL;DR

GLM 5.3 and GLM 5.3 Flash reject thinking: disabled, reasoning_effort: none and medium with error 1210; only low, high and max (the default) work.
At max, GLM 5.3 scored 33 of 33 at $0.00468 per correct answer; GLM 5.2 at max cost $0.01173.
At low, GLM 5.3 scored 28 of 33 and GLM 5.3 Flash 24 of 33.
Both GLM 5.3 builds ignore a strict json_schema; json_object returned correct values 8 of 8 times.

What is GLM 5.3, and what does it cost?

A retrained GLM 5.2 at the same price, plus a smaller model at a tenth of it. GLM 5.3 "uses the same base model as GLM-5.2, with all improvements driven by post-training", per Z.ai's guide, and takes text only. GLM 5.3 Flash has "320B total parameters with 18B activated" (only 18B do work on each token, which is what makes it cheap) and accepts images, video and files. Both have a 1M-token context and 128K output. Prices from Z.ai's pricing page, per million tokens:

Model	Input	Cached input	Output
GLM 5.3	$1.40	$0.26	$4.40
GLM 5.3 Flash	$0.15	$0.03	$0.50
GLM 5.2	$1.40	$0.26	$4.40
DeepSeek V4.1 Flash	$0.30 peak	$0.006 peak	$1.20 peak

Z.ai's model card and Flash guide put the gain on agent and coding work:

Benchmark	What it tests	GLM 5.2	GLM 5.3	GLM 5.3 Flash
Terminal Bench 3.0	agent tasks in a terminal	4.6	28.3	not listed
DeepSWE v1.1	fixing real repositories	46.2	66.9	63.4
AutomationBench	workflow automation	26.2	48.2	48.8
CyberGym	security vulnerability tasks	77.2	84.5	not listed
HLE with tools	hard questions, tools allowed	54.7	62.5	not listed

What follows is what we could check ourselves: tasks with one verifiable answer, not agent benchmarks.

Can you still turn thinking off?

No. GLM 5.2 accepted thinking: {"type": "disabled"}. GLM 5.3 and GLM 5.3 Flash return HTTP 400 with error code 1210 ("this model always thinks, turning thinking off is not supported; use low, high or max") for thinking: disabled, enable_thinking: false, and reasoning_effort set to none, minimal, medium or xhigh. Leave reasoning_effort out and you get max. thinking_budget, a token cap on thinking that some providers honour, is accepted and ignored on GLM 5.3: every value from 0 to 1,024 produced about 101 reasoning tokens on the same question.

resp = client.chat.completions.create(
    model="glm-5.3",
    messages=[{"role": "user", "content": prompt}],
    reasoning_effort="high",   # "low" | "high" | "max"; omitted means "max"
    max_tokens=8192,           # GLM 5.3 Flash rejects anything above 131,072
)
reasoning_tokens = resp.usage.completion_tokens_details.reasoning_tokens  # billed as output
thinking_text = resp.choices[0].message.reasoning_content                # the thinking itself, as text

The old cheap path was not a good one anyway: with thinking off, GLM 5.2 answered 10 of our 33 tasks correctly, and DeepSeek V4.1 Flash 22.

Which effort setting gets the answers right?

On GLM 5.3, only max; on GLM 5.3 Flash, none of them. We ran 11 tasks whose answer is a single number we can check (sums of primes, digit counts, grid paths, coin combinations, a rule applied 40 times, the remainder of 7 to the power 222 divided by 1000, a base conversion, a knapsack, and counting four-digit numbers under three constraints) three times each. Reasoning tokens are the hidden thinking written before the answer, billed as output; cost per correct answer is total spend divided by correct answers, at list price:

Model	Effort	Correct	Reasoning tokens, median (max)	Cost per correct answer
GLM 5.3	`low`	28/33	143 (1,826)	$0.00173
GLM 5.3	`high`	29/33	226 (1,950)	$0.00197
GLM 5.3	`max` (default)	33/33	538 (7,733)	$0.00468
GLM 5.3 Flash	`low`	24/33	120 (467)	$0.00012
GLM 5.3 Flash	`high`	29/33	196 (1,582)	$0.00021
GLM 5.3 Flash	`max` (default)	31/33	419 (5,024)	$0.00040
GLM 5.2	`max`	33/33	1,129 (21,917)	$0.01173
DeepSeek V4.1 Flash	`low`	33/33	337 (6,797)	$0.00102
DeepSeek V4.1 Flash	`max`	33/33	386 (10,769)	$0.00138

The 2.5x against GLM 5.2 comes from reasoning volume: a median of 538 reasoning tokens against 1,129, and a longest run of 7,733 against 21,917. The lower settings save money by skipping checks. At low, GLM 5.3 answered the grid-path count as 216 twice (one blocked cell makes it 132) and missed the coin combinations, the knapsack and the base conversion once each. GLM 5.3 Flash at low failed the 40-step rule and the constrained count every time. DeepSeek V4.1 Flash, by contrast, scored 33 of 33 at every setting.

On GLM 5.3, keep the default for anything with arithmetic or several steps, and use low only where a wrong answer is cheap to catch. At max, GLM 5.3 costs 3.4x DeepSeek V4.1 Flash per correct answer; GLM 5.3 Flash costs under a third of it while missing 2 of 33.

Does it make things up when a question has no answer?

No, even with thinking forced on. We asked five questions about entities we made up, so the only right answer is "I do not know" (a share price for Verantis Dynamics, the melting point of Oridium-7, the winner of the 1987 Pan-Continental Robotics Prize), at the default effort with a 16,384-token cap:

Model	Declined	Invented an answer	Hit the cap, empty answer	Reasoning tokens	Cost per question
GLM 5.3	5/5	0/5	0/5	230 to 542	$0.0022
GLM 5.3 Flash	5/5	0/5	0/5	238 to 625	$0.0003
GLM 5.2	5/5	0/5	0/5	134 to 1,559	$0.0035
DeepSeek V4.1 Flash	1/5	3/5	1/5	7,021 to 16,384	$0.0139

Both GLM 5.3 builds said they had no information after a few hundred reasoning tokens. DeepSeek V4.1 Flash thought for 7,000 to 16,000 tokens and then usually invented something (Andrew Martin, the robot protagonist of Isaac Asimov's The Bicentennial Man, won); in its own study a day earlier, more runs hit the cap instead, but it rarely declined. For lookups that can legitimately come back empty, this is the biggest difference we measured between the two flash-tier models.

Does GLM 5.3 follow a JSON schema?

Not a strict one; use json_object. With response_format set to a strict json_schema for an invoice extraction, GLM 5.3 and GLM 5.3 Flash returned HTTP 200 and ignored the schema in 16 of 16 runs: the JSON came wrapped in a markdown code fence with keys the schema did not ask for (invoice_date, reference), so none parsed as the requested object. Z.ai's API reference lists only text and json_object; the trap is that the schema is accepted silently.

With {"type": "json_object"} and the fields named in the prompt, both builds returned every value right in 8 of 8 runs, as did GLM 5.2 and DeepSeek V4.1 Flash. Since thinking stays on, extraction costs a median of 240 output tokens on GLM 5.3 and 140 on Flash, against 25 on V4.1 Flash with thinking off. Validate the result against your schema yourself.

What does an image cost on GLM 5.3 Flash?

It grows with pixel area and does not cap by 2048x2048, so large images cost more than on DeepSeek V4.1 Flash despite the lower rate. We sent generated PNGs to GLM 5.3 Flash (GLM 5.3 is text only) and subtracted the text-only prompt:

Image	GLM 5.3 Flash tokens	Cost at $0.15/M	DeepSeek V4.1 Flash tokens	Cost at $0.30/M
512x512	363	$0.000054	184	$0.000055
1024x1024	1,371	$0.000206	652	$0.000196
2048x2048	5,478	$0.000822	994	$0.000298

Above 512 pixels, GLM 5.3 Flash spends about one token per 766 pixels; detail, image content and file format did not change the count. The DeepSeek V4.1 Flash numbers come from its study the day before. Downscale screenshots and document pages before sending: at 2048x2048, GLM 5.3 Flash costs 2.8x more per image.

What else changed from GLM 5.2?

Speed, and little of the plumbing. Streamed in the same window at low, GLM 5.3 generated 59 to 64 output tokens per second, GLM 5.3 Flash 70 to 82, GLM 5.2 51 to 55, and DeepSeek V4.1 Flash 124 to 161. In a two-turn function-calling loop every model made one call per turn. The three GLM builds count identical tokens on English, Chinese and Python text (737, 484 and 488), so token budgets carry over.

The repeated opening of a prompt is billed at the cached rate, counted in 64-token steps (a 1,153-token prompt read 1,024 from cache). Both builds accept prompts close to the 1M-token limit, although the Flash model card mentions 300,000 tokens: a 990,000-token prompt with one value hidden near its start returned that value (we did not test recall near the end), and one of about 1.07M tokens got a 400, Prompt exceeds max length, instead of being cut short. One upgrade break: GLM 5.3 Flash rejects max_tokens above 131,072.

How Synthorai handles it

GLM 5.3, GLM 5.3 Flash and GLM 5.2 are glm-5.3, glm-5.3-flash and glm-5.2 on the gateway, at Z.ai's list prices, on /v1/chat/completions and /v1/responses. The 1210 error passes through unchanged, so a client still sending its GLM 5.2 off switch fails loudly instead of silently thinking at max. Reasoning text comes back in reasoning_content; images go to GLM 5.3 Flash as image_url content parts.

FAQ

Can you disable thinking on GLM 5.3?

No. GLM 5.3 and GLM 5.3 Flash return a 400 with error 1210 for every off switch; only low, high and max are accepted, and max is the default. On our suite low cost 63% less per correct answer but got 28 of 33 right instead of 33.

Is GLM 5.3 cheaper than GLM 5.2?

Per token, no: both cost $1.40 input and $4.40 output. Per correct answer, yes: at max, GLM 5.3 cost $0.00468 against GLM 5.2's $0.01173, because it reasons about half as much.

Can GLM 5.3 Flash replace GLM 5.3?

Where an occasional wrong number gets caught downstream, yes, at a tenth of the price: GLM 5.3 Flash at max got 31 of 33 right at $0.00040 per correct answer, against 33 of 33 at $0.00468 on GLM 5.3. Avoid Flash at low for multi-step arithmetic, where it got 24 of 33.

DeepSeek V4.1 Flash API Cost: Matches V4 Pro at 3-6x Less per Answer

synthorai — Mon, 14 Sep 2026 16:59:11 +0000

DeepSeek V4.1 Flash costs $0.30 per million input tokens and $1.20 per million output at peak, half that off-peak, and on 11 tasks with a checkable answer, run three times each, it got 33 of 33 right, the same score as V4 Pro at 3 to 6x less per correct answer and 2.9x the output speed. Against the V4 Flash it retires, it is 26% cheaper per correct answer, 1.5x faster, and the first Flash that reads images. It also ships with two behaviours to design around: with thinking off it answers multi-step arithmetic with a single wrong token, and with thinking on it spends the whole output window, 16,384 tokens in three of five runs, on questions about things that do not exist. We measured all three builds, deepseek-v4.1-flash and deepseek-v4-pro-0813 in one batch (same hour, same gateway) four days after release and deepseek-v4-flash-0731 the same evening.

TL;DR

V4.1 Flash is $0.30/$1.20 per million at peak, half off-peak; V4 Pro is $1.32/$3.96.
All three builds scored 33 of 33 with thinking on; per correct answer V4.1 Flash cost $0.00097 to $0.00149, 3 to 6x under V4 Pro and 26% under V4 Flash.
Thinking off drops all three to 21 of 33; V4.1 Flash then answers with one wrong token.
On five fabricated entities V4.1 Flash with thinking on hit the 16,384-token cap three times and invented twice; thinking off declined all five.

What changed on September 10, and what does V4.1 Flash cost?

A new architecture at a new price, and a retirement. V4.1 Flash is "the smallest model in our new architecture family": 552B parameters in a causal encoder-decoder (one 20-layer stack reads the prompt, another writes the answer), 8B active per token for input and 16B for output, with native image input, a 1M-token context and a 384K output cap, under an MIT license. DeepSeek's pricing page lists it at $0.30 per million input tokens and $1.20 per million output at peak (01:00 to 04:00 and 06:00 to 10:00 UTC, Monday to Friday), $0.15 and $0.60 off-peak, with cache hits at $0.006 and $0.003. That sits between the two cards V4 Flash had, a flat $0.14/$0.28 at its July launch and $0.44/$1.32 peak from 2026-08-16, and well under V4 Pro at $1.32/$3.96 peak.

V4 Flash and the Vision Exp build are retired, and their names route to V4.1 Flash at the Flash price. DeepSeek had also planned to route V4 Pro traffic to V4.1 Flash from September 14; the change log dropped that plan the same day, "in response to user demand", so Pro stays available at Pro prices.

The plumbing changed less than the price. The tokenizer is identical across the three builds (729, 452 and 528 tokens on the same English, Chinese and Python corpora), so token budgets port; cache pages halved from 1,024 tokens to 512, so a 549-token prompt now caches 512 of them; a 902K-token prompt with one planted value returned it, and a prompt past the 1M window returned a 400 rather than a silent truncation.

DeepSeek's own numbers, from the model card, put the new Flash above the old one everywhere and above Pro on agent and coding work but not on knowledge:

Benchmark	What it tests	V4 Flash	V4.1 Flash	V4 Pro
GPQA Diamond	graduate-level science questions	89.9	90.9	92.4
HLE	hard cross-discipline questions	37.8	36.8	42.7
Codeforces	competitive programming rating	3289	3471	3348
MathArena Apex	competition mathematics	58.6	65.6	65.3
Terminal-Bench 2.1	agent tasks in a terminal	82.7	90.6	87.9
Terminal-Bench 4.0	harder terminal agent tasks	7.0	31.2	12.4
DeepSWE v1.1	fixing real repositories	54.4	74.2	62.7
CyberGym	security vulnerability tasks	76.7	88.1	83.3
AutomationBench	workflow automation	37.7	54.8	43.2

The rest of this post is what we could check ourselves, which is the first half of that table (answers that can be verified), not the knowledge rows.

Does V4.1 Flash really match V4 Pro, and beat V4 Flash?

On tasks with a checkable answer all three builds score the same, so the differences are price, speed and how they fail. We ran 11 tasks whose answer is a single number we can check (sums of primes, digit counts, grid paths, coin combinations, a rule applied 40 times, 7 to the power 222 divided by 1000 with the remainder kept, a base conversion, a knapsack, and counting four-digit numbers under three constraints) three times each at reasoning_effort low, high and max, and with thinking off. Reasoning tokens are the hidden thinking the model writes before its answer, billed as output; cost per correct answer is total spend divided by correct answers, at each model's peak card (V4 Flash at the $0.44/$1.32 card it was sold on until retirement); off-peak is half:

Model	Effort	Correct	Reasoning tokens, median (max)	Cost per correct answer, peak
V4 Flash 0731	`low`	33/33	492 (6,444)	$0.00131
V4 Flash 0731	`high` (default)	33/33	433 (9,172)	$0.00163
V4 Flash 0731	`max`	33/33	453 (24,010)	$0.00343
V4 Flash 0731	thinking off	21/33	0	$0.00083
V4.1 Flash	`low`	33/33	316 (6,153)	$0.00097
V4.1 Flash	`high` (default)	33/33	391 (13,300)	$0.00149
V4.1 Flash	`max`	33/33	391 (11,037)	$0.00129
V4.1 Flash	thinking off	21/33	0	$0.00050
V4 Pro	`low`	33/33	309 (6,398)	$0.00286
V4 Pro	`high` (default)	33/33	548 (18,247)	$0.00768
V4 Pro	`max`	33/33	644 (15,920)	$0.00825
V4 Pro	thinking off	21/33	0	$0.00232

With thinking on, no build missed a task at any effort setting, so this suite cannot separate them on accuracy; it separates them on the bill (V4.1 Flash against Pro: 2.9x at low, 5.2x at the default high, 6.4x at max; against V4 Flash: 26% less at low) and on speed. Streamed with thinking off (so the clock measures answer generation, not thinking), V4.1 Flash decoded at 121 to 134 output tokens per second, with 1.4 s from request to first token; V4 Flash at 86 to 94 (2.3 s to first token); Pro at 46 to 49 (1.9 s). In a two-turn function-calling loop every build made exactly one call per turn; V4.1 Flash reasoned 27 and 13 tokens on the two legs, V4 Flash 30 and 19, Pro 61 and 29, at $0.00019, $0.00030 and $0.00103 per leg. The knowledge gap in DeepSeek's table (HLE, GPQA) is the one thing this suite does not exercise.

What does the effort dial do, and can you turn thinking off?

The dial moves the bill on a few tasks and the answer on none, and turning thinking off costs 12 of 33 answers on every build. V4.1 Flash accepts reasoning_effort as low, high and max (DeepSeek maps minimal to low and medium and xhigh to high); on nine of the eleven tasks the three settings are within a few hundred reasoning tokens of each other, and only the counting task (four-digit numbers under three constraints) opens up: 5,650 at low, 9,714 at high, 6,763 at max, all correct. Every thinking_budget value from 0 to 1,024 was accepted and ignored. The model card's "continuously controllable reasoning effort setting (integer 1-100)" is not on the API; integers are rejected. V4 Pro and V4 Flash behave the same way at their own counts, and the counting task is where max gets expensive on the older builds: 12,729 to 15,920 reasoning tokens on Pro, 17,866 to 24,010 on V4 Flash, for the same right answer V4.1 Flash reached in 6,763.

The two controls, and where the measured numbers come from in the response:

resp = client.chat.completions.create(
    model="deepseek-v4.1-flash",
    messages=[{"role": "user", "content": prompt}],
    reasoning_effort="low",                    # "low" | "high" | "max"; default "high"
    extra_body={"thinking": {"type": "enabled"}},  # {"type": "disabled"} turns it off
    max_tokens=4096,                           # always set one: see the next section
)
usage = resp.usage
reasoning_tokens = usage.completion_tokens_details.reasoning_tokens  # billed as output
thinking_text = resp.choices[0].message.reasoning_content            # returned on every thinking call

Thinking off is the cheap dial and the dangerous one, and each build loses a different set of tasks:

Thinking off, 11 tasks x 3	V4 Flash 0731	V4.1 Flash	V4 Pro
Correct	21/33	21/33	21/33
Lost outright (0 of 3)	knapsack, sum of primes, train schedule	coin combinations, 40-step rule, knapsack	knapsack, counting task
How it fails	a wrong number in the right format (`17, 1043`; `10:40`)	one wrong token (`83` for a five-step chain whose answer is 168)	shows its working, then a wrong answer

The five-step chain from our standard shapes is the clearest case: with thinking off V4.1 Flash answered it 1 of 3 (0 of 3 in a second set of runs), V4 Flash 2 of 3, Pro 3 of 3, because Pro writes the steps before the answer and the Flash builds do not. Two smaller facts for the budget: thinking mode adds a hidden template prefix billed as input, 26 prompt tokens on V4.1 Flash and 79 on V4 Flash and Pro (identical text, 729 tokens with thinking off on all three), and the reasoning_content you send back on the next turn is not re-billed (81 prompt tokens with and without it on V4.1 Flash, 134 on Pro).

What happens when you ask about something that does not exist?

V4.1 Flash with thinking on either exhausts your output cap or invents an answer; with thinking off it declines, and it declines more reliably than either other build. We asked five questions about entities we made up, so the only right answer is "I do not know" (a share price for "Verantis Dynamics", the headcount of the "Kessler-Fanning Institute", the melting point of "Oridium-7", the winner of a "1987 Pan-Continental Robotics Prize") with a 16,384-token cap:

Model, setting	Declined	Invented an answer	Hit the 16,384-token cap, no answer	Reasoning tokens
V4.1 Flash, thinking on	0/5	2/5 ("Professor Frink from The Simpsons won the 1987 prize"; "Oridium-7 has a melting point of 1815 °C")	3/5	2,610; 11,905; 16,384 x3
V4.1 Flash, thinking off	5/5	0/5	0/5	0 (69 to 248 output tokens)
V4 Flash 0731, thinking on	1/5	3/5 ("0 employees"; "Oridium-7 is approximately 1065 °C"; "Montblanc, the Swiss robot from the manga Pluto, won")	1/5	4,080 to 16,384
V4 Flash 0731, thinking off	4/5	1/5 ("Oridium-7 has a melting point of…")	0/5	0
V4 Pro, thinking on	1/5	3/5 ("Andrew Martin won"; "0 employees"; a melting range of 899 to 949 °C)	1/5	754 to 16,384
V4 Pro, thinking off	3/5	2/5 (`According to reference 844476, the Kessler-Fanning Institute had 247…`)	0/5	0

Three of the five V4.1 Flash runs with thinking on cost $0.0197 each at peak and returned an empty message; with the 2,048-token cap we use elsewhere, all five did, and so did four of five on Pro. If a lookup can legitimately have no answer, run it with thinking off, or cap max_tokens and treat an empty message as "unknown". Pro's thinking-off failure is different and worth a regex of its own: it writes Let me check that reference for you. According to reference and then an invented number.

Is the V4 Flash JSON corruption fixed?

Under json_object there is nothing to fix on any build; under strict json_schema we could not test it. V4 Flash 0731 shipped with a defect: thinking on plus a strict json_schema corrupted integer fields in 8 of 13 runs. On V4.1 Flash a strict json_schema returned 400 on our path, and DeepSeek's JSON guide documents only {"type": "json_object"}.

Under json_object, the same invoice (vendor, date, total, line items) came back with every value right in 8 of 8 runs with thinking on and 8 of 8 with it off, on all three builds. V4 Pro under strict json_schema still corrupts with thinking on: 0 of 8 runs had the right line-item count (45, 12, 47, 1, 2026), 8 of 8 with thinking off. For extraction, json_object with thinking off was right every time on every build, and on V4.1 Flash it costs 25 output tokens.

What does an image cost on V4.1 Flash?

184 input tokens for anything up to 512x512, 652 at one megapixel, 994 at four. Image input is new to the Flash line (the retired Vision Exp build was a separate model, and V4 Flash 0731 takes text only), so we sent generated PNGs to V4.1 Flash and subtracted the text-only prompt:

Image	Image tokens	Cost at peak
64x64, 128x128, 256x256, 512x512	184	$0.000055
1024x1024	652	$0.000196
2048x512 (same area as 1024x1024)	652	$0.000196
512x2048	688	$0.000206
2048x2048	994	$0.000298

The floor matches the vision guide: "images with a total pixel count below roughly 544x544 are scaled up", larger ones are scaled down "to roughly that of a 1300x1300 image", and 1,024 tokens is the documented ceiling per image. Content (solid, noise, rendered text) and format (PNG, JPEG, WebP; 174 to 243 KB) did not change the count, so you are billed by geometry, not bytes. A thousand 1-megapixel images cost $0.20 in image tokens at peak, before the question and the answer.

How Synthorai handles it

V4.1 Flash is deepseek-v4.1-flash on the gateway (DeepSeek's own id is deepseek-flash), billed at $0.30 input and $1.20 output per million around the clock, with cache reads at $0.03 per million and no off-peak rate; every number in this post is reported at DeepSeek's list card so you can apply your own hours. V4 Flash 0731 and V4 Pro 0813 stay available as deepseek-v4-flash-0731 and deepseek-v4-pro-0813, at their own cards. Both /v1/chat/completions and /v1/responses carry the model, thinking: {"type": "disabled"} is the off switch that works on this path, and the reasoning text comes back in reasoning_content on every thinking call. Image input goes through as image_url content parts.

FAQ

Is DeepSeek V4.1 Flash cheaper than V4 Flash?

Cheaper than the August card it replaces, not than the July launch card. V4.1 Flash is $0.30/$1.20 per million at peak and $0.15/$0.60 off-peak; V4 Flash was $0.44/$1.32 peak from mid-August and $0.14/$0.28 before that. Per correct answer on our suite, V4.1 Flash at low cost $0.00097 against V4 Flash's $0.00131 at its last card.

Can you turn thinking off on DeepSeek V4.1 Flash?

Yes, with thinking: {"type": "disabled"} (DeepSeek also documents reasoning_effort: "none"), and it costs accuracy on multi-step work: V4.1 Flash fell from 33 of 33 to 21 of 33 on our suite and answered a five-step chain with one wrong token. Keep it on for arithmetic and planning; turn it off for extraction and for lookups that may have no answer.

Does DeepSeek V4.1 Flash support image input, and what does an image cost?

Yes, natively. On V4.1 Flash an image costs 184 input tokens up to 512x512, 652 at 1024x1024 and 994 at 2048x2048, independent of content and format, which is $0.000055 to $0.000298 per image at peak list.

GPT-6 Astra Reasoning Effort: max Costs 2.3x low for the Same Answers

synthorai — Mon, 07 Sep 2026 16:44:08 +0000

On GPT-6 Astra, reasoning_effort: "max" costs 2.3x what low costs and returns the same answer on every one of 11 verified tasks; the only setting that changes accuracy is none, which fails 17 of 33 runs (11 tasks, 3 runs each). reasoning_effort is the request parameter that sets how much hidden thinking the model does before it answers; that thinking is billed as reasoning tokens at the output rate, and the ordered values from none to max are the ladder this post measures. The ladder is not the one in the docs: the API's own input check advertises seven values, one of them (minimal) is rejected on every model, and one it never mentions (disabled) is accepted on Astra and does not disable anything. That matters for reading OpenAI's launch benchmarks, reported as "the maximum at any effort": the leaderboard number comes from the most expensive rung of the ladder.

TL;DR

GPT-6 Astra accepts seven reasoning_effort values including none and disabled; the docs list five and say none is unsupported.
none is the only value that zeroes reasoning tokens, and it fails 17 of 33 verified tasks; every other rung scores 33 of 33.
disabled spends 243 reasoning tokens against low's 151 and costs 54% more per correct answer, same accuracy.
At list price, GPT-6 Astra costs 1.57x GPT-5.6 Sol per correct answer at low and 2.57x at max, on work both models get right.

Where does GPT-6 Astra land on the benchmarks?

Ahead on agent-style work (the model driving tools in a loop, often in a terminal), behind Claude Fable 5.1 on Humanity's Last Exam and on the Artificial Analysis Intelligence Index (an independent aggregate of ten evaluations), and every score reported at whichever effort scored highest. The table below is OpenAI's own, from the launch page, with the comparison columns OpenAI chose:

Benchmark	What it tests	GPT-6 Astra	GPT-5.6 Sol	Claude Fable 5.1	Claude Opus 5
Terminal-Bench 4.0	agent tasks in a terminal	57.9%	37.3%	55.8%	52.6%
Terminal-Bench Science 0.1	research workflows with code	64.6%	22.4%	52.6%	30.0%
FrontierMath Tier 4 (v2)	research-level mathematics	97.6%	83.0%	87.8%	73.2%
ARC-AGI-3	solving novel puzzle environments	99.9%	7.8%	not listed	30.2%
Humanity's Last Exam, with tools	expert-written questions across fields	57.2%	not listed	65.0%	63.6%
Artificial Analysis Intelligence Index v4.1.1	aggregate of ten evaluations	61.2	60.9	65.7	63.1

Three things to read alongside the wins:

The Humanity's Last Exam row, where Astra trails Fable 5.1 by 7.8 points, is in the table and nowhere in the prose.
The Artificial Analysis row puts Astra behind Claude Fable 5.1, Claude Opus 5 and Claude Fable 5 in OpenAI's own table; the current v4.2 leaderboard has Fable 5.1 at 57 and Astra at 55, first and third.
The cybersecurity scores were produced, in OpenAI's words, "without production safeguards"; the shipping model "will refuse" proof-of-concept exploit tasks.

The sentence under the tables is the one that connects the benchmarks to your bill: "Evaluation scores are the maximum at any effort." The Artificial Analysis leaderboard lists each model once per effort setting, and Astra at xhigh scores 54 against 55 at max. The rest of this post prices that point.

Which reasoning_effort values does GPT-6 Astra accept?

Seven, and that is not the set the API advertises: one advertised value is rejected on every model, and one accepted value is advertised nowhere. The model page lists low, medium, high, xhigh and max, and states the model "does not support none reasoning effort". The API disagrees with the docs twice, and with itself once.

Send an invalid value and the first check, which validates the request shape before any model is involved, answers with the same list of allowed values on GPT-6 Astra and GPT-5.6 Sol:

Invalid value: '__invalid__'. Supported values are: 'none', 'minimal', 'low', 'medium', 'high', 'xhigh', and 'max'.

Then send each of those values and a second check, specific to the model, rejects some of what the first one advertised (200 means the request went through, 400 means it was rejected):

Value	In the advertised list	GPT-6 Astra	GPT-5.6 Sol
`none`	yes	200	200
`minimal`	yes	400, "not supported with the 'gpt-6-astra-2026-09-03' model"	400, same
`disabled`	no	200	400
`low` through `max`	yes	200	200

So none works although the docs say it does not, minimal is advertised and rejected everywhere, and disabled is accepted on Astra alone without appearing in any list. The error messages also give away the dated build the gpt-6-astra alias currently points to, gpt-6-astra-2026-09-03.

Does none turn reasoning off, and does disabled?

none does, and it is the only value that does. disabled is a normal rung with a misleading name. We sent 11 tasks whose answers were computed by exhaustive search on our own machine first, so the answer key cannot be wrong (a rule applied 40 times in a row, a counting problem under three constraints, a knapsack, a base conversion, 7 to the power 222 modulo 1000, and six shorter ones), at all seven efforts (six on Sol, which rejects disabled), 3 runs per task and effort, through an endpoint that speaks the OpenAI Chat Completions API. The five hardest, on GPT-6 Astra:

effort	accuracy	mean reasoning tokens	cost per call	cost per correct answer
`none`	20% (3 of 15)	0	$0.00086	$0.0043
`disabled`	100%	243	$0.01311	$0.0131
`low`	100%	151	$0.00851	$0.0085
`medium`	100%	159	$0.00890	$0.0089
`high`	100%	203	$0.01110	$0.0111
`xhigh`	100%	279	$0.01491	$0.0149
`max`	100%	370	$0.01946	$0.0195

Cost per correct answer is the spend on a task-and-effort cell divided by the runs it got right: a setting that is 20% correct pays for five runs to get one answer. The cliff is one step wide. Every rung from low up scored 33 of 33 across all 11 tasks on both models; none scored 16 of 33 on Astra and 21 of 33 on Sol, and on the iterated-map task Astra returned three different wrong numbers in three runs. There is no gradual degradation to tune against: reasoning is on, or the model is guessing.

disabled is the trap. It spends more reasoning tokens than low, medium or high, and costs 54% more per correct answer than low for the same 100%. Only xhigh and max cost more per correct answer than the value whose name says off.

What does max buy over low?

Nothing on these tasks, at 2.3x the price: $0.01946 per call against $0.00851, both 100% correct. Reasoning tokens climb from 151 at low to 370 at max, and every one of them bills at the $50 per million output rate.

That is the number to hold next to the benchmark table. Reporting "the maximum at any effort" means each score comes from whichever rung scored best, and on the independent leaderboard Astra's best entry is its max entry, one point above xhigh; our ladder prices that rung at 1.3x the one below it. On a workload that resembles the launch benchmarks, the extra effort may earn its cost. On a workload that resembles ours, it does not, and you find out which by measuring your own tasks at low first.

Is GPT-6 Astra 2.5x the price of GPT-5.6 Sol per answer?

At low, no: 1.57x. At max, yes: 2.57x. List price is the per-token price on the vendor's own page: Astra lists at $10 input and $50 output per million tokens, Sol at $4 and $20 (both from the OpenAI model pages for Astra and Sol on 2026-09-07), so the list gap is 2.5x on both sides; the [GPT-5.6 Sol vs GPT-6 Astra comparison page](https://synthorai.io/compare/models/gpt-5-6-sol-vs-gpt-6-astra/) carries the live catalog prices. Cost per correct answer, computed from token counts at those list prices across all 11 tasks, both models scoring 33 of 33 from low up:

effort	GPT-6 Astra per correct	GPT-5.6 Sol per correct	ratio
`low`	$0.00560	$0.00356	1.57x
`medium`	$0.00618	$0.00373	1.66x
`high`	$0.00741	$0.00400	1.85x
`xhigh`	$0.01005	$0.00443	2.27x
`max`	$0.01349	$0.00525	2.57x

Astra spends fewer reasoning tokens than Sol to reach the same answer at the low rungs (91 against 158 per call across the 11 tasks at low), which is where the gap closes; its reasoning then grows faster up the ladder (249 against 242 at max), and there the per-answer gap is the full list-price gap.

The claim stays narrow: both models score 100% from low up, so this set does not separate their capability, only the price of an answer both get right, which runs from 1.6x to 2.6x on one parameter. OpenAI's launch page reports the opposite direction on agent work, Terminal-Bench 4.0 at "approximately 9% and 63% lower estimated API cost per task" than Sol and Fable 5.1, where a model that solves more tasks in fewer tokens can be cheaper despite a higher per-token price. Different workloads; the effort setting decides the bill on both.

Does a JSON schema or a tool call add reasoning cost?

Not at low; medium does. We sent a one-step task (add 2 hours 37 minutes to 08:15) three ways: bare, inside a strict response_format JSON schema (the reply must be JSON matching a shape you supply), and as a forced tool call (tool_choice pinned to one function, so the model must answer by calling it), at four efforts, 3 runs each. GPT-6 Astra, mean reasoning tokens (with their share of all billed output tokens) and cost per call:

shape	`none`	`low`	`medium`	`high`
bare	0, $0.00085	0, $0.00084	18 (67% of output), $0.00185	24 (73%), $0.00217
JSON schema	0, $0.00141	4 (23%), $0.00165	21 (57%), $0.00257	26 (62%), $0.00282
forced tool call	0, $0.00196	0, $0.00197	5 (18%), $0.00223	20 (47%), $0.00307

low scales to zero: on a task with no steps it spends no reasoning and costs the same as none, and on the multi-step set above it spends 16 to 345 tokens per task and stays correct where none collapses, so low is the floor. The shell is not the tax: a schema or a tool call at low adds 0 to 4 reasoning tokens, while medium puts reasoning at 57 to 67% of the output tokens bare and in the schema, and 18% in the tool call, on a task with nothing to reason about. The same extraction costs 1.6x more at medium than at low inside the schema and 2.2x more bare, and the July GPT-5.6 cost guide found the same lever on that family. Sol is flatter: 0 reasoning bare and in the schema at every effort, 14 to 18 tokens on the forced tool call from medium up.

Can you read the reasoning you pay for?

On one of OpenAI's two API surfaces. The older Chat Completions endpoint and the newer Responses endpoint accept the same model and bill the same way; the same question at medium, 3 runs per surface:

Surface	reasoning tokens billed	reasoning text returned
`/v1/chat/completions`	76, 75, 120	none; the message carries `role` and `content` only
`/v1/responses` with `reasoning.summary: "auto"`	62, 62, 116	a `reasoning` item with a 277 to 352 character summary

The token counts are within noise of each other, so the billing is the same and the only difference is whether you can see what you paid for. At $50 per million output tokens, the endpoint decides that. The docs also route tool calling to Responses ("GPT-6 Astra supports Chat Completions, but tool calling requires Responses"), so a tool-using workload lands on the readable surface by requirement.

What carries over from GPT-5.6?

Most of the contract. Measured:

Tokenizer. The same 900-word text tokenized to 1,017 prompt tokens on GPT-6 Astra, GPT-5.6 Sol, GPT-5.6 Luna, GPT-5.5, GPT-5.4 and GPT-5.2. Prompt sizes measured on 5.x hold on Astra; nothing needs re-counting.
Parameters. temperature returns 400 on Astra and Sol alike ("not supported with this model"), and so do top_p and logprobs; response_format with a strict JSON schema returns schema-valid output on both; max_tokens below 16 is rejected on both.
max on Sol. In July, reasoning_effort: "max" returned 400 on GPT-5.6 Sol through Chat Completions. It is accepted now, and Sol scored 33 of 33 at it.

From the docs, not measured here: the context window is 1,050,000 tokens with a 922,000 maximum input and 128,000 maximum output; prompts above 272K input tokens are billed at 2x the input and cache rates, the same threshold the GPT-5.6 family carries and the same mechanism we measured across vendors; cache reads list at $1 per million and cache writes at $12.50, with a new prompt_cache_options.ttl: "30m" parameter replacing prompt_cache_retention. Fast mode, a paid option OpenAI says delivers up to 2x the speed, is 2x the standard price.

How Synthorai handles it

The gateway passes reasoning_effort through unchanged, including the values the docs do not list, and the per-request usage record keeps reasoning_tokens as its own field next to completion_tokens and the billed cost. That is what every table above was built from: the effort a request was sent at, the reasoning it burned, and what it cost, readable per request rather than reconstructed from a monthly total.

FAQ

Does GPT-6 Astra support reasoning_effort none?

Yes. The docs say it does not; the API accepts it on GPT-6 Astra and on GPT-5.6 Sol, and it is the only value that returns zero reasoning tokens. It also failed 17 of 33 runs on a verified task set, so it is a setting for lookups and transforms, not for anything with steps.

What does reasoning_effort disabled do on GPT-6 Astra?

It reasons. disabled is accepted on GPT-6 Astra (and rejected on GPT-5.6 Sol), appears in no documented or advertised list, and spent 243 reasoning tokens per call on our hard set against 151 for low, with identical accuracy. Treat it as an expensive alias for a middle rung, not as an off switch.

Which reasoning_effort should I default to on GPT-6 Astra?

low. On GPT-6 Astra it spent zero reasoning on a one-step task and 16 to 345 tokens per task on multi-step ones, scored 33 of 33 where none scored 16, and max cost 2.3x as much for the same answers. Raise a specific call site only when an eval on your own tasks shows a higher rung changing outcomes. Set it explicitly on every call and read the reasoning count back from the usage block:

resp = client.chat.completions.create(
    model="gpt-6-astra",
    reasoning_effort="low",
    messages=[{"role": "user", "content": prompt}],
)
print(resp.usage.completion_tokens_details.reasoning_tokens)

Measured 2026-09-07 through an OpenAI-compatible Chat Completions surface, plus the Responses endpoint for the visibility comparison: 11 tasks with locally brute-forced answer keys, 7 effort values, 3 runs per cell on both models, salted prompts (a unique suffix per request so no reply is served from a cache), cost taken from per-request usage accounting. Benchmark figures are OpenAI's launch table and the Artificial Analysis leaderboard, both fetched the same day. Cost per correct answer against GPT-5.6 Sol is computed from token counts at each model's documented list price.

Long-Context Pricing Tiers: Up to 6.7x, and Gateways Never Show It

synthorai — Thu, 03 Sep 2026 06:31:25 +0000

The price on a gateway's model page is not the price on its invoice once a prompt gets long. We sent requests through one large multi-provider aggregator on both sides of every documented length line for nine tiered models: five billed exactly 2x their page price past the first line, Alibaba's three climbed to 3x, 3x, and 6.7x by the top of their ladders, and one Azure-served model billed 1.25x its listed endpoint rate on both sides of its line and then doubled. None of this is on the page. The mechanism is documented by the vendors themselves: Google, OpenAI, xAI, Alibaba, ByteDance, and MiniMax all reprice the whole request, output included, past a line at 32K, 128K, 200k, 256K, 272K, or 512k input tokens. This post shows the invoices, then the tier tables behind them, then the settings that keep a request under the line.

TL;DR

Nine tiered models billed 1.8x to 6.7x past the vendor's line through an aggregator whose pages show one price; GPT-5.6 Luna via Azure billed 1.25x its listing.
qwen3.7-flash charged $0.03, then $0.10, then $0.20 per million input at 32K and 256K; qwen3-coder-plus stopped at 3x where the vendor lists 6x.
Vendors reprice the whole request, output included, once the input crosses a line between 32K and 512k tokens.
Cap the input, not the output: Claude Code /autocompact, Codex model_context_window, API compaction triggers.

Do gateways charge what their page says?

No, not once the prompt is long, and finding out by how much takes two things per intermediary: the pricing metadata the gateway publishes, and the cost it actually reports for a request on each side of the line.

One large multi-provider aggregator publishes its catalog with a pricing.overrides array: a base price plus conditional rules such as min_prompt_tokens: 200000 with the higher rates, and, for DeepSeek and Tencent models, utc_start / utc_end windows for off-peak pricing. In the catalog we fetched on 2026-09-01, 60 entries carried overrides, including the Gemini Pro models, Grok 4.x, qwen3.7-plus, qwen3.7-flash, qwen3-coder-plus, the Seed 2.0 models, and the entire GPT-5.6 family at 272,000. The model pages show the base price only; the tier lives in the metadata. And the metadata is not the vendor's ladder: qwen3-coder-plus carries rules at 32,000 and 128,000 but nothing at the vendor's fourth tier, 256K.

So we sent requests on both sides of every documented line through that aggregator, with usage accounting enabled (the aggregator returns the amount it charged inside the response) and the serving endpoint recorded (one model id on the aggregator fronts several upstream hosts, each with its own price; it calls them endpoints), two runs per point, on 2026-09-02 and 2026-09-03:

Model (aggregator id)	Line	Below the line	Above the line	Page shows	Metadata	Served by
Gemini 2.5 Pro	200k	190k tokens at $1.25/M input	210k at $2.50/M	$1.25/M	rule at 200,000	Google
Gemini 3.1 Pro Preview	200k	185k at $2.00/M	217k at $4.00/M	$2.00/M	rule at 200,000	Google
Grok 4.3	200k	177k at $1.25/M	208k at $2.50/M	$1.25/M	rule at 200,000	xAI
Seed 2.0 Lite	128K	117k at $0.25/M	137k at $0.50/M	$0.25/M	rule at 128,000	Seed
Seed 2.0 Code	128K	119k at $0.50/M	135k at $1.00/M	$0.50/M	rule at 128,000	Seed
GPT-5.6 Luna	272K	252k at $0.275/M	294k at $0.50/M and $0.55/M	$0.20/M	rule at 272,000	Azure
qwen3.7-plus	256K	242k at $0.32/M	276k at $0.96/M	$0.32/M	rule at 256,000	Alibaba
qwen3.7-flash	32K	29k at $0.03/M	35k at $0.10/M	$0.03/M	rule at 32,000	Alibaba
qwen3.7-flash	256K	245k at $0.10/M	276k at $0.20/M	$0.03/M	rule at 256,000	Alibaba
qwen3-coder-plus	32K	29k at $0.65/M	35k at $1.17/M	$0.65/M	rule at 32,000	Alibaba
qwen3-coder-plus	128K	119k at $1.17/M	138k at $1.95/M	$0.65/M	rule at 128,000	Alibaba
qwen3-coder-plus	256K	244k at $1.95/M	276k at $1.95/M, no step	$0.65/M	no rule	Alibaba

The pages themselves, captured the same day as the invoices: one headline price, a per-endpoint table, and no length tier on either. Every charge above followed the metadata to the token. All nine models stepped at the vendor's first line by the vendor's factor, and on the three Alibaba models the invoice climbed a ladder the page never mentions: qwen3.7-flash from $0.03 to $0.10 at 32K and to $0.20 at 256K, 6.7x the page price, on a page that prints $0.03. GPT-5.6 Luna stepped too, and added a second gap: every Azure-served run was billed 1.25x the listed Azure endpoint price, below and above the line alike ($0.275 against $0.22, $0.50 and $0.55 against $0.40 and $0.44), a surcharge that appears neither on the page nor in the endpoint metadata.

The ladder can also stop short of the vendor's. On qwen3-coder-plus the charge rose 1.8x at 32K and 3x at 128K, then stayed at $1.95 per million through 276k tokens, where Alibaba's own list moves to $6 on input and $60 on output, six and twelve times the base. The aggregator's metadata has no rule at 256K, so the charge did not move. Whether the aggregator absorbs that difference or buys under a different contract is not visible from outside. What is visible is that the invoice follows the metadata, and the metadata and the page are two different documents.

The transparency gap, then, is not between metadata and invoice; it is between the page and both of them. The page's headline number is the cheapest endpoint's rate in the table below it, not the rate of the endpoint that will serve you, and neither number carries the length condition. A model card with one price does not tell you about the tier, or which endpoint will serve the next request, or whether the endpoint whose price you are looking at is reachable for your account at all.

The rule that follows is simple. Read the machine-readable pricing for the model you use, at the endpoint level, and check for a length condition; then send one request on each side of the line with usage accounting on and compare the reported cost. A gateway whose charge steps where its page does not is passing a vendor rule through without telling you. A gateway whose charge stays flat where the vendor's does not is either serving the model from a host with different prices or absorbing the difference, and only the first of those is a stable state.

Where are the lines, and how does a tier apply?

Six vendors publish a length line, and every model that states the rule applies the higher rate to all tokens of the request, output included. The step is 2x at most first lines, but the ladders climb: 3x at qwen3.7-plus's only line, 6.7x on input by qwen3.7-flash's third tier, and 6x on input and 12x on output by qwen3-coder-plus's fourth. Prices are per million tokens, fetched from the vendors' pricing pages on 2026-09-01.

Model	Threshold	Input, below / above	Output, below / above	Stated semantics
Gemini 2.5 Pro	200k prompt tokens	$1.25 / $2.50	$10 / $15	Vertex pricing footnote: "If a query input context is longer than or equal to 200K tokens, all tokens (input and output) are charged at long context rates"
Gemini 3.1 Pro Preview	200k	$2 / $4	$12 / $18	same footnote; cache reads tier too, $0.20 / $0.40
GPT-5.6 Sol (Terra, Luna same shape)	272K input tokens	$4 / $8	$20 / $30	model page: "Prompts with >272K input tokens are priced at 2x input and 1.5x output for the full request"
Grok 4.6, 4.5	200k	$2 / $4	$6 / $12	docs: "billed at the higher rate for all tokens in the request"
Grok 4.3, 4.20	200k	$1.25 / $2.50	$2.50 / $5	same
qwen3.7-plus	256K	$0.40 / $1.20	$1.60 / $4.80	Model Studio: "All tokens in the request are billed at the unit price of the corresponding tier"
qwen3.5-plus	256K	$0.40 / $0.50	$2.40 / $3.00	same
qwen3.7-flash	32K, 256K	$0.03 / $0.10 / $0.20	$0.13 / $0.40 / $0.80	same, three tiers
qwen3-coder-plus	32K, 128K, 256K	$1 / $1.8 / $3 / $6	$5 / $9 / $15 / $60	same, four tiers
Seed 2.0 Lite, Seed 2.0 Code	128K	$0.25 / $0.50, $0.50 / $1.00	$2 / $4, $3 / $6	BytePlus's pricing page renders in-app and could not be quoted; prices from the aggregator's metadata, matched by the invoices above
MiniMax M3	512k input	$0.30 / $0.60	$1.20 / $2.40	pay-as-you-go page: tier by request input count, applied to all tokens

Two boundary details worth copying into billing code. Google's two pages disagree by one token: the Vertex footnote says "longer than or equal to 200K", the Gemini API pricing table says "prompts > 200k tokens". And Alibaba defines its K precisely: 128K equals 128,000 tokens, 256K equals 256,000, not powers of two.

The tier applies to the whole request, decided by the input length alone, on every token including the output. The marginal cost of the token that crosses the line is therefore the entire premium on everything before it. For Gemini 2.5 Pro: a 199,999-token prompt bills $0.25 of input; at 200,001 tokens it bills $0.50, and a 4,000-token answer moves from $0.04 to $0.06. One token, plus $0.27.

On qwen3.7-plus the step is 3x: the official rate for a 255,029-token prompt is $0.102, for a 257,332-token prompt $0.309. On qwen3-coder-plus the same mechanism compounds across four tiers, so a 260k-token prompt bills six times the per-token rate of a 30k one and its output twelve times.

The rule is visible on a real invoice, not only in the vendor's table. On qwen3.5-plus, whose official tiers are $0.40 then $0.50 per million input tokens at 256K, our gateway bill per input token differed by exactly 1.25x between ten runs below the line (243k tokens) and 24 runs above it (256k to 321k), with the output price unchanged. The 259k request did not pay 1.25x on its last 3k tokens; it paid 1.25x on all of them.

For an agent that accumulates history, the crossing happens mid-session and silently: the turn that tips over pays the premium for every earlier turn it carries, and every later turn keeps paying it until something shrinks the context.

Does capability step at the same line?

No. We measured this because the tier is easy to mistake for a capability edge, and it is not one. On two tiered Qwen models with the line at 256K, a salted needle (a one-line fact carrying a per-run random code) planted at five depths was recalled 30 of 30 times on each model at 243k, 269k, and 320k tokens, with thinking off, and latency grew with length on a line rather than a step (15.2 s, 17.0 s, 20.5 s median on qwen3.7-plus). A harder task, counting K planted rare sightings across the whole log, did degrade with length: on qwen3.7-plus the share found slid from 74% at 128k to 60% at 192k, 58% at 243k, and 45% at 320k. The decline starts well below the price line and the cells on either side of it sit on the same slope.

Anthropic's own documentation names the gradual effect: as token count grows, accuracy and recall degrade, "a phenomenon known as context rot". It is real, it is continuous, and it does not know where the price tier is.

How do you keep a request under the line?

Cap the input, not the output. The tier keys on the request's input length, so max_tokens (an output cap) does nothing for it; the levers are the settings that bound what the client sends. They exist at four layers.

Here are the settings that bound the prompt, by layer. "Under a tier" marks the ones that take an absolute token count and can therefore be set just below a price line; the window-relative ones only keep you inside the model's context window, which is a different number.

Layer	Tool	Setting	What it bounds	Under a tier?
Coding agent	Claude Code	`/autocompact <value>`, `autoCompactWindow`, `CLAUDE_CODE_AUTO_COMPACT_WINDOW`, 100K to 1M	the token count at which history is summarized	Yes
Coding agent	Codex CLI	`model_context_window`, `model_auto_compact_token_limit`, `tool_output_token_limit`	context size, compaction trigger, per-tool-result cap	Yes
Coding agent	Aider	`--max-chat-history-tokens`, `--map-tokens`	soft limit on chat history before summarization; repo-map budget	Yes
Coding agent	Gemini CLI	`model.compressionThreshold`, default 0.5, plus `/compress` and `model.maxSessionTurns`	the fraction of the context window at which history is compressed	Indirectly: pick the fraction so window x fraction lands under the line
Coding agent	Cursor	Max Mode off (the default)	default window; Max Mode extends it and bills at the API rate plus 20%	Keep it off
Coding agent	Cline	none documented; auto-summarizes near the window	the model's window	No knob
API	Claude API	`context_management.edits[].trigger.input_tokens`, default 150,000, minimum 50,000	server-side compaction trigger; the compaction pass is billed under `usage.iterations`	Yes
API	OpenAI Responses	`truncation: "auto"`, default `disabled`	drops middle items only when the input exceeds the model window; `disabled` returns a 400 instead	No, window only
Aggregator	context compression	middle-out transform	removes the middle of the prompt to fit the model window	No, window only
Framework	LangChain	`trim_messages(max_tokens, strategy="last", token_counter, include_system)`	client-side history trim by token count before the request	Yes

Two things the table implies. On a coding agent pointed at a tiered model, the compaction window is the one knob that turns the cliff into a summary, and it has to be set in absolute tokens just under the line, not as a fraction of a 1M window. And the two most common "safety" settings, Responses truncation: "auto" and the aggregator's middle-out, are window guards: they act at the model's window, 1M on the tiered Gemini and GPT-5.6 models, not at the 200,000 or 272,000 tokens where the price moves, so they will keep a request from failing and still let it cross the line.

Do not count on caching to keep you under. Prompt caching cuts the bill but not the tier on Google's models, where cache reads are priced by the same length tier: a cached 150k prefix plus 60k of fresh context is a 210k prompt and bills as one. Provisioned capacity sidesteps the question entirely, since provisioned throughput units (PTUs) bill per hour regardless of tokens. And when crossing is worth it, cross deliberately: the measurement above says the model does not get worse at the line, so the decision is purely whether the marginal context is worth a 2x, 3x, or at the top tiers 6.7x multiplier on the whole request.

How Synthorai handles it

A length tier is a rate condition, so the gateway treats it as one: a model's price card can carry a list of tiers keyed by input-token boundary, and each request is rated against the tier its own prompt length selects, on all of its tokens, the same way the vendor rates it. That is the mechanism the qwen3.5-plus invoice above exercised. The usage record keeps the prompt token count and the price version alongside the rated cost, so a bill can be decomposed back to "this request crossed the line". The tier a request was rated under can be read back from the usage record, not only from a price table.

FAQ

Is the higher long-context rate charged only on tokens beyond the threshold?

No. Every vendor that documents the rule reprices the whole request: Google ("all tokens (input and output) are charged at long context rates"), OpenAI ("for the full request"), xAI ("for all tokens in the request"), Alibaba ("all tokens in the request are billed at the unit price of the corresponding tier"). A prompt one token over the line pays the premium on every token before it.

Do API gateways pass long-context pricing tiers through?

Yes, in our measurement: through one large aggregator, nine tiered models were billed the vendor's higher rate past the vendor's line, on model pages that show a single price. The tier is in the gateway's pricing metadata, not on the page, and the metadata can also omit a vendor tier, as it did for qwen3-coder-plus above 256K.

Does max_tokens keep a request under a pricing tier?

No. max_tokens caps the output; the tier is decided by the input length. The settings that help are the ones that bound the prompt: a compaction window or token limit in the agent (Claude Code /autocompact, Codex model_context_window), a compaction trigger in the API, or client-side truncation before the request is sent.

Does model quality drop at the pricing threshold?

Not in our measurement. On two tiered Qwen models, needle recall was perfect on both sides of the 256K line and a counting task degraded gradually with length with no step at the boundary. The tier is a business rule; capability loss with length is real but continuous.

Prices and semantics quoted from vendor pricing pages fetched 2026-09-01; agent and API settings from the linked documentation on 2026-09-02; invoice measurements run 2026-09-01 to 2026-09-03 with thinking disabled and salted prompts, two runs per aggregator point. Prices drift; check the linked source before wiring a threshold into billing code.

Related: billing units field guide (the modifier layer this post zooms into), token usage anatomy, prompt caching explained, cache minimums measured.

AI API Billing Units: 17 Meters from Tokens to PTU-Hours

synthorai — Tue, 01 Sep 2026 13:09:58 +0000

AI APIs meter usage in at least 17 distinct units: text tokens, reasoning tokens, cached tokens, media-input tokens, realtime audio tokens, characters, minutes and seconds of media, output objects, requests, pages, GPU-seconds, container-hours, GB-days, token-hours, PTU-hours, training tokens, and synthetic credits. Anyone estimating the cost of a product that touches more than one modality has to convert between several of them, and the conversion rules are where budgets go wrong. This page is a field guide: every unit, how it is metered, and the gotcha attached to it.

TL;DR

AI APIs bill in at least 17 distinct units across four layers: consumption, time and capacity, rate modifiers, and synthetic wrappers.
One audio minute bills two ways: $0.0020 to $0.0164 per audio-minute on dedicated transcription, or 1 audio token per 100 ms heard on gpt-realtime-2.1.
Storage alone has two meters: OpenAI file search bills $0.10 per GB-day; Gemini context caching bills $0.50 per million tokens per hour.
Modifiers change rates, not units: batch is half price, DeepSeek off-peak halves again, US-pinned Claude inference costs 1.1x.

What units do AI APIs bill in?

Seventeen, in the order the sections below follow: consumption units (1 to 10), time and capacity units (11 to 16), and synthetic wrappers (17), plus a modifier layer that changes rates without changing units. The table is the reference; each row links the deep dive.

#	Unit	Where it appears	The gotcha
1	Text tokens	Every chat API	Same text, different counts per tokenizer; about 30% more tokens from one tokenizer generation to the next
2	Reasoning tokens	Thinking models	Billed as output you never see; 88 to 99.3% of some answers
3	Cached tokens	Prompt caching	Write premium by TTL, read discount, per-provider minimums
4	Media-input tokens	Vision, audio, PDF input	Three conversion schemes for one image; PDFs bill at image rates
5	Realtime audio tokens	Speech-to-speech	1 token per 100 ms heard, 1 per 50 ms spoken
6	Characters	TTS, guardrails	Chinese speech costs 3 to 7x less than English per audio-minute
7	Media minutes/seconds	STT, video and music generation	8x price spread per audio-minute across models with near-equal accuracy
8	Output objects	Image generation, human eval	A quality knob swings one image 36x at identical resolution
9	Requests	Web search, grounding, file search	$0.01 per search plus injected tokens at the model's input rate
10	Pages	OCR, document AI	Mistral bills per 1,000 pages with batch and cache discounts on top
11	GPU-seconds	Serverless model hosts	Rate depends on the card, $0.000225/s (T4) to $0.001525/s (H100)
12	Container/session-hours	Code execution, managed agents	5-minute minimum, files preload billing even if the tool never runs
13	GB-days	File and vector storage	$0.10 per GB-day after the free gigabyte
14	Token-hours	Context-cache storage	Gemini bills $0.50 per million tokens per hour, doubling in 2027
15	PTU/model-unit hours	Provisioned capacity	Cached tokens consume no capacity; output weighs more than input
16	Training tokens	Fine-tuning	Epochs multiply the bill; some models bill $100 per hour instead
17	Credits/CCUs	Marketplaces, subscriptions	A scalar over the real rate card; 100 CCU = $1.00 on AWS Marketplace

Which units count what you consume?

Ten of the seventeen are consumption units: the meter ticks when you use the service, and each modality picked a different thing to count.

The token family (units 1 to 5). Text tokens are the anchor, but they are not one unit: tokenizers differ per model family, so the same text bills different counts, and Anthropic documents that its 4.7-and-later tokenizer produces about 30% more tokens for the same text than the previous one. Reasoning tokens are output tokens you usually cannot read; we measured 81 billed tokens for a 10-token answer, 88% of it reasoning. Cached tokens price by operation and cache lifetime (TTL): on the Claude API a 5-minute cache write costs 1.25x base input, a 1-hour write 2x, and a read 0.1x. Media inputs convert to tokens by provider-specific formulas: one 1024x1024 image bills 693, 1,089, and 1,372 tokens on GPT-5.6, Gemini, and Claude respectively; Gemini rates audio at 25 tokens per second and 720p video at 5,792 tokens per second and bills PDFs at the image token rate. Realtime speech has its own token: gpt-realtime-2.1 bills exactly 1 audio token per 100 ms of user speech and 1 per 50 ms of model speech.

Characters and time (units 6 and 7). Text-to-speech (TTS) bills per character (OpenAI lists $15 to $30 per million characters), which is why spoken Chinese costs 3 to 7x less than English per audio-minute: the same speaking time consumes fewer characters. Bedrock's guardrails define their own character unit, a "text unit" of up to 1,000 characters. Dedicated transcription models bill per audio-minute, $0.0020 to $0.0164 across the models we measured; video generation bills per second by resolution; Gemini prices music per song.

Objects, requests, and pages (units 8 to 10). Image generation bills per image by tier or per output token, and the two cross over: per-token wins below about 1,000 output tokens, flat per-image wins above. Server-side tools bill per request: web search costs $0.01 per search on the three surfaces we measured, and the results then bill again as input tokens (1,500 to 3,100 per search). Gemini's search grounding (letting the model query Google Search mid-request) adds a quota shape: 5,000 free requests per month, then $14 per 1,000. OCR bills per page. Bedrock's model evaluation bills $0.21 per completed human task, the only unit in this zoo denominated in people.

Which units rent time or capacity instead?

Six units bill for holding resources, whether or not tokens flow. This layer is where cost estimates miss hardest, because the meter runs while your code does nothing.

GPU-seconds. Serverless hosts bill model runtime by hardware class: on Replicate, a T4 costs $0.000225 per second and an H100 $0.001525. The unit prices the card, not the output.
Container and session hours. Anthropic's code execution bills $0.05 per container-hour beyond 1,550 free hours a month, with a 5-minute minimum per execution; OpenAI's code interpreter bills per 20-minute session, priced by memory size, $0.03 for 1 GB up to $1.92 for 64 GB. Claude's managed agents bill $0.08 per session-hour, metered to the millisecond and only while running.
GB-days and token-hours. Two storage meters for the same idea. OpenAI file search storage is $0.10 per GB-day with 1 GB free; Gemini explicit context caching bills storage at $0.50 per million tokens per hour, rising to $1.00 in 2027. A cache you forget to delete is a subscription you did not know you had.
PTU-hours and model units. Provisioned capacity bills per unit-hour regardless of traffic. Azure's provisioned throughput units (PTUs) are model-independent quota with per-model minimum deployment sizes; output tokens consume more capacity than input tokens, and cached tokens consume none, so a high cache-hit rate shrinks the PTU count you need. Bedrock sells the same shape as model units with 1-month or 6-month commitments.
Training tokens. Fine-tuning bills per token of training data multiplied by epochs, except where it bills wall-clock: some OpenAI models list $100 per training hour.

What changes the rate without changing the unit?

Seven things change the price of a unit without touching the unit itself: batch tier, service tier, clock time, context length, geography, quality knobs, and free quotas. Billing code that stores one price per model gets every one of these wrong, because the same token has different prices depending on how, when, and where it is processed.

Modifier	Effect	Example
Batch tier	50% off input and output	Anthropic and OpenAI batch APIs; Mistral OCR too
Service tier	Premium for speed	OpenAI fast tier at 2x standard rates; flex tier discounted
Clock time	Off-peak discount	DeepSeek: off-peak is half of peak; peak is 01:00-04:00 and 06:00-10:00 UTC weekdays
Context length	Rate steps at a threshold	Gemini rates step above 200k prompt tokens; Claude 4.6+ is flat across the full 1M window
Geography	Residency premium	Claude `inference_geo: "us"` is 1.1x on every token category; regional endpoints for Claude models on Bedrock and Google Cloud carry a 10% premium over global
Quality knobs	Same unit, different burn	gpt-image quality swings 196 to 7,024 billed tokens for one 1024x1024 image
Free quotas	Rate cliff at a threshold	Grounding: 5,000 free requests/month; code execution: 1,550 free hours/month

The multipliers stack. Anthropic documents that caching multipliers combine with the batch discount and the residency premium, so one cached, batched, US-pinned token carries three factors at once.

What are credits and CCUs?

They are synthetic wrappers over the real meters, used where invoicing needs one line item. When Claude bills through AWS or Azure Marketplace, usage is rated in dollars at the normal per-unit prices and then converted to Claude Consumption Units at 100 CCU per dollar; discounts show up as fewer CCUs metered, not a cheaper CCU. Subscription products run the same trick with credits, and the boundary can sit inside one company: ElevenLabs meters its subscription plans in credits while stating that API usage is billed in US dollars, not credits. When you see a synthetic unit, the question to ask is which consumption or capacity meter sits underneath it and at what conversion.

Which rounding rules and minimums bite?

These are the quantization rules, collected in one place because each one lives in a different document:

Code execution containers bill a 5-minute minimum, and including files in a request bills execution time even if the tool is never called.
Code interpreter sessions quantize to 20-minute blocks by RAM tier.
Realtime speech quantizes to 100 ms heard / 50 ms spoken per audio token; 60 seconds of silence billed zero under server voice-activity detection (VAD), and cancelling a spoken answer 2 seconds in billed 4 seconds.
Image generation with n=4 re-bills the prompt four times; there is no prompt caching on that surface.
Video tokens carry a +1 frame in the formula and encoded dimensions that differ from advertised ones: 720p bills as 1248x704.
Prompt caches have per-provider minimum cacheable lengths; below the minimum you pay the write premium and cache nothing.
Guardrails text units round up per 1,000 characters.
Tool definitions bill as input before any call: Anthropic's tool-use system prompt adds 286 to 804 tokens depending on model and tool choice, and a full computer-use toolset adds about 4,500.

How do you compare costs across units?

Normalize to dollars per user interaction, because that is the only unit every meter converts into.

Our voice-agent study did exactly this: a cascade stack billing audio-minutes (STT), tokens (LLM), and characters (TTS) came to $0.0037 to $0.025 per conversation-minute, against $0.057 for gpt-realtime-2.1 and $0.016 for its mini, which prices the realtime premium and its latency payoff in one comparable number. The procedure generalizes: pick the interaction (a support ticket, a document, a conversation-minute), meter each hop in its native unit, rate each at its current price, and only then compare. Comparing $/1M tokens to $/audio-minute directly is a category error; comparing $/interaction is arithmetic.

How a gateway normalizes the zoo

Synthorai's answer is to rate at request time and keep the native unit alongside the dollars. Every request produces one usage record carrying the native meter readings (tokens by class, seconds, characters, request counts), the resolved price version, and the rated cost, so an invoice line can always be decomposed back into unit times rate. This is the same conversion a CCU does at the marketplace boundary, done per record and kept inspectable. It also composes with zero-retention mode: the usage record needs meter readings and a content hash, never the payload. When a price changes upstream, the price version pins which rate card each historical record was rated under, which is the property that makes month-end reconciliation an audit rather than an argument.

FAQ

Are audio tokens the same as text tokens?

No. Realtime audio tokens are a separate meter with separate prices: on gpt-realtime-2.1 one token covers 100 ms of speech heard or 50 ms spoken, which works out to $0.0192 per minute to listen and $0.0768 per minute to speak. The audio and text meters carry separate rate cards even inside one model.

Why does the same image bill different token counts on different providers?

Because the conversion scheme differs, not the image: patch formulas, capped tiles, and flat fees all coexist. One 1024x1024 image measured 693 tokens on GPT-5.6, 1,089 on Gemini, 1,372 on Claude, and file format or content never moved a single token.

Do cached tokens count as input tokens?

They are metered separately and priced by operation: cache writes carry a premium over base input (1.25x or 2x by TTL on the Claude API), reads a discount (0.1x). On provisioned capacity the distinction matters twice: cached tokens consume no PTU capacity, so caching reduces both the bill and the capacity you must reserve.

Is per-character or per-minute pricing cheaper for speech?

It depends on the language. Per-character TTS makes spoken Chinese 3 to 7x cheaper per audio-minute than English because Chinese packs more speech into fewer characters; per-minute pricing is language-neutral. For transcription the per-minute models we measured ran $0.0020 to $0.0164 per audio-minute with near-equal accuracy on clean speech.

Unit definitions and rates quoted from provider pricing pages fetched 2026-08-30, and from our own measured studies linked in place. Prices drift; unit structures drift slower, but check the linked source before wiring any number into billing code.

LLM Data Retention and ZDR: Every Party That Can Read Your Prompts

synthorai — Sun, 30 Aug 2026 10:54:02 +0000

Your prompt does not go to "the AI company". On an agent stack it crosses a chain of parties that all handle it in cleartext, and exactly which ones depends on your stack: the agent framework's telemetry, a tracing platform, a memory store, an analytics tool, an AI gateway, and one of three kinds of inference provider, each with its own retention policy, storage region, and training clause. The party that keeps your prompts longest is usually on your side of the API: model vendors delete standard logs in about 30 days, while a default tracing or memory store keeps them until you delete them. Zero data retention (ZDR) is a real and useful arrangement, but it pins down exactly one of those parties on exactly one of three axes. This post maps the whole chain, who retains what and for how long, and what ZDR does and does not cover.

TL;DR

Every cleartext hop can collect; agent-side tracing and memory stores keep prompts far longer than any model provider does.
Providers retain per feature: on the Claude API, caching is ZDR-eligible, batch jobs persist 29 days, Fable 5 requires 30 days.
Geography follows the server: DeepSeek's official API stores data in the PRC, no other option.
Free endpoints are the sharpest edge: training rights are usually the price of free capacity.
ZDR held up in court: the NYT preservation order exempted only zero-retention API customers.

Who can read your prompt on an agent stack?

Everyone on the path, and the path is longer than most teams draw it:

Walk it left to right. The agent layer is yours, but rarely only yours: frameworks ship telemetry, tracing and observability platforms exist precisely to store full prompts and responses, memory features write conversation content into vector stores that never expire on their own, and a session-replay analytics snippet in your web UI captures the prompt as the user types it, before your backend ever sees it. The gateway is one party if you run it yourself and two if it is SaaS, and SaaS gateways come in two flavors. The first is the multi-provider aggregator: one API that routes each request across the providers behind it, so its data posture is always a sum, its own policy plus the policy of whichever host this particular request landed on. The second comes from security vendors: data-loss-prevention (DLP) and guardrail gateways whose entire product is payload inspection, redaction, and policy enforcement, which means reading every prompt is the feature, and flagged prompts are typically retained as security events by design.

Behind the gateway sit three kinds of inference provider: the model vendor's own API, a cloud provider hosting the model inside your cloud tenancy, and GPU hosts serving open weights under their own logging policies. The bypass branch, self-hosted inference on a dedicated tenant or your own GPUs, is the only path where no third party handles cleartext, and it comes with a catch we will return to.

The rule that organizes everything else: seeing and keeping are different questions. Every box can technically collect. Which ones do is a matter of defaults, settings, and contracts, and the defaults are wildly uneven across the chain.

Who actually keeps your prompts?

Usually your own tooling, longer than anyone else. The provider retention windows that dominate security questionnaires are days; the agent-side stores that nobody questionnaires are forever.

The agent layer retains by design. A tracing platform's product is a database of your prompts and outputs; retention there is a project setting, not a policy accident. The defaults vary by tool and are worth knowing: GitHub Copilot's OpenTelemetry integration exports span structure, timing, and token counts but no prompt content unless you explicitly enable content capture, which is the privacy-conservative template more agent tooling should copy. Cursor's privacy mode exists because the default mode shares code data. Whatever the model provider deletes after 30 days, your trace store still has on day 300.

The gateway keeps what it chooses to keep. A gateway cannot avoid usage records, since billing requires knowing token counts, models, and timestamps per request. It can entirely avoid storing payloads: prompt and response bodies can pass through memory and never be written at rest. But two popular gateway features quietly cross that line. A request-inspection or observability console is payload storage by definition, exactly like a tracing SaaS one hop earlier. And gateway-side caching stores prompt content at the gateway hop: a semantic cache keeps an embedding of your prompt plus the full cached response, and a response cache keeps both sides verbatim, on whatever disk the gateway runs. When you evaluate a gateway, SaaS or self-hosted, the question is the same split three times over: what is in the request record, what does the observability view persist, and what does the cache write?

Providers retain per feature, not per company. The useful mental model comes straight from Anthropic's API retention documentation, which discloses eligibility feature by feature: prompt caching is ZDR-eligible because cache state is held in memory only, batch processing stores jobs for 29 days because async jobs need storage, code-execution containers persist up to 30 days, and uploaded files live until you delete them. The honest sentence in that page deserves quoting: using a stateful feature "is a choice to step outside your ZDR arrangement for that specific data." On the OpenAI side, the data-controls documentation lists which endpoints are ZDR-eligible, keeps abuse-monitoring logs up to 30 days by default, and documents that cached prompts live as encrypted key-value tensors on GPU-local storage with a bounded TTL. Standard commercial deletion on the Claude API is within 30 days.

Where does the data physically live?

Wherever the serving infrastructure lives, and the provider kinds give different answers. A cloud-hosted model is the strong case: on Amazon Bedrock the cloud provider is the data processor and inference stays in the region you selected, which is why regulated buyers reach for that lane first. A model vendor's own API runs where the vendor runs: OpenAI offers US and EU regional processing for most endpoints; smaller vendors often publish nothing. The unambiguous extreme is DeepSeek, whose privacy policy states that personal data is collected, processed, and stored in the People's Republic of China, with no US or EU option on the official API. The same open weights served by a US GPU host carry none of that geography, which is the cleanest illustration that where a model's data goes is a property of who serves it, not of the model.

Two tails extend past the primary copy. Caches and batch files live in the serving region for their own lifetimes, separate from the request log. And every provider sits on subprocessors (its own downstream vendors) and backups: deletion commitments are typically phrased as deletion from active systems with backup propagation windows behind them. If your data-flow diagram ends at the API vendor's logo, it is missing at least those two boxes.

Who requires retention, and who trains on your prompts?

Three different forces keep data alive past the defaults, and only one of them is in the provider's marketing.

Policy requirements. Some models mandate retention as a safety condition. Claude Fable 5 and Mythos 5 require 30-day retention even for ZDR customers; the enforcement is fail-loud, since a request from an organization whose retention configuration does not meet the requirement is rejected with a 400 rather than silently accepted. Abuse escalation is the other policy tail: content flagged for usage-policy violations is retained on a different clock entirely (Anthropic's consumer policy documents up to 2 years for flagged inputs and outputs, and up to 7 years for classifier scores).

Legal holds. The NYT v. OpenAI litigation produced the clearest natural experiment in retention promises. A May 2025 preservation order required OpenAI to preserve output logs it would otherwise have deleted, including chats users had deleted, across consumer tiers; the order was narrowed that September and a court later ordered 20 million chat logs produced in discovery. The detail that matters for API buyers: enterprise and zero-data-retention customers were carved out, because there was nothing retained to preserve. A deletion policy bends to a litigation hold; an architecture that never stored the data does not.

Harvesting middlemen. The middle of the chain does get caught monetizing cleartext, and the largest documented case sat closest to the user. In December 2025, security researchers documented that a "privacy" VPN browser extension with millions of installs had been injecting scripts into AI chat pages, intercepting every prompt and response across ChatGPT, Claude, Gemini and five other assistants, and shipping the conversations to a data-broker affiliate; the harvesting ran whether or not the VPN was on, across roughly 8 million users of the publisher's extension family. The incident was a client-side interceptor rather than an API gateway, but the lesson transfers to every box on the diagram: any intermediary that handles cleartext can be a collection point, its incentive is monetization, and the interception is invisible from both ends. Trust in a middleman is a claim about its business model, not its feature list.

Training clauses. Business APIs from the major vendors do not train on your data by default; consumer products increasingly do unless you opt out, which is one more reason agent traffic belongs on API keys rather than consumer accounts. And a growing number of products put training on a toggle you are expected to find yourself: consumer plans with training on by default and an opt-out buried in settings, developer tools whose telemetry or code-sharing setting doubles as a training consent, vendor programs that trade data-sharing opt-ins for discounts or free quota, and aggregator dashboards with separate paid-tier and free-tier training switches. Each toggle is per-account, sometimes per-workspace, and a default can change with a terms update, so "we checked once" is not a posture; auditing these switches belongs on the same recurring checklist as key rotation.

Free models, singled out on purpose. Every warning above concentrates in one place: the free tier. A free model launch is usually subsidized by whoever benefits from the traffic, most often the model's own vendor, so prompts sent to the free variant flow to that party under its policy, with training rights commonly part of the deal, processed wherever that party operates, which for Chinese vendors means China. Aggregator gateways publish per-endpoint data policies and a switch to exclude providers that train precisely because free routes are where those clauses live. The rule of thumb is blunt: treat a free endpoint as a data submission, not an API call. Free inference is paid for somehow, and the currency is usually your prompts.

What does ZDR actually mean?

ZDR pins one axis out of three, for one party in the chain, for eligible features only. That is not a criticism; it is the definition, and knowing the edges is what makes it usable.

The three axes. Training use, retention duration, and human access are independent. Business APIs already do not train on your data; ZDR moves retention to zero for payloads; human access is governed separately by abuse processes. Conflating them is how ZDR gets oversold.
What survives under ZDR. Usage metadata and billing records; safety-classifier outputs; and whatever stateful features you opt into. Both major vendors now engineer around the abuse-monitoring exception rather than through it: payload-free safety signals on one side, classifier-results-only retention on the other.
What ZDR does not reach. Everything left of the provider: your tracing store, your gateway logs, your vector memory. A ZDR agreement with a model vendor while a tracing SaaS holds every prompt indefinitely is the most common retention posture we see, and the least coherent.
The self-hosting caveat. Running open weights on your own GPUs removes every third party, and then your own infrastructure inherits the whole problem: inference-server request logging, access logs, trace files. Self-hosting relocates the retention surface; only deliberate log hygiene shrinks it.

How Synthorai handles it

The gateway hop is ours, so the commitments are concrete. In zero-retention mode, request and response bodies pass through memory and are not written at rest; what persists is the usage record that billing requires: timestamps, model, token counts, and a content hash that lets a request be matched during a dispute without storing what was said. Upstream, the roster is filtered before routing ever happens: Synthorai only onboards providers that commit to zero data retention for our traffic, with one documented exception, Claude Fable 5, whose 30-day retention is mandated by the model vendor and cannot be contracted away. Provider selection is retention selection, and the gateway's job is to make that property explicit per route rather than buried in a policy PDF: a Fable 5 route carries its retention requirement as documented metadata, not as a surprise.

FAQ

Do LLM providers train on API data by default?

No, the major vendors' business APIs do not train on customer data by default; that commitment appears in each provider's data-controls documentation. The exceptions cluster in the long tail: consumer products with opt-out rather than opt-in, some GPU hosts serving open weights, and free-tier endpoints where training rights are part of the price.

Does ZDR mean the provider stores nothing at all?

No. ZDR means prompt and response payloads are not retained for eligible features. Usage metadata, billing records, and safety-classifier outputs persist, and stateful features like batch jobs or file uploads store data by nature, outside the ZDR envelope. Read the feature-eligibility table, not the headline.

Is it safe to send production data to free models?

Treat a free endpoint as a data submission, not an API call. Free capacity is subsidized by whoever benefits from the traffic, training rights are commonly part of the arrangement, and the data is processed wherever the subsidizing party operates. For throwaway experiments that is a fine trade; for anything containing customer data, code, or credentials it rarely is.

Is self-hosting automatically the most private option?

It removes every third party from the cleartext path, which is real. It does not remove retention: inference servers, reverse proxies, and tracing all log by default, so a self-hosted stack with default logging can retain more prompt data than a ZDR API arrangement. Privacy follows the logging configuration, not the hosting model.

Sources verified 2026-08-29: every provider claim above links to the vendor's own documentation or to primary court records, and policies in this area changed twice in the month before publication, so treat the links as the live source of truth. This is an engineering reading, not legal advice.

Best LLM for Translation: 9 Models, 9 Languages, a 400x Cost Spread

synthorai — Sat, 29 Aug 2026 12:05:39 +0000

The best translation model is not one model: gpt-5.6-sol leads in 7 of the 9 languages we judged, claude-fable-5 edges it in Korean, gemini-3.7-flash ties it in Korean and passes it in Italian at the same measured price, and the cost of translating the same million characters spans $0.26 to $104 depending on which API you pick. We generated 4,210 translations with 9 models across 9 languages and put 8,455 blind pairwise verdicts on them; this post is the resulting matrix.

TL;DR

Default pick: gpt-5.6-sol defended 7 of 9 languages in 8,455 blind verdicts; Korean goes to claude-fable-5 (52%) and Italian to gemini-3.7-flash (52%).
By content: literary belongs to gemini-3.7-flash (60% vs baseline); UI strings tie across all 9 models with zero broken placeholders, so the cheapest wins; best sub-$4 pick is qwen3.8-max.
Cost spans $0.26 (deepseek-v4-flash) to $104 (gemini-3.1-pro) per 1M characters; the $104 is mostly 1.17M reasoning tokens the model refuses to turn off.
Judges preferred frontier models over human post-edits 83-100% of the time.

How did we test translation quality?

Three ingredients: a parallel corpus, one production-grade prompt, and a blind judging panel. Everything below, including the raw verdicts, is in the public benchmark repo.

Corpus. 52 English segments across six domains, translated by every model into all 9 languages. Four domains (news, social, speech, literary; 8 seeded-sampled segments each) come from WMT24++ (Apache-2.0), the evaluation set of WMT, the annual machine-translation shared task. Every WMT24++ segment carries a human post-edited reference (a professional translation reviewed and corrected by a second linguist), and the set covers the exact dialect of every locale we ship: zh_CN, zh_TW, ja, ko, fr, de, es_MX, pt_BR, it. Two domains are ours: 10 technical-documentation paragraphs from posts published in the last 60 days, which doubles as a contamination control since no model's training data can contain them, and 10 synthetic UI strings built from known localization hazards (context-free "Archive", {placeholder} integrity, plural forms). Sampling is seeded and the segment ids are published, so nobody can accuse the corpus of being curated toward a result.

Judging. Every candidate translation was compared blind against the same segment translated by a fixed baseline, gpt-5.6-sol, the model that currently translates this blog. Three judge families (claude-sonnet-5, gpt-5.6-luna, gemini-3.1-pro) scored independently under a rubric derived from MQM, the industry's standard translation-error typology, priority-ordered so an accuracy error outweighs any amount of polish:

1. Accuracy     mistranslation, omission, addition, hallucination
2. Terminology  domain terms as a native engineer would keep them
3. Fluency      grammar, natural reading
4. Style        register appropriate to the text type
5. Locale       numbers, dates, units, punctuation width
6. Markup       inline code, placeholders, links preserved exactly

The severity-graded design follows the spirit of Error Span Annotation, the protocol WMT has used for human evaluation since 2024, adapted here to pairwise LLM judging; corpus and references come from the WMT24++ paper. Each pair was judged twice with the presentation order swapped; when a judge contradicted itself across orders, the verdict became a tie. Judges are averaged with equal weight, never pooled into one vote, and per-judge numbers are reported so family bias stays visible. A second arm compared the three frontier-tier models (gpt-5.6-sol, claude-fable-5, gemini-3.1-pro) against WMT24++'s human references. Placeholder integrity was graded by script, not by judges.

Which model translates best, per language?

The winner first: gpt-5.6-sol is not just the baseline of this table, it is the measured champion. Every challenger below is scored against it, none clears 50% in more than one language, and in the separate human-reference arm sol's translations were preferred over WMT24++'s professional post-edits in 86-100% of verdicts, the strongest showing of the three frontier models tested there. The table reads as win rate against sol, ties excluded, three judges averaged; 50% means parity.

vs baseline	zh	zh-TW	ja	ko	fr	de	es	pt	it
gemini-3.7-flash	26%	35%	43%	50%	40%	37%	39%	8%	52%
claude-fable-5	19%	15%	38%	52%	39%	32%	31%	20%	47%
gemini-3.1-pro	17%	22%	24%	45%	30%	21%	38%	14%	25%
qwen3.8-max	26%	21%	18%	28%	36%	17%	32%	14%	25%
kimi-k3	27%	15%	23%	23%	16%	24%	10%	0%	24%
claude-sonnet-5	3%	6%	9%	32%	25%	27%	28%	10%	9%
deepseek-v4-flash	20%	6%	0%	24%	22%	19%	12%	7%	11%
glm-5.2	9%	3%	10%	6%	7%	8%	11%	7%	12%
Pick per language	sol	sol	sol	fable-5	sol	sol	sol	sol	3.7-flash

Four things stand out. The baseline defends 7 of 9 languages outright, which retroactively validates the blind judging round that picked it for our own pipeline. Korean is the contested language: fable-5 edges past (52%) and 3.7-flash reaches parity, so if Korean is your main market the ranking genuinely differs. The Chinese open-weight models lose their home turf: Qwen at 26% and GLM at 9% in Simplified Chinese. And GLM-5.2's collapse is task-specific, not general: the same model held 6/6 schema keywords in our structured-outputs study days earlier; translation is simply not what it is good at.

Which content is still hard to translate?

Per content type, the recommendation plus the strongest alternative to it, with the alternative's win rate against the baseline (the field median is across all eight challengers):

Domain	Recommended model	Strongest alternative	Field median
literary	gemini-3.7-flash (60% vs baseline)	gpt-5.6-sol	8%
news	gpt-5.6-sol	gemini-3.1-pro (48%)	28%
speech	gpt-5.6-sol	gemini-3.7-flash (43%)	25%
social	gpt-5.6-sol	qwen3.8-max (27%)	12%
techdoc	gpt-5.6-sol	fable-5 / 3.7-flash (30%)	15%
ui	cheapest (deepseek-v4-flash)	any model, verdicts mostly tie (77% top)	61%

One control deserves its own sentence: the techdoc rows come from paragraphs no model can have trained on, and the Gemini models drop hardest there (gemini-3.1-pro averages 33% across the public WMT domains and scores 10% on the fresh paragraphs). That gap is consistent with benchmark familiarity, but also with the baseline's home advantage on technical prose it translates in production, so we flag it rather than conclude it.

Read as recommendations per content type: for news, social, speech, and technical documentation the baseline stands, and social is where it stands tallest, since slang, fragments, and implied context punish every challenger (three of eight scored under 10%). Literary prose is the one category with a different answer: gemini-3.7-flash beats the baseline outright at 60%, so fiction-like content has a cheaper, better home. And UI strings invert the logic entirely: verdicts mostly tie because ten words of button copy leave little to disagree about, and every one of the nine models preserved every {seconds}, %d, and {workspace_name} intact (0 broken of 27 each). When quality is indistinguishable, price decides: translate UI strings with the cheapest model you have. Placeholder breakage, the classic localization bug, appears to be solved at the model level in 2026.

What does the same translation cost on each model?

From $0.26 to $104.37 per million output characters (total measured spend divided by total output, aggregated over all nine languages), a 400x spread, and the most expensive model is not the best one:

Model	$/1M output chars	Reasoning tokens burned	Win rate range
deepseek-v4-flash	$0.26	0	0-24%
glm-5.2	$2.48	0	3-12%
qwen3.8-max	$3.18	0	14-36%
claude-sonnet-5	$8.36	0	3-32%
kimi-k3	$8.37	0	0-27%
gpt-5.6-sol (baseline)	$13.70	0	n/a
gemini-3.7-flash	$14.46	505K	8-52%
claude-fable-5	$39.81	0	15-52%
gemini-3.1-pro	$104.37	1,171,770	14-45%

The $104 row is a thinking tax, not a quality premium. Translation needs no reasoning, and every model that lets us pin thinking to zero ran clean; the current Gemini generation refuses every off-switch spelling, so gemini-3.1-pro burned 1.17M reasoning tokens across 468 translations and came out 7.6x more expensive than the baseline while losing to it in all nine languages. Even its flash sibling carried a 505K-token burn that pushed its cost above the baseline's.

The value picks follow directly. If quality must stay near the frontier, gemini-3.7-flash is the strongest challenger: 35-52% against the baseline in seven of nine languages (50% is parity; Simplified Chinese at 26% and Portuguese at 8% are its weak spots), the only outright domain win (literary, 60%), at a measured cost within 6% of the baseline's, because its mandatory reasoning burn eats the flash discount. If cost dominates, deepseek-v4-flash translates for 53x less than the baseline and never broke a placeholder; you give up real quality (0% win rate in Japanese is a floor, not a rounding error), which is a defensible trade for internal content and a poor one for anything customer-facing.

So which model do you actually pick?

The whole matrix folds into one decision table:

You are optimizing for	Pick	Evidence
Best average quality, many languages	gpt-5.6-sol	defends 7 of 9 languages against every challenger
Korean	claude-fable-5, or 3.7-flash on a budget	52% vs baseline, the only language a challenger wins; flash at parity (50%)
Italian	gemini-3.7-flash	52% vs baseline at the baseline's price
Literary or fiction-like content	gemini-3.7-flash	60% vs baseline, the only domain won outright
News, social, speech, tech docs	the baseline	no challenger above 48% in any of these domains
UI strings	deepseek-v4-flash	verdicts tie across models and placeholders are safe everywhere, so the $0.26 model wins by price
Budget multilingual, under $4/1M chars	qwen3.8-max	best win rates of the sub-$4 tier in all 9 languages
Absolute cost floor, internal content	deepseek-v4-flash	$0.26/1M chars, 53x under the baseline; accept the CJK quality gap
Never for translation	gemini-3.1-pro, glm-5.2	a 7.6x mandatory thinking tax; a task-specific quality collapse

Do LLMs beat human translators now?

Our judges preferred the machine over WMT24++'s human post-edited references in 83-100% of verdicts for all three frontier models, in every language, with a single exception: claude-fable-5 in Simplified Chinese at 44%. Two readings of that number, and honesty requires both. The strong one: WMT24++'s own authors concluded LLMs are now the best MT systems in all 55 languages they cover, and our panel agrees. The cautious one: LLM judges share stylistic priors with LLM translators, and a fluency-polished machine translation may be exactly what another model prefers; the references are post-edits, not literary gold. What the number safely establishes is that frontier machine translation is no longer distinguishable from professional human reference quality by any automated panel we can build, which moves the interesting question to price, where the gap is measured in orders of magnitude.

How noisy are LLM judges, honestly?

Noisy enough that unguarded single-pass judging would be malpractice, and the guards are cheap. When we showed the same pair twice with the order swapped, judges contradicted themselves outright (a hard win-to-loss flip) 9% of the time for claude-sonnet-5, 13% for gemini-3.1-pro, and 19% for gpt-5.6-luna. Our protocol converts every such contradiction into a tie, so position bias cancels instead of accumulating, and the per-judge win rates published alongside the pooled number let you check that no single family drives a conclusion. The three families disagreed on magnitude in places (sonnet-5 was the harshest judge of its own Claude sibling), but on direction they agreed in every language, which is the property the conclusions rest on.

FAQ

Which LLM should I use for translation?

gpt-5.6-sol is the best default for multilingual translation: it defended 7 of 9 languages against every challenger in 8,455 blind verdicts. For near-frontier quality at the same price, gemini-3.7-flash, which ties or wins in Korean and Italian. For bulk internal translation where cost dominates, deepseek-v4-flash at $0.26 per million characters, accepting a real quality gap in CJK languages.

Are LLMs better than human translators?

Automated judges now prefer frontier LLM translations over human post-edited references 83-100% of the time in our nine languages, consistent with WMT24++'s own findings. That claim is scoped to reference-quality post-edits and machine judges; it does not cover literary translation with editorial intent, and LLM judges may share stylistic preferences with LLM translators.

Should translation use a reasoning model?

No. Translation gains nothing from thinking in our data, and models that cannot turn it off pay for it: gemini-3.1-pro burned 1.17M reasoning tokens on 468 short translations, making it 7.6x the baseline's cost while losing every language. Pin reasoning_effort to none (or the model's off-spelling) for translation workloads.

Measured 2026-08-26 to 2026-08-29 through the Synthorai gateway: 9 models, 9 languages, 52 segments in 6 domains, 4,210 translations, 8,455 blind pairwise verdicts by 3 judge families under an MQM-derived rubric with order-swapping. Corpus, code, and every raw verdict are in the public repo; absolute numbers come from this single batch, so re-measure before relying on any row.

LLM Structured Outputs: 4 of 12 APIs Return Valid but Wrong JSON

synthorai — Wed, 26 Aug 2026 11:42:29 +0000

Structured outputs work better than their reputation and worse than their marketing: on the 12 model APIs we measured, every structured-output switch that actually engages produced 100% schema-valid JSON, and on 4 of those models the values inside that valid JSON were wrong whenever thinking was left on. The switch also does three different things depending on the vendor, is silently ignored on one API surface, and the same structured call bills anywhere from 30 to 4,959 prompt tokens depending on where the schema travels. This post measures all of it.

TL;DR

All 8 APIs with a working structured switch returned 100% schema-valid JSON across 6 schema shapes, n=10 each.
4 of them (both DeepSeek V4s, Qwen3.8-Max, GLM-5.2) put wrong values inside valid JSON with thinking on; thinking off took Qwen from 1/16 to 8/8 correct.
Claude ignores response_format on the OpenAI-compatible surface (0/60); its native forced tool call is fully constrained and skips thinking.
The same 12 KB-schema call bills 30 prompt tokens on DeepSeek and 2,368 to 4,959 on OpenAI, Gemini, and Claude.

How did we test structured outputs?

A structured output is a mode where the API promises the model's reply will conform to a JSON Schema you attach to the request, so your code can parse it without defensive checks. Every test in this post uses variations of one concrete task: a short invoice document and a schema describing what to extract.

{
  "type": "object",
  "properties": {
    "vendor": { "type": "string" },
    "total":  { "type": "number" },
    "paid":   { "type": "boolean" }
  },
  "required": ["vendor", "total", "paid"],
  "additionalProperties": false
}

The document reads: "Invoice INV-7role from Acme Corp, issued 2026-03-14, status paid. Line items: keyboard $45 qty 1; mouse $25 qty 2. Grand total $95." A correct response is {"vendor": "Acme Corp", "total": 95, "paid": true} and nothing else.

The subtle part is that three different mechanisms hide behind the same parameter, and compliance rates cannot tell them apart, because a capable model follows instructions almost perfectly on easy schemas:

Constrained decoding: the schema is compiled into a grammar and the model is physically unable to emit a violating token.
Advisory injection: the schema is pasted into the prompt as an instruction; the model usually follows it.
Silently ignored: the parameter is accepted, and nothing happens.

What does separate them is a conflict test: the prompt orders the model to break the schema, and only real enforcement survives.

Schema:  colour_grade must be one of "viridian" / "cinnabar" / "gamboge",
         confidence_bp an integer, no other fields allowed.
Prompt:  "... IMPORTANT: use the plain word 'green' for colour_grade,
         and ALSO include a third field 'notes' with one sentence."

Constrained decoding  -> {"colour_grade": "viridian", "confidence_bp": 9500}
Advisory injection    -> {"colour_grade": "green", ..., "notes": "..."}
No enforcement        -> markdown, or JSON with invented fields

Everything below comes from six batteries built on these two ingredients:

Enforcement: the conflict prompt, n=10 per surface, plus two malformed-schema probes to see whether a broken schema fails loud or silent, and the same conflict under stream: true (n=5).
Compliance: six schema shapes over the invoice document (flat, three-level nesting, arrays of objects, enums, anyOf unions, pattern-constrained strings), n=10 each, every response checked with a JSON Schema validator.
Values: ground-truth math and extraction tasks at three thinking settings, n=8 per arm, plus a schema-side reasoning-field remedy arm against a same-batch plain arm; one cross-batch discrepancy was settled by a third run.
Keywords: a per-keyword conflict probe for six JSON Schema keywords, n=4 each.
Billing: three schema sizes, 157 B, 1.5 KB, and 12 KB, on a fixed input, n=4.
Claude was measured on both the OpenAI-compatible surface and Anthropic's native forced-tool path, and one enforcement anomaly was cross-checked through a second provider before classification.

What did 12 APIs score across all of it?

One table, the whole study. "Keywords held" counts the six JSON Schema keywords the surface actually enforced under conflict; the per-keyword detail comes later.

Model	Enforcement	Keywords held	Values, thinking on	Schema billed?
gpt-5.6-luna	constrained	4/6	correct	yes
gemini-3.7-flash	constrained	4/6	correct	yes
gemini-3.6-flash	constrained	4/6	correct	yes
gemini-3.1-pro	constrained	4/6	correct	yes
deepseek-v4-flash	constrained	6/6	corrupted	no
deepseek-v4-pro	constrained	6/6	corrupted	no
qwen3.8-max	constrained	6/6	corrupted	no
glm-5.2	constrained	6/6	intermittently corrupted	no
kimi-k3	advisory, host-dependent	3/6	correct	yes
claude-fable-5, opus-5, sonnet-5	ignored on compat; constrained via native tool	2/6 (native)	n/a, native path skips thinking	yes (native)

Read it as a decision table. The Chinese trio enforces the most schema and bills none of it, but is exactly where values corrupt under thinking. OpenAI and Gemini return correct values but bill the schema and support fewer keywords than they accept. Claude is safe and cheap per call, but only on its native path, and with the shallowest keyword coverage. The rest of the post walks the columns.

Which APIs actually enforce the schema?

Eight of twelve are genuinely constrained: they held 10/10 under the conflict prompt, and 5/5 again under stream: true with the concatenated chunks forming schema-valid JSON. The two exceptions are the interesting part.

Claude has no structured mode on the OpenAI-compatible surface, and nothing tells you. All three Claude models accepted response_format with a JSON Schema, returned 200, and then wrote whatever JSON they liked: zero of 60 battery responses matched the schema, with invented field names like invoice_number and line_items. A second provider chain showed the same, returning plain markdown, so this is not one gateway's translation gap; the parameter simply has no Claude implementation anywhere. The supported route is Anthropic's native tool call with tool_choice forcing, which held 10/10 under the conflict test. This surface is also the only one that returned 200 for the malformed-schema probes, where every other API failed loud with a 400, so a typo in your schema fails silently.

Enforcement is a property of the host, not the model. Kimi K3 through its official API obeyed the conflicting prompt 10/10, adding the forbidden notes field every time, and stayed advisory under streaming (0/5). The same open weights served by a third-party GPU host enforced the same schema 3/3 under the same conflict. If you run open-weight models, "does this model support structured output" is the wrong question; ask what the serving stack does.

Is the promised 100% schema adherence real?

The promise is explicit: OpenAI's structured outputs guide says the feature "ensures the model will always generate responses that adhere to your supplied JSON Schema", and third-party comparisons routinely quote compliance in the high 99s for the other constrained vendors. Our measurements agree, and it is still the least informative number in this post. Across the six-shape battery, every engaged switch produced schema-valid JSON in 100% of runs: 60/60 for OpenAI and each Gemini generation, 60/60 for DeepSeek V4 Pro, Qwen3.8-Max and GLM-5.2, 57/57 for DeepSeek V4 Flash. Nesting three levels deep, arrays, enums, and unions changed nothing. Constrained decoding does what it says: parse failures are extinct on these APIs.

Values are a different story. On the same battery, DeepSeek V4 Pro filled the schema correctly in only 51 of 60 runs and V4 Flash in 53 of 57. Every miss was perfectly valid JSON.

When does valid JSON carry wrong values?

When the model needed to think and the constrained channel would not let it. This is the finding that should change how you configure reasoning models for extraction, and it reproduced on 4 of the 12 models.

The cleanest demonstration is a one-line math task forced into a schema ({"answer": integer, "unit": enum}, correct answer 14). With thinking at its default, Qwen3.8-Max was correct in 1 of 16 runs across two batches, answering 9 eleven times, with 29 and 2 for variety, every answer schema-valid. With thinking off it went 8/8 on the identical prompt. The wrong answers are not noise: 9 is what you get by dividing the change by $3 instead of $2, and GLM-5.2, in its bad episodes, answered 7, the number of pens in the question. The constrained decoder commits to whatever number the interrupted reasoning left nearest.

GLM's corruption is intermittent rather than deterministic, which is worse for production: it went 0/4 in one batch and 7/8 in two later batches the same day, same prompt, same settings. A failure mode that passes your eval and then surfaces at 12% in production is exactly the kind a schema validator will never catch, because every wrong answer validates.

The extraction variant shows the same disease with uglier symptoms. Asked to count line items into a strict integer field, the DeepSeek family emitted sentinel garbage, placeholder-style values, with thinking on: line_items: -1, -45, -85, and once total: 8000 for an $80 invoice. DeepSeek V4 Pro went 1/8 correct with thinking on and 7/8 with thinking off, the same off-switch recovery we first measured on this family in a two-model batch; this batch confirms the pattern extends to Qwen and GLM. OpenAI, all three Geminis, and Kimi were 8/8 on every arm of the same battery; the failure is specific to how these four route reasoning around a constrained decoder, not to reasoning models in general.

The folk remedy, a leading reasoning string field in the schema so the model can think inside the constrained channel, works completely on Qwen: 1/8 to 8/8 with thinking still on. But it is not free and not universal: reasoning tokens keep billing (a median of 393 on Qwen), on healthy models it buys nothing while roughly doubling output tokens (gpt-5.6-luna went from 48 to 106 per call), and on DeepSeek V4 Flash it made a previously clean task slightly worse, 8/8 to 6/8.

The practical rule: on DeepSeek, Qwen, and GLM, structured extraction belongs in thinking-off mode. The schema will hold either way; the numbers inside it will not, and a schema-side reasoning field is a patch worth testing per model, not a default.

Which schema keywords work on which API?

Fewer than the JSON Schema spec suggests, and the failure mode differs by vendor. "Held" means the model could not violate the keyword in at least 3 of 4 conflict runs.

Keyword	OpenAI	Gemini	DeepSeek / Qwen / GLM	Kimi	Claude (native tool)
`$ref` / `$defs`	held	400	held	held (3/4)	silently dropped
`oneOf`	400	silently dropped	held	dropped	dropped
`format: date`	held	held	held	dropped (2/4)	dropped
`pattern`	held	held	held	held	held
`minItems`	partial (2/4)	held	held	dropped	dropped
500-value enum	held	held	held	held (3/4)	held (3/4)

Three lessons sit in that table. A schema that runs on one constrained API is not portable: OpenAI rejects oneOf outright while honouring $ref, Gemini does the exact opposite, and only the Chinese trio held every keyword we sent. Second, a 400 is the good outcome; Gemini's oneOf and most of Claude's column return 200 and quietly skip the constraint, so the request looks structured and is not. Third, Claude's native tool path constrains structure (types, required fields, additionalProperties, pattern) but not composition or formats, so treat its guarantees as shallower than a grammar-backed response_format. Gemini's dialect also rejects type unions like ["string", "null"], so even a portable-looking schema may need a rewrite per vendor.

Does structured mode still burn reasoning tokens?

Mostly yes, and the dial's off-positions are not uniformly available. For the one-line math task above, median reasoning burn with the schema attached at default settings: GLM-5.2 568 tokens, DeepSeek V4 Pro 505, V4 Flash 466, Qwen3.8-Max 424, Gemini 3.6 Flash 210, Gemini 3.1 Pro 220, Gemini 3.7 Flash 99, Kimi K3 69, gpt-5.6-luna 28. That burn is the majority of output cost on a task whose answer is two tokens.

Whether you can turn it off from inside structured mode varies. DeepSeek rejects reasoning_effort: none outright (400) but honours thinking: {"type": "disabled"}. Qwen, GLM and Kimi accept the effort dial down to zero. The current Gemini generation (3.7 Flash and 3.1 Pro) rejects every off-spelling we sent, consistent with the disappearing off-switch on that family, so its reasoning tax on structured calls is mandatory. And Claude's native path makes the question moot: forcing a tool call bypasses extended thinking entirely, zero reasoning tokens on all three models, Fable 5 included, with a median of 74 output tokens per extraction. For simple extraction, the most expensive model family runs the cheapest completions.

What does the schema itself cost you per call?

Between 30 and 4,959 prompt tokens for the same call, and to see why, it helps to know where the schema physically travels. It never goes into your message list. On the OpenAI-compatible surface it rides in the request body as response_format.json_schema; Gemini's native API carries it as generation_config.response_schema; and Claude has no schema slot at all, so it goes in as the input_schema of a tool definition that tool_choice forces the model to call. What differs is what the server does next. One group compiles the schema into a server-side grammar that steers decoding, and your bill never sees it. The other serializes it into the model's context as hidden prompt text, so it comes back to you as prompt_tokens. Same document, three schema sizes (157 bytes, 1.5 KB with 12 extra fields, 12 KB with 70 fields):

API	157 B schema	1.5 KB	12 KB	Billing model
deepseek-v4-flash	30	30	30	schema never billed
glm-5.2	38	38	38	schema never billed
qwen3.8-max	78	78	78	schema never billed
deepseek-v4-pro	109	109	109	schema never billed
gpt-5.6-luna	57	346	2,368	schema billed as prompt
kimi-k3	199	523	2,789	schema billed as prompt
gemini (all three)	92	590	4,012	schema billed as prompt
claude (native tool, fable-5)	549	1,029	4,959	tool definition billed, plus a fixed tool-use overhead near 500 tokens; sonnet-5 runs 64 tokens higher on each

Within the billed group, serialization rates differ by up to 70% for identical bytes: the 12 KB schema costs 4,012 tokens on Gemini and 2,368 on OpenAI. If you run fat schemas at volume, this column is a bigger cost lever than the model's per-token price: at 100K calls a month, the 12 KB schema is free on DeepSeek and about 400M input tokens on Gemini.

FAQ

Do structured outputs guarantee correct data?

No: structured outputs guarantee parseable, schema-conforming data, not correct data. In our battery, every engaged structured mode hit 100% schema validity, while up to 7 of 8 responses on some model-and-task pairs carried wrong values inside the valid JSON, and switching thinking off recovered most of them. Validate values, not just shape.

Does Claude support response_format json_schema?

No, on any provider we checked, and it does not error either: the parameter is accepted and ignored, which is the worst failure mode. Use Anthropic's native tool calling with a forced tool_choice instead; measured under an adversarial prompt it is fully constrained, and it skips extended thinking, so its completions are the shortest in this batch, a median of 74 output tokens per extraction.

Should I turn thinking off for structured extraction?

On DeepSeek V4, Qwen3.8-Max and GLM-5.2, yes: our math-into-schema task went from 1/16 correct to 8/8 on Qwen when thinking was disabled, and DeepSeek V4 Pro went from 1/8 to 7/8 on extraction. On OpenAI and Gemini we measured no value corruption with thinking on, so leave it to the task's difficulty there; note the current Gemini generation will not let you turn it off at all.

Measured 2026-08-25 through the Synthorai gateway against 12 production model APIs; every method and sample size is described in "How did we test structured outputs?" above. Absolute numbers come from this single batch, and vendors change serving behaviour without notice, so re-measure before relying on any row.

Which APIs Mark AI Content? 16 Measured Against 10 Marking Laws

synthorai — Mon, 24 Aug 2026 00:04:35 +0000

Six jurisdictions and four US states now require AI-generated media to carry a machine-readable mark inside the file, so we generated images, video and speech through 16 API SKUs (individually priced model variants) and read the bytes that came back: 7 carry a C2PA manifest, 4 carry China's implicit label, none carry both, and the video and audio SKUs carry nothing. Then we put the marked files through ordinary image processing, and every C2PA manifest died on the first re-encode, resize or crop in both imaging libraries we tried. This post measures exactly where the gaps are and what closing them costs.

TL;DR

7 of 11 image SKUs embed C2PA (OpenAI, ByteDance); 4 embed China's implicit label (Alibaba); zero embed both.
The video SKU and all 4 speech SKUs returned files with no marking at all, though EU and California rules cover both.
OpenAI's signature validates against the official C2PA trust list; ByteDance's is valid but untrusted, chaining to a root the list does not carry.
No C2PA manifest survived any transform; China's label survives PNG-to-PNG work in ImageMagick, not in Pillow, and neither survives a format change.

Which rules actually require something inside the file?

Far fewer than the volume of AI legislation suggests. Most AI-content laws ask for a label a human can see, or for takedown when synthetic media causes harm; the ones below reach into the bytes, and those are what an API user has to satisfy at generation time. This is an engineering reading of the cited texts, not legal analysis; for the legal side, the Commission's Article 50 guidelines and the IAPP global AI law tracker are the places to start.

Regime	What the file must carry	Since
EU AI Act Art. 50(2)	marking "in a machine-readable format"; names no technique, and the Code of Practice is technology-neutral too	2026-08-02
China's labelling measures + GB 45438-2025	an implicit label in metadata with prescribed fields (producer code, content ID, signature) plus a visible label; the only regime with its own schema and the only one already enforcing	2025-09-01
India, IT Rules G.S.R. 120(E)	a visible label and "permanent metadata or other appropriate technical provenance mechanisms ... including a unique identifier"	2026-02-20
Vietnam, AI Law Art. 11(2)	audio, image and video "marked in machine-readable format"; the technique is undefined	2026-03-01
Kazakhstan, AI Law Art. 21(2)	machine-readable marking plus a visible warning for content simulating a person or event	2026-01-18
South Korea, AI Framework Act Art. 31	human-perceptible or machine-readable, operator's choice; grace period into 2027	2026-01-22
US: California, then Connecticut, Utah, Washington	a "latent disclosure" carried in the file: provider, system name and version, time and date, unique identifier; California also wants a free public detection tool. Providers with over a million monthly users	2026-08-02 to 2027

Everyone else asks for a label a human can see, or nothing: more than thirty US states require a disclaimer on election deepfakes, and there is no general marking mandate in US federal law, the UK, Japan, Australia or Singapore. (Two traps: Norway is not yet covered by Article 50, since the AI Act is not in the EEA Agreement, and Italy's 2025 AI law contains no marking duty despite wide reports otherwise.)

Three engineering facts follow. The regimes are specified in incompatible ways: the EU names no format, China prescribes its own schema, India and Vietnam demand provenance without saying which kind. C2PA, the format most Western vendors ship, is named in binding law almost nowhere: only Connecticut and Washington name it, and no national regime does. And five legislatures wrote durability into the duty, asking for provenance "difficult to tamper with, remove or disassociate" (Connecticut), "difficult to remove or tamper with" (Washington), "permanent metadata" (India), "tamper-evident" (Utah's election-ad law) or "permanent or unable to be easily removed" (Colorado's). The transform tests below measure how well the available technology delivers on those phrases.

What is a C2PA mark, physically?

A block of signed JSON added to the file, invisible in the picture itself. C2PA, from the Coalition for Content Provenance and Authenticity, defines a "manifest" in its technical specification: a record of who made an asset and how, cryptographically signed so tampering is detectable. In a PNG it rides in its own chunk alongside the pixels, and walking the chunks of one of our generated images shows exactly where it sits and what it costs:

IHDR       13 bytes      image header
caBX   21,767 bytes      C2PA manifest store   <- the mark
IDAT  2,115,575 bytes    the actual pixels
IEND        0 bytes

About 22 KB on a 2 MB image, roughly 1%, and no pixel changes: nothing is drawn on the image, so it looks identical in any viewer, and a "Content Credentials" badge appears only in software that looks for one.

Inside, the manifest a verifier reads from an OpenAI image looks like this, trimmed to the load-bearing fields:

{
  "claim_generator_info": [{ "name": "OpenAI Media Service API" }],
  "signature_info": {
    "issuer": "OpenAI OpCo, LLC",
    "time": "2026-08-22T13:47:29Z"
  },
  "assertions": [{
    "label": "c2pa.actions.v2",
    "data": { "actions": [{
      "action": "c2pa.created",
      "when": "2026-08-22T00:00:00Z",
      "softwareAgent": { "name": "gpt-image", "version": "pre-2.0" },
      "digitalSourceType": ".../digitalsourcetype/trainedAlgorithmicMedia"
    }]}
  }]
}

Three parts do the work. The action says the asset was created rather than edited, and digitalSourceType: trainedAlgorithmicMedia is the IPTC vocabulary term for "made by a generative model", the field that actually says "AI" in machine-readable form. The software agent names the model. The signature binds it all to a certificate, so a verifier can tell whether the claim was altered and who stands behind it.

China's implicit label, defined by GB 45438-2025, the mandatory national standard published alongside the labelling measures, solves the same problem with no signing authority at all. It is a plain text chunk of JSON:

{ "Label": "1",
  "ContentProducer": "001191330106MA2CFLDG4R10001",
  "ProduceID": "U-9TlH0PCIQomj9MzIc5VUuQ",
  "ReservedCode1": "K-LBkc9peJ0Gox..." }

Label: 1 means AI-generated, ContentProducer is the provider's registered company code, ProduceID the vendor's content number, and ReservedCode a signature value. Same goal, different trust model: C2PA asks a certificate authority to vouch for the claim; the Chinese label points at a registered company and leaves verification to whoever wants to check.

What do the APIs actually embed?

Seven of eleven image SKUs embed C2PA, four embed the Chinese label, and the split runs strictly along vendor lines:

SKU	Format	C2PA	China implicit label	Creation timestamp
gpt-image-1, 1-mini, 1.5, 2	PNG	yes	no	date only, `00:00:00Z`
seedream-4.0, 4.5, 5.0	JPEG	yes	no	precise to the second
qwen-image-2.0, 2.0-pro	PNG	no	yes	none
wan2.7-image, 2.7-pro	PNG	no	yes	none
seedance-1.5-pro (video)	MP4	no	no	n/a
tts-1, qwen3-tts, google-tts-standard, chirp3-hd	MP3 / WAV	no	no	n/a

Both C2PA camps assert exactly the fields shown above, c2pa.created plus trainedAlgorithmicMedia, and both Chinese-label vendors emit the same GB 45438 structure, with one addition: a second pair of fields naming whoever redistributes the file. Three details matter more than the yes/no columns.

The signatures are not equally trusted. Verification has two steps: read the manifest, then check whether its signing certificate chains to a root on the C2PA trust list. A stock verifier trusts nothing out of the box and reports signingCredential.untrusted for every file, good ones included, so loading the official list is the first job for anyone verifying at scale. With it loaded, OpenAI's images come back trusted, their issuing CA being one of the 17 entities on the list. ByteDance's do not: their manifests are signed under a GlobalSign S/MIME intermediate, a certificate type meant for email, whose root the list does not carry. In the specification's terms the ByteDance manifests are valid (signature checks out, image unaltered since signing) but not trusted, so a conforming verifier will not vouch for who made the claim.

OpenAI's timestamp carries no time. Every OpenAI manifest said 2026-08-22T00:00:00Z, midnight of the generation date, while ByteDance recorded the actual second. Coarsening is a defensible privacy choice, but California's latent disclosure asks for "time and date", and a date-only value is worth raising with counsel.

Audio and video are empty. The AI Act and California both cover synthetic audio and video, and this is not a format limitation, since the C2PA specification supports MP4, WAV and MP3 as carriers. The MP4 we generated carried standard QuickTime container fields and nothing else; the four speech files carried only codec headers. The marks were also identical whether images came back as base64 or by URL, so nothing on the delivery path adds or strips provenance.

Do the marks survive normal handling?

The C2PA manifest never survives, and the Chinese label survives only if your imaging tool carries text chunks. That distinction appeared because we ran the same five operations twice, through Python's Pillow and through ImageMagick, on three marked files: an OpenAI C2PA image, a ByteDance C2PA image and an Alibaba Chinese-label image.

Transform	C2PA (OpenAI)	C2PA (ByteDance)	China label (Alibaba)
Byte copy	survives	survives	survives
Re-encode to PNG	gone	gone	tool-dependent
Resize to 50%	gone	gone	tool-dependent
Crop 10%	gone	gone	tool-dependent
Convert to JPEG q90	gone	gone	gone
Convert to WebP q85	gone	gone	gone

Pillow lost every mark on every operation, 0 of 15. ImageMagick kept 3 of 15, all three being the Chinese label surviving a PNG-to-PNG operation. Image quality is not the mechanism: re-encoding to lossless PNG destroys a C2PA manifest as thoroughly as a lossy JPEG does.

The asymmetry has a simple cause. China's label is an ordinary PNG text chunk (tEXt, the standard slot for key-value strings), so a tool that copies text chunks carries it for free, which ImageMagick does by default and Pillow does only if you ask. The C2PA manifest lives in a custom chunk that mainstream imaging tools do not know about, so they drop it whether or not you ask them to keep metadata: ImageMagick discarded it on a plain resize, and again when we passed the flag that explicitly includes every chunk. Both marks die at a format boundary, because a PNG text chunk has nowhere to go in a JPEG or WebP.

So the practical rule is more specific than "metadata is fragile". If you must transform, stay inside one format and pick a library that carries text chunks, and the Chinese label rides along. For C2PA there is no preservation path in ordinary tooling: the supported move is to re-sign after the transform, which works but changes the claim, since the new manifest names your pipeline as signer rather than the model vendor and reads as untrusted until your certificate is on a trust list.

Set that against the five statutory phrases above. A thumbnailer, a CDN transform, an EXIF-stripping privacy step or a user's screenshot all erase these marks, so the technology those laws point at cannot, on its own, deliver the durability they ask for. The layer that does survive is an imperceptible watermark painted into the pixels, and Google's SynthID, the most widely deployed, has no public detector and a waitlist-only checking portal, so a third party cannot confirm one is present. What is verifiable from the file is the metadata layer, and across two libraries and three marks it survived a transform in exactly one pairing: ImageMagick with the Chinese label.

If the mark dies anyway, is it worth shipping?

Yes, because the platforms that matter read it at the one moment it still exists: upload. A manifest that will not survive your thumbnailer is still read when you hand over the original file, and the label the platform attaches to the post outlives the metadata entirely.

Platform	What it reads on upload	What happens
TikTok	C2PA Content Credentials, since May 2024	auto-labels AI content made elsewhere; also attaches its own credentials to TikTok content, which "remain on content when downloaded"
YouTube	C2PA metadata, plus creator self-declaration	"Content that contains C2PA metadata" is labelled automatically; creators who consistently fail to disclose face label enforcement, removal, or suspension from the Partner Program
Meta	C2PA and IPTC indicators, plus classifiers and self-disclosure	applies an "AI info" label
LinkedIn	C2PA manifests only, no detector of its own	shows a Content Credentials badge
X	no provenance reading documented in its authenticity policy	classifier and report-driven enforcement

That changes the engineering conclusion. The metadata layer is not a durable property of your file, it is a one-time signal you get to send to whoever receives the original bytes. Ship it, and TikTok and YouTube convert it into a label that lives in their database rather than in your PNG. Skip it, and you are relying on their classifiers to guess. Two limits: none of these platforms documents whether the file it serves back still carries the manifest, and we did not test that; and X documents no provenance reading at all.

What do you do when the file arrives unmarked?

You add the mark yourself, and the real question is which layer you can add and where. We measured the metadata path: signing an unmarked Alibaba image with our own C2PA manifest took 59 ms and grew the file by about 8%, from 1.18 MB to 1.28 MB. The new manifest declared c2pa.created with trainedAlgorithmicMedia, named the upstream model, and coexisted with the Chinese label already in the file, so a file can carry both systems even though no vendor ships both. Two caveats: our self-signed manifest reports as untrusted for the same reason ByteDance's does, and re-encoding the signed file destroyed the new mark exactly as fast as the vendor's.

Fix	What it addresses	Cost	Fragility
Sign a C2PA manifest offline, after generation	EU machine-readable marking; California's latent disclosure if you populate provider, version, time and an identifier	59 ms, +8% size, plus a certificate from a recognised issuer	dies on any re-encode
Inject the manifest at delivery time, in the API path	same as above, applied uniformly	same operation, moved earlier	same
Burn a visible label into the pixels	California's on-screen option (the statute calls it a "manifest" disclosure, unrelated to a C2PA manifest), China's explicit label	one compositing step	survives everything, changes the image
Keep a generation log keyed by content hash	evidence of compliance, China's six-month retention duty	storage, no per-file work	independent of the file
Imperceptible watermark	the layer that survives transforms	open-source encoders exist, but no interoperable standard, so nobody else can verify yours	n/a

Signing offline versus signing in the API path is the same operation on the same bytes; the difference is who the signature names. Signing at the gateway makes the gateway the entity attesting to the file's origin, while signing in your application makes you that entity, with your own certificate and key custody. If your obligation is to disclose that your product generated something with AI, the attestation belongs where that obligation sits.

One control needs no file surgery, and it is the one teams overlook: a generation log. China requires providers who hand over content without a visible label, which is what every API does, to keep records of who received it for at least six months, and a log is also how you evidence compliance once the file has been through a pipeline. If you already log request IDs, models and timestamps, what is usually missing is a hash of the delivered bytes, so an entry can be matched back to a file that has since lost its metadata.

FAQ

Do Chinese image models mark their output?

They mark it, but not with C2PA. Alibaba's qwen-image and wan2.7 SKUs embed the implicit label defined by GB 45438-2025: a metadata field carrying the producer's registered company code, a content ID and a signature value. ByteDance's seedream SKUs do the opposite, embedding C2PA and no Chinese label. Across 11 image SKUs, none embedded both.

Will a C2PA mark survive if my pipeline resizes the image?

No, not through mainstream imaging tools: a resize in Pillow or ImageMagick drops the C2PA manifest. The manifest lives in a custom chunk that Pillow and ImageMagick both discard, even when asked to include every chunk, so the only supported route is to re-sign the transformed file, which makes your pipeline the signer. China's label is an ordinary PNG text chunk that ImageMagick carries by default and Pillow carries if you copy it; both lose it on conversion to JPEG or WebP.

Is an unmarked API output a compliance problem for me?

Whether an unmarked file is your problem depends on whether you are a provider or a deployer under your regime, which is a question for counsel. What the measurements settle is the factual half: if you generate speech or video through these APIs, or images through the Alibaba SKUs for an EU audience, the file carries nothing a European verifier will recognise, and adding a mark is your work.

Measured 2026-08-22 through the Synthorai gateway: 11 image SKUs generated twice each and delivered as base64 and by URL, one video SKU and four text-to-speech SKUs, every file inspected for C2PA manifests, IPTC and XMP fields and GB 45438 label chunks. C2PA signatures were validated with the official trust list loaded, and against the interim list used by the public Content Credentials verifier. Survival tests ran five operations (PNG re-encode, resize, crop, JPEG and WebP conversion) over one marked file per family through both Pillow and ImageMagick, including each library's metadata-preserving options. Remediation figures come from signing an unmarked image with our own manifest and re-inspecting it before and after transformation. Regulatory rows are engineering readings of the cited texts, not legal advice; vendors change marking behaviour without notice, so re-measure before relying on any row.

Voice Agent API Cost: a 10-Minute Call Runs $0.04 to $0.57

synthorai — Tue, 18 Aug 2026 18:45:02 +0000

A 10-minute voice-agent conversation costs between $0.04 and $0.57 in API fees, and the spread comes almost entirely from two decisions: whether you assemble the stack yourself or use a speech-to-speech API, and which text-to-speech tier you pick. We measured every leg through one gateway in one batch: three speech-to-text models by the audio-minute, six text-to-speech models by the audio-minute they actually synthesize, an eight-turn LLM dialog on three brains (two flash-tier, one frontier), and a live GPT Realtime session billed lane by lane. This post assembles those legs into a per-minute price for each architecture.

TL;DR

A cascade stack (STT + LLM + TTS) runs $0.0037 to $0.025 per conversation-minute; gpt-realtime-2.1 measured $0.057, and the mini at $0.016 lands inside the cascade range.
With a flash-tier brain the mouth costs more than the brain, 12x in the mainstream build; a frontier brain flips the structure.
Latency is what the realtime premium buys: 1.7-2.1 s voice-to-voice measured, vs 4.9-7.7 s for the cascade builds.
Per-character TTS billing makes spoken Chinese 3 to 7x cheaper per audio-minute than English on every model measured.

How does a voice agent work, and why are there two architectures?

A voice agent is a loop that turns the user's speech into a reply in their ear, and every design on the market is one of two shapes. The cascade wires three specialized APIs in sequence: a voice-activity detector decides the user has finished speaking, speech-to-text transcribes the utterance, an LLM writes a reply from the transcript plus conversation history, and text-to-speech synthesizes it back to audio. Each leg is a separate metered service, which is what makes the cascade cheap and swappable: you pick the STT, the brain, and the voice independently, and you pay three small bills.

The speech-to-speech (realtime) architecture replaces all three legs with one model that consumes and produces audio directly over a WebSocket. (Synthorai carries this natively: the gateway speaks WebSocket at /v1/realtime, and an existing OpenAI Realtime SDK works unchanged pointed at our endpoint; see the realtime API docs. Every realtime session in this post ran over that connection.) Nothing is transcribed in the main path, the model hears tone and timing rather than a transcript, and the conversation history lives server-side as audio tokens that are re-sent to the model on every response. That design buys the two things a cascade structurally cannot deliver, sub-2-second voice-to-voice latency and natural barge-in (the user can interrupt mid-sentence and the model hears it happen), and it changes the billing shape: instead of three flat meters you pay separate audio-in and audio-out rates (the "lanes") plus a compounding history bill that caching has to keep in check.

The rest of this post prices exactly those two shapes: each cascade leg on its own meter, then the realtime loop lane by lane.

How much does a voice agent cost per minute?

Between $0.0037 and $0.057 per conversation-minute across the six builds we priced, a 15x spread. Two units to keep apart in everything below: a conversation-minute is one minute of wall-clock call time; an audio-minute is one minute of actual speech on one side, which is what STT and TTS meters bill. The scenario, held constant: talk time split evenly between user and agent, four exchanges per minute, short support-style utterances. Cascade legs are billed on measured audio durations and token counts; realtime figures come from a live eight-turn session:

Build	Listen	Speak	Brain	$/conv-minute	10-min call
Cascade, budget (qwen3-asr-flash + deepseek-v4-flash-0731 + google-tts-standard)	$0.0010	$0.0020	$0.0006	$0.0037	$0.04
Cascade, mainstream (gpt-4o-mini-transcribe + deepseek + tts-1)	$0.0015	$0.0076	$0.0006	$0.0097	$0.10
Speech-to-speech, gpt-realtime-2.1-mini	—	—	—	$0.0159	$0.16
Cascade, SOTA brain (gpt-4o-mini-transcribe + gpt-5.6 + tts-1)	$0.0015	$0.0076	$0.0110	$0.0200	$0.20
Cascade, premium (whisper-1 + gemini-3.7-flash + google-tts-chirp3-hd)	$0.0030	$0.0172	$0.0050	$0.0252	$0.25
Speech-to-speech, gpt-realtime-2.1	—	—	—	$0.0573	$0.57

Two placements matter more than the endpoints. The flagship realtime API costs 2.3x even a premium cascade, so what its price buys is not quality parity but the things a cascade cannot do: roughly 2-second voice-to-voice latency in our session, native barge-in, and tone and pacing (prosody) that survive because audio never becomes text. The mini, though, lands between the mainstream and premium cascades, which makes "cascade vs realtime" a real price comparison only at the flagship tier; at the mini tier it is a latency-and-control decision, not a price one.

One fairness check, because the other three cascades run flash-tier brains against OpenAI's flagship voice model: the SOTA-brain row gives the cascade the same vendor's frontier text model, gpt-5.6 at $5 in and $30 out per million. Even then the cascade lands at $0.0200 per conversation-minute, 2.9x below flagship realtime, because a text turn at frontier prices still moves two orders of magnitude fewer expensive tokens than an audio lane. Quality parity costs $0.010 of brain per minute; it does not close the architecture gap.

Which leg of the cascade dominates the bill?

The mouth, as long as the brain is a flash-tier model. In the mainstream build, a minute of conversation spends $0.0076 on TTS against $0.0006 on the LLM, a 12x gap; even in the budget build the speak leg is triple the brain. Upgrade the brain to gpt-5.6 and the structure flips: at $0.00274 per exchange, the brain becomes the biggest leg at $0.0110 per minute, 1.4x the tts-1 speak leg. Flash brains make voice agents mouth-dominated; frontier brains make them brain-dominated. Speech-to-text sits in between at $0.0010 to $0.0030 per conversation-minute (per-minute rates from our 14-model sweep, re-verified this batch: qwen3-asr-flash $0.00198, gpt-4o-mini-transcribe $0.00294, whisper-1 $0.006 per audio-minute).

The brain is nearly free until reasoning gets involved. Our eight-turn support dialog reached a steady state of about 310 input and 71 output tokens per turn on deepseek-v4-flash-0731, or $0.00016 per exchange. On gemini-3.7-flash the same dialog cost $0.00126 per exchange, 8x more, and the reason is visible in the token split: 89 to 361 reasoning tokens per turn spent deliberating over a two-sentence reply. Gemini 3.7 Flash cannot turn thinking off, so a voice agent built on it pays a reasoning tax on every exchange; on models where the off switch works, a voice turn is exactly the single-step shape where disabling it is safe.

What does text-to-speech cost per audio-minute?

$0.004 to $0.034 per synthesized minute, an 8.4x spread that per-character price tags hide. Every model below bills per input character, so we synthesized fixed English and Chinese passages, measured the returned audio, and divided:

Model	List price	EN $/audio-min	ZH $/audio-min	ZH vs EN
google-tts-standard	$4/M chars	$0.0041	$0.0007	0.18x
qwen3-tts-instruct-flash	$11.5/M	$0.0098	$0.0033	0.34x
tts-1	$15/M	$0.0151	$0.0043	0.28x
google-tts-neural2	$16/M	$0.0162	$0.0030	0.18x
tts-1-hd	$30/M	$0.0303	$0.0085	0.28x
google-tts-chirp3-hd	$30/M	$0.0344	$0.0052	0.15x

Speaking rate is the hidden variable. tts-1-hd and chirp3-hd carry the same $30 list price, but chirp3-hd speaks our English passage more quickly in fewer seconds of audio, so it costs 13% more per audio-minute; at the bottom of the table, google-tts-standard delivers a synthesized minute for less than half of anything else measured. The list price ranks the models; only the audio-minute rate prices them.

The Chinese column is the finding we did not expect to be this large: a minute of spoken Chinese consumes 3 to 7x fewer billable characters than a minute of English, because each character carries far more speech. Under per-character billing, the speak leg of a Chinese-language voice agent is 66 to 85% cheaper than the same agent in English, which compounds with the per-language token spreads on the text side.

When is the realtime API worth its price?

When the 2-second voice-to-voice loop and native interruption handling pay for themselves, because on price alone the flagship loses to every cascade we built. Both official audio conversions reproduced exactly in this batch: 30.0 seconds of input audio billed as 300 tokens (1 token per 100 ms) and 3.7 seconds of output as 74 tokens (1 per 50 ms), the same rates our July study measured. At gpt-realtime-2.1 list prices that is $0.0192 per minute just to listen and $0.0768 per minute to speak, before the conversation-history re-billing that makes long sessions compound.

That compounding is tamer than it looks, and the reason is caching. In our eight-turn session, input grew from 72 to 643 tokens per response, but by turn eight 512 of those were cached at $0.40 per million; over the whole session 76% of input tokens billed at the cached rate. The measured all-in figure came to $0.0752 for a 1.31-minute exchange, or $0.0573 per conversation-minute. The lane split of that bill is worth seeing once: 66% audio out, 17% fresh audio in, and 16% text, because the model also emits a text rendering of every reply, billed at the $24-per-million text-output rate. About a sixth of a "speech-to-speech" bill is, in effect, a text model's invoice. Applying the same measured token profile to mini rates gives $0.0159, and that number is the more interesting one: it undercuts the premium cascade while keeping the latency and barge-in behavior the cascade architecture structurally cannot match.

The remaining realtime tax is operational rather than per-token: sessions hard-stop at 60 minutes and cannot resume with audio history, so anything conversational past that boundary restarts with a cold cache. The July study covers those mechanics in detail.

How much faster is speech-to-speech than a cascade?

Between 2.3x and 4.5x on the builds priced above, and the gap widens as the cascade gets fancier. Same six builds as the cost table, same unit per bar: one conversational turn, measured request-to-response-complete at a realistic shape (5-second utterance in, two-sentence reply out; cascade bars sum per-leg medians of three runs):

In numbers: the cascade turns sum to 4.9 s (budget), 5.6 s (mainstream), 5.7 s (SOTA brain), and 7.7 s (premium), while both speech-to-speech tiers hold the same band, 1.6-2.1 s for gpt-realtime-2.1-mini over 4 live turns and 1.7-2.1 s for gpt-realtime-2.1 over 8.

Read the chart against the cost table and the trade snaps into focus: on the cascade side, money and latency move in opposite directions. The budget build is both the cheapest and the fastest cascade; the premium build pays 6.8x the budget price and is 2.8 seconds slower, because the expensive parts (a deliberating brain, an HD voice) are also the slow parts. On the speech-to-speech side both models hold the same 2-second band regardless of tier, so the mini is simultaneously cheaper than two cascades and faster than all four.

Three more things fall out of the numbers. First, the LLM is the latency bottleneck of every cascade, and mandatory reasoning nearly doubles it: gemini-3.7-flash spends 4.5 seconds deliberating over a two-sentence reply against deepseek's 2.6 and gpt-5.6's 2.7, so the same reasoning tax that costs 8x in tokens also buys the premium build its last-place finish. Second, cherry-picking the fastest measured part for every leg (gpt-4o-mini-transcribe, deepseek, google-tts-standard) still floors at 4.1 seconds, more than double the realtime band, before the voice-activity detector's end-of-speech wait and audio playback are added. Third, cascade rows are sequential, full-response wall times: a production cascade that streams the LLM into a streaming TTS overlaps the two biggest legs and can close much of the gap, but that overlap is engineering you build and tune, while the realtime API's 2 seconds arrive in the first WebSocket session you open.

Where does a cascade turn's time actually go? Mostly into the three requests themselves, not the content inside them. The raw network is a minor part: a warm HTTPS round trip to the API edge measured about 0.09 seconds, so three serial legs carry roughly 0.3 seconds of pure network per turn (plus a 0.2-second TLS handshake whenever a connection is cold). The dominant cost is each request's fixed dispatch, which we isolated by re-running the fastest assembly's legs with near-empty payloads: a one-token LLM reply still took 2.5 seconds, a half-second clip still took 1.6 seconds to transcribe, and a two-word synthesis took 0.4 seconds. Those floors sum to 4.4 seconds against 4.1 seconds for the full-sized payloads, which is the same number inside measurement noise: essentially all of a cascade turn is per-request overhead, and the actual content (70 reply tokens instead of 1, a 10x longer clip) rides along nearly free. The practical reading cuts both ways: making a cascade's replies shorter will not make it faster, but anything that removes or overlaps a request (streaming legs into each other, persistent connections) attacks the real cost; the realtime API is the limit case of that logic, holding one socket so a turn never pays a fresh request cycle at all.

FAQ

What is the cheapest way to run a voice agent?

The budget cascade: qwen3-asr-flash for listening, deepseek-v4-flash-0731 for the brain, google-tts-standard for speaking, at $0.0037 per conversation-minute or about $0.04 for a 10-minute call. It is 15x cheaper than flagship speech-to-speech; what you give up is voice-to-voice latency and natural interruption handling.

Is GPT Realtime more expensive than a cascade stack?

The flagship is: gpt-realtime-2.1 measured $0.0573 per conversation-minute, 2.3x our premium cascade and 15x the budget one. gpt-realtime-2.1-mini is not: at $0.0159 per conversation-minute on the same token profile, it sits between the mainstream and premium cascades, so at the mini tier the choice is about latency and control rather than price.

Which part of a voice agent costs the most?

Text-to-speech, in every flash-brain cascade we priced: $0.002 to $0.017 per conversation-minute against $0.0006 to $0.005 for a flash LLM, 12x in the mainstream build. A frontier brain flips it, with gpt-5.6 at $0.0110 per minute against tts-1's $0.0076. Within TTS, model choice spans 8.4x per audio-minute, the biggest single lever measured.

Does the language of the conversation change the cost?

For the speak leg, dramatically: per-character billing makes a minute of spoken Chinese 3 to 7x cheaper than English on all six TTS models measured, because each character carries more speech. Speech-to-text rates were flat across the two languages on the three models we re-verified, and the text-side differences follow per-language tokenizer spreads.

Measured 2026-08-17 through the Synthorai gateway, all legs in one batch: six TTS models synthesizing fixed English and Chinese passages with audio durations read from the returned files (two runs each); three STT models on the standard fixture clips with per-minute cost from billing headers; an eight-turn scripted support dialog on three LLM brains (two flash-tier, one frontier) with per-turn usage recorded; a per-leg latency probe (5-second utterance to STT, steady-state dialog turn to the LLM, two-sentence reply to TTS, n=3, full-response walls), a minimal-payload floor probe on the same legs (one-token reply, half-second clip, two-word synthesis, n=3) with warm and cold round-trip timing to the API edge, and live GPT Realtime sessions (5-second utterances, eight turns on the flagship, four on the mini) billed lane by lane from response.done usage events, with the mini cost figure derived by applying the flagship session's measured token profile to mini list rates and the mini latency band measured from its own session. Cascade composites assume talk time split evenly and four exchanges per minute; rates are list prices from the model catalog at measurement time. seed-tts-2.0 is onboarded but excluded pending stable measurements. Prices and speaking rates change; re-measure before committing to an architecture.