DEV Community: synthorai

Gemini 3.7 Flash API Cost, Measured: Tasks Bill 2.5-8x Less Than 3.6

synthorai — Sat, 15 Aug 2026 21:07:39 +0000

Gemini 3.7 Flash bills the same tasks 2.5x to 8x cheaper than Gemini 3.6 Flash, and only half of that is the advertised price cut. The intro rate of $0.75 per million input and $3.75 per million output is 50% off 3.6, and on our meter the same 6.9K-token prompt billed exactly half ($0.00529 vs $0.01057). The other half is quieter: gemini-3.7-flash burned 26% to 77% fewer thinking tokens than 3.6 on the same four measured tasks. We probed the model on day two of its release: the pricing terms, the thinking dial (one position died), the compatibility breaks Google's docs warn about and the ones they undersell, plus cache, context, and tokenizer continuity.

TL;DR

Intro pricing $0.75/$3.75 runs through December 31, 2026, then $1.50/$7.50; the same prompt billed exactly half of 3.6 on our meter.
Default thinking dropped 26-77% versus 3.6 (144 vs 384 tokens on a 5-step task), so per-task bills fall 2.5-8x, not just 2x.
The off-switch is gone: every off spelling returns a 400 ("Thinking level is unsupported"); low is now the floor.
A two-turn tool loop halved 3.7's deliberation on the tool-result turn (32 vs 61 tokens); refusing an unanswerable question cost both models 500.

What does Gemini 3.7 Flash actually cost?

Half of 3.6's rate until December 31, 2026, and less than half of 3.6's tokens on every task we measured. Google prices the launch at $0.75 per million input and $3.75 per million output as an introductory rate; from January 1, 2027 it reverts to $1.50/$7.50, exactly 3.6's card. Cache reads are $0.075 per million, 10% of intro input. The meter agrees with the card: the same 6.9K-token prompt cost $0.0052875 on 3.7 and $0.0105675 on 3.6, a clean halving.

The larger effect is what the model no longer spends. Identical salted tasks, three runs each, median reasoning tokens and the resulting output bill:

Task	3.7 reasoning tokens	3.6 reasoning tokens	3.7 output cost	3.6 output cost	Task bill ratio
Simple lookup	73	98	$0.0003	$0.0008	2.7x cheaper
2-hop word problem	62	266	$0.0002	$0.0020	8.4x cheaper
5-step arithmetic	144	384	$0.0006	$0.0029	5.3x cheaper
JSON extraction	251	340	$0.0011	$0.0027	2.5x cheaper

Google's launch line is that 3.7 "thinks more diligently"; on our tasks diligence means fewer tokens, not more. All accuracy cells stayed 3/3 on both models. Multiply the halved rate by the halved-or-better thinking and the practical upgrade math is a 61-88% smaller bill per task, before caching. The one caveat sits in the calendar: budget against the January 1 reversion, the same pattern as Sonnet 5's intro pricing.

One task shape resists the trend, and it is the one nobody budgets for: refusing. Asked about five fabricated entities (a company, an institute, a town charter, an alloy, an award), both models declined all five, and both spent more thinking doing it than on any other task we measured, a median 511 reasoning tokens on 3.7 and 494 on 3.6. Saying "there is no record of this" cost 3.5x the thinking of solving the 5-step arithmetic chain. Retrieval-augmented pipelines that frequently hit missing data pay this tax on every miss, and it is the one place 3.7's efficiency gain disappears. It also puts a measured floor under a launch-week worry: reviewers flagged a higher hallucination rate for 3.7, but on fabricated-entity questions it hedged 5 of 5, identical to 3.6.

Which thinking controls survive on 3.7?

Three positions and no off, exactly as documented: low, medium (the default), high. What the docs do not spell out is that 3.6's escape hatches are dead. Every off spelling we sent, reasoning_effort: "none", "minimal", thinking_budget: 0, thinking: {"type": "disabled"}, enable_thinking: false, returned the same upstream 400: "Thinking level is unsupported: THINKING_LEVEL_MINIMAL". A second, independent request path refuses in plainer words, "Reasoning is mandatory for this endpoint and cannot be disabled", so this is the model, not one client's translation layer. On 3.6-flash, measured in the same batch, none and minimal still zero the burn. The Flash line has joined the pro tier's policy: thinking cannot be turned off, and 3.6 is now the last Flash that can.

The surviving dial, on the 5-step task (median of 3, all 3/3 correct):

Position	3.7 reasoning tokens	3.6 reasoning tokens
low	133	138
medium (default)	147	284
high	291	409

Two working notes. The default-is-medium claim checks out: unconfigured runs in the dial matrix burned 154, indistinguishable from its explicit medium arm at 147. And thinking_budget is deprecated for real: every nonzero value we sent (16 through 1,024) was accepted and burned the same 135-138 tokens, a no-op, where 3.6 still maps small budgets to off and larger ones to caps. If your 3.6 integration steers cost through budgets, the 3.7 translation is: pick a level, that is the whole interface. For the cross-vendor context on which dials do anything, see our thinking-controls matrix.

What does the agent pitch cost per step?

Tool selection is unchanged from 3.6; what got cheaper is the deliberation between steps. We ran a two-turn loop against two functions (look up an incident, then restart the service it names), three runs per model:

Loop step	3.7 reasoning / output tokens	3.6 reasoning / output tokens
Turn 1: pick the tool	85 / 110	85 / 110
Turn 2: act on the tool result	32 / 58	61 / 87

Both models chose get_incident first and restart_service after, 3/3, with identical prompt tokens. The gap is the second turn, where 3.7 thinks about half as much before committing to the next call. That is the step an agent repeats: a 20-step loop at these medians bills about $0.0043 of output on 3.7 against $0.0131 on 3.6, and the ratio holds after the January price reversion because it comes from tokens, not rates. This is the mechanism behind Google's agentic benchmark jump being pitched as a cost story: fewer deliberation tokens per hop, multiplied by many hops.

Where the model lands against rivals is benchmark-dependent and worth reading before switching a coding agent over: independent roundups put 3.7 Flash ahead on FrontierCode (43.6% vs Sonnet 5's 42.7%) and near 3x Sonnet 5 on AutomationBench, while GPT-5.6 Terra leads Terminal-bench and Sonnet 5 leads the desktop-task exam, per the same roundup and launch-week coverage. The cost floor is where 3.7 is unambiguous.

What breaks when you upgrade from 3.6?

Two hard 400s and several things that fail quieter than the docs suggest. Google's migration note says to remove temperature, top_p, top_k, candidate_count, and prefilled model turns. Measured:

Change	Documented as	Actually happens
Prefilled assistant turn	must remove	400: "Requests ending with a model turn are not supported" (3.6 rejects it too; the docs just say it out loud now)
`n` > 1	must remove	400 on the surface we measured
`temperature` / `top_p` / `top_k`	must remove	accepted silently, on 3.7 and 3.6 alike
`max_tokens` above the 64K output cap	64K limit	accepted with a 200 up to 200,000 on both request paths we tested; the cap applies silently at generation time
`thinking_budget`	replaced by levels	accepted, inert (constant 135-token burn at any value)

The asymmetry is worth noticing: the thinking dial validates loudly (an unsupported level is a clean 400 naming the value), while sampling params and output caps swallow whatever you send. If your client library sets temperature by default, nothing breaks today; if it prefills assistant turns for guided output, it broke before you upgraded.

Do cache, context, and tokenizer carry over?

Yes, with one slowdown. The implicit cache hits at the same shape as 3.6, 4,076 of 6,905 tokens cached on the second call of an identical prompt, cutting the call's cost 52%, and reads price at 10% of intro input. But the build lag grew: 3.6 served a hit 4 seconds after the priming call, 3.7 returned nothing at 4 seconds and hit at 30. Rapid-fire duplicate traffic lands before the cache is warm; the write-side economics are otherwise unchanged.

Context accepted 708,912 input tokens in one call and answered a needle question correctly, consistent with the 1M claim and with no long-context price tier on the rate card. The text tokenizer is byte-identical across four generations, gemini-3.1-pro-preview, 3.5-flash, 3.6, and 3.7 all counted the same mixed English-Chinese-code corpus at 50 tokens, so token budgets port unchanged. Image inputs still bill the flat 1,089 tokens at every size we measured across the Gemini line. Neither generation returns its reasoning text: reasoning_content came back empty on every call, so the thinking you pay for stays invisible on both, a continuing complaint in launch-week threads. Structured output holds: strict json_schema returned valid, correct JSON in 3/3 runs, and reasoning_effort: "low" drove the extraction's thinking to zero, the same single-step safe zone our dial matrix found everywhere.

FAQ

How much does the Gemini 3.7 Flash API cost?

$0.75 per million input tokens and $3.75 per million output as an introductory rate through December 31, 2026; from January 1, 2027 it reverts to $1.50/$7.50, the same card as Gemini 3.6 Flash. Cache reads are $0.075 per million. On identical prompts our meter billed 3.7 at exactly half of 3.6.

Can you turn off thinking on Gemini 3.7 Flash?

No. Every off spelling returns a 400 ("Thinking level is unsupported"); the dial is low, medium (default), high, and low still burned 133 reasoning tokens on our 5-step task. Gemini 3.6 Flash remains the newest Flash where reasoning_effort: "none" works, at 3.7's post-intro price.

Is Gemini 3.7 Flash actually cheaper than 3.6 per request?

Cheaper than the 50% headline suggests: it also thinks 26-77% less on the same tasks, so our measured per-task bills dropped 61-88% (a 2-hop word problem fell from $0.0020 to $0.0002). After December 31 the rate halves back to parity and only the thinking savings remain.

Will code written for Gemini 3.6 Flash run on 3.7?

Mostly. Sampling params (temperature, top_p, top_k) are still accepted despite the docs' removal list, and oversized max_tokens is silently clamped. The two hard breaks: prefilled assistant turns return a 400 (as they already did on 3.6), and every thinking off-switch now 400s, so cost steering must move from thinking_budget to the three levels.

Measured 2026-08-15 through the Synthorai gateway, two days after release: dial and off-switch matrix (7 effort values, 5 budgets, 2 off params, n=3, gemini-3.6-flash rerun in the same batch for every comparison), four-task reasoning-tax sweep, strict-JSON structured output, deprecated-parameter probes, output-cap and 708K-context acceptance, implicit-cache pairs at two wait times, a fixed-corpus tokenizer comparison across four Gemini generations, a two-turn function-calling loop (n=3 per model), and a five-question fabricated-entity refusal probe. Task dollar figures price measured completion tokens at each model's current output rate. Two claims were cross-checked on a second, independent request path: the oversized max_tokens acceptance and the absence of an off position. Intro pricing and dates are Google's published terms; behavior may change as the rollout matures.

How Many Tokens Is an Image? 15 APIs Measured, Same Icon 6 to 1,298

synthorai — Sat, 15 Aug 2026 06:01:54 +0000

The same 1024x1024 image costs 693 input tokens on GPT-5.6, 988 on Qwen 3.8 Max, 1,089 on Gemini, and 1,372 on Claude, and after each provider's input rate is applied one image runs $0.00005 to $0.0137 across 15 vision models, a gap that comes almost entirely from each model's input rate rather than from its image tokenizer. Five vendors publish a billing rule for images and our meter contradicts three of them. This is the image-side companion to our text-tokenizer study: the same locally generated PNGs sent to every vision model in the catalog, image cost read as prompt tokens with the image minus prompt tokens without it, across six sizes, five aspect ratios, three content types, three file formats, and one-to-four image stacks.

TL;DR

One 1024x1024 image: 693 tokens on GPT-5.6, 1,089 on Gemini, 1,372 on Claude; dollars run $0.00005 (qwen3-vl-flash) to $0.0137 (claude-fable-5).
Three schemes: patch formulas (Qwen: (side/32)²+2 exactly), tiles with a cap (GPT stops at 693), and flat fee (Gemini 1,089 at every size, thumbnails included).
Three documented rules fail measurement: Claude flagships downscale at ≈1,920px not 1,568, Gemini bills a flat 1,089 where docs say 258, Qwen's grid is 32px not 28.
File format and content never moved a single token: billing is geometry only.

How do the vendors say an image is billed?

Five of the seven families publish a rule, and three of those five rules do not survive measurement. This table is the article in miniature; every section after it is either the receipts for a disagreement or a cost behavior no documentation mentions at all:

Family	The docs say	We measured
OpenAI	32px patches with a per-model patch budget (docs)	shape matches: 6 tokens at 64px, hard 693-token cap
Anthropic	(w x h)/750, downscale past a 1,568px long edge (docs)	formula exact at 512-1,024px; the 1,568 ceiling holds only on Haiku, flagships keep billing to ≈1,920px
Google	images up to 384px bill 258 tokens, larger ones 258 per 768px tile (docs)	flat 1,089 at every size, a 64px icon included; the documented `media_resolution` dial has no working spelling on the surface we measured
Alibaba	one token per 28x28px, minimum 4 (docs)	a 32px grid fits to the token on qwen3-vl ((side/32)²+2), floor 66, cap 1,600px
Moonshot	dynamic tokens, no formula published, images up to 4K accepted (docs)	consistent: quadratic growth, no cap found through 3,072px (11,674 tokens)
MiniMax / ByteDance	no public formula we could find	quadratic with a 2,048px cap; flat 1,298 per image

The pattern in the right column is worth naming: the documented rules are all about geometry (patches, tiles, divisors), and geometry is measurable, so we measured it. Where the columns disagree, your budget spreadsheet inherits the error: a Claude flagship pipeline sized to the documented 1,568px ceiling under-budgets large images by about 45%, and a Gemini pipeline expecting 258-token thumbnails pays 4.2x that on every icon.

How many tokens does one image cost?

Between 6 and 5,486 in our matrix, depending on model and size, and the token count is only half the bill. Here is the same 1024x1024 PNG on every vision model in the catalog, with each model's input rate applied:

Model	Tokens (1024²)	≈ words of English text	× the model's per-1K-token input price	Input rate /1M	Dollars per image
qwen3-vl-flash	1,026	≈770	1.03x	$0.05	$0.00005
qwen3-vl-plus	1,026	≈770	1.03x	$0.20	$0.0002
minimax-m3	1,371	≈1,030	1.37x	$0.30	$0.0004
Dola-Seed-2.0-pro	1,298	≈970	1.30x	$0.50	$0.0006
gpt-5.6-luna	693	≈520	0.69x	$1.00	$0.0007
gemini-3.7-flash	1,089	≈820	1.09x	$0.75	$0.0008
claude-haiku-4-5	1,373	≈1,030	1.37x	$1.00	$0.0014
gemini-3.6-flash	1,089	≈820	1.09x	$1.50	$0.0016
qwen3.8-max	988	≈740	0.99x	$2.00	$0.0020
gemini-3.1-pro-preview	1,089	≈820	1.09x	$2.00	$0.0022
claude-sonnet-5	1,372	≈1,030	1.37x	$2.00 intro	$0.0027
kimi-k3	1,379	≈1,030	1.38x	$3.00	$0.0041
claude-opus-5	1,372	≈1,030	1.37x	$5.00	$0.0069
claude-fable-5	1,372	≈1,030	1.37x	$10.00	$0.0137

Three readings. First, the idiom bills literally: at 0.75 English words per token, one 1024px image occupies the context budget of a 520 to 1,030-word document, which is why image-heavy conversations exhaust context windows and budgets so much faster than text. Second, the dollar column is almost entirely the rate: token counts cluster within 2x (693 to 1,379), so on any model one image costs 0.69x to 1.38x of whatever it charges per 1,000 input tokens; the dollar column mostly restates each model's input rate. Third, families share tokenizers exactly: both qwen3-vl builds, both GPT-5.6 variants, all three Geminis, and all four Claude models returned identical or near-identical counts on every square image, the same one-tokenizer-per-family pattern we measured on text.

What decides how many tokens an image costs?

Five factors move the bill and three widely assumed ones do not. Everything else in this study is a deep dive on one row of this table:

Factor	Effect	Where it applies
Billing scheme	patch formula vs tiles-with-cap vs flat fee	scheme table below
Resolution (area)	the primary driver, roughly quadratic	every model except the flat-fee pair
Downscale cap	pixels past the ceiling are never billed	1,600px (Qwen), ≈1,920px (Claude flagships), 1,568px (Haiku), 693-token cap (GPT); Kimi has none
Aspect ratio	second-order: bounding grids charge strips more, long-edge caps refund extreme ones	GPT +84% at 3:1 then -11% at 8:1; Haiku -71% at 8:1
Image count	strictly additive, no volume discount	all 15 models
Content (photo vs text vs blank)	no effect	every model measured
File format (PNG/JPEG/WebP)	no effect	every model measured
File size in bytes	no effect	every model measured

The three non-factors are worth stating because both myths circulate: no API in this matrix reads your compression ratio or your image's complexity into the bill. Geometry in, tokens out.

What are the three billing schemes?

Patch formulas, tiles with a cap, and flat fees, and they price small images completely differently. We ran a six-step size ladder, 64px to 2,048px squares:

Model	64px	128px	256px	512px	1,024px	2,048px	Scheme
qwen3-vl (both)	66	66	66	258	1,026	2,502	patch: (side/32)²+2, min 8x8, cap 1,600px
qwen3.8-max	28	28	28	220	988	2,464	patch, smaller floor
gpt-5.6 (both)	6	21	78	309	693	693	tiles, hard cap 693
gemini (all three)	1,089	1,089	1,089	1,089	1,089	1,089	flat fee, any size
Dola-Seed-2.0-pro	1,298	1,298	1,298	1,298	1,298	1,298	flat fee, any size
kimi-k3	17	33	108	369	1,379	5,486	quadratic, no cap found
minimax-m3	18	27	102	363	1,371	5,186	quadratic, caps at 2,048px
claude (all four)	12	28	103	364	1,372	4,764	(w x h)/750, downscale cap

The Qwen formula is exact enough to budget from: a 1,024px square bills (1024/32)² + 2 = 1,026, verified to the token, with a padded minimum of 8x8 patches (66) and a 1,600px downscale ceiling (everything from 1,600 to 1,920px measured exactly 2,502). GPT tiles until 693 and never more: a 1,024px and a 2,048px image cost the same. The flat-fee pair is the trap for thumbnail traffic: Gemini bills a 64px icon 1,089 tokens, the same as a 4K screenshot after its downscale, and Seed bills 1,298. On the other end, Kimi K3 kept scaling past everyone's caps: a 3,072px square billed 11,674 tokens, the only model in the matrix we never saw downscale.

Where exactly do the downscale boundaries sit?

Every family except Kimi resizes large images before billing, and the boundaries sit where the meter says, not where the docs do. The Claude boundary deserves the detail, because it prices real money on claude-sonnet-5 and up: 1,568px bills 3,139 tokens, 1,728px bills 3,847, 1,920px bills 4,764, and it stops there (2,048px and 2,304px bill 4,764 too). The flagship trio keeps billing real pixels about 45% past the documented ceiling; claude-haiku-4-5 is the one model that behaves as written. If you control the upload pipeline, resize to each model's measured ceiling before encoding: pixels past the boundary either bill extra (Kimi) or are silently discarded (everyone else), so oversized uploads buy bandwidth and nothing more.

Does shape, content, or file format change the bill?

Two factors explain every shape effect we measured, and neither is file size: whether the model counts area or a bounding grid, and where its long-edge downscale threshold sits. Content and format, meanwhile, moved nothing anywhere: a 512px solid color, noise field, and text page billed identically on every model we measured, as did the same image at 243KB (PNG), 176KB (JPEG), and 174KB (WebP).

The factor probe holds area constant at one megapixel and stretches the shape:

Model	1:1	2:1	3:1	4:1	8:1 (2,896px long edge)
gpt-5.6 (both)	693	1,271	1,278	1,230	616
claude flagship trio	1,372	1,355	1,411	1,409	1,107
claude-haiku-4-5	1,373	1,356	1,068	788	396
minimax-m3	1,371	1,352	1,410	1,298	650
kimi-k3	1,379	1,361	1,417	1,415	1,361
qwen3-vl (both)	1,026	1,037	992	1,026	992
gemini (all three)	1,089	1,081	1,083	1,056	1,034
Dola-Seed-2.0-pro	1,298	1,277	1,304	1,298	1,315

Read the rows against the two factors. Qwen, Kimi, Gemini, and Seed are flat: pure area (or a fixed fee) with no shape term. GPT is the one bounding-grid biller, charging strips up to 84% more than the same pixels square, until 8:1 crosses its long-edge limit and the downscale refunds the premium: 616 tokens, cheaper than the square. The downscale-threshold family shows the same crossover at each model's own boundary: Haiku starts discounting at 3:1 (long edge 1,774 past its 1,568 ceiling: 1,068, then 788, then 396), the Claude flagships only at 8:1 (2,896 past their roughly 1,920px boundary: 1,107), MiniMax at 8:1 past its 2,048 (650). The practical read for wide-document and screenshot traffic: on GPT, split or downscale strips yourself; on Haiku, extreme shapes are quietly the cheapest Claude pixels sold.

What levers actually cut image input costs?

Three, in order of leverage. First, resize to the model's ceiling: every pixel past the downscale boundary is billed on Kimi (no cap) and wasted everywhere else. Second, detail: "low" on GPT: at 512px it changes nothing (309 either way), but at 2,048px it pins the image to 309 tokens against 693 at high or auto, a 55% cut, and it is the only per-request image dial we found on any model. Third, match the scheme to the workload: flat-fee models (Gemini, Seed) are the wrong place for thumbnail and icon traffic and the right place for consistently large scans; patch and tile models price small images honestly (a 64px icon is 6 tokens on GPT, 17 on Kimi).

Multi-image requests offer no discounts anywhere: stacking 1, 2, and 4 copies in one message billed strictly additively on all 15 models, each copy at its full single-image price. That arithmetic punishes flat-fee schemes hardest: four 256px thumbnails in one Gemini request cost 4,356 image tokens, the same four on qwen3-vl-flash cost 264.

FAQ

How many tokens is a 1024x1024 image?

Measured on the same PNG: 693 on GPT-5.6, 988 on Qwen 3.8 Max, 1,026 on qwen3-vl, 1,089 on Gemini, 1,298 on ByteDance Seed, 1,371 on MiniMax, 1,372 on Claude, 1,379 on Kimi K3. The 2x token spread matters less than the rate applied to it: the dollar cost runs $0.00005 (qwen3-vl-flash) to $0.0137 (claude-fable-5).

Does image file format or compression affect token cost?

No, on every model we measured: the same 512px image as PNG (243KB), JPEG (176KB), and WebP (174KB) billed identical tokens, and solid-color, noise, and text-dense content billed identically too. Billing is a function of pixel dimensions only; compress for bandwidth, not for tokens.

Does detail: "low" reduce image tokens?

On GPT-5.6, yes, and only above the small-image threshold: a 2,048px image billed 309 tokens at low against 693 at high or auto (55% less), while at 512px all three settings billed 309. No other model in the matrix exposed a working per-request image-cost dial.

Are images cheaper if I batch several into one request?

No: 1, 2, and 4 copies of the same image billed strictly additively on every model, each image at full price. Batching saves request overhead and latency, not image tokens, and on flat-fee models (Gemini, Seed) many small images in one request is the most expensive shape there is.

Measured 2026-08-13/14 through the Synthorai gateway on 17 models (15 vision, 2 text-only controls): locally generated PNGs at exact dimensions, image cost read as prompt tokens with image minus a salted same-text baseline; six-size ladder (64-2,048px), boundary probes to 3,072px, six 1MP aspect shapes (1:1 through 8:1 plus a 1:4 tower), three content types, PNG/JPEG/WebP, detail low/high/auto on GPT, 1/2/4-image stacks, and a code-word sight check on every model. Dollar figures multiply measured tokens by each model page's listed input rate on the measurement date (Sonnet 5 at its $2 introductory rate). Vision billing rules change without notice; re-run the ladder before relying on any single cell.

LLM Thinking Controls: What 13 Models Accept, Ignore, or Enforce

synthorai — Wed, 12 Aug 2026 16:28:09 +0000

The same thinking-control parameter means three different things depending on which model you send it to: thinking_budget: 16 burns exactly 16 reasoning tokens on Qwen 3.8 Max, GLM 5.2, and both DeepSeek V4 builds, is silently ignored on Kimi K3 and MiniMax M3, and is rejected with a 400 by GPT-5.6. We probed 13 models from nine vendors with every control spelling the OpenAI-compatible surface accepts, then measured what each dial position costs in reasoning tokens and what it breaks in accuracy, on the same four salted tasks, three runs per cell.

TL;DR

thinking: {"type": "disabled"} zeroes reasoning on 11 of 13 models; the two exceptions (Gemini pro, GPT-5.6) reject it.
Qwen, GLM, and DeepSeek burn thinking_budget: 16 as exactly 16; Kimi and MiniMax accept the field and change nothing: Kimi burned 8-97 against that cap, never 16.
Thinking off dropped 5-step arithmetic from 3/3 to 0-1/3 on eight models; DeepSeek V4 Pro and Claude kept 3/3 by writing steps into the visible answer.
JSON extraction scored 3/3 with thinking off on all 12 models that support off.

Which thinking controls does each API accept?

Three control families circulate on OpenAI-compatible surfaces, and no model honors all of them. reasoning_effort takes an enum (none through max), thinking_budget takes a token count, and thinking: {"type": "disabled"} (with its cousin enable_thinking: false) asks for a hard off. Here is the acceptance map we measured, one trivial salted question per cell:

Model	`reasoning_effort`	`thinking_budget`	`thinking: disabled`
kimi-k3	all 7 values	accepted, ignored	works (rt=0)
qwen3.8-max	all 7 values	exact (16 → 16; 0 rejected)	works
deepseek-v4-flash-0731	5 values, no off position	exact (16 → 16)	works
deepseek-v4-pro	5 values, no off position	exact	works
gpt-5.6-luna	5 of 7 values (`minimal`/`max` rejected upstream)	rejected (400)	rejected (400)
glm-5.2	all 7 values	exact (16 → 16)	works
gemini-3.6-flash	all values; `none`/`minimal` truly off	translated, coarse: 0-64 = off, 1,024 caps	works (ct=2)
gemini-3.1-pro-preview	`none`/`minimal` rejected (pro cannot disable)	lowers burn, floors high (64 → 204)	rejected (400)
minimax-m3	accepted; `none` ignored	accepted, ignored	works (ct=2)
Dola-Seed-2.0-pro	4 values; `minimal` truly off	0 = off; nonzero ignored (16 → 36-64)	works (rt=0)
claude-sonnet-5	`output_config.effort`	`budget_tokens` rejected (400)	works
claude-opus-5	`output_config.effort`	`budget_tokens` rejected (400)	works
claude-fable-5	`output_config.effort`	accepted	accepted

Two rows deserve a flag. Google splits its own line: the flash tier turns off cleanly while the pro tier rejects every off-spelling with a 400, matching Google's position that pro-class thinking cannot be disabled. And the two claude-fable-5 cells marked "accepted" differ from Anthropic's published contract for that model, which specifies thinking cannot be disabled; treat those cells as in flux.

Does "accepted" mean "enforced"?

No, and the gap between the two is billed. A 200 response tells you the parameter parsed, not that it reached the model. The test that separates the two is trivial: send a budget of 16 and read the meter.

Qwen, GLM, and both DeepSeek builds burned exactly 16. Kimi burned 8, 19, 79, and 97 against the same cap across four runs, never 16, and MiniMax behaved the same at 18-44, billed as usual, with nothing in the response hinting the cap had been dropped. Gemini translates budgets into its native control at coarse granularity: on flash, caps of 0 through 64 behaved as a full off while 1,024 allowed thinking (median 141 on our 5-step task); the pro tier cut its burn under a cap but floored around 200 against a requested 64 and cannot reach zero. GPT-5.6 and Claude sit at the honest end of the spectrum: numeric budgets are rejected with a 400 and you know where you stand immediately.

The practical rule: after setting any thinking control, read completion_tokens_details.reasoning_tokens on the next response and confirm the dial moved. A control that fails loud costs you one retry; a control that fails silent costs you the reasoning bill you thought you had capped, on every call.

Is there a universal off-switch?

thinking: {"type": "disabled"} is as close as it gets: it zeroed reasoning on 11 of the 13 models, spanning Kimi, Qwen, both DeepSeeks, GLM, Gemini flash, MiniMax, ByteDance's Seed line, and all three Claude models. Its lookalike enable_thinking: false matches it almost everywhere, with one silent exception: MiniMax accepts it and keeps thinking (31 reasoning tokens on our probe).

The two exceptions fail loud rather than silent: Gemini pro returns a 400 for every off-spelling (the tier cannot disable thinking), and GPT-5.6 rejects the field too. GPT-5.6 needs no off-switch in the same sense: gpt-5.6-luna burns zero reasoning tokens on simple lookups and extractions by default (the two-lever pattern of that family), and reasoning_effort: "none" pins that behavior for math-shaped inputs too.

What does turning thinking off cost in accuracy?

On a 5-step arithmetic chain, everything: eight models fell from 3/3 to 0/3 or 1/3 the moment thinking went off. On a 2-hop word problem, much less: most models held 3/3 with thinking off, and only Kimi and MiniMax dropped to 0/3 there, the same fragile off state the K3 study found on its launch build. The cliff sits where the step count exceeds what the model can carry in a single visible pass.

Model	5-step multi, thinking on	5-step multi, thinking off
kimi-k3	3/3 (52 rt)	1/3
qwen3.8-max	3/3 (96 rt)	0/3
deepseek-v4-flash-0731	3/3 (70 rt)	1/3
deepseek-v4-pro	3/3 (112 rt)	3/3 (answer grew to 142 tokens)
gpt-5.6-luna	3/3 (33 rt)	0/3
glm-5.2	3/3 (237 rt)	0/3
gemini-3.6-flash	3/3 (338 rt)	0/3
minimax-m3	3/3 (66 rt)	1/3
Dola-Seed-2.0-pro	3/3 (128 rt)	0/3
claude-sonnet-5	3/3 (70 out)	3/3 (output grew to 139 tokens)
claude-opus-5	3/3 (60 out)	3/3

The three survivors share one trick: with thinking disabled they write the intermediate steps into the visible answer. DeepSeek V4 Pro's median reply grew from 115 to 142 tokens, Sonnet 5's from 70 to 139. You stop paying for hidden reasoning and start paying for visible reasoning, which on most price cards is the same output rate; the "off" switch relabels the spend more than it removes it. The models that obediently answer in 1-4 tokens when off are the ones that fall off the cliff.

Where the cliff bites, a small budget buys the accuracy back: thinking_budget: 256 restored 3/3 on Qwen and both DeepSeek builds at 77-128 median reasoning tokens, the same floor-rescue pattern we measured on the DeepSeek retrain and on Qwen 3.8's hidden caps.

One cell in our matrix produced no numbers at all: claude-fable-5 returned stop_reason: "refusal" (category cyber) for our exact arithmetic phrasing in 12 of 12 runs, at every effort setting, while a semantically identical rephrase passed 12 of 12. Anthropic documents refusal as a first-class stop reason with an opt-in fallback mechanism; if fable-class models are in your rotation, handle that stop reason before you need it.

What do the effort levels actually buy?

A different curve at every vendor, and only Google's slopes upward. We ran each model's full accepted enum on the same 5-step task, three runs per position, and plotted the medians on one scale:

The lines sort into four shapes. Real throttles: both Geminis climb monotonically, 137 to 390 reasoning tokens on flash and 180 to 387 on pro (saturating at high), and low held 3/3 accuracy at a third to a half of the top positions' burn, which makes it the default worth pinning on Gemini pipelines. Flat lines: DeepSeek (78 at low, 54 at max, drifting down), Kimi (94 at minimal, 67 at max), and MiniMax (61-93 in no particular order) expose a multi-position dial where no position changes anything; DeepSeek's own model card quotes benchmarks at "max reasoning effort", a setting we already found indistinguishable from default. Caps that do not bind: Qwen's levels are budget ceilings, invisible on a task this size (85-156 with no trend) and real only on deep work, as measured in isolation. Non-monotonic: GLM's high burned 116 against low's 184 and max's 345; until that mapping settles, treat its middle positions as unordered. The two adaptive models barely need the dial at all: gpt-5.6-luna spans 32 to 41 tokens across its whole enum, and Opus 5's output_config.effort moved visible output only within noise (54-63 tokens, Sonnet 5 the same at 71-92), adaptive thinking making the real decision.

The operational rule falls out of the shapes: on Gemini, pick a level deliberately, because each step is real money. On Qwen, GLM, and DeepSeek, steer with thinking_budget (exact) and the off-switch, not the enum. Everywhere else the enum is decoration between off and default, and the only way to know which kind you have is the ladder test above: same task, every position, read the meter.

The task-shape lesson generalizes: on our single-step JSON extraction, thinking-on burned the highest counts in the whole matrix (377 reasoning tokens on GLM, 332 on Gemini flash) and bought nothing, because every one of the 12 models that can turn thinking off scored 3/3 on that task with it off. Structured extraction pays the largest pointless reasoning tax, and it is precisely the workload where the off-switch is safe.

Can you see what you paid for?

The billing meter is universal; the thinking itself is not. Six of the open-weight-family models return reasoning text in reasoning_content: GLM 5.2 and DeepSeek V4 Pro sent back what reads as the full chain (508 and 312 characters for 167 and 100 reasoning tokens), while Kimi, Qwen, DeepSeek Flash, MiniMax, and Seed returned shorter traces roughly in line with their small burns. GPT-5.6 and both Geminis return nothing: the reasoning tokens are billed and invisible. Claude returns thinking blocks whose content is omitted by default on the surface we measured, so you see that thinking happened, but not what it was.

That visibility split matters for debugging budget behavior: on the models that return nothing, reasoning_tokens in the usage details is your only instrument, which loops back to the rule above. Trust the meter, not the 200.

FAQ

How do I turn off thinking on an OpenAI-compatible API?

Send thinking: {"type": "disabled"}; in our 13-model matrix it zeroed reasoning on 11 (Kimi, Qwen, DeepSeek x2, GLM, Gemini flash, MiniMax, Seed, and the Claude family). Gemini pro cannot be turned off and returns a 400; GPT-5.6 rejects the field but barely thinks on simple tasks by default. Verify by reading reasoning_tokens on the next response.

Does thinking_budget: 0 disable thinking?

It depends on the model. On Gemini flash and ByteDance Seed, 0 behaves as a clean off; Qwen and DeepSeek reject 0 with a 400; Kimi and MiniMax accept any budget and ignore it. Where budgets are enforced to the token (Qwen, GLM, DeepSeek), the minimum useful value is a small positive number: 256 held 3/3 on Qwen and both DeepSeek builds on our 5-step task, while GLM wobbled to 2/3 at the same setting.

Is it safe to disable thinking for JSON extraction?

In our runs, yes: single-step extraction scored 3/3 with thinking off on every model that supports off, while thinking-on burned up to 377 reasoning tokens for the same output. The boundary is step count, not output format: multi-step tasks collapsed without thinking on 8 of 11 models. One caveat from our DeepSeek study: on the 0731 retrain, thinking-on actively corrupted strict-JSON values, so there the off-switch is a correctness fix too.

Which models enforce thinking budgets exactly?

Qwen 3.8 Max, GLM 5.2, and both DeepSeek V4 builds: request 16, the meter reads 16. Kimi K3 and MiniMax accept the same field and ignore it; Gemini translates it coarsely (small caps act as off on flash, pro floors high); GPT-5.6 and the Claude 5-generation models reject numeric budgets outright (Claude's budget_tokens returns a 400 pointing at adaptive thinking).

Measured 2026-08-11/12 through the Synthorai gateway: acceptance probes for reasoning_effort (7 values), thinking_budget (0/16/1024), enable_thinking, and thinking:{"type":"disabled"} across 13 models, re-verified hours before publication; then a 552-call tax matrix (four salted task shapes x 3 runs per arm, arms limited to each model's verified-working controls), a 183-call full-enum ladder on the 5-step task (the chart's data), top-up cells, and a per-model reasoning-visibility probe. Accuracy graded from raw answers; token medians of n=3; reasoning tokens read from completion_tokens_details.reasoning_tokens (Claude models report output tokens only). Prompts salted per call. Dial semantics and enums are the surface we measured on this date and may change; re-probe before relying on any single cell.

Web Search API and Web Fetch API: How They Work and What $0.01 Buys

synthorai — Mon, 10 Aug 2026 13:45:35 +0000

Every major server-side web search tool now charges the same fee, $0.01 per search, and that number is the least interesting thing on the bill. The searches we metered attached 1,500 to 3,100 tokens of results to each request, those tokens bill at whatever model you route to, and what happens on the follow-up turn depends on a protocol design difference most teams never look at: whether the endpoint carries the search evidence forward, or quietly drops it and leaves the model to search again on your dime. We measured Synthorai's synthorai:web_search and synthorai:web_fetch across four model families, then put the results against Anthropic's and OpenAI's published pricing for their own hosted search.

TL;DR

The per-search fee is $0.01 across Synthorai, Anthropic, and OpenAI; billing records itemize it at $0.0100.
One search injects 1,500-3,100 result tokens at the model's input rate: on cheap models the fee dominates, on premium ones tokens take up to half.
max_uses is a cap, not a quota: allowed 10, the model stopped at 3; fees scale with searches run.
Turn two splits by endpoint: /v1/messages replays results (≈3,900 tokens, no new fee); /v1/chat/completions drops them, so a follow-up needing evidence buys a fresh search.

Why do models need web search and web fetch at all?

Because a model's knowledge stops at its training cutoff, and most production questions do not. Prices, release notes, exchange rates, sports scores, the contents of the URL a user just pasted: none of it is in the weights, and confidently answering anyway is how hallucinated "current" facts ship to users. Web search closes the gap for discovery (the model does not know where the answer lives), web fetch closes it for reading (the model knows the exact page and needs its contents). You can build both yourself with a search API, a scraper, and a tool-calling loop, and teams do; the server-side versions exist because that loop is undifferentiated plumbing, and running it inside the provider removes the round-trips, the parsing code, and the operational surface, for a fee that turns out to be the same everywhere.

How does server-side web search actually work?

The loop runs inside one API request, and that is the whole trick. With client-side tools, the model asks your code to search, your code round-trips results back, and every hop re-sends the conversation. A server tool cuts your code out: you declare {"type": "synthorai:web_search"} in tools, the model emits a query, the gateway dispatches it to a dedicated search provider, injects the returned results into the model's context, and the model reads them and answers (or searches again, up to max_uses). One request in, one response out, citations and the bill included.

Using it is one line in an ordinary request; no loop code, no callbacks:

curl https://synthorai.io/v1/chat/completions \
  -H "Authorization: Bearer $SYNTHORAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash-0731",
    "messages": [{"role": "user", "content": "What is one AI news headline from this week? Name the source."}],
    "tools": [{"type": "synthorai:web_search", "max_uses": 1}]
  }'

Web fetch is the same declaration with a different type; put the URL in the message and the gateway retrieves the page:

"messages": [{"role": "user", "content": "Fetch https://example.com/pricing and summarize the tiers."}],
"tools": [{"type": "synthorai:web_fetch", "max_uses": 1}]

The diagram traces the search request above exactly as we measured it on deepseek-v4-flash-0731. The meter read 2,059 input tokens (a 30-token question plus roughly 2,000 tokens of injected results), 169 output tokens, and $0.0104 total: $0.01 of search fee and about $0.0004 of tokens. Two billing events, and only two: the fee when the search runs, and the result tokens at the model's ordinary input rate. A fetch bills identically, with the fetched page taking the place of the snippets.

The declaration itself costs nothing (a request that declares the tool but never searches bills zero fee), and the tool works on any model in the catalog, which is the structural difference from the first-party alternatives: Anthropic's hosted search serves Claude models, OpenAI's serves OpenAI models, and a gateway-level tool serves whatever you route to.

What does one search really cost?

$0.01 of fee plus a token bill that flips who dominates as the model gets pricier. We ran the same one-search news question three times each across four families, solved each model's exact token rates from tool-free baselines, and read the fee as the residual:

Model	Result tokens injected	Token cost	Fee residual	Fee share of total
deepseek-v4-flash-0731	≈2,020	$0.0003	$0.0101	97%
gpt-5.6-luna	≈1,500	$0.002	$0.0104	83%
qwen3.8-max	≈1,780-2,060	$0.005-0.012	$0.0112-0.0116	49-68%
claude-sonnet-5	≈2,700-3,090	$0.008-0.010	$0.0118-0.0121	54-59%

On the cheapest model the search fee is 97% of the call; on premium models the injected tokens claim half the bill. The fee itself is not an inference: our billing records itemize the tool charge as its own line at exactly $0.0100 per call, alongside a separate count of tool-injected tokens (3,454 of the 3,836 input tokens on one fetch call, to pick a real row). Measured all-in, a single search landed between $0.010 and $0.012 depending on the model carrying it; budget the top of that band and you will not be surprised. The practical read: pairing server-side search with a cheap model makes the search itself the product, and the model almost free.

Which three dials set your token bill?

Search count, result length, and page weight; everything else is rounding. Each dial is directly measurable and two of the three are directly yours to set.

Dial one: how many searches run. max_uses is a ceiling, not a target. On a five-vendor pricing question, allowing 1, 2, and 3 searches billed $0.0106, $0.0216, and $0.0319, fees stacking linearly at $0.01 per search actually run; allowing 5 and even 10 changed nothing, because the model stopped at three searches on its own. Unused headroom is free, exactly like the thinking budgets we measured on reasoning models. The default is 3 with a hard cap of 10, so the unconfigured worst case on a search-happy question is three fees plus three result loads; pin max_uses: 1 on single-fact routes.

Dial two: how long the results run. Injection size follows the breadth of the question, not chance. Across twelve single-search runs on deepseek-v4-flash-0731:

Query type	Injected result tokens (3 runs)
Single fact	1,650 (stable to within one token)
Technical doc lookup	1,920-1,950
Current news	1,790-1,990
Multi-entity comparison	2,060-2,410

Narrow questions pull compact result sets; comparative questions pull wide ones, about 45% heavier than a single fact. For budgeting, 2,000 tokens per search with a 20% band covers everything we saw, which at typical model rates prices the token side of one search between a twentieth and half of the $0.01 fee.

Dial three: how heavy the fetched page is. The fetch fee never moves; the page decides the rest. Our ladder, all through the same model:

Page	Injected tokens	All-in cost (DeepSeek)	Same fetch at a $2/1M model
Minimal test page	209	$0.0101	$0.0104
Lean homepage	1,421	$0.0103	$0.0128
Long-form guide	3,460	$0.0106	$0.0169
Data-heavy post	5,196	$0.0108	$0.0204
Long Wikipedia article	27,380	$0.0139	$0.0648

On a cheap model even the Wikipedia article is a cent and a half; route the same fetch through a $2-per-million model and that one page costs six and a half cents, five times the fee. One supporting fact for the budget sheet: the same page tokenizes differently per model family. The data-heavy post was 5,196 tokens on DeepSeek and 5,508 on qwen3.8-max, a 6% tokenizer premium consistent with our density measurements.

What happens to search results on the next turn?

The split is a design difference between the two API protocols themselves, and it decides your follow-up bill. The /v1/messages protocol defines an assistant turn as a list of content blocks, so server-tool activity is part of the turn's official record: the response carries server_tool_use, web_search_tool_result, then text, and the protocol expects those blocks back on the next turn. The evidence rides forward as input tokens by design. The /v1/chat/completions protocol defines an assistant message as a single content string with no slot for server-tool results, so injected results are absorbed server-side; only the visible answer enters history, and the evidence is gone.

We ran the same search-then-follow-up pair on claude-sonnet-5 through both surfaces. Via /v1/messages, the follow-up billed 3,911 input tokens and $0.0109, all of it token cost: no new search, because the model still had the results to draw on. Via /v1/chat/completions, the follow-up billed $0.0204, more than the replay, because the model, holding nothing but its own one-line summary, ran a fresh search: a new $0.01 fee plus a new result load. And the re-search is a choice, not a law: an earlier run of the same absorbed-style follow-up on deepseek-v4-flash-0731 answered straight from its summary at 460 input tokens and $0.00009. Absorption makes your follow-up bill bimodal: nearly free when the summary suffices, fee-plus-load when the model decides it needs the web again.

So the honest framing is not that one endpoint is cheaper; it is that they move the cost to different places. The block style pays a predictable token toll each turn and never re-buys evidence it already has; the absorbed style bets that follow-ups will not need the evidence, and pays a fresh fee whenever the bet loses. High-turn chat with shallow follow-ups suits /v1/chat/completions; research agents that interrogate sources suit /v1/messages, where the prompt-cache layering discipline applies to carried results like any other bulky context.

What do Anthropic and OpenAI charge, and where do their tools run?

The comparison is short: everyone charges $0.01 per search, and after that the bill is just your model's token price. Anthropic bills $10 per 1,000 searches with results as input tokens (and its web fetch is fee-free, tokens only); OpenAI bills the same $10 per 1,000 calls with token treatment varying by model tier. The fee is settled; the model's input rate on 1,500-3,100 injected tokens is the variable.

	Synthorai	Anthropic	OpenAI
Fee per search	$0.01	$0.01	$0.01
Result tokens	model's input rate	model's input rate	model's input rate (tiers vary)
Web fetch fee	$0.01	none	-
Models served	any in the catalog	Claude only	OpenAI only

The row that actually separates the three is not on the price sheet: it is where the tools run. First-party server tools are built for their vendors' own agent stacks and stay on first-party surfaces: Anthropic's docs state web search is not available on Amazon Bedrock, only basic search reaches Google Cloud, and Microsoft Foundry requires a hosted-on-Anthropic deployment; web fetch skips Bedrock and Google Cloud entirely. OpenAI's search is bound to its Responses API. Move your workload across a cloud boundary or a model family and the first-party tool stays behind, which is also why a gateway does not pass those tools through: they cannot follow the traffic. A gateway-level tool takes the opposite trade: one declaration that survives both a model swap and a platform move.

FAQ

How much does the web search API cost per request?

$0.01 per search executed, itemized as its own line in the billing records, plus the injected results billed as input tokens at your model's rate (1,500-3,100 tokens per search in our runs). A request that declares the tool but never searches bills no fee. Budget $0.010-0.012 all-in per search on typical models; on very cheap models the fee is 97% of the total.

Do search results get re-billed on later conversation turns?

Depends on the endpoint. Via /v1/messages the result blocks replay with history (about 3,900 input tokens per turn in our runs) and no new fee accrues. Via /v1/chat/completions results are absorbed after the turn; the follow-up is nearly free if the model answers from its summary ($0.00009 in one run) but incurs a fresh fee and result load if it re-searches ($0.0204 in another). Anthropic's first-party search documents the replay behavior as standard; there, results re-bill every turn.

How do I cap web search spend on a request?

max_uses. It is a hard ceiling the model cannot exceed (default 3, cap 10), fees scale only with searches actually executed, and unused headroom is free. For single-fact routes, max_uses: 1 bounds the worst case at one fee plus one result load.

When should I use web fetch instead of web search?

Fetch when you already know the URL and want the full page; search when you need discovery. Both cost $0.01 per use through the gateway, but a fetch injects the entire page (209 tokens for a minimal page up to 27,380 for a long article in our ladder) while a search injects snippets from several sources. For read-and-summarize pipelines on Claude models specifically, Anthropic's own fetch tool is fee-free and cheaper.

Measured 2026-08-10 through the Synthorai gateway using synthorai:web_search and synthorai:web_fetch on deepseek-v4-flash-0731, gpt-5.6-luna, qwen3.8-max, and claude-sonnet-5: per-model token rates solved from tool-free two-point baselines, search fees read as billed-cost residuals over token value (n=3 per model), a max_uses ladder, a query-type result-load sweep (4 types x 3 runs), follow-up-turn probes on both endpoints with identical questions, and a five-page fetch ladder (three of our own pages plus a minimal external page and a long article). Fees were confirmed against itemized billing records (per-call tool-charge and tool-injected-token lines), not inferred from totals alone. Anthropic and OpenAI figures are their published docs rates at publication time. The diagram's numbers are one unedited measured call. Fees and behavior may change; verify against your own usage records.

Qwen-Image 3.0 API, Measured: 9 Scenes for One $0.03 Image

synthorai — Thu, 06 Aug 2026 15:06:03 +0000

Qwen-Image 3.0 costs $0.03 per output image, undercutting its own predecessor's $0.035, and the prompt is free: a roughly 5,000-token prompt billed exactly the same as an eleven-word one in our probes. That single fact separates it from token-metered rivals, and it makes the model's flashiest trick real economics: ask for nine different scenes in a 3x3 grid and you get all nine, billed as one image. We probed qwen-image-3.0 on the day it reached a commercial API route, because the launch itself shipped with no rate card, no benchmarks, no weights, and no technical report: the billing shape, the nine-in-one claim, the 10-pixel text claim (including the Japanese small-print dispute), the 4,500-token prompt window, and where it lands against its own predecessor and the rest of the image lineup.

TL;DR

qwen-image-3.0 bills $0.03 per output image, under its predecessor's $0.035; prompts are unbilled at any length we tried.
The "nine images at once" claim is real as a single-image grid: 9 of 9 scenes rendered, billed as one image; the API batch cap is n=4.
Dictated small print is the weak spot: the footnote misspelled in 3 of 3 English runs; self-invented text renders clean.
Generation is slow: 58-85 seconds standard against 10 for qwen-image-2.0; long prompts stretched to 241 seconds.

How does Qwen-Image 3.0 billing actually work?

Per image, by resolution tier, with the prompt riding free. Every response carries an image-count-and-tier usage block instead of token fields: a 1024x1024 or aspect-variant request bills one image at the 1K tier, a 2048x2048 request bills the 2K tier, and nothing in the usage accounting touches the prompt. We sent the same scene request with an eleven-word prompt and with fillers estimated at 1,000, 4,200, and 5,000 tokens: identical billing every time. The rate card DashScope published after launch week matches that shape exactly: $0.03 per output image at either tier for qwen-image-3.0, $0.003 per input image in editing workflows, and the tier boundary pinned at 2,250,000 pixels of output area. The tier split only prices differently on the pro SKU ($0.04 for 1K-class output, $0.075 for 2K). The quietly remarkable part is the direction: $0.03 undercuts qwen-image-2.0's $0.035, a price cut attached to a capability upgrade, in a launch everyone expected to premium-price.

The contrast with token-metered image APIs is structural. gpt-image-2 bills prompt tokens plus output image tokens, and those output tokens are not even stable: the same eleven-word prompt billed 215 output tokens on one run and 744 on another, a 3.5x swing in the metered cost of identical requests. Flat per-image pricing has no such variance; what you predict is what you pay. The same trade-off ran through our five-model cost study: per-image billing is boring, and boring is a feature in a budget.

Is "nine images at once" real?

Yes, but not the way the phrase suggests, and the real version is better for your bill. The API's batch parameter rejects anything above four (n must be between 1 and 4), so you cannot get nine separate files in one call. What the launch demos actually show is compositional: ask for a 3x3 grid of nine different scenes in one image and the model delivers. Our probe requested a lighthouse at dusk, a chess match, a bowl of ramen, a paper crane, a subway platform, a cactus in rain, a violin, a snow fox, and a hot air balloon:

Nine for nine, each cell coherent, billed as one image: $0.03 for the sheet, $0.0033 per usable cell. Alibaba's own demos push the same mechanism harder, packing nine full infographics with text and formulas into one grid. If your workflow needs thumbnails, mood boards, or option sheets rather than nine standalone files, this is a real cost trick: one flat-rate generation replaces nine, and slicing a grid is a one-line crop job.

Does the 10-pixel text claim survive a zoom?

Half of it does, and the half that fails is the half you would ship. The claim covers rendering text down to 10 pixels, and at the layout level the model over-delivers: our spec-sheet probe came back with invented sidebars, port diagrams, package-contents rows, and security badges, dozens of small labels, all legible and all spelled correctly. The failure hides in the one string we dictated verbatim. Across three runs, the requested footnote "All measurements at 25 degrees Celsius" rendered as "Celkaus", "Celuse", and "Celsue": zero of three correct, every error in the smallest type on the page.

Japanese behaves the same way, which settles a dispute from launch week: independent testing reported Japanese small print looking "somewhat unnatural" against the official demos, and our runs reproduce it. Of three Japanese label runs dictating a two-line storage warning, one came back fully correct, one malformed a single character, and one swapped two characters outright (読 became 認, 保管 became 保宜), on an otherwise flawless minimalist label:

The pattern is consistent and oddly specific: text the model authors for itself is clean; text you dictate for it to transcribe is where glyphs break. A Chinese-prompt run made the point from the other side, inventing its own three bin labels (加急, 普通, 存档) in perfect hanzi without being asked:

Self-authored text held up in every language we tried. For mockups, storyboards, and layout comps, the text rendering is genuinely production-grade. For anything where the exact dictated string is load-bearing, a compliance notice, a spec value, a legal line, proofread every glyph or composite the real text on afterward.

What does the 4,500-token prompt window actually buy?

Room, for free, but paid in seconds. The headline capability of this release is instruction volume: prompts 4.5x longer than the previous generation. Our acceptance ladder found no wall: filler prompts estimated at 4,200 and even 5,000 tokens were accepted without complaint, so the documented ceiling is not enforced with an error at the boundary we could find, and none of it costs anything extra under per-image billing. What long prompts do cost is latency, noisily: our standard short-prompt generations ran 58-85 seconds, two 1,000-token runs came back in 85 and 120, the 4,200-token probe took 241, and the 5,000-token one only 88. The trend is upward but the variance is wide; budget wall-clock, not dollars, when you use the window.

Where does it land in the image-model lineup?

Slowest in the catalog, by a wide margin, which is the honest price of the "useful, not pretty" positioning. Same-day, same-prompt timing across the lineup:

Model	Seconds per image (2 runs)	Billing	List price per image
qwen-image-2.0	9.8 / 10.2	per image	$0.035
qwen-image-2.0-pro	13.4 / 14.2	per image	$0.075
wan2.7-image	20.4 / 23.0	per image	-
seedream-5-0	30.8 / 34.1	per image	-
gpt-image-2	32.3 / 35.1	per token	$0.007-$0.023 measured
qwen-image-3.0	58-85 typical	per image	$0.03

Two placement notes. First, the predecessor is not obsolete: qwen-image-2.0 generates in a sixth of the time, and for high-volume simple assets that throughput still wins. 3.0 earns its seconds on layout-heavy, text-heavy, instruction-dense work that 2.0 cannot follow. Second, Alibaba now fields two image lines side by side: Qwen-Image and wan2.7-image from the Tongyi Wanxiang family, and the sibling is three times faster; if you are already in the Alibaba ecosystem for aesthetics rather than documents, the other line deserves the comparison.

Capacity is still beta-thin, which belongs in any rollout plan: we hit upstream rate quotas after roughly a dozen generations mid-study, and reports of the free tier describe the same 429 walls on back-to-back calls. One verification note in place of the usual benchmark table: this launch shipped without benchmarks, weights, or a technical report, and the rate card arrived only after launch week, so most claims have nothing official to check against. Everything above is from our own meter and probes, and the per-image rates on your provider may differ; read them from your first invoice, not from third-party guesses.

FAQ

How much does Qwen-Image 3.0 cost per image?

$0.03 per output image at either resolution tier on DashScope's published rates, with input images (editing workflows) at $0.003 and text prompts unbilled at any length. That undercuts qwen-image-2.0's $0.035. The pro SKU prices its tiers apart: $0.04 for 1K-class output and $0.075 for 2K.

Can Qwen-Image 3.0 really generate 9 images at once?

As one image, yes: a 3x3-grid prompt returned all nine requested scenes distinctly, billed as a single 1K-tier image. As an API batch, no: n is capped at 4, and each image in a batch bills separately. The grid form is the cost-efficient one when you need contact-sheet output rather than standalone files.

Is Qwen-Image 3.0's small text usable in production?

For self-authored text, yes: labels and copy the model invents rendered correctly across our runs, in English, Chinese, and Japanese. For dictated strings, no: the exact footnote we specified came back misspelled in 3 of 3 English runs, and 2 of 3 Japanese runs altered at least one character. Proofread dictated small print or composite it in with real type.

Should I upgrade from Qwen-Image 2.0?

Only for the work 2.0 cannot do: dense layouts, document-style pages, long multi-part instructions, in-image text. 2.0 generates a standard scene in about 10 seconds against 3.0's 58-85, on the same per-image billing shape, so throughput workloads should stay put. Treat 3.0 as a specialist for instruction-heavy generations, not a drop-in replacement.

Measured 2026-08-05 to 2026-08-06 through the Synthorai gateway on the day qwen-image-3.0 reached this commercial route (the -pro tier was not yet available): billing-shape and size-tier probes, batch-cap and grid-composition probes, a long-prompt acceptance ladder (filler lengths estimated at 1,000 / 4,200 / 5,000 tokens by word count; the images API returns no prompt token count), Chinese and Japanese prompts, small-text renders with repeated runs, and same-day same-prompt timing arms across six catalog models. Generated images shown are unedited probe outputs. Dollar figures use DashScope's rate card, which was published after launch week and matches the usage-accounting structure we measured; gpt-image-2's per-image range is its measured token counts priced at rates verified in our earlier image study. Behavior may change as the beta matures.

Build an LLM Chatbot: Streaming, Context Compression, Memory

synthorai — Tue, 04 Aug 2026 18:49:59 +0000

This guide walks through a complete chatbot you can run locally in about two minutes and read in an afternoon: a FastAPI server, one static page, no database, no build step. It covers the design of each subsystem, the order things happen in on every turn, and the failure modes we hit while building it. The full source is at github.com/synthorai-io/use-cases under chatbot/. Every request goes through one OpenAI-compatible endpoint, so the models in the picker are a dropdown, not separate integrations.

git clone https://github.com/synthorai-io/use-cases
cd use-cases
cp .env.example .env    # put your API key in it
cd chatbot
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn server:app --reload

What must an MVP chatbot do?

Eight things, and each one is there because of a specific way chatbots fail without it:

Feature	The failure it prevents	Where it lives
Persistent conversations (list, search, rename)	a chat that forgets everything on reload is a demo, not a tool	`storage.py`
Model selection, switchable mid-conversation	one model priced for the hardest question overpays every easy one	`config.py`
An editable persona, with presets	the system prompt is the product; changing it must not need a deploy	`presets/`
Streaming, with a Stop that reaches the provider	silence before the first token reads as broken; a fake Stop keeps billing	`server.py`
A context budget, with compression at the limit	every model has a window; hitting it silently is how bots "get stupid"	`context.py`
Long-term memory	re-introducing yourself every session is the most-felt chatbot annoyance	`context.py` + `.data/memory.md`
Web search and web fetch	the model's knowledge froze at training time; chat questions skew current	`tools.py`
Cost visibility, per turn and per session	the bill grows with history; you cannot tune what you cannot see	`server.py`

Web search and fetch deserve the longer argument, because they are the pair most MVP feature lists cut first. A chat model's knowledge stops at its training cutoff, months before today, and chat questions skew heavily toward the current: prices, versions, releases, "does X support Y yet". A chatbot that answers those from training data is not degraded, it is confidently wrong, and the user cannot tell which answers are stale. The failure goes deeper than missing facts: while building this we found that a model with no date anchor assumes its cutoff is "now", so it even searches wrong, putting a stale year into its own queries. Retrieval is not an enrichment feature for a chatbot; it is the difference between an interface to a snapshot and an assistant.

Why two tools and not one: search and fetch answer different questions. Search returns snippets a few hundred characters long that regularly disagree with one another; it answers "what is out there". Fetch retrieves one page in full and answers "what does that page say", which is what settles an exact figure. One discovers, the other verifies. The model decides per turn whether to use either, so a turn that needs neither costs nothing extra, and per-use caps bound the ones that do.

The rest of the table stakes come along too: Regenerate, markdown rendering that does not break mid-stream, named recovery paths for failures. The sections below walk the list roughly top to bottom, in the order a request flows through the code.

How is it put together?

Three pieces: a static page, six small Python files, and the gateway. The page posts a message; the server assembles context, compresses if needed, streams the completion back as server-sent events, and records measured usage. Conversations are JSON files you can cat.

static/index.html   the chat UI (vanilla JS: picker, budget bar, markdown, activity trail)
server.py           routes; the per-turn pipeline: project → compress → stream → record
context.py          token budgeting, compression call, memory file, message assembly
tools.py            the /v1/messages transport used by tool-enabled turns
storage.py          one JSON file per conversation under .data/ (settings included)
config.py           env-driven settings: model lineup, budgets, prompts
presets/            system-prompt presets, one .txt each

The per-turn pipeline in server.py is the part worth internalizing. Every message the user sends flows through the same five steps, and each step maps to a section below:

user message
    │
    ▼
[1] project the next request's size     last measured prompt_tokens
    │                                   + estimate(new message, pessimistic)
    │                                   + completion reserve
    ▼
[2] over budget? ──yes──▶ compress      old turns ──▶ rolling summary
    │                                       └───────▶ durable facts ──▶ memory.md
    ▼
[3] assemble and send                   [persona][memory][summary][date][history]
    │                                   cache mark after the system blocks
    ▼
[4] stream SSE back to the page         delta / reasoning / search / fetch /
    │                                   compression notice / warning / error
    ▼
[5] record measured usage               usage.prompt_tokens becomes step [1]'s
                                        input on the next turn

The loop closes on itself: step 5's measured number is what step 1 trusts next turn, so the budget is always grounded in what the API actually counted rather than a local guess.

One architectural fork to know about up front: plain turns go to /v1/chat/completions, and turns with web search or fetch enabled go to /v1/messages. That split is not a stylistic choice; the section on tools explains the measured behavior that forces it.

Which models go in the picker?

Seven ship in .env, grouped by tier in the picker (fast & cheap, balanced, frontier), and you can add any id the gateway serves from Settings; added ids persist in .data/models.json. The choice of default matters more than the lineup: the MVP starts on Gemini 3.6 Flash with its reasoning dial pinned to the floor. Chat replies are single-step work, and on single-step tasks reasoning_effort: "minimal" cut measured per-call cost 91-97% versus the default with output a reader could not tell apart. That setting lives in config, per model, because not every model accepts it:

# config.py — extra request params per model
MODEL_PARAMS: dict[str, dict] = {
    "gemini-3.6-flash": {"reasoning_effort": "minimal"},
}

DeepSeek V4 Flash is in the lineup twice over: as the budget option in the picker, and as the default summarizer for compression, because summarization is also single-step, quality-tolerant work. Claude Sonnet 5 is the pick when writing quality is the product; budget it by token counts rather than sticker price, since identical text tokenizes to 41% more tokens than on Sonnet 4.6. GLM-5.2 earns its slot on tool-heavy flows: a warm tool-call turn measured $0.0009 against $0.0051 on Claude Opus 4.8, with the caveat that its 6.6s median warm turn is felt latency in a chat window.

Per-model capability is measured, not assumed, and the code routes around the differences rather than pretending they are not there. In this lineup, only DeepSeek and GLM stream their working out as reasoning_content; the Claude models return no thinking blocks through this gateway even with the thinking parameter set, so the UI only offers a live "Thinking" panel where one can exist.

Switching models mid-conversation costs nothing structurally. The history is provider-neutral {"role", "content"} messages, so the same conversation can start cheap and escalate to the quality pick when a hard question arrives. The one real cost is invisible: a model switch abandons the previous model's prompt cache, so the first turn after a switch re-reads the full context at the cold price.

What does the model actually see each turn?

A layered prompt, assembled in one place and in a deliberate order: most-stable content first.

Layer	Comes from	Changes
System prompt (persona)	`presets/*.txt` or free text	never, unless edited
Long-term memory	`.data/memory.md`	rarely (facts appended)
Rolling summary	compression	only when compression fires
Date anchor	the server clock	daily
Tool instructions	Settings, tool turns only	rarely
Recent turns	the conversation	every turn

The ordering rule is: most stable first, because of caching (next section). The persona never changes, memory changes rarely, the summary changes only on compression, and the history changes every turn, so each layer sits behind everything that changes less often than it does.

The date anchor earns its place in the stack, and its position in it. A model with no date anchor assumes its training cutoff is "now": it writes search queries with a stale year in them and reads a release table as if the newest row were current. But the anchor is deliberately a date, not a timestamp: it lands in the prompt prefix, so anything finer than day granularity would break the cache on every single request. And it sits after persona, memory, and summary, so those keep their cache across midnight.

How does prompt caching work, and which models honor it?

The principle: providers cache the byte-identical prefix of a request and serve it back at a steep discount, charging a small premium to write it the first time. Chat is the single best workload for this, because each request is the previous request plus two messages: the entire past is a stable prefix by construction. The write premium pays back on the very next turn (on Anthropic models, 1.25x to write for the 5-minute TTL against roughly 0.1x to read; the write-side economics are measured here), and cache_control markers cut measured cost 88-89% across three Claude tiers. On a conversation carrying real memory and a summary, later turns cost roughly a tenth of a cold first turn.

The catch is that "caching" is not one mechanism. Providers split into two camps, and this lineup contains both:

Model	Caching style	You do	Hits reported as
Claude family	explicit: `cache_control` breakpoints	place the mark	`cache_read_input_tokens`
DeepSeek V4 Flash	implicit: automatic prefix	nothing	`prompt_tokens_details.cached_tokens`
GLM-5.2	implicit: automatic prefix	nothing	`prompt_tokens_details.cached_tokens`
Gemini 3.6 Flash	implicit; the marker is accepted and ignored	nothing works	not reported through this gateway

Explicit caching (Anthropic's model) makes you name the breakpoint but tells you exactly what happened: separate fields for tokens written and tokens read, so the discount is auditable per turn. Implicit caching (the OpenAI-style camp) needs no markers and just happens when your prefix repeats, but you get whatever the provider decides, and the only evidence is a cached_tokens field after the fact; how reliably that fires varies widely by provider. The MVP's strategy is to serve both camps at once: order the prompt most-stable-first (which is what implicit caching keys on) and always send the explicit marker (which the implicit camp ignores harmlessly). One request shape, every model gets whatever caching it is capable of.

Where the explicit mark goes came out of measurement, not documentation: on this gateway, cache_control is honored on a system block and silently ignored anywhere else. The common multi-turn pattern of marking the newest user message (which would let the whole history be cached) reads back zero cached tokens at full price. So the MVP marks the end of the system section, and the cacheable prefix is persona + memory + summary. Two consequences follow. Caching only engages once that prefix clears the model's minimum (about 1,024 tokens), so a fresh conversation caches nothing and one with accumulated memory and a summary caches everything. And anything that edits the prefix has a price: editing the system prompt goes cold from byte one, compression re-writes the summary (one cold read, covered below), and switching models abandons the old model's cache entirely.

The TTL is a config decision with the same shape everywhere: the default 5-minute cache covers an active conversation, and the 1-hour tier covers a user who stepped away, at double the write premium. CACHE_TTL=1h is the one-line change; whether it pays depends on whether your users actually come back inside the hour.

How do you know you are near the context limit?

Measure, don't guess: the only token count that is true for the model you are talking to is the usage.prompt_tokens the API returned for the previous request. Local estimators mislead across vendors, and the spread is not small: the same English text tokenizes 41% apart between two models from the same vendor (we measured it), more across vendors. The MVP counts tokens locally only for the one thing the API has not seen yet, the message being sent right now, and that estimate rounds up on purpose:

# context.py
def needs_compression(last_prompt_tokens, pending_text, message_count, budget=None):
    if message_count <= config.KEEP_RECENT_MESSAGES:
        return False  # nothing old enough to fold away
    projected = (
        last_prompt_tokens                    # measured, last response
        + estimate_tokens(pending_text)       # estimated, pessimistic
        + config.MAX_COMPLETION_TOKENS        # worst-case reply
    )
    return projected > (budget or config.CONTEXT_BUDGET_TOKENS)

Overestimating triggers compression one turn early, which costs one cheap summary call. Underestimating overflows the window, which costs a failed or silently truncated request. The asymmetry decides the rounding direction.

There is a trap inside "measure": providers disagree about whether prompt_tokens already includes the cached tokens. Some report the full prompt, others only the uncached delta. Getting this wrong is not cosmetic, because this number is the budget's ground truth: under-count it by the size of the cache and compression never fires, and the window silently overflows. The server normalizes (when prompt_tokens is smaller than the reported cached count, it cannot be the whole prompt, so the parts are summed) and keeps its own estimate as a floor under the measured value. What usage actually reports per provider is its own topic; LLM Token Usage Anatomy covers it.

The default budget is 102,400 tokens, which a normal conversation will not reach; it is a working budget, not a model limit, and the number to set is the point where a turn costs more than it is worth. To watch the mechanism work, open Settings and drop the window to a few thousand for one conversation (the budget is per-conversation), then paste a few long messages. The context bar in the header is drawn from the measured number and segmented by what is filling the window: persona, memory, summary, history.

What happens at the limit?

Compress, don't truncate. Truncation makes the bot forget the beginning of the conversation, and users experience that as the bot being stupid. Instead, everything except the most recent messages (default: last 8) is folded into a rolling summary by the cheap summarizer model, and the summary rides along as a system block. The recent window survives verbatim, so the bot's short-term voice does not change; only the deep past gets lossy. Nothing is silently dropped: a notice appears in the transcript when compression fires, with the message count and the summary's size.

One compression, end to end:

before   (projected next request 103.1k > 102.4k budget)

  [persona][memory][date][ m1 ..................... m34 │ m35 ....... m42 ]
                            old enough to fold            last 8, kept

one summary call to deepseek-v4-flash:
  in:   prior summary + m1 ... m34
  out:  {"summary": "one dense paragraph: topics, decisions,
                     open questions, promises",
         "facts":   ["prefers Python", "timezone is UTC+8"]}

after

  [persona][memory + new facts][summary][date][ m35 ....... m42 ]
   unchanged  grows rarely      replaced         verbatim

Three design points in the compression call carry most of its reliability:

The summary absorbs the previous summary. The second compression folds in the first summary along with the newly old messages, so it stays one rolling paragraph rather than a chain of summaries-of-summaries growing without bound.
Failure does not cost the turn. If the summary call errors, the server drops the oldest turns unsummarized, tells the user exactly what was lost, and answers the message anyway. A sloppy memory beats a dead chatbot, and the error path that never runs in demos is the one that pages you in production.
The prompt is a setting, with one guard. What the summarizer is asked to keep decides what the conversation remembers, so the compression prompt is editable per conversation. The server rejects an edit that removes the {transcript} placeholder, because that prompt would summarize nothing, and the failure would surface much later, on the first overflow. Same logic behind COMPRESS_MAX_TOKENS: the output cap has headroom over what the prompt asks for, because a summary truncated mid-sentence is what every later turn inherits.

For the cache, compression charges you exactly one cold re-read: the summary block changes, so everything after the memory block goes cold for one request, then the new, smaller prefix is cached again. That is the honest accounting: a summary call plus one cold read buys every subsequent turn priced on a smaller, warm context.

How does memory survive across sessions?

Step back and the bot has three memory stores, distinguished by scope and lifetime, and every design choice in this section falls out of that split:

Store	Scope	Lifetime	Size on the wire
Recent turns, verbatim	this conversation	until compression folds them	full text of the last 8 messages
Rolling summary	this conversation	dies with the conversation	one paragraph, under 200 words
`memory.md` facts	every conversation	until a human prunes them	a few lines

The summary and the facts look similar but age differently, which is why they are separate. The rolling summary is conversation-shaped (decisions, open questions, what the assistant promised) and dies with the conversation, correctly. Durable facts about the user ("prefers Python", "timezone is UTC+8") are still true next week and wasted if they die here.

The compression prompt asks for both at once and returns JSON: a summary string and a facts array. Facts append to .data/memory.md, and every conversation loads that file as a system block. Extraction happens at compression time rather than as a separate pass for a cost reason: compression is the moment a model is already re-reading the old turns, so the harvest rides on tokens you were paying for anyway. One call, two outputs.

The MVP's dedup is an exact-line match, and the limitation shows quickly: one compression stores "User prefers Python over Node.js" and a later one adds "The user prefers Python over Node.js." Semantic dedup means embedding or LLM-comparing each candidate against the store, which is a real feature with real cost, so the MVP does the dumb thing and says so. The memory file is markdown precisely so a human can read and prune it; Settings exposes it as an editable text box, which doubles as the transparency panel: what the bot knows about you is a file you can open.

What happens when the user hits Stop?

The provider stops generating. That sentence is the feature, and most chat UIs do not have it: a Stop that only hides the output while the completion runs on means paying for tokens nobody will read.

The chain has three links. The Send button becomes Stop while a reply streams (same position, no confirmation) and clicking it aborts the browser's fetch. The server polls for disconnect between events, notices, and breaks out of its streaming loop, which closes the upstream connection so the provider halts. Whatever text already arrived is persisted and marked interrupted, shown with a dashed border in the transcript. The same chain fires when the tab closes or the connection drops, because from the server's side those are the same event.

Persistence has to be exactly-once here, and the code is explicit about it: the save runs from inside the streaming loop on a clean finish, from the exception handler on a broken stream, and from the finally block on a hard disconnect (where the framework cancels the generator and nothing after the loop runs). The function is idempotent, so whichever path fires first wins.

Stop pairs with Regenerate: interrupt a reply you do not want, then redo it without retyping. Regenerate drops trailing assistant turns (including an interrupted one) so the conversation ends on the user message again, and re-answers it, with the current model. Since the model dropdown applies to the next turn, Stop plus Regenerate is also how you re-ask the same question on a stronger model.

What happens when a turn fails?

One of six things, and each gets its own recovery path rather than one red line. classify() in server.py maps upstream exceptions onto failure kinds; the UI maps kinds onto actions:

Failure	What the user sees	Recovery
Network error	named as such	Retry button
Rate limit	wait time from `retry-after` when present	Retry button
Bad API key	the env var to fix, by name	edit `.env`
Content filter	"retrying the same text will be refused again"	Edit & resend
Broken mid-stream	the partial reply, kept, marked interrupted	Retry
Compression failed	warning naming what was dropped	none needed; the turn continues

Two invariants do most of the work. First, the user's message is never lost: if the request fails before the first event, the server rolls the message back out of the conversation and the UI puts the draft back in the composer, so a retry cannot double-send it. If the stream breaks after text arrived, the partial is kept and marked. Second, a refused reply gets rewind, not retry: the server pulls the user message back out of the history and returns it to the composer for editing, because retrying verbatim into a content filter fails identically forever.

A subtler storage rule sits behind both: a blank assistant turn is never stored. An empty assistant message replayed upstream breaks the user/assistant alternation some providers enforce, and the error it eventually causes names the wrong message, on a later turn, which makes it miserable to debug. The MVP strips empty content from the wire format and refuses to store text-free replies (unless the turn has search or reasoning activity worth keeping, in which case the activity is stored and the empty text stripped on the way out).

How does web search work without a tool loop in your code?

The gateway runs the tools server-side, which changes whose code does the work. Classic function calling is a loop you own: the model returns a tool_calls block, your code executes the tool, appends a tool_result message, and re-sends the whole conversation, once per call. Server-side tools move that loop into the gateway: declare synthorai:web_search or synthorai:web_fetch on the request and the gateway sits between the LLM and the tool providers, relaying the model's tool calls to third-party search and fetch APIs and folding the results back into the model's context. Nothing about the tools is magic: each search and each fetch is itself a call to an external API, which is exactly why they are billed per use. There is no tool_result round trip in this codebase, only events to render:

browser        server (tools.py)     Synthorai gateway       LLM / tool providers
   │ POST /chat    │                     │
   ├──────────────▶│ declare tools +     │
   │               │ budget note         │
   │               ├────────────────────▶│── question + tools ──▶ [LLM]
   │               │                     │◀── tool_use: search ── [LLM]
   │               │                     │── query ─────────────▶ [search API]
   │               │  search results     │◀── results ─────────── [search API]
   │ SSE: search ◀─┤◀────────────────────┤── results ───────────▶ [LLM]
   │               │                     │◀── tool_use: fetch ─── [LLM]
   │               │                     │── URL ───────────────▶ [fetch API]
   │               │  fetch result       │◀── page text ───────── [fetch API]
   │ SSE: fetch ◀──┤◀────────────────────┤── page text ─────────▶ [LLM]
   │ SSE: delta ◀──┤◀────────────────────┤◀── answer tokens ───── [LLM]
   │ SSE: done     │                     │

The roles split cleanly: the model decides (whether to search at all, whether a snippet settles the question or a page needs fetching, when to stop and write), the gateway executes (calls the search or fetch provider, feeds the result back to the model), and your server just renders the events streaming past it. Both tools are on by default and a turn that needs neither costs nothing extra, so arithmetic stays free while "what is the current…" searches. The loop on the gateway side is bounded too: if the server-side search loop pauses at its own iteration limit (pause_turn), the transport echoes the turn back to resume it, at most three times.

Four things we learned building this are worth more than the happy path:

The endpoint decides what you can see. Both endpoints run the search and bill for it (measured 2026-08-04, on Anthropic and Gemini channels alike), but only /v1/messages surfaces the search itself: the query and the result URLs arrive as typed blocks your code can render and store. On /v1/chat/completions the same search runs invisibly: HTTP 200, an answer that opens "Based on the search results…", and zero citations or annotations on the streaming and non-streaming paths alike. A search you paid for but cannot audit is silent degradation, the worst failure shape (nothing errors, the provenance is just gone), and it is the entire reason tools.py exists as a second transport instead of one extra field on the normal request.
The model must be told its budget, in words. The caps (default: 3 searches, 2 fetches per turn) are enforced silently by the gateway, so a model that does not know about them plans as if tools were unlimited and burns the last round mid-thought; the turn ends on "Let me search for..." with no answer. The MVP injects one sentence stating the budget, and the wording is a measurable cost lever: in testing, no note at all spent the whole tool budget and finished mid-sentence, an over-strict note gave up and told the user to go read the page, and the shipped wording skipped search entirely, fetched the two authoritative pages, and cost the least of the three. It is editable in Settings; change it and watch the per-turn cost line.
Caps are the only brake. Both tools are billed per use, and the model decides how many rounds it wants. One uncapped turn in testing ran three searches and two fetches before a single output token was billed.
Degrade, don't die. Two failures get the same treatment: drop the tool, retry once, tell the user. Web fetch needs an entitlement on the key, and without it the whole request fails with web_fetch_not_enabled rather than degrading. And Gemini accepts the tool declaration, then errors the moment it actually calls one (Function call is missing a thought_signature). In both cases, losing the user's turn over an optional tool is the wrong trade.

Everything the model did to reach an answer (thought, searched, read) streams live into an activity trail above the reply, collapsed to one muted line ("Thought for 5s · Searched the web · Read 2 pages") and expandable into a timeline with queries, result domains, and reasoning text. The trail is stored with the message, which is the point: a trail that vanishes on reload cannot be used to audit an answer later. Citations render as numbered domain chips under the reply.

How does markdown render without breaking mid-stream?

By splitting the accumulated text into a stable prefix that is safe to render and a tail that is not. Rendering a half-open construct makes the layout jump when the next token closes it, so the renderer cuts at the last complete line, and if an opening code fence has no closer yet, everything from that fence on stays plain text until it closes. The bubble re-renders only when the stable prefix actually grows, so streaming does not rebuild the DOM per token. Code blocks get a copy button; the renderer is a hundred lines of vanilla JS, no library, and that is a statement about what an MVP needs rather than about markdown libraries.

What does a conversation cost?

Whatever the gateway's usage says, and the MVP's job is to read that honestly. Two normalization problems turned out to be load-bearing:

Vendors disagree on field names. For cached tokens alone: Anthropic models report cache_read_input_tokens, DeepSeek and GLM only prompt_tokens_details.cached_tokens, Gemini neither. The stats line reads whichever is present so "cached" means one thing.
A missing cost is not zero. The gateway reports cost on some turns and omits it on others (reproducibly, on turns where a tool is declared but not used). Turns that arrive without a cost field are counted and shown as "+N unreported" instead of being summed as zero, because a total that silently omits turns reads as the bill when it is only a floor.

The header shows running session totals (input, output, cached share, searches, fetches, cost, turns) and each reply carries its own line. The cached share is the number to watch: it is the caching section above, measured on your own conversation.

Where do all the knobs live?

In one Settings panel, and the scoping is the design decision worth copying: almost everything in it belongs to a single conversation.

Only two things are global, because they describe the user rather than a conversation: the model lineup (add any id the gateway serves; it persists in .data/models.json) and the long-term memory file, exposed as an editable text box. Everything else is scoped to the open conversation: the system prompt (with the presets from presets/ as a dropdown), the context budget, the web search and fetch toggles, the tool instructions, and the compression prompt. Two conversations can therefore run side by side with different personas, budgets, and tools against the same models, which is how you reproduce a claim from this guide instead of taking our word for it.

Each field carries a small info marker (hover to peek, click to pin) stating what changing it costs, because most of these knobs have a price that is not visible in the UI otherwise: editing the persona invalidates the cache from byte one, so it costs one cold turn; rewording the tool instructions moves the per-turn bill; lowering the budget makes compression fire sooner. The model dropdown in the top bar is the one setting that saves immediately, since switching models mid-conversation is a first-class action rather than a configuration change.

What is left out, and where would it attach?

Deliberate omissions, each with its attachment point:

Auth and multi-user — a session layer in front of the routes; conversations already have ids, so scoping them to a user is a filename prefix, not a redesign. Until then this is a localhost tool: every request it serves spends your API key, so do not expose it to the public internet as-is.
Rate limiting — same place, same reason: it guards a shared deployment, and this is not one yet.
RAG — retrieved documents belong after the summary and before the recent messages: volatile content goes late in the prefix so it does not churn the cached persona and memory blocks. Model-side, this is where the 1M-context picks earn their windows.
Client-side function calling — the streaming loop grows a tool_calls branch and an executor; the GLM-5.2 write-up covers the cross-provider contract differences waiting there. The gateway-side tools above deliberately avoid that loop.
Semantic memory dedup — embed each candidate fact, compare against the store, keep the novel ones. The memory file format does not change.
Syntax highlighting — the copy button is the feature people use; highlighting is a library decision for a real frontend.
A real database — storage.py is a handful of functions; porting them to SQLite is an afternoon, and the JSON files were the point until you have users.

DeepSeek V4 Flash API Cost: Thinking Mode Corrupts Strict JSON

synthorai — Tue, 04 Aug 2026 15:17:01 +0000

DeepSeek V4 Flash costs $0.14 per million input tokens and $0.28 per million output, with cache hits at $0.0028, and the retrained 0731 build that now ships under that name has a defect you need to route around: with thinking on (the default) and a strict json_schema, integer fields came back corrupted in 8 of 13 default-thinking runs across two independent request paths. Turning thinking off fixed every run and cut the extraction to a seventh of the tokens. We measured deepseek-v4-flash-0731 on day one: the corruption, the sharper off-switch cliff the retrain introduced, the budget floor that rescues it, the 1,024-token cache pages, and what still separates the preview build and V4 Pro.

TL;DR

With default thinking plus strict json_schema, deepseek-v4-flash-0731 corrupted integer fields in 8 of 13 runs on two request paths; V4 Pro corrupted 2 of 4, and only the preview stayed clean.
The 0731 retrain sharpened the off-switch cliff: 2-hop math fell 6/6 to 0/6.
The cache serves 1,024-token pages from roughly a 1.1K-token floor, hits 0.3 seconds after priming, and entries outlive 45 minutes.
enable_thinking: false fixed every structured run at a seventh of the tokens; for 2-hop math the safe thinking budget is 256.

How do the three V4 builds compare on paper and on the meter?

Same tokenizer, same cache, same thinking machinery; different prices, different failure modes. Everything measured below comes from identical probes run against all three (a dash means we did not probe that cell); the day-one analysis threads focus on benchmarks, so this is the operational half of the comparison:

	Flash 0731	Flash preview	V4 Pro
List price, in / out per 1M	$0.14 / $0.28	$0.14 / $0.28	$0.435 / $0.87
Cache-hit input per 1M	$0.0028	$0.0028	$0.003625
Thinking default	on	on	on
Strict JSON with thinking on	5/5 corrupted (our path)	4/4 clean	2/4 corrupted
2-hop math with thinking off	0/6	2/6	4/4
`thinking_budget`	token-exact	token-exact	honored (4/4 at 16)
Cache pages	1,024 tokens, hit at 0.3s	same	same
Tokenizer and prompt overhead	identical, 5 tokens	same	same
Needle recall probed to	838K tokens	-	-

Does thinking mode corrupt structured output on V4 Flash?

On the 0731 build, yes, and the failure is quiet enough to reach production. A four-field invoice extraction under strict json_schema (vendor, date, total, line-item count) with default thinking returned schema-valid JSON whose numbers were wrong: line_items came back as -1, 1, -1, and -19 against a document that plainly lists three items, 0 of 5 default-thinking runs correct through our gateway. Capping the budget does not dodge it: a 64-token-budget run corrupted the same way, and even at a 256-token budget 1 of 3 runs returned a line-item count of 670. The corruption follows the presence of thinking, not its size. On a second, independent request path the same probe corrupted 3 of 8 runs across two batches, once returning a total of 519.95 against the document's $520.00 and once a line-item count of 22; that path kept reasoning on even when asked to disable it, so the clean fix below is verified on the primary path. The JSON always parses and always passes the schema; only the values are wrong, which is the worst possible failure mode for a pipeline that trusts validation.

The fix is one line: enable_thinking: false produced correct, valid JSON in every run, in about 44 completion tokens against the default's 328. The spread across the family is telling: the preview build, probed identically with thinking on, went 4/4 clean, while V4 Pro corrupted 2 of 4, so the failure spans the V4 thinking line and hits the retrained flash hardest. It is also not the thinking-plus-schema bug already on record: vLLM fixed a plumbing issue last April where DeepSeek JSON landed in the reasoning field with empty content; here the plumbing is fine and the values are wrong, a strictly nastier failure. Until DeepSeek addresses it, treat thinking and strict structured output as mutually exclusive on these models, which costs you nothing: single-step extraction is exactly the workload where thinking off is safe and 7x cheaper.

Which thinking controls does the API accept?

Two off switches, an exact budget, and an effort dial with no off position. The surface we measured accepts reasoning_effort values low, medium, high, xhigh, and max; unlike Qwen 3.8 Max, none and minimal are rejected, so the dial alone cannot silence the model. Turning thinking off takes either enable_thinking: false or thinking: {"type": "disabled"}, which behave identically (9-token answers on a trivial question). thinking_budget is honored to the token, exactly as we measured on Qwen 3.8: request 16 and the meter reads 16. The full chain of thought comes back in reasoning_content, and the fixed prompt overhead is a lean 5 tokens per call.

The effort dial, meanwhile, did nothing we could measure. On a deep prime-counting task, low and high burned 31,374 and 31,370 reasoning tokens against the default's 26,897, all three correct: ordinary variance, no cap in sight. Where Qwen 3.8's levels are hidden budget caps that bind on deep work, V4 Flash's levels changed nothing at any depth we probed. On this model the two controls that matter are the off switch and thinking_budget; treat the dial as decoration. There is an irony here: DeepSeek's own model card pins its agent benchmarks at "max reasoning effort", a setting that produced nothing distinguishable from the default on the surface we measured.

How much thinking does two-hop math actually need?

More than before the retrain, which inverts the cheap-mode advice we gave for other models. On our quotable 2-hop arithmetic batch (1850 crates times 24 parts, 75% shipped, 3,120 arrive), the three V4 builds behave like three different models:

Configuration	0731	Preview flash	V4 Pro
default (thinking on)	6/6	6/6	4/4
thinking off	0/6	2/6	4/4
`thinking_budget: 16`	2/6	5/6	4/4
`thinking_budget: 64`	5/6	-	-
`thinking_budget: 256`	6/6	-	-

Two lessons. First, the agent retrain moved arithmetic into the thinking channel: the preview build limps through with thinking off, 0731 collapses outright, and Pro does not care at all. Second, the budget floor is model-specific: 16 thinking tokens fully rescue Qwen 3.8 Max on this same batch, but 0731 needs 256, at which point it is both perfect and cheaper than the default (completion totals of 53-188 against the default's 100-200). If you port a thinking-budget config between models, re-run the accuracy check; the lever is universal, the threshold is not.

What does the implicit cache deliver?

1,024-token pages behind a low floor, served fast and held long. Salted prefix pairs produced hits of exactly 1,024, 2,048, 4,096, and 7,168 tokens as the prompt grew: page-aligned quantization at 1,024. The floor sits just above one page (a 704-token prompt never hit; 1,166 tokens hit 1,024). A hit landed 0.3 seconds after priming, so there is no build lag to engineer around, and the entry was still serving at +45 minutes, the longest-lived implicit cache we have probed at this tier; Qwen 3.8 Max's entry died between 15 and 45 minutes and its floor is near 4.3K tokens. DeepSeek lists cache-hit input at $0.0028 per million, 2% of the miss price, with no write premium; the usual layering discipline applies: stable prefix first, volatile content after.

Are the 1M context and 384K output caps real?

The window held everywhere we probed it: planted override codes came back verbatim at 137,638, 465,238, and 838,198 prompt tokens, in 7 to 20 seconds, the fastest long-context recall we have measured at this price tier. The output side is looser than documented: DeepSeek's published maximum output is 384K tokens, but requests with max_tokens of 393,217 and even 524,288 were accepted in both thinking and non-thinking modes, so the cap is not enforced at request time and an oversized budget will not fail loudly. Set your own ceiling if you depend on one.

Preview, 0731, and Pro: which build answers your calls?

DeepSeek's pricing page now lists one SKU: deepseek-v4-flash, model version DeepSeek-V4-Flash-0731, at unchanged prices. In practice, the preview build is still being served under its own name on some routes, and the two are easy to tell apart from the outside even though they share a tokenizer (identical counts on English, Chinese, and code corpora, so budgets port). The cleanest fingerprint is the off-switch probe: thinking off on 2-hop math scores about 2/6 on preview and 0/6 on 0731 in our batches, and the preview is the only build that passed the structured-output probe clean (4/4, against 0731's 5/5 corrupted and Pro's 2/4). If your traffic depends on either behavior, probe the endpoint you actually call rather than trusting the name; V4 Pro ($0.435/$0.87) shrugged off the math cliff but not the structured-output one.

One planning note: the trio's shared tokenizer runs about 6-9% more tokens than Qwen 3.8 on identical corpora, so cross-vendor budget comparisons need the per-language density numbers, not just the rate card.

FAQ

Is structured output safe on DeepSeek V4 Flash?

With thinking off, yes: strict json_schema was enforced and every extraction in our batch was correct. With default thinking on the 0731 build, integer fields came back corrupted in 8 of 13 runs across two request paths while still passing schema validation. V4 Pro corrupted 2 of 4 on the same probe; only the preview build stayed clean. Pin enable_thinking: false on structured-output routes until the defect is fixed.

Can you turn off thinking on DeepSeek V4 Flash?

Yes, two spellings: enable_thinking: false or thinking: {"type": "disabled"}. But the 0731 retrain made the off state fragile on multi-step work (0/6 on 2-hop math); use a thinking_budget of 256 as the floor for anything beyond single-hop lookups, which measured both perfect and cheaper than the default.

What is the minimum prompt size for the V4 Flash cache?

Just over 1,024 tokens: a 704-token prompt never cached, an 1,166-token one hit exactly 1,024. Hits quantize in 1,024-token pages, arrive 0.3 seconds after priming, and survive past 45 minutes; cache-hit input is listed at $0.0028 per million, 2% of the miss price.

Did prices change with the 0731 release?

No. DeepSeek kept $0.14 per million input (cache miss), $0.0028 (cache hit), and $0.28 output, and folded 0731 into the existing deepseek-v4-flash name as its current model version. What changed is behavior, not price: sharper thinking dependence and the structured-output defect above.

Measured 2026-08-04 through the Synthorai gateway against deepseek-v4-flash-0731 (contrast arms on deepseek-v4-flash preview and deepseek-v4-pro): strict-schema extraction batches with per-run payload logging, verified on a second independent request path; dial-acceptance and garbage-value probes; the 2-hop accuracy batch (n=4-6 per arm, salted) with a thinking-budget rescue ladder; salted cache pairs at 2.5s pacing with floor, page, lag, and gap probes; max_tokens boundary probes in both modes; and identical three-corpus tokenizer counts across the trio plus Qwen 3.8 Max. Dollar figures are DeepSeek's published list rates at publication time; verify against your own provider's meter. Behavior may change as DeepSeek iterates on the 0731 build.

Qwen 3.8 Max API Pricing: 16 Thinking Tokens Beat the Off Switch

synthorai — Mon, 03 Aug 2026 17:33:46 +0000

Qwen 3.8 Max bills $2 per million input tokens and $6 per million output, and its cheapest reliable configuration is not the one the API seems to offer. Turning thinking off with reasoning_effort: "none" dropped our two-step arithmetic task from 4/4 correct to 1/6, while granting a hard budget of just 16 thinking tokens restored 6/6, averaging a fifth fewer output tokens than the default. Launch week produced loud capability claims and little to check them against: no model card, no public benchmark table, internal-only evals, which Hacker News was quick to flag. Billing behavior is different, because anyone with an API key can measure it. We measured qwen3.8-max on day one through the Synthorai gateway: every thinking control the surface accepts, the reasoning tax at each setting, the implicit cache's floor and build lag, the 1M-context claim, and what carries over from Qwen 3.7.

TL;DR

qwen3.8-max's seven reasoning_effort values collapse into four measured behaviors: off, a 4,096-token cap, a 16,384 cap, and unbounded.
thinking_budget is exact: request 16 tokens and the meter reads 16; the ceiling is 262,144.
With thinking off, 2-hop math fell to 1/6 correct; a 16-token budget scored 6/6 for less.
The implicit cache builds in under 0.3 seconds and reads at $0.25/1M, but nothing caches below roughly a 4,300-token prompt.
Input caps are exact and fail-loud: 991,808 (thinking off), 983,616 (thinking on).

What do the thinking controls on Qwen 3.8 Max actually do?

Three parameters are live, and they are not the three the documentation lists. Third-party docs describe reasoning_effort with three values (low, medium, xhigh, default xhigh). The surface we measured accepts seven: none, minimal, low, medium, high, xhigh, and max; an invalid value is rejected with that exact allowlist. Alongside it, the native thinking_budget (a positive integer up to 262,144) and enable_thinking (boolean) pass through to the provider, which validates them itself: a budget of 0 or 262,145 comes back as a 400 quoting the bound.

On ordinary tasks the effort levels are indistinguishable. Across trivial Q&A, 2-hop math, and a medium combinatorics problem, low, medium, high, xhigh, and the default all burned reasoning tokens in the same noisy band; nothing separated them. The separation only appears on a task that wants to think for tens of thousands of tokens. On a prime-counting problem whose unconstrained run burned 45,129 reasoning tokens, the levels finally started to bind:

Setting	Reasoning tokens on the deep task	Correct?
default (omitted)	45,129	yes
`minimal`	4,096 (exact cap)	no
`low`	4,096 (exact cap)	no
`medium`	16,384 (exact cap)	no
`high`	44,348 (no cap reached)	yes
`xhigh`	38,029 (no cap reached)	yes
`max`	35,300 (no cap reached)	yes

The mental model that fits every observation: each effort level is a preset thinking-budget cap, and the seven names collapse into four behaviors. Off is one (none and enable_thinking: false behave identically). Minimal and low share the same 4,096-token cap. Medium quadruples it to 16,384. High, xhigh, max, and the default form the fourth tier: none of them bound on this task, burning between 35K and 45K tokens across runs, which is ordinary variance at this depth. If the top three differ at all, the split sits above 45K thinking tokens, deeper than most production traffic ever goes; the docs' claim that the default is xhigh is consistent with everything we measured. The split is stark: every capped run answered wrong, every uncapped run answered right. Below the cap, behavior is identical across levels, which is why the dial feels dead on everyday traffic. Above it, the cap truncates thinking mid-task. If you want a specific ceiling, skip the presets and set thinking_budget directly; the next section takes that parameter apart.

How does thinking_budget actually work?

It is enforced to the token, it is a ceiling rather than a quota, and a mid-sized cap can make a task cost more than setting no cap at all. thinking_budget is DashScope's native integer parameter (1 to 262,144, documented default 131,072, inherited from the open-weight Qwen3 family) that sets the maximum reasoning tokens for a call. Requests of 0 or 262,145 are rejected with a 400 quoting the range. Below the ceiling nothing changes: a budget of 8,192 on a task that naturally thinks for a few hundred tokens burned 331 and 485, exactly as if no budget were set. At the ceiling, enforcement is token-exact: budgets of 16, 64, and 256 stopped reasoning at precisely 16, 64, and 256 in every run.

What happens at the cutoff is the interesting part. The model does not abandon the work; it stops reasoning and finishes the job in the visible answer. On a medium combinatorics task (count domino tilings of a 2x12 grid), every budget produced the right answer, but the totals are not what you would predict:

Budget	Reasoning burned	Total completion tokens
none set (default)	226-303	234-311
16	16 (exact)	368-406
64	64 (exact)	399-408
256	256 (exact)	776-787
8,192	331-485 (never bound)	339-493

The curve is not monotonic. A 256-token budget cost 2.5x more than no budget: the model spent its allowance starting a reasoning chain, lost it mid-thought, and re-derived the answer step by step in the visible channel. The tightest budget beat the mid-sized ones because a 16-token allowance is too small to start anything, so the model goes straight to compact visible work. Three rules fall out. First, tiny budgets are a real lever on shallow-to-medium work: 16 tokens went 6/6 on our 2-hop math batch at 98-161 total tokens against the default's 126-207. Second, never deploy mid-sized caps on traffic of unknown depth; they land in the dead zone where the cap truncates real reasoning and you pay for the work twice (the deep-task rows above, 4,096 and 16,384, are the same failure at scale, and there they also answered wrong). Third, a binding budget changes the shape of the output: capped runs answer with their work shown, which matters if a parser expects a bare result.

How much of the day-one documentation survives measurement?

About half, and the split is worth publishing because nothing else about this launch is independently checkable yet. Every number below is from our own meter and probes:

Documented claim	Measured verdict
Input caps: 991,808 (non-thinking) / 983,616 (thinking)	Exact; oversized requests 400 quoting the bound
`thinking_budget` range: positive integers up to 262,144	Exact; 0 and 262,145 both rejected
List price $2 in / $6 out per 1M	Meter matched to the fourth decimal on every call
Cache reads at 0.25x credits	Exact: $0.25/1M, no write premium
`reasoning_effort` values: low, medium, xhigh	Wrong: seven values accepted, including a full off switch
Max output 131.07K "in both modes"	Wrong both ways: thinking off rejects `max_tokens` above 65,536; thinking on accepted every value we tried, up to 393,216
Multi-turn clients "must return unmodified `reasoning_content`"	Unenforced: omitted and tampered histories accepted
"Context caching supported" (no details)	Real, but the load-bearing spec is undocumented: ≈4.3K floor, 15-45 min lifetime

The pattern favors the billing plane: everything that decides what you pay is precise and honestly enforced, while the parameter documentation lags what the surface actually does.

Does turning thinking off save money?

It saves tokens and costs correctness, and there is a better trade two lines away. In our quotable batch of 2-hop arithmetic (1850 crates times 24 parts, 75% shipped, 3,120 arrive), the default configuration went 4/4 at 126-207 completion tokens per call. reasoning_effort: "none" produced 4-5 token answers and went 1/6. The same prompt with thinking_budget: 16 went 6/6 at 98-161 completion tokens, cheaper than the default and as accurate, on this task class. One-hop arithmetic stayed 3/3 even at none, so the off switch is safe for lookups and single-hop transforms; it is multi-step work that collapses. This is not a 3.8 regression: qwen3.7-max with thinking off went 3/6 on the identical batch.

The dead zone described in the budget section has a dollar figure on hard tasks. On the deep prime-counting run, the low preset burned its 4,096 thinking tokens, spilled 13,882 more of visible candidate-checking, and still answered wrong: $0.11 for an incorrect answer, versus $0.27 for the default's correct one. The off switch shows the same physics on hard tasks: with thinking disabled outright, both 3.8 and 3.7 poured a 13-15K-token enumeration into the visible answer, one landing the count and one missing it by a single prime, in single shots each. A budget that binds mid-reasoning can raise total spend while lowering quality. Cap thinking on tasks you know are shallow; let deep tasks think.

Is the 1M-token context window real?

Effectively yes, with exact and honest limits. The API accepts up to 991,808 input tokens with thinking off and 983,616 with thinking on, and both bounds are enforced fail-loud: an oversized request is rejected with a 400 that quotes the exact limit, rather than silently truncating your document. Needle recall worked at every size we probed, 161K, 677K, and 919K tokens, returning the planted override code verbatim in 11 to 63 seconds. A 919K-token request costs about $1.84 at list price, so the window is real but a full-window call is a design decision, not a default.

What does the implicit cache deliver, and where is the floor?

The fastest cache build we have measured, behind an unusually high floor. Repeating a salted 6,103-token prompt produced a hit on the very next request 0.3 seconds later; there is no warm-up window to engineer around, unlike Gemini's tens-of-seconds build. Hits kept coming at +5 and +15 minutes with no re-prime, and the entry was gone by +45, so the working lifetime sits somewhere between 15 and 45 minutes of silence. Reads bill at $0.25 per million, 0.125x the input rate, and there is no write premium; the discount arrived automatically in the cached_tokens field and the metered cost.

The floor is the catch. A 4,221-token prompt never produced a hit; a 4,360-token one did, and every first hit was exactly 4,096 tokens. Below roughly 4.3K tokens of prompt, this cache does not exist for you, a sharp contrast with Claude's 1,024-token minimum and Kimi K3's small-block automatic caching. Above the floor, hits quantize in 128-token blocks (we observed 4,096, 8,320, 12,544, and 16,768), but coverage of the primed prefix ranged from 51% to 96%, so budget on discounting most of a long prefix, not all of it.

Do structured output and tool calls pay the reasoning tax?

By default yes, and they are the safest place to cut it. Strict json_schema output works, and it is doing real enforcement: the same extraction without a schema came back wrapped in markdown fences. With the default configuration, a four-field invoice extraction burned 252 reasoning tokens before emitting 57 tokens of JSON. With reasoning_effort: "none" it produced valid, correct JSON in 54 total tokens, a 5.7x cut; thinking_budget: 16 sat in between. Tool selection behaved the same way: the model called the right function with thinking off at a third of the default's tokens. Single-step extraction and routing are exactly the shape where the off switch is safe, and at $6/1M output the habit compounds.

One more billing note for agent builders: the API returns the full chain of thought in reasoning_content, and the docs instruct multi-turn clients to send it back unmodified. The instruction is not enforced. We replayed turns with the reasoning included, omitted, and deliberately tampered; all three were accepted, and short-chain accuracy was unaffected. Replayed reasoning bills as ordinary input tokens, so omitting it is a real saving on multi-turn traffic until you see quality reasons not to.

What carries over from Qwen 3.7, and what changed?

The tokenizer is unchanged and your token budgets port directly. Identical English, Chinese, Japanese, and code corpora tokenized to identical counts across qwen3.8-max, qwen3.7-max, qwen3.7-plus, qwen3.6-flash, and qwen3.5-flash, so per-language cost planning from our tokenizer-by-language study carries over unchanged.

Two things did change. First, 3.8-max carries a fixed prompt overhead its siblings do not: the same one-character message counted 49 prompt tokens on 3.8-max against 11 on every other Qwen we probed, a constant +38-token rider per call. It is noise on long prompts and a measurable percentage on short, high-frequency ones. Second, thinking is always available rather than a mode switch, with the seven-level dial and exact budget parameter above; 3.7's controls were coarser. One pricing clarification, because two schemes circulated at the preview: the Token Plan subscriptions ($6 to $68 monthly, with deep off-peak discounts) price Alibaba's own apps, not the API. Through the API you pay the $2/$6 list rate, and our gateway meter matched it to the fourth decimal on every call in the study.

FAQ

Can you turn off thinking on Qwen 3.8 Max?

Yes, fully: reasoning_effort: "none" (or enable_thinking: false) eliminates reasoning tokens entirely. Reserve it for single-hop work. On 2-hop arithmetic it scored 1/6 in our batch while a 16-token thinking_budget scored 6/6 at comparable or lower token counts, so the floor setting for multi-step traffic should be a small budget, not the off switch.

What is the minimum prompt size for Qwen 3.8 Max's cache?

About 4,300 tokens in our probes: a 4,221-token prompt never hit, a 4,360-token one did, and first hits are always exactly 4,096 tokens. Below the floor no discount exists; above it, reads bill at $0.25/1M with no write premium and the entry is readable 0.3 seconds after priming.

Is reasoning_effort supported on Qwen 3.8 Max?

Seven values are accepted (none, minimal, low, medium, high, xhigh, max), but levels behave as thinking-budget caps that only differ once a task thinks past them. For deterministic control, set thinking_budget directly: it is enforced token-exactly, rejects 0, and tops out at 262,144. Client tooling currently disagrees about which tiers exist; the list above is what the surface accepted on day one.

Do you have to send reasoning_content back in multi-turn conversations?

The documentation says yes; the API does not check. Omitted and even tampered reasoning history was accepted without error or short-chain accuracy loss in our probes, and replayed reasoning bills as normal input. Skipping the replay is a legitimate cost lever until your own evals show quality loss on long chains.

Measured 2026-08-03 through the Synthorai gateway against qwen3.8-max (comparison arms on qwen3.7-max, qwen3.7-plus, qwen3.6-flash, qwen3.5-flash): dial-acceptance, garbage-value, and max_tokens boundary probes, a 45K-natural-burn deep task to bind the effort caps, a fixed quotable batch for the accuracy cliff (n=4-6 per arm, salted), salted cache pairs with 2-3s pacing plus gap ladders, needle and overflow probes at 161K-919K tokens, and identical four-corpus tokenizer counts. Dollar figures are billed-cost readings from the gateway meter at list rates ($2/$6 per 1M). Preview-period discounts, rates, and behavior may change; verify against your own usage records.

MCP Tool Overhead, Measured: 26 Tools, $0.03 Every Call

synthorai — Fri, 31 Jul 2026 17:44:41 +0000

Connecting one MCP server to your agent adds a per-call tax before anything happens: the GitHub server's 26 tools cost $0.0302 on every single Claude Opus 4.8 call, whether or not a tool is used. The mechanism is invisible until you read the meter, the size depends more on which model you run than on how many tools you connect, and most of it is recoverable. We measured the whole chain: five model families, five real MCP servers, synthetic ladders from 0 to 40 tools, and the caching and slimming levers that shrink the bill.

TL;DR

Tool definitions are re-billed as input tokens on every API call; one small tool measured 401 tokens on Claude Opus 4.8.
The same toolset costs 2.7x more tokens on Claude than on GPT-5.6 (173 vs 64 per tool measured).
The real GitHub MCP server (26 tools) bills $0.0302 per call on Opus 4.8 vs $0.0029 on Gemini 3.6 Flash, a 10x spread.
Explicit caching cuts the carried block to a tenth, but sub-floor toolsets cannot cache, and changing one tool re-bills the whole block.

How does an MCP server turn into input tokens?

Through a three-stage chain, and only the last stage costs money, but it repeats on every call. One scoping note first: MCP servers expose three primitives, tools, resources, and prompts, and tools are the only one with a standing bill, because the API has a tools parameter that must carry every schema on every request. Resources and prompts have no API field of their own; they cost tokens only when their content is actually pulled into the conversation. Stage 1 is the MCP protocol itself: your agent client asks the server tools/list over JSON-RPC and gets back schemas; this happens once per session and touches no model. Stage 2 is the client converting those schemas into the API's tools parameter, dropping MCP-only fields like annotations and outputSchema; still free. Stage 3 is the provider rendering those schemas into hidden prompt text on every request, because the API is stateless and the model cannot remember what tools exist. That rendered text bills at the full input rate, every call, used or not.

The smallest possible demonstration: a request whose user message is just "Reply OK" costs 11 prompt tokens on Opus 4.8. Add one modest get_weather tool, two parameters, one-line description, and the same request costs 412 tokens. That single tool is a 401-token, $0.002 rider on every call it travels with. Note the asymmetry with what you sent: the tool's JSON is only about 130 tokens of text; the provider's rendering nearly tripled it.

What does one tool cost on each model?

It depends on the model far more than most teams expect. We sent identical synthetic toolsets (three parameters, one-line descriptions) at ladder sizes from 0 to 40 tools through five families and read the billed deltas:

Model	Fixed overhead (tools on)	Marginal per tool	40 identical tools
Claude Opus 4.8	290	173	7,210
GLM 5.2	96	120	4,896
Kimi K3	37	99	3,997
Gemini 3.6 Flash	≈0	72	2,871
GPT-5.6 Terra	98	64	2,658

Same tools, 2.7x spread: Claude renders the most verbose tool harness in the group, GPT-5.6 the tersest. The relationship to the JSON you send also cuts both ways: Claude bills about 1.36x a naive bytes-over-four estimate of the wire payload, while GPT-5.6 and Gemini bill roughly half of it. Token counts for tool blocks are not portable across families, so budget per model, not per schema.

What do real MCP servers cost per call?

Here is the table for five real, unmodified servers from the official MCP repository, converted with the standard field mapping and measured as carried overhead per call:

MCP server (tools)	Opus 4.8	GPT-5.6 Terra	Gemini 3.6 Flash	Kimi K3	GLM 5.2
GitHub (26)	6,043 tok / $0.0302	2,076 / $0.0052	1,931 / $0.0029	3,152 / $0.0095	4,077 / $0.0022
Filesystem (14)	2,785 / $0.0139	1,254 / $0.0031	1,200 / $0.0018	1,574 / $0.0047	1,772 / $0.0010
Everything (13)	1,942 / $0.0097	798 / $0.0020	663 / $0.0010	970 / $0.0029	1,176 / $0.0006
Memory (9)	1,670 / $0.0083	554 / $0.0014	491 / $0.0007	815 / $0.0024	1,057 / $0.0006
Sequential-thinking (1)	1,764 / $0.0088	912 / $0.0023	815 / $0.0012	870 / $0.0026	1,015 / $0.0006

Two lessons hide in the last row. Sequential-thinking exposes a single tool and still costs more than Memory's nine on four of the five families, because its one description runs enormous: tool count is not the metric, rendered size is. And the top row is the one that surprises agent builders: carrying GitHub's toolset through a 10-call agent episode is $0.30 on Opus 4.8 and $0.03 on Gemini, a full order of magnitude, before any real work happens. Note that GLM 5.2 lands lower still in dollars while billing more than twice Gemini's tokens; a cheap list rate outweighs a verbose renderer, so budget tokens and dollars separately.

How much of the toolset do agents actually use?

A small fraction per call, which is what makes the tax feel unfair. In our agent suite's tooling scenario, an 8-tool set rides all three calls of a typical episode while each call invokes at most one tool: every call pays for eight schemas to use one. At Opus 4.8's measured rates that 8-tool block is about 1,674 tokens per call, roughly 5,000 tokens per episode of pure schema carriage. The tool-loop scenario is the favorable case, three tools with near-every-call usage, and even there the block re-ships on all four to five calls of the loop. The general rule from the records: utilization per call rarely exceeds one tool, so the per-call cost of your toolset scales with what you connected, not what the agent does.

Can caching absorb the tool tax?

Mostly yes on the right models, with three sharp edges we measured. On Claude Opus 4.8, the tool block is a first-class cacheable prefix: marking the last tool with cache_control wrote our 20-tool block once (3,682 tokens at the 1.25x write premium) and every subsequent call read it back at 0.1x, cutting the effective carry by 90%. Kimi K3's automatic caching did even better with zero configuration: 2,048 of the block's 2,112 tokens, 97%, came from cache on the second call, in its familiar 256-token blocks. GPT-5.6's explicit breakpoints price the same shape, 1.25x writes against 0.1x reads, in our day-one measurements of that family. Gemini's implicit cache, consistent with everything else we have measured about it, produced zero hits in three primed probes over the tool block; treat any discount there as a rebate, not a plan.

The edges. First, the floor trap: a 2-tool block is about 655 tokens on Claude, below the 1,024-token cache minimum, so small toolsets cannot cache at all even when explicitly marked; the per-model floors decide whether this lever exists for you. Second, volatility: the tool block sits at the very top of the prompt, so changing the tool list busts everything after it. We measured the penalty directly: adding one tool to a cached 20-tool set on Claude Opus 5 re-wrote the entire 4,082-token block at the write premium. Keep the tool list frozen for the session, or pay the re-write on every change. Third, that freeze requirement is exactly what Anthropic's mid-conversation tool changes beta (documented for Opus 5) is built to relax, letting tools change between turns without invalidating the cache; it is the feature to watch for tool-heavy agents.

How do you shrink the tool block itself?

Cut parameters before prose. On our 20-tool block, trimming descriptions to a terse single clause saved 10%; cutting each tool from three parameters to one saved 34%; both together saved 44% (3,768 tokens down to 2,128). Parameter schemas (names, types, nested descriptions) are where the tokens live, which inverts the usual instinct to polish descriptions and let schemas sprawl.

The bigger lever is not connecting what you will not use. Every MCP server you attach adds its full block to every call; a second config line can double the tax. Registering a subset of a server's tools, where your client supports filtering, shrinks the rider roughly in proportion: keep five of the GitHub server's 26 and Opus 4.8's $0.0302 falls to an estimated $0.007 at the server's average tool size. And if you run models from more than one family, remember the 2.7x spread above: the same agent, moved from Gemini to Claude without re-budgeting, nearly triples its tool carriage in tokens.

FAQ

Does MCP itself add token cost?

No. The MCP protocol, discovery, JSON-RPC, tool invocation plumbing, never touches the model and bills nothing. The cost appears when your client forwards the server's schemas into the API's tools parameter, which the provider re-renders into billed prompt text on every call. MCP's contribution is scale: it made attaching 26 tools a one-line decision, and 26 schemas ride every call thereafter.

Do unused tools cost anything?

Yes, exactly as much as used ones. The model must read every schema on every call to know what it could invoke; our agent-suite records show at most one tool invoked per call while the full set is billed each time. An attached-but-idle server is pure carrying cost, which caching can discount but only slimming or disconnecting removes.

Do MCP resources and prompts cost tokens like tools do?

No. Tools are unique in having a per-call carriage cost, because the tools parameter re-ships every schema on every request. Resources bill only when the client reads one and inserts its content into the conversation, at which point it is ordinary input, priced like any retrieved document, and the same layering discipline applies: volatile resource content belongs after the cache breakpoints. Prompt templates likewise bill only when invoked, as the rendered text they produce. A connected server's unused resources and prompts cost nothing, which makes tools the number to audit when an MCP integration shows up on the bill.

Does changing the tool list invalidate the prompt cache?

Yes, entirely. The tool block renders at the top of the prompt, so any change re-writes it and everything cached behind it; we measured a one-tool addition re-billing a 4,082-token block at the 1.25x premium. Freeze the tool list per session, batch tool-list changes together, and watch Anthropic's mid-conversation tool changes beta, which is designed to remove exactly this penalty on Opus 5.

Measured 2026-07-31 through the Synthorai gateway: synthetic tool ladders (0-40 tools, n=2) and five real MCP-server toolsets (schemas captured live from the official servers via JSON-RPC tools/list) across five model families; caching probes with salted prefixes and per-call cache itemization; agent utilization from the suite records behind our agent cost studies. GPT-5.6 caching multipliers cite our day-one cost guide measurements. Dollar figures are billed-cost deltas read from the gateway meter, which prices at list input rates. Rates and rendering behavior may change; verify against your own usage records.

Prompt Cache Write Cost: When Does the 1.25x Premium Pay?

synthorai — Wed, 29 Jul 2026 16:29:32 +0000

A prompt-cache write pays for itself after a single re-read within the TTL: the write bills 1.25x the input rate, the read bills 0.1x, so one hit already swings the two-call total from a 25% premium to a 32.5% saving. And yet on one of the five scenarios in our agent suite, the same premium produced a measured 6% loss. The difference is a single number, the read:write ratio of your traffic, and this post is about measuring it before the invoice does. Everything below was measured through the Synthorai gateway on live billing meters; the agent numbers come from a 150-episode suite, the TTL and clustering results from dedicated salted probes.

TL;DR

Explicit cache writes bill 1.25x (5-minute TTL) or 2x (1-hour); GPT-5.6's breakpoints converged on the same 1.25x/0.1x shape; break-even is one re-read.
Across our agent suite the premium's net effect ranged from +6% (RAG, read:write 0.2) to -83% (batch, 15.7).
Cache hits refresh Claude's TTL for free: steady traffic pays the write once per idle gap, not per window.
Implicit caching spans a spectrum: GPT-5.5's sustained 7/8 hits after one primed call, Gemini's 1-3 hits per 12 even clustered; no premium either way.

What does a cache write actually cost?

Three pricing shapes exist across the providers we compare, and two of them charge for writing. Explicit caching on Claude bills cache creation at 1.25x the input rate for the default 5-minute TTL and 2x for the 1-hour tier, with reads at 0.1x; GPT-5.6 moved to explicit breakpoints with the same 1.25x write and 0.1x read multipliers, so the industry's two largest explicit implementations now price identically. The third shape is implicit caching (Gemini, Kimi, the pre-5.6 OpenAI models): no write premium at all, only discounted reads when the provider happens to hit.

The break-even arithmetic for explicit writes is short. A prefix of P tokens costs P at the normal rate uncached. Cache it and the first call costs 1.25P; every re-read within the TTL costs 0.1P instead of P. One re-read brings the two-call total to 1.35P against 2P uncached, a 32.5% saving, and each further hit saves 90% of its prefix cost. The premium is not the risk. The risk is writing blocks that never get read back, and that risk turns out to be workload-shaped.

When does the write premium lose money?

When your prefix changes faster than it repeats. We measured the read:write token ratio across five agent scenarios (identical models, identical marker discipline), then computed the net effect of caching at the 1.25x/0.1x rates:

Scenario	Read:write ratio	Net effect vs no caching
Batch (stable instructions, many jobs)	15.7	-83% on prefix spend
Long chat (growing history)	6.8	-75%
Tool loop / tooling	5.4	-72%
RAG (retrieved docs in the prefix)	0.2	+6%: caching costs more
Suite overall	3.5	-64%

The RAG row is the one to internalize. Retrieved documents differ per query, so every call re-wrote a prefix that the next call could not reuse: five written tokens for every one read back, and the 1.25x premium compounds into a net loss. Nothing was misconfigured; the workload's shape simply cannot amortize a write. The fix is layering, not abstinence: mark the system prompt and tool definitions (stable across queries) and leave the retrieved documents after the last breakpoint, unmarked. The same discipline applies to anything volatile, timestamps, user names, per-request context, and to settings that silently invalidate the prefix: changing output_config.effort between turns re-renders the prompt and busts the cache, which is why effort should stay constant within a cached session. Our LangChain study found the same failure mode framework-side: builders that make marking everything as easy as marking the right thing.

How often do you actually re-pay the write?

Once per idle gap, not once per TTL window, because hits refresh the clock for free. We proved this with a salted 4,981-token prefix on Claude Opus 4.8:

Call	Timing	Cache write	Cache read
1	t=0	4,981	0
2	+3 min	0	4,981
3	+6 min	0	4,981
4	+9 min	0	4,981
5	after 7 min of silence	4,981	0

Call 4 lands well past the original 5-minute TTL and still reads clean, because calls 2 and 3 each renewed the window at no charge; only the 7-minute silence forced a re-write. The practical consequence: for any route whose inter-arrival time stays under the TTL, the write premium is a one-time cost, and the 5-minute tier behaves like an indefinite cache. The 1-hour tier (2x write, verified billing into its own ephemeral_1h_input_tokens bucket through the gateway) is for the other traffic: pay 2x once instead of 1.25x repeatedly, which wins as soon as it saves you a single re-write, so it fits routes with idle gaps that fall between five minutes and an hour. Past an hour of silence, neither tier carries you across runs. A scheduled job should budget by what happens inside one run: a cron that fires a single request gains nothing from writing, while a cron that fans out into many same-prefix requests is just a batch, and caches exactly like one.

Can you rescue implicit caching by shaping your traffic?

It depends entirely on whose implicit cache, because best-effort spans a spectrum. At the good end, GPT-5.5's automatic caching behaved like a mechanism: after one priming call the entry was readable two seconds later (first probe hit, on both salts we tried), and a subsequent burst read 4,864 of a 5,170-token prompt from cache on seven of eight probes with no premium paid for any of it. That is genuinely free money, and it demands nothing beyond sending the same prefix twice. At the other end sits Gemini 3.6 Flash: in the same agent suite where explicit caching served 77-78% of Claude's input tokens, Gemini's implicit cache served 4%, on workloads with genuinely repeating prefixes.

We tried to rescue the bad end with traffic shaping: cluster same-prefix calls so the provider's cache is still warm. Twelve back-to-back calls on Gemini 3.6 Flash produced exactly one hit (call 10; call 11 missed again), and eight calls spread at 3-minute intervals produced zero. Suspecting a fluke, we re-ran the clustered arm on a second, independent request path and on Gemini 3.5 Flash: three hits and one hit out of twelve, respectively, never consecutive for long, and the one block that did cache came back at the same 4,073 tokens on both paths. The mechanics are consistent; the probability is not.

Part of the miss rate is ingestion lag, and its size is provider-specific: GPT-5.5's entry was readable two seconds after the prime, while Gemini's takes tens of seconds to build, so bursts that follow the first call immediately outrun its cache (no Gemini arm in any of our runs hit before the fourth call). Priming the prefix once and waiting 45 seconds before the burst lifted hits to 3 of 8, the best any shaping achieved, but a 90-second wait returned zero of 8: by the time we came back, the entry was already gone. Days earlier, a longer warm-up sweep on the same model had sustained hits, so the rate also drifts with time and load. Between the build lag, the short lifetime, and the drift, clustering on the bad end helps only in the sense that three hits beat one. (A testing note: probe implicit caches with paced calls and a prime-and-wait arm, and measure the build lag itself with a post-prime ladder; back-to-back probing measures your request rate, not the cache.)

So the honest rule is per-provider, not per-mechanism: measure whether your provider's implicit cache behaves like GPT-5.5's (prime it, then trust it) or like Gemini's (treat any hit as a rebate). Kimi K3 sits between them, no premium, a small floor, and 57-62% of input served from cache without configuration. And there is one telling data point about which end wins: OpenAI had the best implicit cache we measured and still moved GPT-5.6 to explicit breakpoints, trading its own luck-free rebate for customer-controlled marks.

Which workloads amortize the premium?

Match the decision to the read:write ratio and the idle gap, both of which you can read from your own usage records before committing:

Workload	Verdict	Why
Multi-turn agent sessions	Cache	History re-reads every turn; suite measured R:W 3.3-3.6 overall
Shared system prompt, steady QPS	Cache, 5m tier	Inter-arrival < TTL means one write, then free refreshes
Bursty sessions with 5-60 min gaps	Cache, 1h tier	2x once beats 1.25x per gap
Batch jobs, same instructions	Cache, and cluster the jobs	R:W 15.7 measured, -83%; clustering keeps the window warm
RAG with volatile documents	Layer it	Mark instructions/tools only; docs after the last breakpoint
Scheduled crons, one request per run	Skip	Runs are hours apart; every write expires unread
Scheduled crons, many requests per run	Cache within the run	The run is a batch: first request writes, the rest read; hits keep the TTL alive for the run's duration
Sub-floor prompts	Skip	Below the per-model cache minimums nothing caches anyway

Two closing rules from the meters. First, judge caching by cache_read versus cache_creation in your own usage itemization, not by whether hits feel frequent; the RAG scenario felt cache-friendly and measured 0.2. Second, when the ratio is healthy, the premium disappears into rounding: at read:write 3.5, the suite's overall figure, prefix spend drops 64% including every write ever paid.

FAQ

Is prompt caching worth it for RAG?

Not for the retrieved documents. In our measurement RAG wrote five tokens for every one it read back, turning the 1.25x premium into a 6% net loss. Cache the stable layers, system prompt and tool definitions, and keep retrieved content after the last breakpoint; the complete caching guide covers prompt layering in detail.

Should I use the 5-minute or the 1-hour cache TTL?

By idle gap, not by session length. Hits refresh the 5-minute TTL for free, so any route with sub-5-minute inter-arrival never re-writes and the cheap tier behaves indefinitely. Pay the 1-hour tier's 2x write only when your gaps fall between five minutes and an hour; it wins as soon as it avoids one 1.25x re-write. Gaps beyond an hour: budget with no cache at all.

Does implicit caching charge a write fee?

No, and structurally it cannot: a write premium only makes sense where the caller opts in, so premiums exist exclusively on the explicit side (Anthropic's 1.25x/2x multipliers, GPT-5.6's breakpoints, and the storage-rent shape of explicit cached-content APIs like Gemini's per-token-hour fee). Every implicit implementation we measured (Gemini, Kimi, pre-5.6 OpenAI) and every major published rate card (DeepSeek, Qwen, Grok) bills implicit caching as discounted reads only. What varies is reliability: GPT-5.5's implicit cache sustained 7 hits in 8 primed probes, while Gemini's served 4% of input in our agent suite and clustering barely moved it. Zero premium on a discount you reliably receive beats everything; zero premium on one that arrives at random is worth less than a small premium on one you control.

Measured 2026-07-25 to 2026-07-29 through the Synthorai gateway: scenario read:write ratios from 150 Claude-arm agent episodes with per-call cache itemization (Gemini and Kimi shares from their own suite runs); TTL-refresh and 1-hour-bucket probes on claude-opus-4-8 (salted 4,981- and 3,742-token prefixes); implicit-cache probes on gemini-3.6-flash and gemini-3.5-flash (salted ~6,800-token prefixes; clustered, spread, and prime-and-wait arms) and on gpt-5.5 (salted 5,170-token prompt, primed burst). Net-effect percentages computed from measured ratios at 1.25x write / 0.1x read multipliers. Rates and cache behavior may change; verify against your own usage records.

Claude Opus 5 vs Opus 4.8, Measured: Same Price, 3x Apart

synthorai — Sun, 26 Jul 2026 17:41:35 +0000

Claude Opus 5 and Claude Opus 4.8 bill the same $5 per million input tokens and $25 per million output, and on identical prompts the default Opus 5 configuration cost 3.1x more. The reason is adaptive thinking: Opus 5 thinks by default, bills the thinking as output, and never shows it to you. One request setting closes the gap to exact parity, and it is a setting the bigger Fable 5 refuses to accept. Opus 5 went GA on 2026-07-24, positioned as Fable-5-level intelligence at half the token price; whether your bill actually halves depends almost entirely on this one choice.

TL;DR

Default Opus 5 billed 3.1x the same-priced Opus 4.8 on our five-task matrix; 42-95% of its output tokens were hidden thinking.
thinking: {"type": "disabled"} brought Opus 5 to exact parity with 4.8 (384 vs 384 output tokens), accuracy held; Fable 5 rejects that parameter.
On agent traffic the tax collapses to +33%, with tool/batch scenarios near parity: adaptive thinking barely fires in tool loops.
The 1M context is real (needle recalled at 969,950 tokens) and the cache floor is 512 tokens, half of 4.8's 1,024.

How do Opus 5, Opus 4.8, and Fable 5 differ as platforms?

Before the cost deep-dive, the orientation map: three current Claude tiers, two axes of difference, rates and request shape. Side by side (measured items marked, the rest from the model docs):

	Opus 4.8	Opus 5	Fable 5
List price (in/out per M)	$5 / $25	$5 / $25	$10 / $50
Thinking default	off unless requested	adaptive, on (measured)	always on
`thinking: disabled`	accepted, independent of effort	accepted at effort `high` or below	rejected with 400 (measured)
Effort ladder	`low`-`max`, default `high`	`low`-`max`, default `high` (cost side measured below)	`low`-`max`, default `high`
Thinking content returned	n/a by default	never (measured)	never
Cache floor	1,024 tokens	512 tokens (measured)	512 tokens
Context window	1M	1M, default and max (969,950-token needle, measured below)	1M
Assistant prefill	rejected	rejected, named 400 (measured)	rejected
Fast mode	available (research preview)	available, $10/$50	not offered
Refusal fallbacks	serves as the default fallback target	`fallbacks` incl. new `"default"` mode (beta)	introduced here (explicit lists)
Data retention	standard options	standard options	30-day retention required

Three rows deserve a sentence. The thinking: disabled row is the migration trap: on Opus 5 it is coupled to effort, accepted only at high or below per the docs, where 4.8 treated the two settings as independent; version-gate migration scripts. The fast-mode row sets up a comparison that matters later: Opus 5 in a hurry costs exactly Fable 5's rate card, so "fast Opus 5 vs default Fable 5" is a pure speed-vs-capability trade at equal token prices. And the retention row is the quiet compliance win: Fable-class intelligence on Opus 5 comes without Fable 5's 30-day retention mandate.

Two more platform notes that resist table form. Anthropic documents edge cases when thinking is disabled (tool calls occasionally written into visible text, leaked internal tags); we did not hit either across 84 thinking-off calls in the agent suite measured below, but the guidance reinforces the routing rule: keep thinking on for tool-heavy routes, where the tax is small anyway. And mid-conversation tool changes (beta, Opus 5) let you add or remove tools between turns without busting the prompt cache, which protects the cached-prefix economics our prompt caching guide is built on for long agent sessions.

What does Opus 5 cost out of the box vs Opus 4.8?

3.1x more for the same work at the same rate card. We ran the three current Claude tiers through a five-task matrix (n=3 per cell, salted prompts) at their defaults, on the native Messages API, with Fable 5 alongside for scale:

Task	Opus 5 default	Opus 4.8	Fable 5	Accuracy
Trivial arithmetic	12	3	12	3/3 all
Factual one-liner	40	6	11	3/3 all
Small code function	70	38	44	—
Multi-step word problem	152	102	52	3/3 all
120-word paragraph	1,031	236	264	—
Total out tokens (cost per set)	1,305 ($0.03427)	384 ($0.01120)	383 ($0.02233)

Same answers, same rates as 4.8, three times the bill. Opus 4.8 does not think unless you ask it to; Opus 5 ships with adaptive thinking on, and the thinking bills at the full $25/M output rate.

The Fable 5 column holds the counterintuitive result: the model with double the rates ($10/$50) billed 35% less than default Opus 5 in absolute dollars, because it answered the same tasks in 383 output tokens to Opus 5's 1,305. All three models run the same documented default effort (high), so the gap is thinking calibration, not configuration. Two mechanisms fit the numbers. First, a more capable model needs less deliberation to be sure of an easy answer: Fable 5 spent 52 tokens on the word problem where Opus 5 spent 152, and 264 on the paragraph where Opus 5 spent 1,031. Second, Opus 5's headline capability is test-time compute scaling, converting extra deliberation into quality on hard problems, and its default calibration buys that insurance on every request, including the ones that need none. On easy traffic you are paying for insurance you do not use; Fable 5 mostly declines to buy it.

Where does the extra spend go?

Into reasoning you cannot read. Comparing billed output tokens against the visible answer text, 42-95% of Opus 5's default output spend was hidden thinking, and it fires even on questions that need none: the answer to 17*23 carried 11 thinking tokens behind a 1-token answer, and the 120-word writing task spent about 806 of its 1,031 output tokens deliberating. The thinking content is never returned in any form, no summary, no trace, which puts Opus 5 at the most closed end of the visibility spectrum we mapped in our token usage anatomy study, alongside Fable 5. You can see the count in the usage itemization; you cannot see what it bought.

What does the thinking switch actually do?

It turns Opus 5 into Opus 4.8's bill. Sending thinking: {"type": "disabled"} drove thinking to zero on every task, and the totals landed at exact parity with 4.8, 384 output tokens to 384, $0.01130 to $0.01120 per set:

Arm	Output tokens (set)	Cost (set)	vs Opus 4.8	Accuracy (3 checkable tasks)
Opus 5 default (= effort `high`)	1,305	$0.03427	3.1x	9/9
Opus 5, effort low	1,019	$0.02720	2.4x	9/9
Opus 5, effort medium	1,167	$0.03089	2.8x	9/9
Opus 5, explicit effort high	1,514	$0.03956	3.5x	9/9
Opus 5, effort xhigh	1,633	$0.04262	3.8x	8/8
Opus 5, effort max	1,569	$0.04093	3.7x	9/9
Opus 5, thinking disabled	384	$0.01130	1.0x	9/9
Opus 4.8 default (= effort `high`, no thinking)	384	$0.01120	1.0x	9/9
Fable 5 default (= effort `high`, thinking always on)	383	$0.02233	2.0x	9/9

A labeling note first: every model here documents the same API default, effort high, and Anthropic states an explicit high is identical to omitting the parameter. Our implicit-default and explicit-high arms still differ by 16%, which is run-to-run variance on the writing task (the noisiest cell in every batch we have run), not a real distinction; read those two rows as one arm measured twice. What separates the three defaults is not the effort level but what thinking does at that level: none on 4.8, frugal and adaptive on Fable 5, aggressive and adaptive on Opus 5.

Two things stand out. First, the effort dial is a quality ladder, not a cost knob: the ladder itself (low through max) is not new, 4.8 accepts the same range, but on tasks this simple every step above low just buys more deliberation at identical accuracy, with xhigh topping out at 3.8x the 4.8 bill. The top tiers exist for test-time compute scaling on genuinely hard problems, which a five-task sanity matrix cannot exercise; what it can show is the cost side, and the cost side says the dial never gets you back to 4.8 parity. Only the off switch does, at 67% below the default. Second, the switch exists at all: Fable 5 rejects thinking: {"type": "disabled"} with a 400, so this is a genuine Opus 5 differentiator, not a family constant. The same thinking object works on both gateway surfaces, /v1/messages and the OpenAI-compatible /v1/chat/completions; on the latter, disabled runs report zero reasoning_tokens in completion_tokens_details:

{"model": "claude-opus-5", "thinking": {"type": "disabled"}}

The honest caveat: our checkable tasks are retrieval- and single-step-shaped, and accuracy held 9/9 with thinking off, including the multi-step word problem. Harder agentic work is exactly what adaptive thinking exists for, so treat the switch as a per-route decision, the same rule we landed on for Kimi K3 and Gemini 3.6 Flash: off for extraction, formatting, and single-step calls; default where your evals say the thinking earns its bill.

Does the 3x tax hold on agent workloads?

No, and the difference is the point. Across our agent scenario suite (tool loops, RAG, tooling, batch, long chat; 50 episodes per arm), default Opus 5 cost only 33% more than Opus 4.8, not 210%, and the tool-shaped scenarios ran near parity (1.01-1.22x). Adaptive thinking is genuinely adaptive there: about 88 thinking tokens per call inside agent loops versus 806 on a bare writing prompt. The outlier is long chat at 1.58x, which is where the off switch still pays (1.22x with thinking disabled). Function calling showed zero thinking tax at all: at the default, a tool-call request came back in 52 output tokens with no thinking attached, the same budget a non-thinking model would spend.

So the practical split is by traffic shape, not by model: bare completions and chat-shaped calls carry the 3x default tax and want the switch; tool-heavy agent traffic mostly does not need it.

Is Opus 5 really half the price of Fable 5?

Only after you flip the switch. Per token, yes: $5/$25 versus Fable 5's $10/$50. In practice, on our bare-task matrix, default Opus 5 billed $0.03427 per set against Fable 5's $0.02233, 53% more in absolute dollars, because Fable 5 answered the same tasks in 383 output tokens to Opus 5's 1,305. With thinking disabled, Opus 5's $0.01130 is almost exactly half of Fable 5's bill, which is the launch promise made real, through a parameter Fable 5 itself does not accept.

Context, cache, and tokenizer: what else did we verify?

The 1M window is real and fails loud. A recall needle at the front of a 969,950-token prompt came back correct in 39 seconds, and a 1,010,221-token prompt returned a clean prompt is too long: … > 1000000 maximum rather than silently truncating.

The cache floor halved. Anthropic documents a 512-token minimum cacheable prefix for Opus 5 (and Fable 5), down from 1,024 on Opus 4.8 and Sonnet 5; our sweep matched it, with prefixes near 511 tokens never caching and 547 caching reliably. Cached reads bill $0.50/M (0.1x), writes 1.25x, 5-minute TTL. Shorter system prompts are now cacheable, which quietly matters for high-QPS routes.

The tokenizer is unchanged across Opus 5, Opus 4.8, Fable 5, and Sonnet 5: identical token counts on our multilingual and code samples, so per-language budgets and prompt-size estimates carry over with no re-baselining.

FAQ

Can you turn thinking off on Claude Opus 5?

Yes, at effort high or below; the docs state that combining it with xhigh or max returns a 400. In our probes it drove thinking tokens to zero and brought cost to parity with Opus 4.8 (384 vs 384 output tokens on our matrix). This is specific to Opus 5: Fable 5 rejects the same parameter at any effort with a 400. The effort dial (low through max) also works but never reaches parity: from -21% at low to +24% at xhigh versus the default in our matrix.

Why is my Opus 5 bill higher than Opus 4.8 at the same list prices?

Because Opus 5 thinks by default and the thinking bills as output at $25/M. On bare prompts, 42-95% of its billed output was hidden reasoning in our measurements; a two-digit multiplication carried 11 thinking tokens behind a 1-token answer. Read reasoning_tokens from the usage itemization to see the share on your own traffic, and disable thinking on routes that do not need it.

Should agent workloads disable thinking on Opus 5?

Usually not. In our agent suite the default cost only +33% versus Opus 4.8, with tool and batch scenarios near parity, because adaptive thinking barely fires inside tool loops. The exception is long chat-shaped sessions (1.58x), where the switch still pays. Measure your own mix; the tax lives in bare completions, not tool calls.

Measured 2026-07-25 through 2026-07-27 on claude-opus-5, claude-opus-4-8, and claude-fable-5 via the Synthorai gateway: five-task matrix and effort/switch ablation from one canonical batch (n=3 per cell, salted prompts, native Messages API), agent numbers from a 150-episode scenario suite, context and cache probes from needle-recall and prefix sweeps, API-shape rows (prefill, switch acceptance) from direct request probes. Accuracy counts use tasks with a single checkable answer. Prices and behavior may change; verify against your own usage records.

Gemini 3.6 Flash: the Thinking Dial That Moves Cost 30x (Measured)

synthorai — Fri, 24 Jul 2026 11:16:46 +0000

Gemini 3.6 Flash charges you for thinking tokens on top of the answer, and how many it spends is a dial you control per request. On the same 120-word writing task, the default setting billed $0.03316 and the minimal setting billed $0.00110, a 30x swing for output a reader could not tell apart. That dial is the most important cost decision on this model, and it comes with one sharp edge. Gemini 3.6 Flash went generally available on 2026-07-21 at $1.50 per million input tokens and $7.50 per million output, down from $9 output on 3.5 Flash. It shipped alongside Gemini 3.5 Flash-Lite and a security-tuned 3.5 Flash Cyber; this post measures the two general-purpose tiers, 3.6 Flash and Flash-Lite.

TL;DR

reasoning_effort: "minimal" cut per-call cost 91–97% versus the default (a 30x swing on a 120-word task), free on single-step, structured-output, and tool-calling work but breaking multi-step math 3/3 → 0/3.
Google's "17% fewer output tokens" is workload-dependent: our reasoning-heavy tasks ran 19% lighter (32% cheaper), our agent suite 9% heavier (6% cheaper).
The 1M context is real (a needle at 972K tokens recalled) and prompt caching matches Google's published 4,096-token floor exactly, a clean spec match unlike some "1M context" models that undershoot what they advertise.

Everything below was measured on 2026-07-24 through the Synthorai gateway, with repeated prompts salted to defeat caches; raw usage records back every number.

What does Gemini 3.6 Flash cost per task at default settings?

Reasoning dominates the output bill, and it is charged whether or not you see it. At the default effort, the model spends far more tokens thinking than answering, and those reasoning tokens bill at the full $7.50/M output rate:

Task	Answer tokens	Reasoning tokens (billed)	Cost per call
Factual one-liner	2	69	$0.00056
Trivial arithmetic	3	167	$0.00131
Small code function	29	379	$0.00312
Multi-step word problem	4	472	$0.00368
120-word paragraph	139	4,274	$0.03316

The pattern is the one to internalize: a two-token factual answer still carried 69 tokens of reasoning, and the 120-word paragraph spent 30x more tokens thinking than writing. Reasoning tokens are itemized in completion_tokens_details.reasoning_tokens, so you can see the count, but never the content. Gemini returns no thinking summary or trace at all, the most closed end of the spectrum we mapped in our token usage anatomy study, where Kimi K3 returns its full chain of thought and GPT-5.6 a summary. The next section is about turning that spend down.

What does the thinking dial actually do?

It is a genuine, monotonic cost lever, and on most task types it is nearly free money. Setting reasoning_effort (or the native thinking_config.thinking_level) to minimal drove reasoning tokens to zero and cut cost 91–97% per task:

Task	Default cost	`minimal` cost	Swing	Accuracy default → minimal
Factual one-liner	$0.00056	$0.00005	12x	3/3 → 3/3
Trivial arithmetic	$0.00131	$0.00006	22x	3/3 → 3/3
Small code function	$0.00312	$0.00028	11x	—
Multi-step word problem	$0.00368	$0.00014	26x	3/3 → 0/3
120-word paragraph	$0.03316	$0.00110	30x	—

The dial is real and the accepted values are minimal, low, medium (the default), and high; each step bought monotonically more reasoning in our probes (minimal 0 tokens, low ~180, medium ~530, high ~650). The one thing minimal cannot do is think, and multi-step arithmetic needs to: forced to answer the pencils-and-bags word problem tersely, the model got it wrong all three times, with scattered wrong answers rather than one systematic slip. On retrieval, classification, formatting, and single-step questions, minimal held accuracy and cut the bill by an order of magnitude.

The practical rule mirrors what we found on Kimi K3: minimal is a defensible default for extraction, lookup, and formatting, and a footgun for anything that needs intermediate steps. Set it per route, not globally, and verify accuracy on your own tasks before shipping it on a reasoning-heavy one.

Two high-volume production shapes make the case concrete: structured output and function calling both spend reasoning at the default, and both are safe to run at minimal. A schema-constrained extraction (response_format with a JSON schema) billed 337 reasoning tokens at the default and returned valid JSON; at minimal it billed zero reasoning, still returned valid schema-conforming JSON, and cost 9x less. A function call behaved the same way: 74 reasoning tokens and a correct get_weather(city) call at the default, versus zero reasoning and the same correct call at minimal, 4x cheaper. These are single-step tasks dressed up as "structured," and the model does not need to think its way to a field it was told to fill, so if your traffic is extraction or tool routing, minimal is close to free money.

Does the "17% fewer output tokens" claim hold up?

It depends on the workload, and the split is instructive. Google's launch positioned 3.6 Flash as spending about 17% fewer output tokens than 3.5 Flash on the Artificial Analysis Index (up to 65% on individual agentic evals). We ran both models through two of our own testbeds and got opposite signs:

Testbed	Output tokens 3.6 vs 3.5	Cost 3.6 vs 3.5
Task matrix (five short tasks, reasoning-heavy)	−19%	−32%
Agent suite (tool loop, RAG, batch, long chat)	+9%	−6%

On the reasoning-heavy short tasks, the claim not only reproduced but beat its headline: total output fell 19%, close to Google's 17%, and it came almost entirely from thinking, not the answer. Splitting the output tokens in a paired rerun of both models, the visible answer shrank only 4% while reasoning fell 19%, concentrated in the math and writing tasks where 3.6 reaches the same result with less deliberation. That is the mechanism behind the benchmark: on work that leans on the thinking budget, 3.6 is genuinely more efficient at the same answer.

On agentic, multi-turn traffic the sign flips: 3.6 spent about 9% more output than 3.5 across the suite. The efficiency gain lives in the reasoning phase, and agent loops spend proportionally less of their budget there, so there is less to save and 3.6's slightly longer turns win out. Either way the bill drops, because the two effects stack differently: reasoning-heavy tasks save on both tokens and the $9→$7.50 rate cut (−32%), while agent traffic saves on price alone (−6%). The honest summary is that "17% fewer output tokens" is real where thinking dominates the output and inverts where it does not, so measure your own mix rather than assume the headline, and remember the dial from the previous section moves this far more than the version bump does.

Is the 1M context window real?

Yes, and it fails loud rather than silent. We placed a recall needle at the front of prompts of increasing size: it was still recalled correctly at 972K input tokens, and a prompt past the limit returned a clean 400 input token count exceeds the maximum rather than silently dropping content. That is worth stating because not every "1M-context" model on the market actually serves the window it advertises. One testing note for anyone reproducing this: pad with varied, sentence-shaped filler, because a prompt built from a single repeated token pushed the model into degenerate gibberish well before the size limit.

Prompt caching is automatic and matches the spec on the number that matters. Google documents a 4,096-token minimum for context caching on the Flash models, and our sweep landed exactly there: prefixes at or below ~2.1K never cached, hits began around 4.1K tokens, and each hit left roughly the last 2.1K uncached, after a 5-to-8 call warm-up. Cached input reads at $0.15/M, a 10x discount off the $1.50 fresh rate. This is worth stating plainly because it is the reassuring case: unlike some models we have measured whose advertised numbers overstate what the endpoint actually delivers, Gemini 3.6 Flash's cache floor and its 1M window both do what the docs say. Caching still only pays for genuinely long, stable prefixes, and note the Flash tiers support only automatic (implicit) caching, not the explicit cached-content API, so you cannot manually pin a big document and reuse it below the floor.

Where does Gemini 3.5 Flash-Lite fit?

Flash-Lite is the predictable-cost tier. It never spends reasoning tokens silently, so its bill tracks visible output one-to-one. On the same multi-step math problem Flash-Lite billed $0.00057 against 3.6 Flash's $0.00368 default, roughly 6x cheaper, and it worked the answer out in the open rather than in a hidden reasoning field. At $0.30/M input and $2.50/M output it is the right default for high-volume, latency-sensitive, single-step work; step up to 3.6 Flash when a task needs the reasoning the dial can add back. The tokenizer is unchanged not just across the three new models but back to Gemini 2.5 Flash: identical token counts on English, Chinese, Japanese, Korean, and Python across every generation we checked, so per-language budgets built for 2.5 carry to 3.6 without re-baselining.

FAQ

Can you turn reasoning off completely on Gemini 3.6 Flash?

reasoning_effort: "minimal" (or thinking_level: "minimal") drove reasoning tokens to zero in our probes and is the floor of the dial; the accepted steps are minimal, low, medium, and high. There is no separate "disabled" state, and attempts to hard-disable reasoning are rejected upstream, so minimal is as low as it goes, and for single-step tasks it is low enough.

Why is my Gemini bill higher than the visible answer suggests?

Because reasoning tokens bill at the full output rate and are not part of the text you get back. A two-token answer can carry dozens to thousands of billed reasoning tokens; read completion_tokens_details.reasoning_tokens (or reconcile total_tokens − prompt − completion) to see the real output charge, and turn the dial down where the task allows.

Gemini 3.6 Flash or Claude Haiku 4.5?

They occupy the same fast-tier slot at similar prices, and the split is by workload, not by a single winner. On our cost lens, 3.6 Flash's thinking dial is the differentiator: minimal makes it an order of magnitude cheaper on single-step traffic, while its default spends reasoning that Haiku 4.5, at $1/$5, does not. Published benchmarks give Haiku 4.5 the edge on coding depth and 3.6 Flash the lead on math and raw token price; pick by which your traffic is made of, and measure both on your own tasks before committing.

Is Gemini 3.6 Flash cheaper than 3.5 Flash?

Yes, in every workload we measured, though by how much depends on the shape. Output dropped from $9/M to $7.50/M, and on reasoning-heavy short tasks 3.6 also spent fewer output tokens, so cost fell about 32%; on agent traffic it spent slightly more tokens and the saving came from the rate cut alone, about 6%. Either way it is cheaper; migrate and re-measure your own mix. For per-token cost decomposition across families, see our token usage anatomy study.

Measured 2026-07-24 on gemini-3.6-flash, gemini-3.5-flash, and gemini-3.5-flash-lite via the Synthorai gateway; task-matrix and agent-suite token counts from per-call usage records, effort-dial results from a salted five-task ablation (n=3 per cell), context and cache probes from needle-recall and prefix sweeps. Accuracy counts use tasks with a single checkable answer. Prices and behavior may change; verify against your own usage records.

DEV Community: synthorai

Gemini 3.7 Flash API Cost, Measured: Tasks Bill 2.5-8x Less Than 3.6

What does Gemini 3.7 Flash actually cost?

Which thinking controls survive on 3.7?

What does the agent pitch cost per step?

What breaks when you upgrade from 3.6?

Do cache, context, and tokenizer carry over?

FAQ

How Many Tokens Is an Image? 15 APIs Measured, Same Icon 6 to 1,298

How do the vendors say an image is billed?

How many tokens does one image cost?

What decides how many tokens an image costs?

What are the three billing schemes?

Where exactly do the downscale boundaries sit?

Does shape, content, or file format change the bill?

What levers actually cut image input costs?

FAQ

LLM Thinking Controls: What 13 Models Accept, Ignore, or Enforce

Which thinking controls does each API accept?

Does "accepted" mean "enforced"?

Is there a universal off-switch?

What does turning thinking off cost in accuracy?

What do the effort levels actually buy?

Can you see what you paid for?

FAQ

Web Search API and Web Fetch API: How They Work and What $0.01 Buys

Why do models need web search and web fetch at all?

How does server-side web search actually work?

What does one search really cost?

Which three dials set your token bill?

What happens to search results on the next turn?

What do Anthropic and OpenAI charge, and where do their tools run?

FAQ

Qwen-Image 3.0 API, Measured: 9 Scenes for One $0.03 Image

How does Qwen-Image 3.0 billing actually work?

Is "nine images at once" real?

Does the 10-pixel text claim survive a zoom?

What does the 4,500-token prompt window actually buy?

Where does it land in the image-model lineup?

FAQ

Build an LLM Chatbot: Streaming, Context Compression, Memory

What must an MVP chatbot do?

How is it put together?

Which models go in the picker?

What does the model actually see each turn?

How does prompt caching work, and which models honor it?

How do you know you are near the context limit?

What happens at the limit?

How does memory survive across sessions?

What happens when the user hits Stop?

What happens when a turn fails?

How does web search work without a tool loop in your code?

How does markdown render without breaking mid-stream?

What does a conversation cost?

Where do all the knobs live?

What is left out, and where would it attach?

Related reading

DeepSeek V4 Flash API Cost: Thinking Mode Corrupts Strict JSON

How do the three V4 builds compare on paper and on the meter?

Does thinking mode corrupt structured output on V4 Flash?

Which thinking controls does the API accept?

How much thinking does two-hop math actually need?

What does the implicit cache deliver?

Are the 1M context and 384K output caps real?

Preview, 0731, and Pro: which build answers your calls?

FAQ

Qwen 3.8 Max API Pricing: 16 Thinking Tokens Beat the Off Switch

What do the thinking controls on Qwen 3.8 Max actually do?

How does thinking_budget actually work?

How much of the day-one documentation survives measurement?

Does turning thinking off save money?

Is the 1M-token context window real?

What does the implicit cache deliver, and where is the floor?

Do structured output and tool calls pay the reasoning tax?

What carries over from Qwen 3.7, and what changed?

FAQ

MCP Tool Overhead, Measured: 26 Tools, $0.03 Every Call

How does an MCP server turn into input tokens?

What does one tool cost on each model?

What do real MCP servers cost per call?