DEV Community: hc_xshh

I read China's FX rulebook for foreign passports. The $50,000 allowance runs one way.

hc_xshh — Thu, 05 Nov 2026 02:24:06 +0000

A foreign national working in China can convert foreign currency into yuan inside an annual allowance and with nothing but a passport. Going the other way, the same person has no allowance at all unless they hold a permanent-residence ID card. The difference sits in two articles that bank staff work from at the counter.

The document is the Current-Account Foreign-Exchange Business Guide, 2020 edition, issued as 汇发〔2020〕14号 and amended three times since (full text in the State Council gazette, 2020-08-28, read 2026-11-05). The provisions used here run from Article 54 to Article 71, in the guide's personal-conversion section. Everything below comes from that text unless another document is named.

A foreign passport gets the $50,000 allowance one way only

A foreign passport carries a $50,000 annual allowance into yuan and no allowance out of yuan without a permanent-residence card (Articles 54, 55 and 59, read 2026-11-05).

Direction	Annual allowance	What the counter asks for	Article
Foreign currency into yuan	$50,000 per person	passport	54, 55
Foreign currency into yuan, above the allowance	none	ID plus documents on what the yuan is for, with the amount	58
Yuan into foreign currency	none without a permanent-residence ID card	ID plus documents on where the yuan came from, tax receipts named	59
Yuan into foreign currency, permanent-residence ID card held	$50,000 per person	ID	54, 59
Yuan converted back to foreign currency	not an allowance	ID plus the original exchange receipt	60

The first line of Article 54 sets both halves in one sentence: individual conversion into yuan, and conversion into foreign currency by a domestic individual, are each under an annual facilitation allowance of $50,000. A foreign national is not a domestic individual, so only the first half reaches them. Article 59 then opens the purchase route for them, and its second paragraph hands the allowance back to anyone holding a 外国人永久居留身份证.

The figure sets paperwork, not a ceiling

The $50,000 marks the point where documents start, and Articles 56 and 58 both provide a route above it for anyone with the transaction on paper.

For a domestic individual, above-allowance conversion into yuan needs identity documents plus source-of-funds documents stating the amount; above-allowance purchase needs identity documents plus purpose documents stating the amount (Article 56). For a foreign national, above-allowance conversion into yuan needs identity documents plus documents on the purpose of the yuan, again with the amount (Article 58). The document follows the money's destination, not its origin.

One sentence in both articles catches people out. Where a single conversion into yuan exceeds $50,000, the yuan must be paid directly into the counterparty's domestic renminbi account (Articles 56 and 58). A salary conversion that large does not land in your own account first.

Capital-account money sits outside all of this. The 2006 framework says current-account business is managed by the principle of convertibility and capital-account business by the process of convertibility, which is a polite way of saying the second list is shorter (个人外汇管理办法, Order No. 3 of 2006, Article 3, in force 2007-02-01). Selling a flat and sending the proceeds abroad is a capital-account question, not the one answered here.

A passport is the whole definition

A passport holder is a 境外个人 in these rules, whatever the visa inside it says, because the definition turns on the document rather than the permission (Order No. 3 of 2006, Article 37).

The same article defines a domestic individual as a Chinese citizen holding a resident ID card, a military ID or an armed-police ID. Hong Kong, Macao and Taiwan residents with their travel permits fall on the foreign-national side. A work permit, a residence permit for work and a permanent-residence card are three different things in this taxonomy: the first two leave you where you were, and only the third moves you into the allowance.

What the visa list means for opening an account in the first place is a separate question, covered in Opening a bank account in China as a visitor.

Salary out: the tax receipt is the document named

A bank selling foreign currency against yuan earned in China asks for your ID and for documents on where the yuan came from. Article 59 names tax receipts as part of that evidence.

The article's words are 税务凭证, tax vouchers. In practice that means the after-tax picture of the income has to exist before the conversion does, which matters if part of your pay arrived informally or the tax was handled by someone else. The rule also says 合法收入, lawful income, and current-account income, both of which are checked rather than assumed.

Article 68 then hands the branch the pen: a bank may decide for itself which documents to examine, in what form, and to what depth, may refuse a transaction whose authenticity it doubts, and must explain a refusal and the appeal channel (Article 68). Two branches of the same bank can therefore ask for different paper. That discretion is the reason this page describes articles rather than promising a checklist.

The declaration form is written for residents

Article 57 binds the domestic individual to the personal purchase declaration form. Article 59 gives a foreign national a documents-based purchase instead.

That distinction is worth knowing when a counter hands over the form anyway. The form is also where the six prohibitions are printed, and the fifth one answers a common question (form and filing instructions, 汇发〔2021〕13号, in use from 2021-04-01, each bank setting its own start date before the end of June 2021):

Line	What it forbids
1	false declarations about the purchase
2	false supporting documents
3	lending your allowance to someone else
4	borrowing another person's allowance, or splitting
5	using purchased foreign currency for overseas property, securities, life insurance, or investment-type insurance that pays returns or dividends
6	money laundering, gambling, tax evasion, underground banking

Line 5 is about the nature of the transaction, not the size of it. A purchase of $10,000 for a deposit on a flat abroad fails on the same line as a purchase of $200,000. It restricts what purchased foreign currency may be spent on; a salary conversion under Article 59 carries its own documentary test.

Splitting puts you on a two-year list

Converting in several slices, or through several people's names, to stay under the allowance is 分拆, and the guide treats lending your allowance and borrowing someone else's differently (Article 62).

Act	Consequence	Provision
lending your allowance to help someone avoid the allowance or the authenticity check	a written risk warning through the bank; a second occurrence puts you on the watch list	62(1)
borrowing another person's allowance, or avoiding the rules another way	watch list directly	62(2)
while on the list	listed year plus the next two; every conversion needs identity documents and amount-bearing supporting documents	62(3)

A 2009 notice that is still quoted online, with six named patterns of splitting, was repealed by the 2020 guide's own annex of abolished rules. The general prohibition survives in Article 7 of the 2006 framework: no splitting to evade the limit, and no false documents to evade the authenticity check.

Keep the exchange receipt

Yuan left over from an earlier conversion turns back into foreign currency with the original exchange receipt, which stays usable for 24 months from the conversion date (Article 60).

Two daily exceptions need identity documents alone: up to $500 equivalent in a day at a bank, and up to $1,000 equivalent in a day at a shop inside the departure area before leaving the country.

Cash and cards have their own numbers

Cash limits are counted per trip and per day, and a card issued in China carries an annual overseas withdrawal cap that follows the person rather than the card.

Action	Limit	Source
bringing foreign cash in	above $5,000 equivalent, declare in writing to customs	guide, Article 89
taking foreign cash out, with nothing on your inbound declaration to cover it	$5,000 to $10,000: permit from a bank; above that, only for travel to countries in financial disorder, with a permit from the local SAFE office	guide, Article 89
taking renminbi cash in or out	20,000 yuan per person per trip	PBOC announcement No. 18 of 2004, cited below
withdrawing foreign cash at a bank counter	above $10,000 equivalent in a day needs prior SAFE filing, and only for travel to countries in financial disorder	guide, Article 86
overseas cash withdrawal on a card issued in China	100,000 yuan equivalent per person per calendar year, 10,000 yuan per card per day	汇发〔2017〕29号, in force 2018-01-01
overseas card spending	ordinary current-account purchases, not counted against the $50,000	same notice, in the accompanying questions and answers

The 20,000 yuan figure predates the baggage rules, which say only that cash above the prescribed amount must be declared. The central bank raised the renminbi limit from 6,000 to 20,000 per person per trip in an announcement effective 2005-01-01 (PBOC announcement No. 18 of 2004, PDF, read 2026-11-05; baggage rules, Order No. 276, in force 2025-04-01).

How I checked

Three passes, all on 2026-11-05.

First, the version. The regulator's list of valid foreign-exchange rules, updated to 30 June 2026, carries the 2020 guide and the 2006 framework, and lists the guide under three amending notices (SAFE list, PDF, read 2026-11-05). It carries 168 documents and no 2026 edition of the guide, although agency pages in search results claim one; those pages attribute the same document number to three different titles, which is enough to leave them out.

Second, the text. I read the gazette version of Articles 54 to 71 and checked the same articles on the regulator's own site, including Articles 62, 68 and 86 to 89.

Third, the pipeline. A draft notice published on 2026-11-02 would add current-account business to what a relative may handle on your behalf, allow a non-relative with a written reason, and exempt foreign athletes, referees and delegation members at accredited sports events from submitting documents for each transaction (draft and covering note, comments due 2026-12-01). It is a draft. Nothing in it changes the allowances above today.

What I could not check

What a specific branch asks for a salary conversion. Article 68 lets each bank set its own document list. I did not survey branches, so the contract, the tax record and the employer letter are described as the shape of the requirement, not as a fixed set.
Whether the 2026-11-02 draft becomes rules, and in what form. The comment window closes on 2026-12-01.
Capital-account routes. Sale proceeds of property, equity and investments follow a different chapter with different approvals, and I did not work through it here.
Whether an unused allowance carries into the next year. I found no article in the guide, the 2006 framework or its implementation rules that addresses carry-over, and I am not repeating the version that circulates on bank pages without a citation.
An official English translation of the guide. The regulator runs an English site, but I read this guide in Chinese, and every article number here refers to that text.

The friendlier version of the same material for Chinese citizens moving money abroad, including the nine categories of source documents, is in the Chinese piece 个人购汇与结汇：5 万美元额度管什么. Where the yuan is priced, and why a bank's counter rate differs from the published parity, is in How the yuan is quoted.

Read next, from the same notebook:

Opening a bank account in China as a visitor — The visa list a branch works from, and what changes after the account exists.
How the yuan is quoted — The parity, the bank's counter rates, and why the spread is the bank's decision.

Originally published at Field Notes.

1,510 impressions, 6 clicks: what a 28-day Search Console teardown actually showed

hc_xshh — Wed, 28 Oct 2026 03:52:49 +0000

1,510 impressions, 6 clicks: what a 28-day Search Console teardown actually showed

A small English site, roughly 70 posts, written for foreigners dealing with practical things in China. It is about three weeks old. Two weeks ago I pulled the query-level data for the first time and got 157 impressions and 4 clicks over 28 days.

Today the same window reads 1,510 impressions and 6 clicks. Impressions up almost tenfold in twelve days.

The easy read is "growth". What I actually found after sorting all 71 pages with any impression is a single, concrete leak: 1,131 of those impressions land on pages already ranking in positions 1 to 10, and that whole group converted into 5 clicks. A 0.44% click-through rate on pages that are already on page one.

The site is no longer in the "not indexed" phase. It gets impressions, it has rankings. What it does not have is anyone clicking.

Three numbers on one screen, and they disagree

Before any conclusion, the accounting. Google Search Console reports this period three ways:

Header: 6 clicks, 1,510 impressions, 0.4% CTR, average position 11.1.
Pages tab: 71 pages with impressions, 8 clicks, 1,559 impressions.
Queries tab: 47 queries, 104 impressions total, zero clicks.

The queries tab accounts for 6.9% of the impressions shown in the header.

This is not a bug. At low volume, Search Console applies privacy thresholding to queries: it only lists the ones it is willing to list, and it rounds the numbers. The consequence matters more than the mechanism: if you treat the query table as a complete picture, you will draw the opposite conclusion from the one the data supports.

Everything below uses the pages tab, because that layer is complete. The three totals are printed here so the discrepancy is visible rather than silently resolved.

Impressions are already on page one

Sorting the 71 pages by average position into four buckets:

Position	Pages	Impressions	Clicks
1-10	51	1,131	5
11-30	15	296	1
31-60	4	131	2
61-100	1	1	0

The four buckets add up to 71 pages, 1,559 impressions and 8 clicks, matching the pages tab.

The first row is the interesting one. More than half the pages already sit in the top 10. They take about 72% of all impressions. They convert at 5 clicks per 1,131 impressions, which is under half a percent.

A new site's usual problem is not being indexed. This one is past that. The bottleneck is the two lines of text Google shows: the title and the description are not giving anyone a reason to click.

So the next move is not writing fifty more posts.

The six biggest pages, and two counter-examples

Top six by impressions (clicks and average position in brackets):

LPR and mortgage rates: 367 impressions, 0 clicks, position 6.1
FX quota and paperwork: 128, 0, position 7.1
Paying in China as a foreigner: 94, 0, position 15.6
Air quality index explained: 84, 0, position 5.8
Sleep data by group: 71, 2, position 31.8
Setting up Alipay and WeChat Pay: 70, 0, position 8.0

Five of these share a shape: they answer a factual question, but the title is written as a topic label while the reader typed a question. The LPR page is the extreme case. 367 impressions, zero clicks, position 6.1. It was seen 367 times and never once looked worth opening.

Two counter-examples are worth copying.

One page, explaining why speed is the default in China, got 4 impressions, 2 clicks, a 50% CTR at position 5.0. Almost no impressions, but when it was seen, people clicked. Its title is a standalone sentence; you do not have to guess what the page argues.

The other is the sleep data page: position 31.8, still 2 clicks, 2.8% CTR. It is picking up non-English long-tail queries, which says the audience is not confined to English-language markets. I have not decided whether to follow that thread yet.

Query-level optimization at this volume is blind work

My first instinct was to go query by query. The numbers killed that plan.

The 47 listed queries total 104 impressions and zero clicks. All 6 header clicks land on words the query table does not show. The terms that actually convert are the ones Search Console is not telling me. Optimizing against the visible list means optimizing traffic that was never going to click.

Two things in the query table are still usable.

The first is a clear "do not bother". Five name queries total 22 impressions at positions 1.0 to 5.7, the best positions on the site, with zero clicks. They come from note pages that quote specific authors; the searcher wants the person, not the page. No amount of title work will change that.

The second is "real demand, far away". Three variants of the word for a Chinese tax invoice total 23 impressions at positions 55 to 91. Four phrasings of the eSIM question sit at 75 to 82. The demand exists, but it is dozens of positions from a click, and working on it now changes nothing.

The usable substitute is the pages tab, which is complete. The rule I settled on: decide per page, grouped by position band, never per query.

What I changed, and the test I will judge it by

Five pages: FX quota, LPR, paying in China, Alipay setup, the leaving-and-returning checklist. All five had heavy impressions and zero clicks.

Two edits only. Move the answer to the front of the title, and put the real question phrasing into the description. Body copy untouched. The reasoning is simple: at position six, the title line is the only thing anyone reads.

Baseline is the table above, window 10 to 25 October, pulled on 28 October.

The pass criteria are fixed in advance. In two weeks I look at exactly two things:

Whether CTR on those five pages moves off 0%.
Whether average position falls out of the top 10.

Not total impressions. Impressions moving is an indexing story; clicks moving is a title story. Mixing them means never knowing which lever did anything.

One risk, recorded up front: three of the five are utility pages. A punchier title that overpromises sends people into a page that does not answer their question, which is worse than no click. The descriptions have to say what the page actually contains, not just bait.

Open questions

Why the query table covers only 6.9% of impressions: no official answer found. I can only confirm the gap at low volume, and I do not know when it closes.

Why the invoice cluster gets 23 impressions but sits at 55 to 91: my guess is the page explains the process but never defines the concept, and English searchers are looking for the concept. That is a guess, untested.

Why the sleep page converts at 2.8% from position 31.8: the non-English long-tail lead. Whether to build a line around it is undecided.

Reproduction

Data: Google Search Console, Performance report, Pages tab, 28-day window (the page itself displayed 10 to 25 October), pulled 28 October 2026, with the dashboard reporting "last updated 3 hours ago".

The bucket table was produced by exporting the 71 page rows and summing by average position. Bucket totals match the tab total.

Drafted with an AI assistant from my own measurements; every number and link here was checked against the source.

My content-gap detector found 38 gaps. Two-thirds were bugs in the detector.

hc_xshh — Mon, 26 Oct 2026 01:33:53 +0000

I write a small English blog about China — 70 posts, a domain registered on 11 October 2026. Like anyone with a new site, I wanted to know what to write next, and the cheapest signal available is autocomplete: send a seed phrase to Google's and Bing's suggestion endpoints, collect what people are actually typing, then compare that against what you have already published.

So I wired it into a status dashboard I check each morning. This morning it ran in 5.2 seconds and collected 466 unique suggestions from 56 requests (51 returned; the other five were dropped by a proxy or a rate limit).

Then it told me I had 38 content gaps.

That number was wrong, and the way it was wrong turned out to be more useful than the number.

The scoring, in one paragraph

Each query is lowercased, tokenised, stemmed, and stripped of question words and place-or-language words — "how", "what", "china", "english", "foreigner" and so on. What is left is a bag of content words. Each post gets the same treatment, but only from its slug and title. The score is matched content words divided by total content words, plus 0.15 if an adjacent pair also matches. Above 0.60 counts as covered, 0.30–0.59 as partial, below that as a gap.

def content(q):
    ws = [stem(w) for w in re.findall(r"[a-z0-9]+", q.lower())]
    return [w for w in ws if w not in GEO and w not in ASK and len(w) > 1]

Three thresholds, one table, a number I can put in a commit message. What could go wrong.

The 38 gaps, by cluster

Cluster	Queries	Examples
Meituan app in English	12	"meituan app english version", "how to change language in meituan app"
Didi	5	"how to call didi in china", "didi alternatives in china"
China bank foreign exchange	4	"china bank foreign exchange policy"
1688	3	"translate 1688 as a year in history"
Company registration	3	"business registration in china"
Remote work	2	"working remotely from another country"
China Mobile	2	"does china mobile support esim"
Hukou, in Dutch or German	2	"wat is het hukou systeem", "was ist das hukou system"
Hukou, in hip hop	2	"hukou has hip hop music"
Fapiao check	1	"china tax bureau fapiao check"
Customs law	1	"customs law of the people's republic of china"
One apartment	1	"apartment in china that holds 30000 people"

Failure one: my stopword list ate the query

Take "how to change language in meituan app". After filtering, the content words are change, language. "Meituan" sat in my geography set, because I had added it while tuning noise from "meituan vs eleme" comparisons. "English" sat there too, for the same reason. "App" was in the ask set, because it appears in half of all autocomplete suggestions and carries no topic.

So "meituan app english download" reduced to download. "login to meituan english" reduced to login. "meituan waimai app english" reduced to waimai.

Those queries then scored 0.00 against every post I have, including one titled Chinese apps in English: what Meituan and Dianping ship, which answers them. The word that would have matched was deleted before the matching started.

It gets better. The seed I typed for that topic was "meituan english app". It produced zero content words, so the loop skipped it entirely — 465 of the 466 suggestions were scored, and the one that was dropped was my own seed phrase.

All twelve Meituan queries were that bug. Same word list, same week, same collector.

Failure two: matching titles instead of pages

The Didi cluster is the opposite mistake. didi survived the filter, but posts are indexed from slug and title only, and my ride-hailing post is called "Ride-hailing in China: licences, fares and invoices". The body mentions Didi repeatedly. The title does not. One content word, zero matches, five queries into the gap bucket.

Both bugs point the same way: the score measures how well my titles match a query, not whether a page answers it.

What survived a human pass

I read all 38 by hand. Twelve were the stopword bug. Five were title-only matching. Two were the same question asked in Dutch and German. Six were entity collisions — Google reading 1688 as a year, "hukou" as hip hop, and one viral-sounding apartment query with no matching page. That is 25 of 38, about two-thirds, that were noise from four causes, none of them editorial.

Three clusters were left that are worth acting on:

China bank foreign exchange — four queries, and no page whose title names the policy.
Registering a company — "business registration in china" and "how long does it take to register a company in china". I have a page titled as a process, not as a duration.
China Mobile — "china mobile international customer service number" and "does china mobile support esim", both answered in sections on a page whose title does not contain either phrase.

That is the useful output: three clusters, not 38 gaps, and the reason each one was invisible.

What I changed in the tool, not the site

Index the body, not only slug and title. The posts are markdown on disk; a bag of body words per post costs nothing and would have caught the Didi cluster.
Never strip a token that appears in fewer than N posts. Brand and language words are exactly the ones that carry intent. If I need to suppress noise, suppress it after matching, not before.
Report clusters, not counts. "38 gaps" is a number I can quote. "Twelve of them are one app's language settings" is the information.
Keep the human pass. The tool's job is to turn 466 suggestions into a list I can read in ten minutes. It is a triage queue, not an editorial calendar.

The collector still runs every morning at 07:00, because as a triage queue it works. It just should not be allowed to name a number by itself.

Method notes, so the arithmetic can be checked

28 seed phrases × 2 engines = 56 requests. 51 returned a body. The 466 suggestions are unique after lowercasing.
Scoring is hits / content_words + 0.15 if any adjacent pair matches; it is a lexical overlap score, not a semantic one. A page that answers a question in different words scores zero.
Read 2026-10-26, from the same dashboard run that produced the site's traffic numbers. No API key is involved: both suggestion endpoints are public.
The blog is at notebookfield.com, and the Meituan page the scorer missed is here.

Five numbers on my dashboard that were true and still wrong

hc_xshh — Fri, 23 Oct 2026 05:21:20 +0000

Every morning at 07:10 a script collects numbers for a dashboard I keep for my own blog: is the site up, what did I publish, how many people came, what Google showed, which posts did anything. None of the numbers is wrong. Several of them were, in practice, useless — because they were true in a way that made me read the wrong story.

These are the five that lied to me this month, with the arithmetic and the fix for each.

1. "Views are down 83% this week"

What it said. One morning: 290 page views, 290 visitors. The next morning: 50 views, 50 visitors. A 83% collapse, the kind of chart that makes you check whether the site is down.

Why it was wrong. The 7-day window had slid by one day. Cloudflare Web Analytics aggregates by UTC day, and the first window (2026-10-14 to 2026-10-20) contained a single crawler sweep on 10-14: about 250 page views across 20+ paths, where every path showed exactly one view and one visitor. The next window (10-15 to 10-21) no longer contained it. Same site, same traffic, different window.

The tell: 250 views and 250 visitors. Real readers occasionally read two pages. A ratio of exactly 1.00 across hundreds of views is a process walking your sitemap.

The fix: print the window next to every number, not in a tooltip. A metric without its window is not a metric, it is a mood. This one is now a labelled field in the data file, so the chart cannot be read without it.

2. "One page per visit means nobody reads past the landing page"

What it said. Pages per visit: 1.0. A classic engagement death signal.

Why it was wrong. The denominator included the crawler. When 250 of 290 "visitors" are one automated client fetching one page each, the average is arithmetic on a population that mostly does not read.

The fix: split automation out before computing anything per-user. Cloudflare's RUM GraphQL exposes a userAgentBrowser dimension, so the collector now queries two buckets and reports "non-bot views" as its own metric, with the bot share shown next to it. This week: 70 views in 7 days, 60 of them non-bot, 14% automated.

The uncomfortable part: after the split, pages per visit is still 1.0. Now the number means what I thought it meant the first time — people arrive on one page and leave. A metric can be broken and then confirm bad news; those are different events and I only learned which one I had by fixing the measurement first.

3. "Upvote ratio 0.61 — so 39% of voters disliked it"

What it said. A post read once gave 17 points and a ratio of 0.61. Read again two minutes later: 18 points, ratio 0.62. On a smaller post a week earlier, the same check gave 5 points at ratio 1.0, then 4 points at 0.84–0.86.

Why it was wrong. At single-digit and low-double-digit scores, platforms fuzz the ratio so outsiders cannot reverse-engineer individual votes. The fuzzing means "ratio 1.0" is not "no downvotes"; it is "not enough votes for us to tell you". Re-reading gives different numbers because the fuzz is recomputed.

The fix: at low counts, report counts and stop. Score and comment count are integers and reproducible; a ratio at n=5 is noise with a decimal point. If a trend matters, require two reads pointing the same way before writing it down.

4. "Zero traffic on two days"

What it said. The daily series had no rows for 2026-10-17 and 2026-10-19. Two flat-zero days between normal days.

Why it was wrong. Nothing was collected those days because the analytics beacon was not live on the site yet — it was added later. A missing row and a zero row rendered identically, and zero rendered as "traffic died".

The fix: missing data gets its own state, not a zero. The collector now writes the expected dates, marks the ones with no data as missing, and the chart draws a gap instead of a line at zero. The alert text lists the missing dates by name, because "no data" is information and "0" is a claim.

5. "4 clicks. No — 6 clicks."

What it said. The same Search Console panel, same 28-day window, two numbers: the headline said 4 clicks and 612 impressions; adding up the per-page table gave 6 clicks and 563 impressions.

Why it was wrong. Neither is wrong. At low volume Search Console applies thresholds and rounding in different places, so the aggregate and the detail do not reconcile. This is not a bug I can fix, and it is not a reason to distrust everything — it is a reason to stop treating one of them as "the" number.

The fix: print both, labelled, and never do arithmetic across them. My dashboard now stores the headline figures and the summed figures as separate fields. If a decision depends on the difference between 4 and 6 clicks, the decision is not ready to be made.

The rule I ended up with

Every number on that dashboard now has to answer four questions before I am allowed to act on it:

What window is it over, written next to the number?
Who is in the denominator — people, sessions, or automated clients?
Is a missing value being rendered as a zero?
At this sample size, is the difference I am looking at even measurable?

The dashboard still has plenty of numbers that are small. That is fine. Small is a fact; a small number presented without its window, its population and its missing days is a story, and usually the wrong one.

Drafted with an AI assistant from my own measurements; every number and link here was checked against the source.

The dashboard itself is a static page rebuilt every morning from the raw sources. The longer write-ups and the measurements live at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

Collecting trending feeds from 20 platforms — The same failure mode one layer down: a job that reports "success" while the data is twelve days old.
What this site ships — The measurement rules this dashboard is built on, written down before the numbers arrived.

I rewrote five article titles to match real search queries. Then Google showed them 122 times and nobody clicked.

hc_xshh — Fri, 23 Oct 2026 05:21:12 +0000

I run a small English blog — 70 posts, a domain registered on 2026-10-11, so twelve days old when I wrote this. Search Console was showing a strange pattern: pages getting impressions and almost no clicks. The cheapest thing I could change was the headline, so I changed five of them.

This is what happened, including the part where the data could not answer the question.

The short version

On 2026-10-16 I rewrote the titles and descriptions of five posts. Frontmatter only; not a word of the bodies changed.
In the first five days of usable data after the change, those five pages collected 122 impressions and zero clicks.
That is not proof the titles failed. It is proof that at this volume I cannot measure a title at all. So I stopped the experiment instead of starting a second batch, which is the more useful outcome.

What I changed, and why those five

The five pages had impressions but no clicks, and the queries they appeared for had a different shape from the headline. People searched questions; my titles were noun phrases.

Page	Before	After
`using-esim-in-china`	Using an eSIM in China: what the official pages say	Can you use an eSIM in China? It depends on the phone
`china-transit-visa-free-rules`	China's transit visa-free rules: 24 hours and 240 hours	Transiting China without a visa: 24 hours and 240 hours
`chinas-fapiao`	China's fapiao: what it carries, and how to check one	China's fapiao: the invoice system, explained
`sleep-duration-what-guidance-says`	Where the sleep-duration numbers come from	How much sleep do adults need? The published ranges
`air-quality-index-explained`	The AQI is a definition: China vs US breakpoints	The AQI is a definition: China's HJ 633 vs US breakpoints

The reasoning was simple: put the words people actually type into the title, keep the promise the page can keep. I verified afterwards that all five new titles were live in the <title> tag, not just in the repository.

What the numbers actually say

Search Console, read on 2026-10-23. The window is 2026-10-10 to 2026-10-20 — console data lags two to three days, so the "week" is really five days after the change.

Whole site, 28 days:

clicks        4
impressions 612
CTR         0.7%
avg position 11.8

Whole site, last 7 days: 0 clicks, 455 impressions, average position 13.4. Every click in the 28-day window happened before 2026-10-14.

The five rewritten pages, same 28-day window:

Page	Impressions	Avg position
air-quality-index-explained	54	6.2
chinas-fapiao	23	38.2
china-transit-visa-free-rules	20	27.8
using-esim-in-china	16	20.8
sleep-duration-what-guidance-says	9	7.2

At the query level, can you use esim in china was still at position 82 with one impression, and can i transit in china without visa at 75 with one impression — the same readings I had before the change.

Why "122 impressions, 0 clicks" is weaker evidence than it looks

Three reasons, in order of how much they matter.

1. Most of those pages sit where nobody clicks. Three of the five average position 20 to 38. At that depth an impression is a rounding error, not an opportunity. The honest expectation for 122 impressions spread across positions 6 to 38 is less than one click.

2. The volume cannot separate a good title from a bad one. Published sample-size arithmetic: to tell a 3% click-through rate from a 6% one with any confidence, each arm needs roughly 750 impressions. The entire site had 612 impressions in 28 days. At a true CTR of 3%, seeing zero clicks on 46 impressions happens about a quarter of the time anyway.

3. The console hides most of the queries. The query table returned 36 rows covering 67 of 612 impressions — about 11%. The rest sits below Google's reporting threshold. So for most pages I do not know which query produced the impression, which means I cannot tell whether the new title matches it or misses it.

There is also a timing problem I can name but not quantify: a twelve-day-old domain is still being crawled and re-evaluated, so any effect of a title change is mixed with the effect of Google simply noticing the site for the first time. I cannot separate the two with this data.

What I changed in the process, not the site

The experiment did produce a decision, just not the one I was hoping for:

No second batch. I had three more titles ready. Changing them now would mean two experiments running at once, and then no result would be attributable to either.
A title change needs a query phrase I can show, not a query theme I can guess. If the console will not tell me the sentence, I do not write the sentence into the title. Guessing produces a plausible-looking string and no information.
The bottleneck is impressions, not click-through rate. 612 impressions in 28 days on 70 posts means almost nobody sees the pages. Editing headlines optimises a conversion step that is not the constraint.

So I stopped editing titles and started spending the same hours on distribution: getting the pages in front of people somewhere other than a search results page they never reach. This post is part of that.

Method notes, so the numbers can be checked

All figures are from Google Search Console for the https://notebookfield.com/ property, read 2026-10-23, window 2026-10-10 to 2026-10-20.
Search Console's own totals disagree with the sum of its per-page table (4 clicks / 612 impressions at the top, 6 clicks / 563 impressions when summed). Both are printed here rather than picking the friendlier one. This is a known behaviour at low volume.
Titles and descriptions live in frontmatter and are not part of the article body, which is why this counted as a metadata change and not a content change. Five social-preview images were regenerated because they render the title.
What I did not measure: click-through rate by position for these specific pages. Google does not publish it per page, and with 122 impressions the confidence interval would be useless anyway.

Drafted with an AI assistant from my own measurements; every number and link here was checked against the source.

I keep the longer write-ups and the raw measurements at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

Can you use an eSIM in China? It depends on the phone — One of the five rewritten pages, and what the carriers actually publish.
China's fapiao: the invoice system, explained — Another of the five: what the tax rules put on the document.

One of China's four big e-commerce platforms publishes its fee card. The rest don't.

hc_xshh — Sun, 18 Oct 2026 02:28:39 +0000

I spent an afternoon opening the merchant fee pages of Douyin, Taobao, Tmall, JD and Pinduoduo as an anonymous visitor. One of them publishes a full rate card. One publishes a notice with the rates inside a PNG. One answers a four-field form. The other two stop at a login.

That distribution is the story: in Chinese e-commerce, the fee is public only when the platform decides to make it public, and the form of publication tells you how comparable the numbers are.

Full tables and the per-platform notes are on my site: What Chinese e-commerce platforms charge a merchant. This post is the short version.

Douyin is the only full card

Douyin publishes a category technical service fee schedule dated 14 July 2026: 269 category rows, base rates from 0.30% (phone top-ups) to 10.00% (a catch-all services row), median total 6.90%.

Those aggregates are mine, not the platform's. I pulled each row out of the rendered DOM, forward-filled the merged category cells, dropped the one placeholder row with no category and a 0.00% rate, then computed the range, median and distribution in Python. The row-level percentages are Douyin's; the arithmetic is mine.

Two columns matter. The base rate comes from the category you list in. A second column adds up to 5.20 percentage points when the order arrives through the Douyin Mall app or Doubao. On 98 of the 269 rows the supplement equals the base exactly, so the same apparel category is 5.00% in a livestream and 10.00% in mall search. On 151 rows the supplement is 1.3x the base, which is where the 4.60%, 6.90% and 9.20% totals come from.

Deposits follow last month's turnover, not the category: ¥500 up to ¥50,000 GMV, ¥2,000 to ¥100,000, ¥5,000 above that. Categories the platform considers risky — top-ups, game services, fuel cards, cinema — carry ¥200,000; second-hand phones ¥20,000.

Tmall: a notice that points at an image

Tmall's only public statement on category fees this year is a cut notice dated 27 March 2026. It says the rates change from 1 April and points to a list of new categories and rates. That list is one PNG, 1552 by 928 pixels.

I did not transcribe it. Percentages read off a screenshot are not a source I would publish, and the rows being cut are online-game cards, digital services, gift cards and video-game software — not apparel or appliances.

Taobao's 0.6% base fee, capped at ¥60 per sub-order, circulates in Chinese trade press from the 2024 rule. I could not open the help-centre article carrying it, so on my site it stays labelled second-hand.

JD answers with a form

JD's recruitment site asks for entity type, shop type, import status and category, then returns three numbers. One query — 企业, 旗舰店, 非进口, 家用电器 / 大家电 / 平板电视 — returned a ¥5,000.00 deposit, an operating support service fee of 3.70% and a transaction service fee of 0.60%.

Two failed attempts taught me the interaction: clicking a category name drills into its children but leaves the selection count at zero. The checkbox beside the name is what ticks it.

Pinduoduo: the rulebook is behind the login

The rules centre answers with a login form. The public onboarding manual (version 6.3, fifteen pages) lists documents, not money: business licence, legal representative's ID, shop manager's ID, brand authorisation. No percentage, deposit or monthly fee.

One clause is useful for a foreign brand: authorisation can come from a trademark owner's wholly owned subsidiary, or from its 境内唯一总代理 — sole agent inside China — which the manual marks as applying only to overseas trademark owners. In practice a foreign brand reaches a Pinduoduo storefront through a mainland company.

The number that is not comparable

A headline rate is only comparable if both sides compute it on the same base. Douyin's settlement base is what the buyer paid including shipping, plus host coupons, platform coupons and subsidies, minus the merchant-funded parts. JD reports a category rate and a separate 0.60% transaction fee. Tmall states one category rate in a picture and nothing else.

So "Douyin charges 5%" and "JD charges 3.7%" are not two points on one scale. They are two different objects, published in two different forms, on two different bases.

What I could not check

Tmall's rates as numbers: the notice points to an image.
Taobao's 0.6% base fee and ¥60 cap: trade press only, no live rule page I could open.
Whether Tmall still charges an annual fee: the old tables paired it with the category rate; status unverified in 2026.
Every Pinduoduo rate and deposit: merchant login required.
Alipay's and WeChat Pay's standard merchant rates: they circulate widely, but I could not pin either to a dated official page, so neither appears.

The full page has the 269-row breakdown, the cross-border card (145 rows, 1.60%–6.00% base), the deposit rule and the source links, each with the date the platform printed on it: notebookfield.com/posts/platform-commission-rates.

Read next, from the same notebook:

China's minimum wage by province, 2026 — The provincial wage floor the same merchants hire against.
How the money actually moves in live commerce — Where the commission sits in the settlement chain.

Measuring error in support AI, and what accuracy hides

hc_xshh — Fri, 09 Oct 2026 07:01:48 +0000

The way I test a support or ticketing system is against a task book, not a demo. The number people
ask for afterwards is accuracy, and I have stopped handing that over on its own: on every realistic
test set I have built, the average was the least informative thing in the results file.

What follows is the method I use now, plus a run on public data so the numbers can be checked.

Five failure modes, five different bills

A support model fails in at least five ways, and they do not cost the same.

Fabrication. The reply asserts something absent from the policy or the record: a fee that does
not exist, a wrong refund window, an invented delivery date. This is what people call
hallucination. The taxonomy I lean on, OlaBench, splits it into four sub-types — factual
hallucination, misuse of retrieved results, relevance hallucination, and logical inconsistency
(OlaBench, arXiv:2510.22143v3, §3.1; v1 25 Oct 2025, v3 25
May 2026, retrieved 9 Oct 2026). A sentence that reads well and is false costs money, and the cost
usually arrives months later.

Missed detection. The ticket contains a legal threat, a regulator's name, a safety issue or a
fraud signal, and the system files it as routine. Nothing false was said. The failure is that no
human ever reads it. On a scoring sheet this often counts as a correct classification, because the
model did pick a queue that exists.

Misrouting. The reply is reasonable and the queue is wrong, so a differently-skilled team
applies its own policy. The measurable cost is time to resolution and repeat contact.

Out-of-scope answers. The system answers a question it was never built for — tax, legal
advice, another provider's product. The content may be accurate. The exposure is regulatory.

Refusal. The system declines, loops, or hands off. There is no answer, so there is no error to
score. This one gets tracked as a transfer or containment rate and drops out of the error count
entirely.

Failure	What is wrong	Where the cost lands	Commonly reported as
Fabrication	the content	refunds, goodwill, disputes	hallucination rate
Missed detection	nothing	escalation, legal, safety	almost nothing
Misrouting	the destination	time to resolution, repeats	accuracy
Out of scope	the remit	compliance	rarely measured
Refusal	nothing	churn, containment	transfer rate

One accuracy figure blends these five at whatever ratio the sample happens to contain, and that
ratio is set by whoever picked the sample.

What I ran

I needed a routing task with a ground-truth label and text a model would really see. Public
complaint data has both.

The US CFPB publishes its Consumer Complaint Database under CC0 (CFPB Consumer Complaint
Database, retrieved 9 Oct
2026); the search API's own metadata reports 18,274,023 records, license CC0, last updated
2026-10-08 (API metadata,
retrieved 9 Oct 2026).

The JSON search endpoint returns 15 fields and no narrative (field
reference, retrieved 9 Oct 2026). That is new: the
September 2026 release removed complaint narratives from the database (CFPB release
notes, Release 24, September 2026). So I took
the text from a Hugging Face mirror instead, BEE-spoke-data/consumer-finance-complaints (dataset
card, CC0, mirror last
modified 29 Dec 2025); its has-text config holds 1,689,573 rows (split
sizes,
retrieved 9 Oct 2026).

Task: read the narrative only, predict the product queue. Ground truth: the product label the CFPB
publishes. I drew 3,894 complaints from it. The oldest is dated 2015-03-20, the newest
2024-01-04.

The label set changed under the data

The raw product field holds 17 distinct strings in this sample, and they are not 17 products. The
date ranges are what give it away.

Product string as published	Rows	Received, first → last in this sample
Credit reporting, credit repair services, or other personal consumer reports	1,892	2017-04-24 → 2023-08-24
Debt collection	547	2015-03-20 → 2023-12-18
Credit card or prepaid card	267	2017-04-24 → 2023-08-21
Mortgage	248	2015-04-21 → 2023-11-02
Credit reporting or other personal consumer reports	245	2023-08-26 → 2024-01-04
Checking or savings account	195	2017-04-24 → 2023-11-21
Credit reporting	92	2015-04-17 → 2017-04-09
Student loan	85	2015-10-06 → 2023-11-13
Money transfer, virtual currency, or money service	79	2017-08-08 → 2023-11-13
Vehicle loan or lease	75	2017-04-24 → 2023-11-01
Credit card	68	2015-04-06 → 2023-12-22
Payday loan, title loan, or personal loan	41	2017-04-24 → 2023-06-20
Bank account or service	34	2015-04-04 → 2017-01-31
Consumer Loan	19	2015-03-25 → 2017-02-03
Payday loan	3	2015-10-08 → 2017-02-24
Money transfers	2	2015-07-17 → 2015-12-05
Payday loan, title loan, personal loan, or advance loan	2	2023-09-20 → 2023-10-06

Read the dates down the table. Bank account or service last appears here on 2017-01-31, and
Checking or savings account first appears on 2017-04-24. Three more strings end in 2017 and their
successors start the same year. The three credit-reporting strings sit end to end: 2015 to 2017,
2017 to 2023, 2023 into 2024. Same category, renamed.

I collapsed the 17 strings into 9 categories before scoring anything. The mapping is in the script,
applied in one place rather than inferred per row. Without that step an accuracy compares a 2016
row and a 2023 row whose label sets do not line up, and the model gets credit or blame for the
taxonomy.

The live database disagrees with the mirror about which names are current. Aggregated over
complaints received in 2026, the CFPB API returns 11 product strings, with Credit card and
Prepaid card listed separately and no Credit card or prepaid card at all (product aggregation,
complaints received 2026-01-01
onward,
retrieved 9 Oct 2026). I did not reconcile the two taxonomies.

Three ways one accuracy figure misleads

Imbalance. After collapsing, the sample looks like this:

Collapsed category	Rows	Share
Credit reporting	2,229	57.2%
Debt collection	547	14.0%
Credit card or prepaid card	335	8.6%
Mortgage	248	6.4%
Checking or savings account	229	5.9%
Student loan	85	2.2%
Money transfer or virtual currency	81	2.1%
Vehicle loan or lease	75	1.9%
Personal / payday / title loan	65	1.7%

A predictor that answers "credit reporting" for every ticket scores 0.572 here. That is a
constant, and it sits close enough to a real model's score to be mistaken for one.

Sample selection. Same data, same model, two splits:

Split	Test n	Accuracy	Macro F1	Credit-reporting share of test set
Random 60/40	1,558	0.756	0.428	57.1%
Chronological, split at 2022-04-21	1,560	0.832	0.377	74.0%

The chronological run scores 7.6 points higher while getting worse on most categories. The test
set became more concentrated in the biggest class, and the dominant-class predictor rides that
concentration up. Anyone reporting the 0.832 as an improvement after retraining would be reading
their own sampling as progress.

Coverage. Sort the test set by the model's confidence and score only the top slice:

Coverage	Tickets scored	Accuracy on that slice
100%	1,558	0.756
75%	1,168	0.854
50%	779	0.904
25%	389	0.900
10%	155	0.897

The 0.904 at half coverage is a real number computed on held-out data, and published alone it
misdescribes the system by 15 points. Note where it stops moving: from 50% coverage down to 10%,
accuracy drifts from 0.904 to 0.897, so the model's confidence and its correctness come apart on
the tail. That tail is where the misroutes are.

Underneath the average, the per-category picture on the random split:

Category	Test n	Recall	Precision
Credit reporting	889	0.879	0.886
Debt collection	225	0.631	0.637
Credit card or prepaid card	129	0.760	0.430
Mortgage	114	0.860	0.737
Checking or savings account	93	0.548	0.607
Money transfer or virtual currency	32	0.000	0.000
Student loan	29	0.276	0.889
Vehicle loan or lease	29	0.000	0.000
Personal / payday / title loan	18	0.000	0.000

Three categories are never once predicted correctly. Accuracy is 0.756. Macro F1 is 0.428. The
distance between those two is the honest headline for this model.

Pulling the high-risk errors out of the average

The useful published idea here is OlaBench's risk axis, Critical Business Risk Rate. It counts only
responses that inappropriately assert one of four things, each a high-stakes failure that may
trigger compliance exposure, user disputes or reputational damage: admitting platform liability
(618 test cases), misidentifying the ICS role (228), overcommitting (138), and disparaging
individuals or merchants (16) (OlaBench,
arXiv:2510.22143v3, §3.1 and Table 1; 25 May 2026, retrieved 9
Oct 2026). It is reported as its own number, beside the hallucination rate.

The shape of that is what to copy: choose failures whose consequences differ in kind, count them
separately, publish both, and never average them.

For the routing test I defined three tiers, and I chose them:

critical — a complaint about funds or account access (checking or savings, money transfer) lands in the reporting queue
material — a credit or loan dispute lands in a different loan or reporting family
routine — a misroute inside the same family

Tier	Random split	Chronological split
Critical	7 / 1,558 = 0.4%	11 / 1,560 = 0.7%
Material	43 / 1,558 = 2.8%	28 / 1,560 = 1.8%
Routine	330 / 1,558 = 21.2%	223 / 1,560 = 14.3%

The two splits swap places depending on which tier you read: the one with the prettier accuracy
hides a critical error rate almost twice as high.

Published work also shows how far the same named metric can move when the population changes. The
OlaBench paper reports a critical business risk rate of 8.7% for its OlaMind-Stage-2 model on the
benchmark, and a critical business risk rate below 0.05% in daily manual annotation of live
dialogues after launch (OlaBench, arXiv:2510.22143v3, §5.2 and
Table 3; 25 May 2026, retrieved 9 Oct 2026). Both are the same metric, measured on different
populations with different annotators. Quoting one without the other is a choice.

How these numbers were produced

Data. 3,894 complaints from a CC0 mirror of the CFPB Consumer Complaint Database (config has-text, 1,689,573 rows). The draw is 40 pages of 100 consecutive rows at random offsets in the split, deduplicated by complaint ID, seed 20261009. Rows with a narrative under 80 characters were dropped. It is a clustered draw, not an independent one.
Label hygiene. 17 published product strings collapsed to 9 categories using the date ranges in the table above. The mapping is written out in the script, not inferred per row.
Model. Multinomial naive Bayes over word unigrams and bigrams, Laplace smoothing α = 1, features seen at least twice in training. Python standard library only.
Splits. Random 60/40 with seed 7; chronological split at the 60th percentile of the received-date order, which lands on 2022-04-21.
Selective accuracy. Test rows sorted by the model's maximum posterior probability, top k scored.
Every number in the tables above is printed by that one script over that sample.
Deliberate choice. No language model appears in this pipeline, including the scoring. A measurement instrument that is itself a model carries its own error, and OlaBench publishes the size of its own: its judge agrees with human annotators at 91.7% on risk identification and 82.6% on hallucination detection, over 5,000 human-annotated instances (OlaBench, arXiv:2510.22143v3, §3.3 and Table 2; 25 May 2026, retrieved 9 Oct 2026). A judge that disagrees with people 8% of the time on the risk axis puts a floor under everything else you can measure with it.

What I could not check

Whether the mirror's product strings match the live database's current taxonomy. The mirror's rows use names the live API no longer returns for complaints received in 2026. I collapsed by date range and did not reconcile the two.
Whether the mirror is complete. Its files are timestamped December 2025 and the newest complaint in my 3,894-row sample is dated 2024-01-04. I did not verify whether later records exist in it.
Whether complaints published with a narrative are representative of all complaints. Narratives appeared only with consent, so every number above sits on that consenting subset.
Whether this sample's class mix reflects the population. Credit reporting is 57.2% of the sample and about 82% of the live database: 14,978,722 of 18,274,023 records carry one of the three credit-reporting strings (product aggregation, retrieved 9 Oct 2026). Most of the gap is the mirror's date cap.
How noisy the product label is. It is chosen by the consumer at submission, not assigned by an expert, and I treated it as ground truth because companies route on it. If it is noisy, my error rates are a floor, not a ceiling.
Whether a published tiering exists for complaint-routing consequences. I could not find one, so the three tiers are mine. The OlaBench categories are a different kind of thing: they score what a reply asserts about liability and commitment, and they do not classify misroutes.
Whether error rates vary with company size, state, or complaint age. A stratified rate is the next thing to compute.
The naive Bayes model is a floor. A trained transformer would score higher on the same task. The run was to see the shape of the error distribution, and a better model changes the level without changing the shape.

I keep the longer write-ups and the raw measurements at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

How an eval set is built, and why it rots — How the test set behind an accuracy number is built, and how it rots.
From token prices to task prices: three jobs, seven APIs — What the same support task costs per ticket across seven APIs.

The Chinese open-weights release timeline, dated

hc_xshh — Fri, 09 Oct 2026 07:01:16 +0000

This is a list of dates, not an argument. Every row is a model whose weights were
published for anyone to download, with a date I could trace back to the organization that
published it, a parameter count as stated by that organization, and a link.

Most timelines of this kind are written from memory or from coverage, which is why they drift.
I built this one from vendor pages, papers and Hugging Face's API. This revision leaves the table
at 67 rows: seven are new, one of them because an entry I dropped in the first pass turns out to
be open-weights after all. The dropped list at the end has eleven entries, each with a reason.

How to read this timeline

Open weights only. A release is in the table only if the organization published
downloadable weights. API-only and hosted-only models are out, including flagship ones:
Qwen3-Max, the served ERNIE models, Kimi's hosted variants, DeepSeek's API-only models.

A date I could trace. The Date column has two kinds of value. A plain date is the date on
the organization's own dated page — a research blog post, a news post, a release page, an
accepted paper. A date marked † is the Hugging Face repository creation timestamp in UTC,
read from the Hugging Face API on 2026-10-09, used when the vendor had no dated page I could
read. The two can differ by a few days, and that is usually preparation rather than a
different release: DeepSeek-V3's weights repository was created 2024-12-25 12:52 UTC and the
announcement is dated 2024-12-26.

Parameter counts come from the vendor, or they are blank. Total parameters with activated
parameters for mixture-of-experts models, as stated on the announcement, the model card, or
the repository holding the same weights. A family that ships many sizes gets a range. Where
none of the vendor's pages for that release states a count, the cell is — rather than a
number carried over from a sibling model, even when the lineage makes the number obvious.

One link per row, to the vendor or the weight repository. No aggregator links, no
news coverage. That makes most rows single-source by construction: each cites the vendor's own
statement about its own release, and every † row is single-source by definition.

Scope. Language models and the flagship multimodal lines that carry the same brand and
weights: Qwen3-VL, Qwen3-Omni, MiniMax-M3. Pure image, video and speech models are out of scope.

When the vendor's page is client-rendered

Two of the labs whose releases I date most often publish announcements as a page that ships no
text to a plain HTTP request. z.ai's blog returns a 598-byte HTML shell that loads a JavaScript
bundle; qwen.ai's blog returns a shell that fetches its content after load. Both render fine in
a browser, which is what a reader clicking the link sees.

For z.ai I did read something: the date string sits inside the page's own bundle, so the bundle
is where I checked GLM-4.5 (2025-07-28),
GLM-4.6 (2025-09-30), GLM-4.7
(2025-12-22) and GLM-5 (2026-02-12). That is weaker evidence than a
rendered page, and it is why GLM-5.2 stays out: its bundle contains
2026-06-16 and no other date, and a date visible only inside a build artifact is not one I will
print as sourced.

Qwen's older blog carries the full text and a date; the newer qwen.ai copies do not. So the
three Qwen3-* rows dated from qwen.ai in the first pass now cite the weight repository and
carry a †.

2023

Q1–Q2

Model	Date	Parameters	Source
ChatGLM-6B	2023-03-13 `†`	6.2B	THUDM/chatglm-6b
Baichuan-7B	2023-06-13 `†`	7B	baichuan-inc/Baichuan-7B
ChatGLM2-6B	2023-06-24 `†`	`—`	THUDM/chatglm2-6b

Q3–Q4

Model	Date	Parameters	Source
Qwen-7B	2023-08-03 `†`	7B	Qwen/Qwen-7B-Chat
Baichuan 2	2023-09-19	7B, 13B	arXiv:2309.10305
ChatGLM3-6B	2023-10-25 `†`	`—`	THUDM/chatglm3-6b
Yi-34B	2023-11-01 `†`	34B	01-ai/Yi-34B
DeepSeek Coder	2023-11-01 `†`	1.3B–33B	deepseek-ai/deepseek-coder-33b-instruct
DeepSeek LLM	2023-11-29 `†`	7B, 67B	deepseek-ai/DeepSeek-LLM-67B-Chat

Baichuan 2 is the only 2023 row dated by a paper: arXiv:2309.10305 is dated 2023-09-19,
while its weight repositories appear earlier, on 2023-08-30 †, so the paper is the single
source for that date. The two ChatGLM cells are blank because their cards do not state a count:
6.2 billion is on the ChatGLM-6B card and not on the cards of its two successors, and copying
it across is the habit that makes timelines of this kind drift.

2024

Q1

Model	Date	Parameters	Source
Qwen1.5	2024-02-04	0.5B–110B	Qwen blog

Qwen1.5 was in the dropped list in the first pass, because I looked for it on the new blog and
found nothing readable. The old blog still carries the post, dated 2024-02-04, listing six
sizes plus 110B.

Q2–Q3

Model	Date	Parameters	Source
DeepSeek-V2	2024-05-07	236B total, 21B active	arXiv:2405.04434
GLM-4-9B	2024-06-04 `†`	9B	zai-org/GLM-4-9B-Chat
Qwen2	2024-06-07	0.5B–72B	Qwen blog
Qwen2-VL	2024-08-29	2B, 7B, 72B	Qwen blog
DeepSeek-V2.5	2024-09-05 `†`	`—`	deepseek-ai/DeepSeek-V2.5
Qwen2.5	2024-09-19	0.5B–72B	Qwen blog

Q4

Model	Date	Parameters	Source
Hunyuan-Large	2024-10-22 `†`	389B total, 52B active	tencent/Hunyuan-Large
Qwen2.5-Coder	2024-11-06 `†`	0.5B–32B	Qwen/Qwen2.5-Coder-32B-Instruct
DeepSeek-V2.5-1210	2024-12-10	`—`	DeepSeek changelog
DeepSeek-V3	2024-12-26	671B total, 37B active	DeepSeek news

The DeepSeek point releases inside a model line are dated on DeepSeek's own changelog, one page
per release. Hunyuan-Large's 389B/52B comes from
arXiv:2411.02265, submitted 2024-11-04, three weeks after
the weights appeared.

2025

Q1

Model	Date	Parameters	Source
MiniMax-Text-01	2025-01-14	456B total, 45.9B active	arXiv:2501.08313
DeepSeek-R1	2025-01-20	671B total, 37B active	DeepSeek news
Qwen2.5-VL	2025-01-27 `†`	3B, 7B, 72B	Qwen/Qwen2.5-VL-72B-Instruct
QwQ-32B	2025-03-06	32B	Qwen blog
DeepSeek-V3-0324	2025-03-25	671B total, 37B active	DeepSeek changelog

The two DeepSeek updates here do not restate a parameter count on their own changelog pages;
the cells carry the count from the repositories that hold those weights, which are the V3 and
R1 repositories listed above.

Q2

Model	Date	Parameters	Source
Qwen3	2025-04-29	0.6B–235B	Qwen blog
DeepSeek-R1-0528	2025-05-28	671B total, 37B active	DeepSeek changelog
MiniCPM4-8B	2025-06-05 `†`	8B	openbmb/MiniCPM4-8B
MiniMax-M1	2025-06-16	456B total, 45.9B active	arXiv:2506.13585
Hunyuan-A13B	2025-06-27	80B total, 13B active	Tencent-Hunyuan/Hunyuan-A13B
ERNIE 4.5	2025-06-30	0.3B–300B; flagship 300B/47B	ERNIE blog

Q3

Model	Date	Parameters	Source
Kimi K2	2025-07-11 `†`	1T total, 32B active	moonshotai/Kimi-K2-Instruct
Qwen3-Coder	2025-07-22	480B total, 35B active	Qwen blog
GLM-4.5	2025-07-28	355B total, 32B active	z.ai blog
Step-3	2025-07-31	321B total, 38B active	StepFun research
Seed-OSS-36B	2025-08-21	36B	ByteDance Seed blog
DeepSeek-V3.1	2025-08-21 `†`	`—`	deepseek-ai/DeepSeek-V3.1
LongCat-Flash-Chat	2025-08-29 `†`	560B total, 18.6B–31.3B active	meituan-longcat/LongCat-Flash-Chat
Qwen3-Next-80B-A3B	2025-09-09 `†`	80B total, 3B active	Qwen/Qwen3-Next-80B-A3B-Instruct
Ling-flash-2.0	2025-09-17 `†`	100B total, 6.1B active	inclusionAI/Ling-flash-2.0
Qwen3-Omni-30B-A3B	2025-09-20 `†`	30B total, 3B active	Qwen/Qwen3-Omni-30B-A3B-Instruct
Qwen3-VL	2025-09-22 `†`	235B total, 22B active	Qwen/Qwen3-VL-235B-A22B-Instruct
DeepSeek-V3.2-Exp	2025-09-29	`—`	DeepSeek news
GLM-4.6	2025-09-30	`—`	z.ai blog

Q4

Model	Date	Parameters	Source
Ling-1T	2025-10-02 `†`	1T total, 50B active	inclusionAI/Ling-1T
DeepSeek-OCR	2025-10-17 `†`	`—`	deepseek-ai/DeepSeek-OCR
LongCat-Flash-Omni	2025-10-23 `†`	560B total, 27B active	meituan-longcat/LongCat-Flash-Omni
MiniMax-M2	2025-10-27	230B total, 10B active	MiniMax blog
Kimi K2 Thinking	2025-11-04 `†`	`—`	moonshotai/Kimi-K2-Thinking
DeepSeek-V3.2	2025-12-01	`—`	DeepSeek news
GLM-4.7	2025-12-22	`—`	z.ai blog

2026

Q1

Model	Date	Parameters	Source
GLM-4.7-Flash	2026-01-19 `†`	30B total, 3B active	zai-org/GLM-4.7-Flash
Step-3.5-Flash	2026-02-01 `†`	196B total, 11B active	stepfun-ai/Step-3.5-Flash
MiniMax-M2.5	2026-02-12	`—`	MiniMax news
GLM-5	2026-02-12	744B total, 40B active	z.ai blog
Qwen3.5-397B-A17B	2026-02-16	397B total, 17B active	Alibaba press release

Q2

Model	Date	Parameters	Source
MiniMax-M2.7	2026-04-09 `†`	`—`	MiniMaxAI/MiniMax-M2.7
Kimi K2.6	2026-04-14 `†`	1T total, 32B active	moonshotai/Kimi-K2.6
DeepSeek-V4 (Pro and Flash)	2026-04-24	Pro 1.6T total / 49B active; Flash 284B total / 13B active	DeepSeek news
Step 3.7 Flash	2026-05-23 `†`	198B total, 11B active	stepfun-ai/Step-3.7-Flash
MiniMax-M3	2026-06-01	~428B total, ~23B active	MiniMax blog

The DeepSeek-V4 row now carries both counts; the Flash model's 284B total is stated on the same
announcement page as the Pro figures. Step 3.7 Flash moved to a † date: its GitHub repository
has no releases, and the page I first cited carries a creation timestamp rather than a release
date.

Q3

Model	Date	Parameters	Source
LongCat-2.0	2026-07-05 `†`	1.6T total, ~48B active	meituan-longcat/LongCat-2.0
Hunyuan Hy3	2026-07-06	295B total, 21B active	Tencent newsroom
Qwen3.8	2026-08-05 `†`	27B; 2.4T total / 95B active	Qwen/Qwen3.8-2.4T-A95B
DeepSeek-V4-Pro GA	2026-08-13	1.6T total, 49B active	DeepSeek changelog
DeepSeek-V4-Flash-Vision-Exp	2026-08-21	`—`	DeepSeek changelog
DeepSeek-V4.1-Flash	2026-09-10	552B total; 8B active for input, 16B for output	DeepSeek changelog

Hy3 is the entry whose status changed in this revision. In the first pass I dropped it because
the page I found did not say whether weights were published. It does: Hy3 is open-sourced under
Apache 2.0 with weights on Hugging Face, and Tencent's newsroom names the parameter counts. The
V4-Pro GA row repeats the count from the April preview announcement, because the GA page does
not restate it. The Qwen3.8 row is dated by the repository of the 27B model, the first of the
line to appear; the flagship 2.4T-A95B repository was created three days later, and Qwen's own
announcement page is one of the client-rendered ones described above.

Two rows elsewhere disagree with third-party timelines for the same reason: LongCat-2.0's
repository was created 2026-07-05 while secondary coverage puts its launch at the end of June,
and DeepSeek-V4-Flash's repository was created 2026-04-22, two days before the announcement
page.

What changed in this revision

Seven rows were added and three cells changed, listed because the edit history is the point of
a table like this:

Qwen1.5 promoted into 2024 Q1 (2024-02-04), after finding the dated post on the old Qwen blog.
Hunyuan Hy3 promoted into 2026 Q3 (2026-07-06). It was dropped in error.
Four DeepSeek point releases added: V2.5-1210 (2024-12-10), V3-0324 (2025-03-25), R1-0528 (2025-05-28) and V4-Pro GA (2026-08-13). All four are dated on DeepSeek's own changelog, the same source as V3.2 and V3.2-Exp.
Qwen3.8 added (2026-08-05 †), which the first pass omitted entirely.
Two ChatGLM parameter cells blanked. The ChatGLM2-6B and ChatGLM3-6B cards do not state a count, so the cells are now —.
DeepSeek-V4-Flash gained its total. The row said "Flash 13B active" and dropped the 284B total, which the same page states.

Three rows changed source or date type for the same structural reason: the Qwen3-Next,
Qwen3-Omni and Step 3.7 Flash rows now cite a weight repository with a †, because the pages
cited in the first pass were either unreadable to a plain request or carried no release date.

How this table gets updated

The table is edited in place. Every quarter I re-read the vendor pages for the labs already
listed, then check Hugging Face for new repositories under those organizations. A release
enters the table when it has a date and a link; until then it sits in the dropped list, with
the reason.

Two rules keep it stable: I do not replace a † date with a later one unless the vendor
publishes its own dated page, and I do not remove a superseded row.

The current table has 67 entries. Twenty-three of them fall in the twelve months from October
2025 to October 2026. That is a count of this table and nothing more; it is not a claim about
the field, because the table only holds releases I could date.

What I could not check

Eleven candidates were dropped, for these reasons:

Yi-1.5 (May 2024) — the weight repository (01-ai/Yi-1.5-34B) was created 2024-05-11 †, and I could not date the announcement itself.
InternLM2 and InternLM2.5 — the weight repositories (internlm/internlm2-7b, internlm/internlm2_5-7b-chat) look like renamed predecessors, so the creation timestamp does not mark the release.
MiniCPM 1 and 2 — same problem; the repository (openbmb/MiniCPM-2B-sft-bf16) is now a monorepo for a whole series.
Baichuan 3 and 4 — I could not find weight artifacts at all.
ERNIE 5.0 and ERNIE 5.1 — Baidu's blog dates them 2026-02-06 and 2026-05-09 (ERNIE 5.0, ERNIE 5.1) and describes ERNIE 5.0 as a 2.4-trillion-parameter model, but I found no open-weight repository for either.
Kimi K3 — the model card (moonshotai/Kimi-K3) carries the full architecture table (2.8T total, 104B activated) and a license, but no release date. The repository timestamp is 2026-06-13 and secondary coverage says mid-July. I could not reconcile that.
Kimi K2.5 — same shape of problem: the weight repository timestamp (moonshotai/Kimi-K2.5) is 2026-01-01 while the launch is reported at the end of January.
GLM-5.2 — the post is client-rendered and the only date string in its bundle is 2026-06-16. I am not willing to print that as the release date.
DeepSeek-V3.1-Terminus — a mid-cycle rename with a 2025-09-22 † repository (deepseek-ai/DeepSeek-V3.1-Terminus); including it would have double-counted the V3.1 line.
Qwen-Image-2.1 — a dated weight repository (Qwen/Qwen-Image-2.1, 2026-09-14 †) but an image model, outside this table's scope by the rule stated at the top.
The Step-Audio line — the same, for speech. The repositories exist (stepfun-ai/Step-Audio-R1.1) and I did not date them, because they would need their own table with its own scope.

Three other limits on this table:

The † timestamps are preparation times, not launch times. A repository can be created days before the announcement, and from outside I cannot see whether weights were uploaded at once or in pieces. Where the gap is large I noted it above the table.
The dates are the vendor's own. No release date here has been confirmed by a second source, and one link per release is deliberate: a vendor correcting itself is the signal worth keeping, and coverage repeating the vendor is not a second source.
License differences are not reflected. Apache-2.0, MIT and the vendor-specific licenses Kimi, MiniMax and LongCat ship under are not the same permission. Each row links to the page that states its license, and for Hy3 that page states Apache 2.0.

I keep the longer write-ups and the raw measurements at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

From token prices to task prices: three jobs, seven APIs — What these models cost per task once they are routed through an API.
How an eval set is built, and why it rots — How to build an eval set for them, and why it ages.

Why there is no Western mini program

hc_xshh — Fri, 09 Oct 2026 07:00:37 +0000

The usual version of this question is about super apps, and it has been answered at length by people with better data than mine; mine is narrower. A mini program is a small bundle of code that runs inside somebody else's application, and the question is which parts of that arrangement exist in what Apple and Google ship.

I read the vendors' own documentation for WeChat Mini Programs, Apple App Clips, Google Play Instant and Progressive Web Apps, and compared them on five dimensions: install and size, identity, payment, entry point, distribution and review.

One mechanism carries most of the answer. A mini program brings neither its own user account nor its own checkout. It borrows both from the application hosting it.

The five dimensions, side by side

	WeChat Mini Program	Progressive web app	Native app (iOS/Android)	App Clip (iOS)
Install and size	Runs inside WeChat. Main package 2 MB maximum, all packages together 30 MB (20 MB for programs developed by a third-party provider) (WeChat subpackages, read 2026-10-09)	No package at all. Chrome offers install after HTTPS plus a manifest carrying a name, 192 px and 512 px icons, `start_url` and a display mode (web.dev install criteria, read 2026-10-09)	Full binary from the store. Apple states the App Store limits the size of apps installable over a mobile connection and gives no number on that page (Apple, read 2026-10-09); Google states a compressed download limit of 200 MB for apps published as app bundles (Google, read 2026-10-09)	10 MB uncompressed on iOS 15 and earlier, 15 MB on iOS 16 and earlier, 100 MB on iOS 17 or later under conditions (Apple, read 2026-10-09)
Identity	`openid` per mini program, `unionid` shared across every app under one Open Platform account. Both come from `wx.login` plus `code2Session`, which WeChat documents as returning the UnionID "without user authorization" (login, UnionID, read 2026-10-09)	Each origin keeps its own account. The browser hands nothing to a site silently	Account per app. Sign in with Apple where the app offers it, as an explicit step by the user (Apple, read 2026-10-09)	Sign in with Apple, the keychain and CloudKit, shared with the parent app (Apple, read 2026-10-09)
Payment	WeChat Pay. A merchant number is bound to the mini program's AppID, at most 50 AppIDs per merchant number (WeChat Pay, page updated 2025-07-02). Virtual goods must go through the mini program virtual-payment channel, where "the platform charges a technical service fee" (WeChat, read 2026-10-09)	Card and wallet APIs called by the merchant, on the merchant's own acquiring contract	In-app purchase is required to unlock features. Apple's guidelines name QR codes among the mechanisms that may not be used instead (Apple, read 2026-10-09)	Apple's in-app purchase. Hosts in the Mini Apps Partner Program "earn 85% of qualifying In-App Purchase sales within qualifying mini apps" (Apple, read 2026-10-09)
Entry	Documented entry paths carry IDs: 1011 scan a QR code, 1012 long-press an image, 1013 pick a code from the album, 1023 the Android home-screen icon, 1005 and 1006 search boxes, 1026 the nearby list, 1183 search inside the PC WeChat mini program panel (WeChat scene values, read 2026-10-09)	A URL, a search result, or a home-screen shortcut the user adds by hand. On iOS there is no install prompt at all (web.dev, read 2026-10-09)	Store search and the home-screen icon	An App Clip Code, NFC tag or QR code at a physical location; Siri suggestions; Maps; Messages; a Smart App Banner on a website in Safari (Apple, read 2026-10-09)
Distribution and review	Published from the mini program console. Virtual payment requires a certified entity: an enterprise, institution or individual merchant (WeChat, read 2026-10-09)	Nothing is submitted. It is HTTPS plus a manifest	Store review against the App Review Guidelines	No listing of its own. It ships inside a full app, which the App Clip documentation says "must include the same functionality as the App Clip" (Apple, read 2026-10-09); Guideline 2.5.16(a) adds that all App Clip features must be in the main app binary and that "App Clips cannot contain advertising" (Apple, read 2026-10-09)

The Android equivalent is gone. "Starting December 2025, Instant Apps cannot be published through Google Play, and all Google Play services Instant APIs will no longer work" (Google Play Instant, last updated 2026-06-24).

Identity arrives before the first screen

The WeChat login flow is one call on the client and one on the server. wx.login returns a code, the server exchanges it through code2Session, and the reply carries an openid for that user in that mini program, a unionid and a session key (WeChat, read 2026-10-09).

The UnionID page says what this costs the user: the developer gets it "without user authorization" (WeChat, read 2026-10-09). No phone number, no password, no consent sheet. The openid is scoped to one mini program; the unionid is identical across every mobile app, website app, official account and mini program bound to the same Open Platform account, so a merchant with several surfaces sees one person.

Compare the alternatives. On the web, an origin cannot read an identifier out of the browser; it asks the user to sign in and then owns a credential to protect. On iOS, an App Clip can use Sign in with Apple and the keychain, which Apple's documentation points developers at sharing with the full app, but those are technologies an app opts into rather than an identifier handed over at launch.

That is why the first screen of a mini program is the product. Onboarding was paid for at install time, by the host.

Payment is a binding between two accounts

Paying inside a mini program does not mean the mini program holds money. WeChat Pay binds a merchant number to the mini program's AppID. One merchant number can bind at most 50 AppIDs, the binding starts on the merchant side and is confirmed from the mini program side, and once established cannot be unbound (WeChat Pay, page updated 2025-07-02). The rail and the code package are joined by an account relationship neither side can create alone.

Virtual goods get a second layer. The documentation is blunt: virtual currency, unlocked features, subscription content, paid services, tips and virtual gifts "require integration with Mini Program's virtual payment system for both purchase and payment", and the platform charges a technical service fee on the amount (WeChat, read 2026-10-09). Physical goods go through ordinary WeChat Pay; digital ones go through the host's channel, with the host's cut.

Apple's version of that cut is recent and documented. The Mini Apps Partner Program was announced on 13 November 2025 (Apple, 2025-11-13): host apps carrying mini apps can join, earn 85 percent of qualifying in-app purchase sales inside those mini apps, and must use Apple's in-app purchase system for them (Apple, read 2026-10-09). Admission is not automatic: a host needs the Advanced Commerce API and a manifest describing itself and its mini apps.

Apple's App Review Guidelines still require in-app purchase to unlock features or functionality, and the same sentence lists QR codes among the mechanisms that cannot substitute for it (Apple, read 2026-10-09). What changed, on 1 May 2025, is that Guidelines 3.1.1, 3.1.1(a), 3.1.3 and 3.1.3(a) were updated for the United States storefront after a court decision about buttons and external links (Apple, 2025-05-01).

The QR code is the offline switch

A WeChat developer does not have to choose between a printed code and a link shared in a chat. The official scene-value table lists both, with IDs: 1011 scanning a QR code, 1012 long-pressing an image, 1013 picking one from the album, 1023 the Android home-screen icon, 1005 and 1006 the two search boxes, 1026 the nearby list, 1183 search inside the PC WeChat mini program panel (WeChat, read 2026-10-09).

Two details make the printed code cheap. Codes come from an API, and WeChat states every generated mini program code is permanently valid. The "one item, one code" interface has no quantity limit and needs a smaller printed area than an ordinary URL code (WeChat, read 2026-10-09). A code already printed and pointing at a URL can be configured to open a chosen page inside a mini program (WeChat, read 2026-10-09). The person scans what is already on the table.

App Clips have the same physical invocations: an App Clip Code, an NFC tag, a QR code at a physical location (Apple, read 2026-10-09). Two constraints pull the other way.

The first constraint trades size against physical entry. The 100 MB ceiling for iOS 17 or later carries four conditions: the App Clip "only supports digital invocations" and not "physical invocations such as App Clip Codes, QR codes, or NFC tags" (Apple, read 2026-10-09), it is used where a reliable connection is likely, and it "doesn't support iOS 16 and earlier". The exception Apple names is a demo link from App Store Connect, which may use the 100 MB limit and still support App Clip Codes, NFC tags and QR codes. An App Clip with a counter code stays inside the smaller limit.

The second concerns persistence. Apple's Human Interface Guidelines say App Clips "remain on the device for a limited amount of time", and that only App Clip Codes produced in App Store Connect or with Apple's own code generator are approved for use (Apple, read 2026-10-09). A merchant cannot print its own.

Who reviews it, and who is answerable

Reviews differ in kind as well as in strictness. A PWA is submitted to nobody; Chrome installs it once its manifest and HTTPS satisfy the criteria (web.dev, read 2026-10-09). An App Clip cannot be submitted alone. It ships inside a full app that must already contain the same functionality, cannot carry advertising, and is reviewed as part of that app (Apple, read 2026-10-09). A mini program is submitted inside WeChat, and the entity requirement arrives with payment rather than with the code: to sell virtual items, the program must be a certified enterprise, institution or individual merchant (WeChat, read 2026-10-09).

The commercial consequence: in the App Store, a mini-app-like product reaches users through a host that already has an account relationship and a reviewed app behind it. In WeChat, the party needing the corporate identity, the certification and the merchant binding is the mini program author.

Where the four pieces sit

In WeChat, four things a mini program depends on sit with one owner: the identity system, the payment rail, the entry surface (camera, search, chat, home screen, desktop) and the distribution channel with its review. An author plugs into a host that has all four and pays for the parts it uses.

In the Apple and Google stacks those four are held by parties with different interests. The operating system vendor issues the platform account and runs the payment system; the browser owns the entry surface for web content; the stores own distribution and review; the rail for physical goods sits with the merchant's bank relationship.

No single party there can hand all four to a third party. Apple's answer in November 2025 was to supply the missing pieces itself and keep the commission: hosts get Apple's in-app purchase for mini apps and keep 85 percent of qualifying sales, in exchange for approval, a manifest and Apple's review.

The mechanism that makes mini programs work is therefore an ownership arrangement rather than a technology gap. Reproducing it outside WeChat means finding one party that holds the users, the identity, the payment rail and the entry surface, and getting it to open all four to third-party code at a price the third party will pay. In the West, the two parties that hold those pieces chose to be the host themselves.

How I checked this

Every figure and quote comes from documentation published by the vendor it describes: WeChat's mini program and WeChat Pay docs, Apple's developer documentation, guidelines and news items, Google's Android documentation, and web.dev.
All pages were fetched on 2026-10-09. Most carry no publication date, so a date beside a source is my reading date unless the page states its own: the WeChat Pay AppID binding page (updated 2025-07-02), the Google Play Instant page (last updated 2026-06-24), Apple's updated-guidelines news item (2025-05-01), Apple's Mini Apps Partner Program news item (2025-11-13) and Tencent's write-up of the 2022 WeChat Open Class (2022-01-07).
Size limits and scene-value IDs come from the pages that state them: WeChat's subpackage page for 2 MB and 30 MB, Apple's App Clip page for the 10/15/100 MB table, WeChat's scene-value table for the ID numbers.
Google Play Instant's page would not load in a plain HTTP client, so I read it through the Internet Archive and then through a rendering fetch that returned the live page.

What I could not check

The fee WeChat charges on virtual payments. The page says a fee is charged on the payment amount and gives no percentage.
The standard WeChat Pay rate for physical goods. I found no official rate page; what I reached describes partner fee schemes and refers to a merchant rate without the base number.
The extra steps for virtual payment on iOS. WeChat's virtual payment page says iOS "requires additional activation and adaptation procedures" and points at separate documentation; I could not load that page, so I do not know whether an Apple-side charge is involved.
Whether an App Clip can take payment for physical goods through Apple Pay. Not stated on the App Clip pages I read.
Any current count of mini programs or mini-program daily users. Tencent's release discloses Weixin monthly accounts and names Mini Games and Mini Shops, but carries no mini-program count. The most recent first-party figure I found: more than 450 million daily users at the end of 2021, from Tencent's write-up of the 2022 WeChat Open Class (Tencent, 2022-01-07). That figure is four years old.
What "certified Mini Program" requires in practice, beyond the entity types named on the virtual payment page, and how long WeChat's review takes.

I keep the longer write-ups and the raw measurements at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

Chinese apps in English: what Meituan and Dianping ship — What Meituan and Dianping actually ship in English.
Why fast is the default in China — The default-speed argument, with the records behind it.

From token prices to task prices: three jobs, seven APIs

hc_xshh — Fri, 09 Oct 2026 07:00:13 +0000

Every vendor publishes a price per million tokens. Almost none publishes what a job costs; the number that decides whether a feature ships is left as arithmetic homework.

I did that arithmetic for three jobs, from first-party price lists read on 2026-10-09. The token counts are assumptions, not measurements, and they are written out below. The money is the vendor's standard-tier list price, in USD.

The three jobs

Job 1, label 1,000 support messages. Each call sends a 400-token instruction block and a 120-token message, and asks for a 15-token answer. Billed per run: 520,000 input tokens, of which 400,000 are the instruction block sent 1,000 times, plus 15,000 output tokens.

Job 2, turn a 100,000-word draft into a structured summary. Google's token guide says a token is about 4 characters and 100 tokens about 60-80 English words, putting 100,000 words at 125,000-167,000 tokens (Google token counting guide, read 2026-10-09). I used 140,000:

One shot: 140,000 tokens in, a 3,000-token structured summary out.
Map-reduce: 14 chunk calls of about 10,000 tokens each, 700 tokens of notes per chunk, then a 9,800-token reduce call. Totals: 149,800 in, 12,800 out.

Job 3, one round of tool-assisted bug fixing. Eight turns against a 20,000-token repository: system prompt, tool schemas, three files. The prompt grows about 2,500 tokens per turn as tool results come back, from 20,000 to 37,500. Cumulative billed input is 230,000 tokens, of which 192,500 are a re-sent prefix a cache can serve; output is 6,000 tokens including reasoning.

How I read the price lists

Five first-party pages, all read on 2026-10-09; the Gemini API page builds its tables client-side, so I read it in a browser.

OpenAI API pricing — standard tier
Anthropic pricing
Gemini API pricing — paid tier, introductory rates through 2026-12-31
DeepSeek Models & Pricing — off-peak column
xAI pricing

The rules I applied:

Standard tier only, unless a row says batch: the real-time tier.
Cache reads are billed at the published cached-input rate. Cache writes are billed where a vendor publishes a write price. OpenAI and Anthropic do: $2.50 per million tokens for the two frontier models here, $0.125 for the two small ones, on Anthropic's 5-minute cache. Google, DeepSeek and xAI publish no write price, so I charged zero, which flatters those three in every cached row.
In the cached rows I charge a write for every input token that is not a cache read: 400 tokens in Job 1, 37,500 in Job 3.
Reasoning tokens are output tokens. OpenAI states they are billed as output tokens (OpenAI reasoning guide, read 2026-10-09). Gemini's output price is quoted "including thinking tokens" and DeepSeek's V4.1-Flash runs in thinking mode by default.
Tool and search fees sit outside the main table.
No prompt hits a long-context threshold; the prompt-length step that does apply, Anthropic's 100,000-token rule, is below.
No aggregator site was used for any price.

What each job costs

Task	Model	Estimated cost	Assumptions	Price source (read 2026-10-09)
Job 1, no cache	gpt-6.1-sol	$1.19	standard tier; 520k in / 15k out	OpenAI
Job 1, no cache	gpt-6-luna	$0.060	standard tier; 520k in / 15k out	OpenAI
Job 1, no cache	Claude Sonnet 5.5	$1.19	standard tier; 520k in / 15k out	Anthropic
Job 1, no cache	Claude Haiku 5.5	$0.060	standard tier; 520k in / 15k out	Anthropic
Job 1, no cache	Gemini 3.8 Flash	$0.45	standard tier, introductory rate; 520k in / 15k out	Google
Job 1, no cache	DeepSeek V4.1-Flash	$0.087	off-peak window; 520k in / 15k out	DeepSeek
Job 1, no cache	Grok 4.7	$1.13	standard tier; 520k in / 15k out	xAI
Job 1, instructions cached	gpt-6.1-sol	$0.43	400k of input served from cache; one 400-token write	OpenAI
Job 1, instructions cached	gpt-6-luna	$0.024	400k of input served from cache; one 400-token write	OpenAI
Job 1, instructions cached	Claude Sonnet 5.5	$0.43	400k cached, 5-minute TTL and write	Anthropic
Job 1, instructions cached	Claude Haiku 5.5	$0.024	400k cached, prompts under 100k, 5-minute TTL	Anthropic
Job 1, instructions cached	Gemini 3.8 Flash	$0.18	400k of input served from cache, storage excluded	Google
Job 1, instructions cached	DeepSeek V4.1-Flash	$0.028	off-peak; cache hits free of a write fee	DeepSeek
Job 1, instructions cached	Grok 4.7	$0.53	400k of input served from cache	xAI
Job 2, one shot	gpt-6.1-sol	$0.31	140k in / 3k out, one call	OpenAI
Job 2, one shot	gpt-6-luna	$0.016	140k in / 3k out, one call	OpenAI
Job 2, one shot	Claude Sonnet 5.5	$0.31	140k in / 3k out, one call	Anthropic
Job 2, one shot	Claude Haiku 5.5	$0.078	140k in / 3k out, above the 100k step	Anthropic
Job 2, one shot	Gemini 3.8 Flash	$0.12	140k in / 3k out, one call	Google
Job 2, one shot	DeepSeek V4.1-Flash	$0.023	off-peak; 140k in / 3k out	DeepSeek
Job 2, one shot	Grok 4.7	$0.30	140k in / 3k out, under the 200k threshold	xAI
Job 2, map-reduce	gpt-6.1-sol	$0.43	14 chunk calls + 1 reduce call; 149.8k in / 12.8k out	OpenAI
Job 2, map-reduce	gpt-6-luna	$0.021	14 chunk calls + 1 reduce call	OpenAI
Job 2, map-reduce	Claude Sonnet 5.5	$0.43	14 chunk calls + 1 reduce call	Anthropic
Job 2, map-reduce	Claude Haiku 5.5	$0.11	15 calls, each prompt under 100k	Anthropic
Job 2, map-reduce	Gemini 3.8 Flash	$0.16	14 chunk calls + 1 reduce call	Google
Job 2, map-reduce	DeepSeek V4.1-Flash	$0.030	off-peak window, 15 calls	DeepSeek
Job 2, map-reduce	Grok 4.7	$0.38	14 chunk calls + 1 reduce call	xAI
Job 3, agent loop	gpt-6.1-sol	$0.25	8 turns; 230k in (192.5k read, 37.5k written) / 6k out	OpenAI
Job 3, agent loop	gpt-6-luna	$0.013	8 turns; 230k in (192.5k read, 37.5k written) / 6k out	OpenAI
Job 3, agent loop	Claude Sonnet 5.5	$0.25	8 turns; 192.5k reads and 37.5k 5-minute writes / 6k out	Anthropic
Job 3, agent loop	Claude Haiku 5.5	$0.013	8 turns, every prompt under 100k	Anthropic
Job 3, agent loop	Gemini 3.8 Flash	$0.065	8 turns; 230k in (192.5k cached) / 6k out, storage excluded	Google
Job 3, agent loop	DeepSeek V4.1-Flash	$0.0098	off-peak window; 8 turns	DeepSeek
Job 3, agent loop	Grok 4.7	$0.21	8 turns; 230k in (192.5k cached) / 6k out	xAI

Two pairs of rows are identical on purpose. gpt-6.1-sol and Claude Sonnet 5.5 both list $2.00 per million input tokens, $10.00 output, $0.10 cached input and a $2.50 cache write (OpenAI pricing and Anthropic pricing, read 2026-10-09); gpt-6-luna and Claude Haiku 5.5 both list $0.10, $0.50, $0.01 and $0.125.

The same jobs with each vendor's biggest discount

Batch is the biggest published discount: 50% on OpenAI (pricing page, read 2026-10-09), on Anthropic ("a 50% discount on both input and output tokens", pricing page, read 2026-10-09) and on Google's paid tier ("Batch API (50% cost reduction)", pricing page, read 2026-10-09), where the Batch column halves the cached-input rate too. Not everywhere: xAI's batch section gives 20% to grok-4.3 and the grok-4.20 variants and says unlisted models have no batch discount, which leaves Grok 4.7 at full price (xAI pricing, read 2026-10-09). DeepSeek publishes no batch tier.

Model	Job 1, cached	Job 2, one shot	Job 3, loop	Batch discount	Price source (read 2026-10-09)
gpt-6.1-sol	$0.22	$0.16	$0.12	50%	OpenAI
gpt-6-luna	$0.012	$0.0078	$0.0067	50%	OpenAI
Claude Sonnet 5.5	$0.22	$0.16	$0.12	50%	Anthropic
Claude Haiku 5.5	$0.012	$0.039	$0.0067	50%	Anthropic
Gemini 3.8 Flash	$0.088	$0.058	$0.033	50%	Google
DeepSeek V4.1-Flash	$0.028	$0.023	$0.0098	none published	DeepSeek
Grok 4.7	$0.53	$0.30	$0.21	none	xAI

Claude Haiku 5.5's one-shot cell is the outlier: batch brings it to $0.039, still three times its cached Job 1 of $0.012, because a 140,000-token prompt sits above the 100,000-token step (Anthropic pricing, read 2026-10-09).

What the numbers say

Every figure here is arithmetic on the rows above, whose rates are linked (OpenAI, Anthropic, Google, DeepSeek, xAI, read 2026-10-09).

Output tokens decide the bill at the frontier tier. In cached Job 1 on gpt-6.1-sol, $0.15 of the $0.43 is output (35%) and $0.04 is cached instruction tokens (9%).

The spread inside one vendor is bigger than the spread between vendors. Cached Job 1 is $0.024 on gpt-6-luna and Haiku 5.5, $0.43 on gpt-6.1-sol and Sonnet 5.5, $0.53 on Grok 4.7: 22x from dearest to cheapest, 18x between the cheap pair and the frontier pair. At the same tier, the vendor changes nothing.

The agent loop is the cheapest job here. Job 3 is $0.25 on gpt-6.1-sol against $0.31 for the one-shot summary and $1.19 for Job 1. On the small models the order flips: Haiku 5.5 pays $0.078 for the one-shot summary against $0.013 for the loop, because 140,000 tokens crosses its threshold and 230,000 over eight turns does not.

Map-reduce costs more than one shot. On the two $2/$10 models it is $0.43 against $0.31, and the gap is all output tokens, 12,800 against 3,000. Chunking buys reliability and pays for it in output.

The absolute numbers are small. The main table spans $0.0098 to $1.19; batch pulls the floor to $0.0067. At this scale, a cost problem is usually a volume problem.

The variables that break the conversion

Thresholds re-price the whole request. Claude Haiku 5.5 charges $0.10 and $0.50 per million input and output tokens up to 100,000 prompt tokens, and $0.50 and $2.50 above it; the length counts cache reads and writes, and each request is priced on its own (Anthropic pricing, read 2026-10-09). Job 2's 140,000-token prompt pays the higher rate on everything: $0.078, against $0.016 under the cap. xAI applies long-context rates to all tokens in a request once the prompt passes 200,000, where Grok 4.7 costs $4.00 input, $1.00 cached input and $12.00 output instead of $2.00, $0.50 and $6.00 (xAI pricing, read 2026-10-09). Gemini 3.1 Pro Preview steps from $2.00 to $4.00 input per million above 200,000 tokens (Gemini Enterprise Agent Platform pricing, read 2026-10-09). OpenAI's line is 272,000 input tokens, published in a column tooltip; above it gpt-6.1-sol's input doubles to $4.00 and its output to $15.00 (OpenAI pricing, read 2026-10-09). All three jobs stay under every line; Job 2 is halfway to OpenAI's.

Tokenizer differences. Anthropic notes that Claude 4.7 and later use a tokenizer that "produces approximately 30% more tokens for the same text" (Anthropic pricing, read 2026-10-09). A comparison built on one words-per-token ratio, as mine is, is off by that much for Anthropic.

Time of day. DeepSeek's peak hours are 01:00-04:00 and 06:00-10:00 UTC, Monday to Friday, excluding Chinese public holidays; every other hour is off-peak at half price (DeepSeek Models & Pricing, read 2026-10-09). Job 1 uncached is $0.087 off-peak and $0.174 at peak. Off-peak is 133 of the week's 168 hours.

Cache economics differ in shape as well as in rate. Anthropic's cache hit is 10% of the input price on most models, 5% on Opus 5.5 and Sonnet 5.5, 2.5% on Fable 5.1 and Mythos 5.1; writes cost 1.25x for five minutes and 2x for an hour, so a 5-minute cache pays for itself after one read (Anthropic pricing, read 2026-10-09). DeepSeek's hit price is $0.003 per million against $0.15 for a miss (DeepSeek pricing, read 2026-10-09). Google charges no write fee but charges storage, $0.50 per million tokens per hour on Gemini 3.8 Flash through 2026-12-31 and $4.50 on Gemini 3.1 Pro (Gemini pricing and Gemini Enterprise Agent Platform pricing, read 2026-10-09); a cached 140,000-token document runs $0.07 an hour on Flash and $0.63 on Pro.

Platform fees sit outside the token meter. OpenAI bills $2.50 per 1,000 tool calls and $10 per 1,000 web searches, so Job 3's eight calls cost $0.02, and Anthropic's tool-use prompt adds 286 to 675 tokens (OpenAI pricing and Anthropic pricing, read 2026-10-09). xAI bills X search per item fetched: $5 per 1,000 posts, $10 per 1,000 profiles (xAI pricing, read 2026-10-09).

Latency and geography carry multipliers. OpenAI's Fast mode, renamed from Priority on 2026-07-30, is 2x, and data-residency endpoints add 10%; Gemini 3.8 Flash's Priority tier is $1.35 input against $0.75 standard (OpenAI pricing and Gemini pricing, read 2026-10-09). xAI's priority tier is 2x and its US regional endpoint 1.1x; Anthropic charges 1.1x for US-only inference; regional Bedrock and Google Cloud endpoints carry a 10% premium (xAI pricing and Anthropic pricing, read 2026-10-09).

The list price itself moves. Google quotes its Flash prices "through December 31, 2026" and doubles them on 2027-01-01: $0.75 to $1.50 input, $3.75 to $7.50 output, $0.075 to $0.15 cached input, $0.50 to $1.00 per million tokens per hour of cache storage (Gemini pricing, read 2026-10-09). Anthropic's $2/$10 for Claude Sonnet 5 was launched as introductory pricing through 2026-08-31 and is now standard, with the rise to $3/$15 cancelled (Anthropic pricing, read 2026-10-09). OpenAI's promotional pricing for GPT-5.6 Sol runs at least through 2026-11-21, on a model not in this table (OpenAI pricing, read 2026-10-09). Any per-task table is a snapshot with an expiry date.

What I could not check

I ran none of these three jobs. The token counts are stated assumptions, and a measured run would move every number here.
Cache-write pricing for Google, DeepSeek and xAI. I found no published write price for any of the three, so I charged zero. xAI's prompt-caching page lists a billing rate per token type with no write line; Google's context-caching price is a read rate plus hourly storage; DeepSeek's table has only a hit price and a miss price. If any of them does charge for writes, its cached rows are too low.
Whether the Haiku 5.5 threshold behaves as written at the boundary. The page counts cache reads and writes in a prompt's length, which would mean a cached 140,000-token document pays the higher rate. I did not test it.
Real token counts. I used Google's words-per-token ratio across every vendor; Anthropic's own note about a 30% tokenizer difference is a warning that this is approximate.
Volume, committed-use and negotiated enterprise pricing, free tiers and trial credits are outside this table. The price on the page is the list price.
Resale through cloud platforms. Anthropic bills Bedrock and Google Cloud in consumption units, and Gemini Enterprise Agent Platform prices are not the Gemini API prices: on 2026-10-09 the two pages disagreed on cache storage, $0.50 per million tokens per hour (Gemini pricing) against $1.00 for one model (Gemini Enterprise Agent Platform pricing). The route you buy through can change the number.

I keep the longer write-ups and the raw measurements at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

Measuring error in support AI, and what accuracy hides — The error side of the same task: what accuracy hides.
The Chinese open-weights release timeline, dated — A dated timeline of the Chinese open-weight releases that set these prices.

How an eval set is built, and why it rots

hc_xshh — Fri, 09 Oct 2026 07:00:02 +0000

An eval set is an instrument. It has a purpose, a scoring rule, a calibration record and an expiry date. Treating it as a file you download is where the trouble starts: the file already made those decisions without writing them down.

What follows is the build document I would want for a set I have to defend six months from now. Each rule is traceable to a public source or marked as my practice.

Where the samples come from

Four sources, each importing a different bias.

Source	What you get	The bias it imports	Documented where
A public benchmark, reused	Comparability with published numbers	Contamination; a task choice someone else made	Sainz et al., 2023-10-27
Public administrative data	A real distribution, a public licence, dated updates	A sample of who reports, not of who is affected	CFPB, read 2026-10-09
Synthetic generation	Volume you can afford, labels you control	Generator artifacts in the surface form	Gururangan et al., ACL 2018
Your own production traffic	Items that match what you ship	A convenience sample that drifts with the product	Lipton et al., ICML 2018

Reuse. LiveBench's authors state the problem in one sentence: test set contamination "can quickly render benchmarks obsolete" (LiveBench, v2 2025-04-18). Their answer was questions from recent competitions, arXiv papers and news, added monthly. LiveCodeBench did the same for code: 400 problems published between May 2023 and February 2024 (LiveCodeBench, arXiv v1 2024-03-12). A static benchmark decays without anyone editing it.

Administrative data. The CFPB complaint database is the best example I know. Its search API reported 18,274,023 records under a CC0 licence on 2026-10-09, with a last_updated of 2026-10-08 (CFPB complaint search API, read 2026-10-09). Complaints are published only after the company responds or after 15 days, whichever comes first, and complaints referred to other regulators, including depository institutions with less than $10 billion in assets, are not published (CFPB API docs, read 2026-10-09). The agency states that the database "is not a statistical sample of consumers' experiences in the marketplace" (CFPB database page, read 2026-10-09). A random draw samples the people who complained and whose complaints survived publication, not the people who were harmed.

Synthetic. Gururangan et al. showed that a text classifier reading only the hypothesis, with the premise deleted, reached about 67% on SNLI and 53% on MultiNLI (Annotation Artifacts in NLI Data, ACL 2018). The label was recoverable from phrasing because the crowd workers followed a template. Every generator has one. My test: delete the part of the input meant to carry the answer, score again, and compare with the majority-class rate. If accuracy stays well above it, the set is partly measuring its own generator. (my practice)

Production sampling. This is the source I trust least and use most. It produces items that match what the system sees, and its composition changes when the product does. Label shift is the named version of that failure: the marginal p(y) moves while p(x|y) does not, measurable without test labels using Black Box Shift Estimation (Lipton et al., ICML 2018).

What counts as correct

The match rule is a parameter, and the same predictions score differently under each choice. Four conventions:

Convention	The rule, as the source writes it	Source
Exact match against any reference	Counts predictions that match any one ground truth answer exactly; punctuation and articles ignored	SQuAD, arXiv v3 2016-10-11
Partial credit against the best reference	F1 over bags of tokens, "the maximum F1 over all of the ground truth answers", then averaged	SQuAD, arXiv v3 2016-10-11
A threshold on localisation	IoU thresholds `[.5:.05:.95]`, 10 of them; 101 recall thresholds; `maxDets` of 1, 10 and 100	cocoapi `cocoeval.py`, read 2026-10-09
A graded rubric with an exclusion flag	0–3 for how well specified the issue is; 0–3 for whether the tests are scoped to it; a separate 0/1 question for any other reason the sample should not be used; a 1–5 confidence rating	SWE-bench annotation instructions, undated

Three things this table taught me.

Say who the ceiling belongs to. In SQuAD the human number comes from the data: the second annotator's answer is treated as the prediction, the rest as ground truth, and humans score 77.0 exact match and 86.8 F1 on the test set (SQuAD, arXiv v3 2016-10-11). A ceiling quoted without its rule says as much about the rule as about the task.

One published number can hide ten thresholds. COCO's headline detection score averages over ten IoU thresholds, three detection budgets and a recall grid. A single figure without a named threshold is a claim nobody can re-check. Pick the threshold, publish it, keep it fixed across versions.

The exclusion question is the part people skip. SWE-bench Verified is 500 samples "verified to be non-problematic by our human annotators", drawn from 1,699 randomly sampled items that 93 Python developers annotated, with the full annotation set and the rubric released alongside (OpenAI, updated 2025-02-24). The rubric's last section asks, 0/1, whether there is any other reason not to use the sample. My handling of it, (my practice):

A closed list of exclusion reason codes, decided before annotation starts.
Two raters must agree before an item is dropped; one rater's "unclear" is not an exemption.
The dropped item stays in the file with its reason code, so the exemption stays auditable.
The exemption rate is reported next to every result; if it moves between releases, the set changed even when the questions did not.

Ground truth built by pooling is biased by construction. Test collections are built by judging only a sample of documents. When the pool is small relative to the collection, the judgment set "can be biased in that they favor relevant documents that contain topic title words", a bias that depends on collection size and not on the number of relevant documents (Buckley et al., NIST, 2007-07-17). In a modern eval, if your labels come from checking only what your current systems returned, the label set favours items phrased the way those systems phrase things. My rule: draw a random slice from outside the candidate pool. (my practice)

Whether the raters agree

Agreement is a property of the pair and the setup.

Krippendorff's alpha is 1 − Do/De, and it accepts any number of coders and missing entries (Krippendorff, 2011-01-25). For two raters, Cohen's kappa, with Landis and Koch's 1977 bands (reproduced in an AHRQ report, 2010).

Here is why the setup has to travel with the number. First-turn results on MT-bench, in the paper's two setups: S1 counts ties and inconsistent votes, S2 excludes ties (MT-Bench, v4 2023-12-24):

Pair (first turn)	S1, R = 33%	S2, R = 50%	Source
GPT-4 pairwise vs human expert	66% (n = 1,343)	85% (n = 859)	MT-Bench, v4 2023-12-24
GPT-4 single-answer vs human expert	60% (n = 1,280)	85% (n = 739)	MT-Bench, v4 2023-12-24
Human vs human	63% (n = 721)	81% (n = 479)	MT-Bench, v4 2023-12-24

The headline everyone quotes is 85% against 81% for human agreement (MT-Bench, v4 2023-12-24). The same table holds 66% for the same judge under the other tie rule. An agreement figure quoted without its setup is not comparable to anything.

Measure agreement per stratum, not pooled. ChaosNLI collected 464,500 annotations, 100 per example, over 3,113 SNLI and MNLI examples and 1,532 ANLI examples (Nie et al., 2020-10-08). Models are near-perfect where humans agree and "can barely beat a random guess" where they do not, and those low-agreement items are most of the errors. A pooled score mixes two measurement regimes whose mixing ratio changes as models improve.

Sizing. For an agreement study, start from the kappa minimum-sample-size tables (Bujang and Baharum, EBPH 14(2), 2017). For comparing two systems, use a power analysis: Miller gives the MDE formula and shows that raising answers per question from 1 to 10 moved the MDE from 13.2% to 7.5%; in his illustrative table, HumanEval's 164 questions at a fictional 83.6% carry a standard error of 3.2% (Miller, 2024-11-01). If items arrive in groups, such as the same document or template, compute clustered standard errors (Miller, 2024-11-01, Table 4); on DROP they were 1.34 against 0.44 naive.

My rule: size the calibration slice for the agreement interval I need, the scoring slice for the effect I want to detect, and refuse to report a gap below the MDE. (my practice)

Why the set rots

Decay	Symptom	What I do
Contamination	One set improves while unrelated sets do not	Keep item dates and prompt hashes; treat a stale set as historical
Label shift	Scores move while the system did not	Estimate the label mix and re-weight or re-cut the slice
Taxonomy and rubric drift	The same question means something different	Version the labels; freeze the rubric text per set version
Saturation	Every system sits near the ceiling	Retire the set and say when

Contamination's extent "is unknown, as it is not straightforward to measure", and where it exists it overestimates performance (Sainz et al., 2023-10-27). I cannot audit training corpora, so I keep the dates and admit the exposure window.

Saturation is not a failure of the set: GLUE's performance passed the level of non-expert humans, which is what prompted SuperGLUE (Wang et al., 2019-05-02). Benchmark choice is fragile on its own: swapping tasks can reorder methods even when the methods do not change (Dehghani et al., 2021-07-14). Both are reasons to keep more than one set.

Taxonomy drift has a public paper trail. The CFPB's September 2026 release notes record that "consumers' complaint narratives ... have been removed from the database"; the June 2026 release removed "Consumer disputed" and "Consumer consent provided" from exports, long after those filters went away; the May 2023 release re-based the ZIP code field on 2019 census estimates (CFPB release notes, read 2026-10-09). A set built on those narratives became unmaintainable one month ago. The same page explains the general rule: the database shows "the consumer's original products, sub-products, issues, and sub-issues selections consistent with the options available on the form at the time the consumer submitted the complaint" (CFPB database page, read 2026-10-09). Labels frozen at submission, against a taxonomy that keeps moving.

Keeping it maintainable

(my practice, with borrowed pieces named)

Version the set. Additions and exclusions get a version bump and a changelog line; never edit an item silently.
Two slices. A frozen slice for comparability, a rolling window for freshness. LiveCodeBench and LiveBench are public versions of the same idea.
Write the metadata down. A datasheet (Gebru et al., 2018, CACM 2021) or a Croissant record (2024-03-28), so the next person need not infer the collection process.
A desensitisation rule that is checkable. The CFPB's ZIP rule is a good template: publish the 5-digit ZIP unless the census area has under 20,000 people, then 3 digits only if the wider area has more than 20,000, otherwise nothing (CFPB field reference, read 2026-10-09). A rule stated as a threshold can be re-run; "we removed personal information" cannot.
Watch the exemption rate. It is the leading indicator that the rubric stopped fitting the data.
State the blind spot in one line. SQuAD's metrics ignore punctuation and articles; COCO's headline averages ten thresholds. Both are fine as long as the limitation is on the label.
Plan the retirement. When a set saturates or its taxonomy moves, freeze it, date it, and build the next one from the same seed items.

What I could not check

How many SWE-bench samples were dropped to leave 500. The announcement gives the annotated total (1,699) and the rubric, not the drop count (OpenAI, updated 2025-02-24).
Whether any specific model saw any specific benchmark item. Sainz et al. state the extent is unknown; I have no method that resolves this, so I do not claim cleanliness.
The CFPB's narrative scrubbing standard. The field is gone and the current field reference no longer documents it; I could not reach a dated scrubbing specification.
The CFPB's historical consumer_consent_provided semantics. The export field and its filter are gone; I found the removal dates, not the values.
Krippendorff's own minimum sample size for alpha. I have the kappa tables from Bujang and Baharum; I found no equivalent table for alpha.
The Landis and Koch bands at first hand. I read a reproduction of the 1977 table in a 2010 AHRQ report, not the original.
LiveCodeBench's current item count. The 400 figure comes from the March 2024 paper; the platform's live count may differ.
The OpenAI SWE-bench Verified page at first hand. The live page served an automated-client challenge when I read it; the same version is readable in an Internet Archive copy, and the annotation instruction PDF is undated.
The CFPB record count above is a live API value and changes daily. It was 18,274,023 when I read it on 2026-10-09.

I keep the longer write-ups and the raw measurements at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

Measuring error in support AI, and what accuracy hides — One eval run, with the error distribution instead of a single accuracy number.
Births counted three ways, and none agree — Three official counts of the same thing, and why none agree.

DeepSeek filled in the Ascend column: six repos, one technical report, and my machine with no NPU

hc_xshh — Wed, 30 Sep 2026 08:35:30 +0000

DeepSeek released a batch of things over the last two days: six repositories plus one technical report in Chinese, all around Huawei's Ascend. I cloned all six, read them, and tried them on my own machine.

This post covers three things: what was actually released, whether those impressive-looking numbers hold up, and exactly where my machine gave up.

The short version

Six components were open sourced this time, and every one of them has a CUDA counterpart: a language for writing kernels, matrix multiply, multi-card communication, a general kernel library, an attention kernel, and a TopK kernel. Together that's filling in a column, not shipping a new model.
The numbers are strong: dense matrix multiply at 99.8% of the hardware limit, sparse attention prefill at 95%, multi-card communication at 90–95% of physical bandwidth. But every one of these was measured on PoC hardware Huawei provided to DeepSeek. The commercial version is planned for mid-October. No third party can reproduce them today.
The most valuable thing in this release isn't the announcement — it's the Chinese technical report published alongside it. It lays out the trade-offs of writing a sparse attention kernel on Ascend step by step: why one AI Core handles 64 heads, why you must pair each CUBE with two vector cores, and why one layout conversion has to detour through UB. I've copied the parts I found most revealing below.
What I could verify locally: all six repos cloned, I read and counted the code, and I could not run a single kernel. The build dies at line 7 of setup.py. That section is below too — it's measured, not paraphrased.

(Component-by-component comparison. The rightmost column is the size of the Ascend-specific part, which I counted from the source.)

1. Getting this layer straight first

Between a large model and the silicon sits a software layer: it tells the chip how to compute (kernels) and how to move data between cards (communication libraries). NVIDIA's version of that layer is CUDA; Huawei's is CANN. Anyone can write the model on top, but this layer has to be rewritten in the chip vendor's own language.

When DeepSeek V4 launched in April, Huawei got there first with Ascend support — but that delivery was about making it run. The pieces that matter most for performance were still open sourced only in their CUDA form. One media analysis at the time put it this way: moving from CUDA to CANN involves rewriting kernels at scale, restructuring the training pipeline, and re-tuning accuracy and performance — not less work than training the model itself. (Source: 21st Century Business Herald, 2026-04-24, in Chinese.)

That is exactly what this release is. Which is why the sentence in the announcement — "all components correspond one-to-one with the previously open-sourced NVIDIA-platform components" — is the most substantive line in it.

2. The six components, one by one

TileLang's Ascend 950 backend. TileLang is a language for writing high-performance kernels in Python, led by a team at Peking University and used for most of the kernels in DeepSeek V4's training. This time it landed in the main repository, not as a bolt-on adapter: PR #3308, "Introduce Ascend 950 backend", merged on 30 September and shipped with v0.1.15. The official support matrix lists Ascend 950 as Supported, built from source with USE_ASCEND=ON USE_CUDA=OFF, requiring CANN, the bisheng compiler and torch_npu on the machine, with torch.npu.is_available() returning True.

DeepGEMM-Ascend (matrix multiply). Same package name and same API as the NVIDIA version, so existing code keeps working. The repo was created on 29 September.

DeepEP-Ascend (multi-card communication). MoE dispatch and combine — sending tokens to the right expert and collecting them back. The kernels are all header-only .hpp files sitting on Huawei's HCCL/HCOMM, UBMEM and URMA. It's also the only Ascend repo in this batch that was built from scratch.

TileKernels v2.0.0 (general kernel library). The small but frequently used kernels: MoE routing, quantization, Engram, hyper-connections. The README's own words: "added an Ascend backend, selected automatically at runtime; the same Python API runs on both NVIDIA GPUs and Huawei NPUs."

FlashMLA (attention). Sparse attention prefill and decode.

DeepSelect (TopK). The step that picks compressed blocks in sparse attention, plus the TopK the sampler needs. The authors claim 2–20× faster than native torch.topk.

3. The numbers are strong — read them together with their sources

(Performance figures and where they come from. Bar length = the fraction of that hardware's theoretical limit; further right means closer to the ceiling.)

The key readings:

Dense matrix multiply at 99.8% of the hardware limit (BF16, FP8 and FP4 all land around there). FP4 measured 1701 TFLOPS on a 4096×7168×16384 shape against a 1730 TFLOPS hardware limit, on Ascend 950DT with CANN 9.2.0.
Sparse attention prefill at 410 TFLOPS, 95% of the theoretical limit; decode at 360 TFLOPS, 83%.
MoE communication: at EP8, dispatch 373–375 GB/s and combine 345–347 GB/s; within 32 cards it reaches 90–95% of physical bandwidth. At EP128 dispatch drops to 313–320 and combine to 272–278, with the vendor saying combine is still being optimized.

The sourcing point has to be stated plainly: DeepEP-Ascend's own README says these numbers were measured on a PoC HDK that Huawei provided to DeepSeek, with additional manual configuration that is not part of any public release. Huawei's commercial HDK for Atlas 850E is planned for mid-October. A third party cannot get the same hardware and configuration right now, so this set of numbers should be read as a vendor claim, not as a reproducible public benchmark.

Two pieces of hardware background, both from official statements: the Atlas 950 supernode supports 8192 950DT chips across 160 cabinets (128 compute, 32 interconnect), with an FP8 rating of 8 EFLOPS and FP4 up to 16 EFLOPS, and interconnect bandwidth of 16 PB/s — Huawei's own line being that this single product's total interconnect bandwidth is more than ten times the peak bandwidth of today's entire global internet. Availability is Q4 2026. A single Ascend 950DT is planned at 144 GB of memory, 4 TB/s of memory bandwidth and 2 TB/s of interconnect bandwidth.

That is the basis for the fine print on DeepSeek's side: the pricing cadence of V4-Pro is tied to the shipping schedule of the Ascend 950 supernode.

Further out, there are moves in progress: Bloomberg reported on 4 September (relayed by Guancha) that DeepSeek plans to deploy at least 160,000 Ascend 950DT chips for inference in an Inner Mongolia data center, with a gigawatt-scale power plan and full delivery possibly taking more than a year. Huawei announced the Ascend 960 supernode at its 17 September Connect conference — 4096 cards, replacing 48,000 800G optical modules with 5,500 in-house optical engines. Both of these are third-party reporting or vendor announcements; I have no independent verification to offer.

4. That Chinese technical report is the part worth reading

FlashMLA shipped with a paper called 《Ascend Sparse Attention Forward 算法与优化技术简析》("A brief analysis of the Ascend sparse attention forward algorithm and optimization"), in Chinese, released today and sitting in the repo. It lays out line by line why the kernel is written the way it is. A few of the parts that stuck with me:

One AI Core handling 64 query heads is not arbitrary. The report puts the compute-to-memory ratio on the table: with CUBE saturated (4096 multiply-adds per cycle), L2 total bandwidth of 5 TB/s, 32 cores and 1.65 GHz, the head count has to be 64 or more before compute becomes the bottleneck. At the same time, 128 heads would overflow L0C (a 128×512 float32 output is 256 KB), forcing the accumulator onto the vector core, which is extremely hard to write; empirically M=64 is already enough to saturate CUBE.

Every CUBE needs two vector cores; one is not enough. All three reasons are hard constraints: a single vector core can have at most 16 MTE2 copy requests in flight; decode has to read FP8 or FP4 KV and dequantize to bf16, and one vector core can't dequantize fast enough to keep up with CUBE; and FixPipe writes out at only 128 bytes per cycle per core, which by the formula forces B_TOPK above 73 — in practice they use 96 or 128. Taken together, the second vector core is not redundancy.

A purely arithmetic optimization saves a large round of data movement. The conventional online softmax approach may need to rescale the accumulated output against a new maximum every iteration. Their approach sets a threshold of 6 for "does this need rescaling" and skips the rescale when it isn't triggered — a trick they call skip-scale. The payoff is bigger on an NPU than a GPU, because on an NPU scale-O has to touch three units: FixPipe, UB and SIMD. The hard part is that the signal is produced by the vector core while the consumers are the CUBE and the vector core, which they solved with an SS buffer plus a cross-core flag.

The KV layout conversion goes the long way around because of a memory access pitfall. CUBE wants NZ layout while KV sits in memory as ND. The normal approach is to convert in flight during the copy, but they fetch KV sparsely token by token, which rules that out. They tried copying tokens as 32-byte blocks directly, and found that the BIU inside the AI Core only coalesces accesses when both source and destination are contiguous — "source contiguous only" doesn't coalesce, L2 bandwidth utilization drops, and the bottleneck becomes L2 bandwidth itself. In the end the path is: memory to UB, SIMD vector instructions to convert into NZ, then into L1.

The report also covers gather2 (fusing two token copies into one to work around the MTE2 queue depth of 16), L1 dual-bank balancing (splitting each tensor into two, one per bank), and the fact that icache is only 32K (CUBE), 16K (VECTOR) and 8K (SIMD VF) — which is why they use hardware loop instructions instead of unrolled loops. None of these details appears in the announcement.

One companion piece worth reading: Huawei's Ascend community has a long tutorial on TileLang from beginner to expert (published 2026-05-08) that lists the expert-mode conversion as six steps — split the requested cache by architecture into UB, L1 and L0C; insert workspace (because UB and L1/L0C cannot be copied between directly, so data used in both places has to exist twice); separate CV cores; synchronize across cores; have the two vector cores each handle half the data; and finally deal with the hardware special case that L0C cannot be written to directly. One of their measurements: hand-written FlashAttention at 44.9 microseconds, 22.28 microseconds after manually pipelining it.

5. How third parties see it

The most systematic external tracking comes from SemiAnalysis's InferenceX. They've been recording DeepSeek V4 1.6T inference performance across hardware from Day 0 through Day 43, with a dedicated section analyzing inference on Ascend 950DT — how compute and communication overlap, and which compute streams Huawei used. One of their own observations is worth keeping: on the CUDA side, vLLM and SGLang were usable the day the model shipped, while ROCm wasn't usable on day one and only caught up on day 26, with an improvement of more than 100×. The ecosystem maturity gap is the genuinely hard part of this road. The same piece also notes that V4's fused MegaMoE kernel carries a theoretical 1.92× speedup over a naive implementation — working backwards, a naive implementation spends close to half its time in dispatch and combine.

Two industry figures as reference points: IDC's data has NVIDIA's share of China's AI accelerator market falling from 95% two years ago to 55% in 2025, with domestic chips passing 41% for the first time — Ascend alone shipping 812,000 cards. China Mobile's April AI supernode tender this year was for 6,208 cards at roughly 2.06 billion RMB, requiring CANN-ecosystem solutions across the board. Both are third-party figures; I'd re-check them before using them.

6. How far I got on my own machine

(After cloning the six repos locally, where the chain breaks. Green is what worked; red is what this machine doesn't have.)

The environment first: Python 3.11.15; /usr/local/Ascend does not exist; import torch_npu raises ModuleNotFoundError; which bisheng ccec finds nothing; there is no nvidia-smi command; and lspci shows zero NVIDIA, Ascend or Huawei devices. This is a pure CPU machine.

Then I actually tried to build DeepGEMM-Ascend. The system python3 doesn't even have setuptools, so it stopped at line 5. I made a clean venv, installed the dependencies, and the second attempt got to line 7:

import torch
ModuleNotFoundError: No module named 'torch'

The Ascend part only starts at line 28: ASCEND_HOME defaults to /usr/local/Ascend/ascend-toolkit/latest, then it imports torch_npu and links against ascendcl, hccl_fwk, torch_npu, opapi and dl. DeepEP-Ascend's CMakeLists is even more direct — under ${ASCEND_HOME_PATH}/aarch64-linux/ alone there are dozens of header directories, plus the clang/15.0.5 headers from the bisheng compiler.

The conclusion is that I never even reached the doorstep. Reproducing any of this yourself starts with owning an Ascend machine with CANN installed.

7. What's in the code but not in the announcement

(Distribution of Ascend-specific kernels in TileKernels. The quant directory is the largest: 12 files, 3,491 lines.)

TileKernels writes two implementations for each kernel. Take SwiGLU: the directory contains both swiglu_forward_kernel.py and swiglu_forward_asc.py. The Ascend one starts with import tilelang.ascend.language as T and from tilelang.ascend.language import simd as S, and its pipeline synchronization is hand-written:

T.ascend_set_flag('MTE3_MTE2', 0)
T.ascend_wait_flag('MTE2_V', input_slot)

The README's "backend selected automatically at runtime" is true, and the mechanism is an is_ascend() check in tile_kernels/config.py; the package can carry both backends in one environment. But a backend is not automatic translation — the Ascend implementation is written by hand. I counted the Ascend-specific files: quant 12 files / 3,491 lines, mhc 9 / 2,375, engram 6 / 969, moe 4 / 652, plus one each in transform and rand — 33 files, 7,729 lines in total.

Constraints scattered through asserts. A few: per-token quantization only supports groups of 32 channels; the E4M3 scale factor is CUDA-only; SwiGLU only supports sf_block=(1, 32); hyper-connections only support 4 streams; randn doesn't support graph capture on NPU because replay would reuse the same random seed; and norm's hidden dimension must be divisible by 128. The announcement's line that "every TileLang kernel has a corresponding high-performance implementation" holds — the price is that list of shapes and preconditions.

FlashMLA's Ascend kernel covers only 6 combinations. Under csrc/ascend_kernels/prefill/sparse there are 4 headers plus 6 script-generated instantiation files: V4.1, h64, prefill and decode, with and without sink, fp8 and fp4. Change the shape and you regenerate. This is not a general-purpose library.

There is no kernel file in DeepEP-Ascend with "ascend" in its name. Its kernels are organized by function under csrc/kernels/ep, comm, bucket, pp and engram — all .hpp, 47 files, 6,335 lines in total.

The same batch of commits carries a breaking change. FlashMLA removed Hopper support and V3, V3.2 and V4.0 in this release, changed the KV cache format, and requires switching back to the 15 September commit if you need older models or formats. The Ascend kernels here only serve V4.1. This version of the attention library moves with V4.1.

8. What this post doesn't do

I reproduced none of the performance numbers — I have no Ascend card. I did no independent verification of second-hand items like "160,000 chips" or the supernode timelines; each is marked with its source in the text. When the commercial HDK lands in mid-October, there will be a version a third party can actually check, and I'll come back and fill this in then.

References

DeepSeek's announcement, "DeepSeek open-sources Ascend foundation components" (in Chinese): https://mp.weixin.qq.com/s/X41mKH4Ds-VXUAnK6M8Eww
FlashMLA Ascend technical report (Chinese): https://github.com/deepseek-ai/FlashMLA/blob/20260930-ascend-open-source/docs/20260930-ascend-prefill-deep-dive-zh.md
DeepGEMM-Ascend, DeepEP-Ascend, TileKernels, FlashMLA, DeepSelect: the same-named repositories under github.com/deepseek-ai
TileLang main repo PR #3308: https://github.com/tile-ai/tilelang/pull/3308
TileLang Ascend backend docs: https://github.com/tile-ai/tilelang/blob/main/tilelang/ascend/README.md
Huawei, "Ascend 960 supernode launch" (2026-09-17): https://www.huawei.com/cn/news/2026/9/hc-ascend960-supernode
Huawei supernode interconnect keynote transcript (2025-09, Atlas 950 specs): https://www.huawei.com/cn/news/2025/9/hc-xu-keynote-speech
Ascend community, "TileLang AscendNPU IR from beginner to expert" (2026-05-08): https://www.hiascend.com/developer/techArticles/20260506-1
Cailian Press, "Huawei's Atlas 950 hardware to appear at WAIC" (2026-07-07): https://www.cls.cn/detail/2418935
Guancha, "DeepSeek reportedly to buy 160,000 Huawei Ascend 950DT chips" (2026-09-05): https://www.guancha.cn/economy/2026_09_05_830131.shtml
Economic Observer, "Huawei computing power opens its chain" (2026-05-01): http://www.eeo.com.cn/2026/0501/860610.shtml
SemiAnalysis InferenceX, "DeepSeekV4 1.6T Day 0 to Day 43": https://inferencex.semianalysis.com/blog/deepseekv4-16t-day-0-to-day-43-performance

Every number, error message and source in this post is in the list above; nothing here is repeated from somewhere else. I write one of these deep dives a week, published first in my own community in Chinese — this is the cleaned-up public version.

I also keep a small paid community where I write these engineering logs up in full, one deep dive a week. Its content is in Chinese — I mention it in case you read Chinese: AI落地实录 (¥25/year with the current new-member coupon, ¥50 list price).

Disclosure: this English version was translated and edited with an AI assistant from my own write-up. Every figure, error message and file count comes from my own runs or the linked primary sources — which is exactly why the parts I could not verify myself are marked as vendor claims.

Originally written in Chinese for my own notes; this English version is a translation.

I keep the longer write-ups and the raw measurements at Field Notes — no ads, no tracking, one-click unsubscribe.

Read next, from the same notebook:

The Chinese open-weights release timeline, dated — The release timeline these repos sit on, dated.
From token prices to task prices: three jobs, seven APIs — What the models cost per task, not per token.