I set out to build a simple thing: a table that answers "which Chinese AI model is cheapest for this task".
It took about a day to discover the question is malformed. Of the 111 models I normalised, only 15 charge per token. The rest charge per image, per generation, or a flat rate per call. Comparing them by their headline number is comparing nothing.
Here is the distribution, from the feed I rebuild daily:
| Billing unit | Models | Share |
|---|---|---|
| Flat (per call) | 49 | 44% |
| Per generation | 33 | 30% |
| Per 1M tokens | 15 | 14% |
| Per image | 9 | 8% |
| Free | 5 | 5% |
Why this breaks most "cheapest model" tables
Take two entries from the feed at the time of writing: one image model at $0.0215 per image, one chat model at $0.075 per 1M tokens.
Neither number is smaller than the other. One answers "what does an image cost", the other answers "what does a million tokens cost". A table that sorts them together is just sorting strings.
This is not a Chinese-provider quirk, but the mix is unusual: outside China most API pricing is token-shaped, so tooling assumes tokens everywhere. Import a feed where 44% of rows are flat-priced into a token-shaped calculator and every total is wrong.
What I did instead
Normalise within a category, never across:
- text → 1M input tokens
- image → per generated image
- video → per 5 second clip
- 3D / audio → per generation
- embedding → 1M tokens
Then keep the unit next to the price, in the same row, so the comparison cannot silently drop it. A row that says $0.054 | per 5s clip cannot be mistaken for $0.054 | per 1M tokens.
The feed itself records platform_markup_pct: 0 on every row, so you can tell list price from resale price.
Where this breaks anyway
Retries are invisible. A video generation that fails twice and succeeds on the third attempt costs three times the listed price. The feed has no notion of retry rate, and neither does any provider's pricing page.
Flat pricing is not comparable to itself. Two models both billing "flat" can be wildly different: one flat rate might cover 4k tokens, another an entire 1080p video. The unit label is honest and still useless.
Regional pricing and provider-side discounts are ignored. I read the public price page; if you have an enterprise agreement, your numbers differ.
And the data can be wrong. I am parsing other people's pages. If a provider changes their table layout, a row can silently keep yesterday's price. That is why the feed carries a generated_at timestamp — check it before trusting a number.
Check the distribution yourself
curl -s https://aiapi-pro.com/china-ai-leaderboard.json \
| jq -r '[.categories[].models[].unit] | group_by(.) | map({unit: .[0], n: length}) | sort_by(-.n)'
Ten seconds, no key, and you do not have to take my numbers on faith.
What I would want next
The useful version of this table is per-workload, not per-model: "what does 10,000 product images with one retry each cost" is a question with an answer; "which model is cheapest" is not.
If you have a workload in mind, I would rather normalise that than argue about a leaderboard.
Top comments (0)