I checked 2,016 AI images letter by letter. Here's how I kept the test fair

GPT Image printed VENON on a death metal T-shirt. The prompt said VENOM. The last M had turned into an N, hidden in the thorns.

Our first check missed it. It was read off a contact sheet at thumbnail size and marked correct. A zoomed re-read caught it later. That one image shaped most of how we run this test, so this post is about the method, not the leaderboard.

Some context. My team runs AIFont, a site that publishes text-effect prompts for image models. Every prompt goes to five models in seven languages, and every image gets checked against the word it was supposed to write. After 59 styles that came to 2,057 first-attempt images, 2,016 of which we could score.

The results, briefly

Correct out of judged, first attempt only:

Script	GPT Image 2.5	Nano Banana 2.1	Grok Imagine 2.0	Seedream 5.0 Pro	Qwen-Image 3.0 Pro
Latin (en, de, es, pt)	235/236	235/235	229/235	230/235	213/235
Chinese	59/59	58/59	58/59	56/59	56/59
Japanese	59/59	58/59	52/59	51/59	45/59
Arabic	48/50	51/51	35/49	27/47	6/53

I give counts, not percentages, on purpose. With about 60 images per cell, one image moves a percentage by almost two points. GPT Image and Nano Banana are one image apart overall. That's a tie.

Same prompt, first image only

Two rules do most of the work.

Every model gets the identical prompt. One prompt per style per language, written in that language, sent unchanged to all five. No per-model tuning. If a model needs a special prompt to spell right, that's a result, not something to fix before measuring.

Only the first image counts. We do rerun wrong images, because the website needs a usable picture. But a retry only exists because the first try failed, and sometimes we draw extra samples after a correct one. Both lean toward correct. Count them and every model looks better than it is, and the weaker models get more second chances.

There was a less obvious trap. New styles get drafted on one model first, and the prompt changes a few times before all five run. Count every prompt version and that drafting model gets extra rows the others never saw. So scoring uses only the prompt version each style uses now. A prompt version is the MD5 of the full text sent, so "same prompt" is something you can check in the data, not something you have to take from me.

Settings were fixed per model: 2K, square, each vendor's API, October 22–29, 2026. GPT Image at xhigh quality, Grok Imagine at medium (its highest). The other three have no quality setting.

The ledger: append-only, latest review wins

Every generation writes a row, failed API calls included. Reviews go in a second table. Nothing in either table gets edited. Trimmed down, it looks like this:

-- one row per generation, success or failure
CREATE TABLE gen_attempts (
  id            bigint PRIMARY KEY,
  font_id       varchar(64),   -- style slug
  locale        varchar(10),
  word          varchar(100),  -- the text the prompt asks for
  prompt_hash   char(32),      -- md5 of the full prompt sent
  attempt_kind  tinyint,       -- 1 first, 2 retry, 3 extra sample
  model_id      varchar(64),   -- upstream model version at generation time
  ok            tinyint(1),    -- did the API return an image
  archive_url   varchar(512),  -- our copy of the original, never deleted
  gen_time      datetime
);

-- one row per review; changing your mind = a new row
CREATE TABLE gen_reviews (
  id            bigint PRIMARY KEY,
  attempt_id    bigint,
  verdict       tinyint,       -- 1 correct, 2 wrong, 3 unsure
  error_types   json,          -- fixed vocabulary
  rendered      varchar(255),  -- what the image actually says
  zoomed        tinyint(1),
  reviewer      varchar(32),
  is_native     tinyint(1),
  supersedes_id bigint,        -- the review this one replaces
  review_time   datetime
);

Three choices I'd make again.

model_id is read from the service config at the moment of generation. No hand-kept mapping table. If a vendor quietly swaps the model behind the same name, the data splits on its own.

A re-review is a new row pointing at the one it replaces. VENON's wrong "correct" is still in there, followed by the correction. You can see who changed their mind and when.

attempt_kind follows the latest reviews. If a first image flips from correct to wrong, the second image in that cell stops being an "extra sample" and becomes a "retry". It's the one field the tooling is allowed to recompute. And the content build refuses to publish a style if any of its images has no review yet.

AI reviews everything, people review where the AI might be wrong

Reading 2,000 images carefully by hand isn't realistic. I'd rather a person read 500 carefully than 2,000 tired. So there are two layers.

An AI reviewer read every image, zoomed in for Chinese, Japanese and Arabic. Then a person went through the images where the AI was most likely to be wrong:

• every image the AI marked wrong or unsure
• Arabic, where AI reading isn't reliable
• Chinese and Japanese images the AI had only seen on a contact sheet
• a random sample of 50 from everything else

The sample of 50 matched the AI on all 50, so the rest keep the AI's verdict and say so (human_reviewed: false). In the scored set, 489 images had a person look at them.

What we skipped: cases a glance settles. A paragraph of extra lettering across the image, or a word that differs only in case, which a rule decides (more below). Human time there buys nothing.

The direction of the AI's mistakes surprised me. It was too strict, not too lenient. About a third of the non-Arabic images it marked wrong were overturned by the human pass. The human pass wasn't perfect either: re-reading the disagreements changed 11 more verdicts. One read isn't enough, for a model or a person.

Wrong only if you can say what it reads as

Most of those overturned calls were the same kind: a letter that looks odd. The ぶ in しぶや drawn like a 3 with two dots. An extra slanted stroke inside 自. A capital F with a flourish that looks a bit like a J. A native reader reads all of them as the intended word without stopping.

So the rule became: a word is wrong only if you can say what it reads as instead. VENON. Konra. Eiwig. 爱の兰. If all you can say is "the shape is weird", it's correct. That's why rendered is required for every spelling error. If you can't fill it in, you don't have an error.

Three calls follow from that:

• Traditional Chinese counts as correct in a Simplified prompt. A reader sees 义氣 and reads 义气. Mixed forms are normal on real signs.
• Case doesn't count unless the prompt asked for it. POTION for Potion is fine.
• Extra text counts as wrong only when it competes with the word: similar size, right next to it, reads like part of the title. A shop sign in the background is background.

I think this is the right rule for any text-rendering test, and I expect people to disagree. A stricter rule, every stroke matching a reference glyph, is easier to automate. It also punishes exactly the stylization people ask these models for. Nobody prompts for a blackletter logo and wants Helvetica.

Arabic: why we don't claim 100%

Nobody on my team reads Arabic natively. The AI doesn't read it reliably either. In the first human pass, the AI and the person disagreed on 17% of the Arabic images they both read.

When they disagreed, we didn't pick a winner. The record becomes unsure, gets disputed: true, keeps both calls, and stays out of the table:

{
  "id": 88,
  "language": "ar",
  "target_text": "شارع",
  "model": "Grok Imagine",
  "text_verdict": "unsure",
  "reviews": [
    { "reviewer": "ai", "text_verdict": "incorrect", "native_speaker": false, "zoomed": true },
    { "reviewer": "maintainer", "text_verdict": "correct", "native_speaker": false, "zoomed": true }
  ],
  "disputed": true
}

41 images in the scored set are like this. It changes how you should read the Arabic row. Nano Banana's 51/51 comes from 58 images; 7 are disputed. GPT Image has 8.

I'm confident in the ranking. 48 of 50 against 6 of 53 is a far bigger gap than any reviewing error. I'm not confident in the exact numbers, and they will move when native readers check them. Writing "100% human-verified" on this would be easy, and it would be false.

That one isn't disputed. The prompt asked for نمر (tiger). Seedream added a ب and a vowel mark.

Five numbers hide the error types

Overall: 99%, 100%, 93%, 91%, 79%. The ranking is right. It's still not enough to pick a model.

Qwen-Image made 86 errors. In 40 of them it spelled the word correctly and then filled the image with text nobody asked for, often a scrambled copy of the prompt.

Grok Imagine and Seedream fail on detail. Of Grok's 14 Arabic errors, 6 are a missing, extra or moved dot and 5 an extra letter. Of Seedream's 20, 9 are dots and 7 extra letters. In Japanese it's dakuten in the wrong place: Seedream turned ばくはつ into ばくばつ.

The error type predicts what a second run does:

	First image wrong	Fixed by one retry
GPT Image	3	1
Nano Banana	2	2
Grok Imagine	28	13
Seedream	36	18
Qwen-Image	86	10

Detail errors are noise: about half disappear on a second try. Qwen's extra text is a habit and survives retries. So "Seedream 91%, Qwen 79%" looks like worse and a bit more worse, but the fixes are different. With Seedream, run it again. With Qwen, rewrite the prompt; rerunning just burns money.

My take: a text-rendering benchmark that reports only an accuracy number is half a benchmark. That's why error_types in the ledger is a fixed list (extra_letter, dots_diacritics, extra_text, and so on), not a free-text note. Free-text notes don't add up.

The data is public

All 2,495 images in the ledger are in the dataset, including retries and test runs, one JSON record each:

{
  "id": 978,
  "style": "death-metal-font",
  "language": "en",
  "target_text": "VENOM",
  "prompt_hash": "035d561d24e21967dda5e64918b7d79a",
  "model": "GPT Image",
  "model_id": "gpt-image-2.5-sunburst",
  "attempt": "first",
  "purpose": "published",
  "image_url": "https://cos.aifont.ai/research/attempts/death-metal-font/death-metal-font-en-sunburst-s1-90094.jpg",
  "text_verdict": "incorrect",
  "error_types": ["wrong_letter"],
  "rendered_text": "VENON",
  "human_reviewed": true,
  "disputed": false
}

Prompts sit in a separate file keyed by prompt_hash. Images are linked, not shipped. License is CC BY 4.0, archived on Zenodo with DOI 10.5281/zenodo.23815210.

If you recompute anything, keep attempt == "first", purpose == "published", the current prompt version per style, and drop disputed. summary.json is already filtered that way.

Corrections are the part I care about most. If you read Arabic, Japanese or any language in there and a verdict is wrong, open a "Wrong verdict" issue with the record id and what the image actually says. An accepted correction goes in as a new review under your GitHub handle, same append-only rule, and the next release recomputes the verdict. Arabic readers first.

If you run a comparison like this without a native reader for some of the languages, say so in the results. We do.

The full per-language results and more examples are in our report on the best image model for text, and the data is on GitHub.