Every LLM writes the same Todo App. And I mean that literally

Jarek — Sun, 05 Jul 2026 17:53:35 +0000

I'm building a benchmark of local models.

Simple task: build a Todo App in a single HTML file - add, delete, mark as done, filters, localStorage. Trivial. Each model gets 6 attempts. 96 files total from 16 models - from Claude Sonnet 4.6 and Haiku 4.5, through the whole Qwen family (9B to 122B), Gemma 4, GPT-OSS 20B and 120B, GPT-4.1-mini, GPT-5.4-mini, Codex-mini, all the way to Z.AI GLM-5-turbo.

I open file after file to judge whether it works, how the code looks, and run it in the browser - just like a caveman, instead of automating it, because I actually want to see what the AI produced.

Somewhere around the Nth one I get this weird déjà vu. After the next one it clicks - it's as if one person wrote all of them. The same function names (addTodo, saveTodos, deleteTodo), the same CSS classes (.filter-btn, .container, .todo-text), the same localStorage key: 'todos'. Models from three different companies, trained on three different datasets - structurally indistinguishable.

I decided to check it - but this time using an agent.

Fingerprint instead of diff

I'm not comparing the code literally - whitespace, quotes, function order are noise. I want to compare structure. For each file I pull out a set of "tokens":

id="..." attributes from the HTML
CSS class names
JS function names
CSS variables

Each file becomes a bag of ~30 labels. For every pair I compute the Jaccard coefficient - |A ∩ B| / |A ∪ B|. 1.0 = identical fingerprint, 0.0 = not a single shared identifier.

Then single-link clustering with a 0.45 threshold: two files land in the same cluster if there's a chain of pairs with similarity ≥ 0.45. A naive algorithm, but on 96 files it runs in a fraction of a second. The script is ~150 lines of Node.js with no dependencies beyond Playwright for generating the PNGs.

The result: one giant monocluster

A 96×96 heatmap. The top-left corner is one huge dark block taking up 62.5% of the whole matrix - 60 files, four companies:

Anthropic Haiku 4.5 - all 6 attempts
Qwen 3.5 (9B, 27B, 35B, 122B) and Qwen 3.6 (27B, 35B)
Gemma 4 (26B and 31B)
Z.AI GLM-5-turbo - all 6 attempts

Four companies, five model families, spanning 9-122 billion parameters. Structurally the same code.

The most surprising numbers:

Jaccard 1.000  qwen3.5_122b_03  ↔  qwen3.5_122b_04
Jaccard 1.000  qwen3.6_35b_01   ↔  qwen3.6_35b_03

Same model, two different runs - an identical set of identifiers. Zero variance. The model has a "favorite piece of code" that it returns almost deterministically.

The most common signatures across the whole corpus: the .active class in 92/96 files, addTodo in 80/96, deleteTodo and .filters in 75/96 each, saveTodos in 74/96. These aren't coincidences.

It's the imprint of one specific canon - TodoMVC plus its hundreds of clones on GitHub, which all these models very likely swallowed during training.

There are other schools, though

Beyond the mainstream you can see two distinct bubbles.

The "OpenAI minimalist" cluster - GPT-OSS 120B and 20B, GPT-4.1-mini, part of Codex-mini. Characteristics: ~200 LOC instead of ~330, fewer CSS classes, simpler structure. Same problem, terser and without the decoration. Interestingly, GPT-OSS 120B and 20B sit together despite a 6× difference in parameters - clearly the same "style school" inside the company.

GPT-5.4-mini stands out differently: it consistently uses CSS variables with a color palette - --primary-color, --bg-color, --danger-color. The other models hard-code their colors. This one ran into newer frontend patterns during training.

Sonnet 4.6 - an interesting case. 6 attempts split into three small subgroups and one singleton. Intra-model Jaccard = 0.441 - the lowest in the whole "class A". Haiku 4.5 sits at 0.815. So the same Haiku gives you the same thing every time, while Sonnet gives you something different every time.

Codex-mini - the lowest score in the set, mean Jaccard = 0.277. Three attempts in the OpenAI cluster, three singletons. A model that doesn't even have a "favorite solution" for a trivial task.

Will Snake be any different?

I said above that todo is a recall task. Snake - with modes (human-vs-human, human-vs-cpu, cpu-vs-cpu), a score, a start menu and a game-over screen - has enough decision dimensions to force the models to design rather than remember. I ran the same script on another 96 files.

The heatmap looks completely different. No big dark blocks. The network view: a few small groups in one corner, surrounded by a ring of 73 singletons.

Metric	Todo App	Snake
Largest cluster	60 files (62.5%)	6 files (6.3%)
Singletons	4 (4%)	73 (76%)
Identical pairs (Jaccard 1.0)	2	0
Cross-company clusters	2	0

All seven Snake clusters are intra-family - not one of them links models from different companies. The exact opposite of todo.

In todo the dominant signatures were things like addTodo, .active, .filter-btn - names of specific design decisions that everyone could have made differently, but everyone made the same. In Snake the dominant ones are startGame, id="gameCanvas", endGame, draw, spawnFood - that's the domain vocabulary, the minimum you can't get around. Everything else - game modes, menu structure, how state is held - is different in every file.

Sonnet 4.6 in Snake: 6 attempts, 6 singletons, max Jaccard = 0.324. The highest intra-model diversity in the whole set.

What follows from this

The Todo App isn't a benchmark of programming skill. It's a recall test. The code exists in the training data in hundreds of copies, and the model reproduces it. Swap the task for something with real complexity and the "model consensus" shatters into 78% individual approaches.

Four companies in one bucket isn't a coincidence - Anthropic, Alibaba, Google and Z.AI very likely trained on overlapping GitHub scrapes (TodoMVC + clones) and/or on synthetic data from a shared stronger teacher. It's not a conspiracy, it's the consequence of "good code" on the web being a finite set.

Low variance between attempts isn't a feature - it's a defect. Codex-mini with a mean Jaccard of 0.277 isn't the "worse" model. It's the model that gives you different approaches on the first answer. In real work, where the first answer is rarely the final one, that has value. On leaderboards with one attempt per task it counts as "instability".

And practically: if you're building CRUDs and wondering whether to grab Haiku, Qwen 9B or Z.AI GLM - it probably makes no difference to code quality. What matters is price, latency and context size. Not "intelligence".

I started this benchmark to check how the agent I'm building works with different models, and which is the "best local model for coding".