Seven Claims in My Recipe Get the Code Wrong

The articles on this blog are produced from a recipe: an instruction file that says which category to draw a topic from, which limits must never be crossed, which validators to run. Last night, before putting that recipe to work, an odd curiosity got the better of me. Is the recipe itself still true?

I pulled up three files — the repository's CLAUDE.md, its AGENTS.md, and my article-writing recipe. Every concrete assertion in them, meaning every number, every file path, every mechanism name, I put to the code one by one. Seventeen claims turned out to be checkable. Seven of them described something the code no longer does.

That wasn't the part that surprised me. What surprised me was that automated scanning found none of them.

Why these files rot differently

What happens when a README goes stale? A human reads it, thinks "something here is off", and either fixes it or shrugs. The document's wrongness runs into a human's suspicion.

Files that instruct an agent have no such brake. They aren't read, they are executed. If one says "set reading time to the word count divided by 250", the thing doing the writing does exactly that; it never asks where the 250 came from or whether it still holds today. The document determines behaviour as much as code does, yet it has none of code's guarantees: it isn't compiled, it has no tests, nothing type-checks it. An executable file carrying all the risks of a file that has never been run.

So I kept the scope to these three files. All of them describe this blog's own infrastructure, and all of them governed that run.

I tried the easy way first: the scanner scored zero

My first instinct was to automate. I wrote a short Python script: from the backtick contents of those three files it collects, with a regular expression, every token that either ends in a known extension (.ts, .mjs, .json, .yml, .astro, .png…) or closes with a slash, then looks each one up on disk. The logic is innocent enough: if a document mentions a file that isn't there, the document has gone stale.

Twenty-five unique tokens came out. The script flagged seven as missing. Seven findings, not bad, I thought — and then I bothered to check them one at a time.

All seven were my script's fault.

personalize.sh appeared bare inside a sentence; its real home is under scripts/, and the document gives the full path two lines below. site.conf was absent because it isn't an input but an output the script creates — line 126 of scripts/personalize.sh writes it. en/ was shorthand; src/pages/en/about.astro was exactly where it should be. feed.json isn't a repository path at all, it's an address on the live site. memory/ lives outside the repository, in my home directory. -en.png is a filename suffix with nothing to do with paths. content-generate.yml existed too; I had simply torn the token off its directory prefix.

All twenty-five tokens were correct. My scanner's precision was zero.

I had learned this lesson before: make a scanning script produce a known-correct answer before trusting it. Mine had never learned to say "found nothing"; it only produced noise. Had I believed those seven flags and started "fixing" the document, I would have broken seven lines that were right.

The real damage was sitting where no scanner was looking.

Four hundred words of extra reading time

The costliest finding was a single number. For reading time my recipe says: divide the word count by 250.

No such divisor exists in the code. The top of scripts/lib/reading-time.mjs reads WORDS_PER_MINUTE = 200; src/utils/content.ts divides by 200 as well; scripts/migrate-content.mjs, which moves content into the database, fills a missing value using 200 too; scripts/notify-subscribers.mjs, which alerts subscribers, uses the same divisor. The same number in four places, 250 in none of them — and the count keeps rising if I keep looking.

What stings here isn't that the two divisors differ. It's that they are close. On a 2,200-word article, 250 gives you 9 minutes and 200 gives you 11. A two-minute drift catches nobody's eye — not a reader's, not mine. The ideal magnitude for being quietly wrong.

So is there a gate that catches it? There is — and this is where it gets interesting.

scripts/validate-en-post.ts, when called with --staged, switches to strict mode and requires reading time to be deterministic: exactly the word count divided by 200. If it isn't, it fails with readingTime is not deterministic. In other words, the recipe's 250 blows up immediately in the hands of anyone who runs that validator.

That validator isn't in the recipe's list of mandatory pre-publication gates. Which is a second wrong claim on its own: the list names two commands, while the repository holds four validators.

Stack the two facts and the picture is this: a wrong number sits in the same document as the omission of the only gate that could catch it. Run the gate and you get an error and learn the document is wrong; skip it and you learn nothing. Either way the document fails to protect you from its own mistake.

One more detail: by default that validator only looks at English files, covering the Turkish ones only with --all-langs. So someone following the recipe to the letter gets an error on the English side, fixes it, and ships the Turkish side with the wrong number. A half-caught bug is more irritating than one never caught, because the caught half gives you the feeling that the system is working.

The two validators missing from the recipe aren't in the same situation either. For validate-mdx-staged.ts there's a pre-commit hook installed in the repository, so it runs by itself at commit time; unwritten in the recipe, but there's a net. For validate-en-post.ts I could find nothing that calls it automatically. Not in the recipe, not in the hook. A gate nobody takes by the hand is not a gate.

The schema I said was rejecting things now belongs to something else

The second cluster of findings: the numbers were right, the reasons were wrong.

My recipe says, for titles, "don't exceed 70 characters, the schema rejects it". Here is what I found in the code: src/content.config.ts no longer has a blog collection. The file's own comment dates it — removed on 29 June 2026, when article content was taken out of the build-time collection and served at runtime from a database instead. Astro's documentation is clear about what schemas do: they validate a collection's entries through Zod, and a violation produces a build error. A file matched by no loader pattern isn't a collection entry, so no schema applies to it.

Blog frontmatter is validated today by src/lib/blog-frontmatter-schema.mjs. I looked at the limits there: 120 for the title, 200 for the description.

So "the schema rejects anything over 70" is false for article titles. The blog schema allows up to 120; three separate validators hold the 70-character line: validate-en-post.ts, scripts/generate-content.ts and validate-editorial-series.mjs. The number is right, the culprit is misidentified — and when a document names the wrong culprit, whoever wants to change the limit goes looking in a place that doesn't exist.

I got this wrong myself on the first pass, because the trap has a second storey. content.config.ts still contains a schema that reads title: z.string().max(70) and description: z.string().max(160) — but that schema belongs to short videos, not to articles. The comment on line 110 of validate-en-post.ts says its own limits are "the same as the content.config.ts schema", and as numbers that sentence still holds; as scope it doesn't. The day the blog collection was removed, the comment quietly began pointing at something else's schema. A stale value you notice; a stale scope you don't, because the figures still land in the same place.

For the description the recipe gives two numbers: "~150-160 characters, DEFINITELY ≤200 (the migration schema rejects it, the deploy falls over)". The advice is right — 150-160 really is what the strict gates enforce. The problem is the part in capitals. Land between 161 and 200 and the migration schema lets you through and the promised deploy failure never happens; meanwhile your English file trips validate-en-post.ts and generated content trips generate-content.ts. The document draws the safe band in the wrong place and promises the wrong consequence on top of it. If you are going to emphasise one number, don't pick the one from the loosest gate in the system.

A rule whose reason has died

One of the recipe's first steps is a pace lock: check the live feed, and if four articles have been published today, produce nothing. The reason is written in parentheses too — the old GitHub cron is supposedly still live, and the total supposedly never passes four a day.

I went looking for that cron. The trigger block of content-generate.yml contains only workflow_dispatch; there is no schedule. Per GitHub's own documentation, workflow_dispatch makes a workflow startable only manually, through the API or through the CLI, and the only time-based trigger is schedule. So this workflow never fires on its own. I checked its run history: it has no recorded runs at all. The file's own comment gives the date — the hourly cron was retired on 9 September 2026.

Articles today come out of a scheduler on my own machine, and that scheduler has three slots a day: 08:30, 13:30, 16:30. The structural ceiling is three.

I have to be honest here, because "so the rule is nonsense" would be the easy conclusion. Four-article days do happen: in the live feed, nine of the last forty days reached four, the most recent being Friday 2 October. There's also a separate publishing workflow that runs on Thursdays, and in September it was the one that pushed three Thursdays up to four — though the last three Thursdays stopped at three. So checking against four isn't pointless. What's wrong is the rule's reason: the document ties a threshold that stands up to a mechanism that can no longer fire.

The cost of that shows up later. Six months from now someone will want to change that threshold, will read the reason, will go hunting for a cron that isn't there, and will either burn time in the wrong place or lack the nerve to remove a rule they don't understand. A dead reason makes a rule untouchable.

A stale number multiplies itself

The last cluster is the most ordinary: copied numbers.

CLAUDE.md, discussing the blog's content, says roughly 730 articles. I counted last night: 1,325 Turkish files — 465 technology, 388 tutorials, 264 career, 208 life. The number has nearly doubled. The practical harm here is small, because the sentence appears in the step about emptying content for a fresh installation; whether it's 730 or 1,325, you delete all of it. But it teaches the wrong scale.

The one with teeth is the deploy duration. The documents say "about 12 minutes", and they say it in five separate places: once in CLAUDE.md, four times in AGENTS.md. I pulled the durations of the last twenty-four successful deploy runs: median 3.3 minutes, fastest 2.6, slowest 4.8. Not one of them comes near even six.

I also found where the 12 came from, because it lives on in two code comments. validate-en-post.ts describes an incident from 1 June 2026: a hand-written description crossed the limit and the build "blew up 12 minutes later". A deploy script notes that the old path took 8-12 minutes. So 12 was the correct number for the era when building really was slow. The build got faster, the number stayed — and while staying, it multiplied. A wrong number, once written, starts looking more correct the more it is copied; seeing the same figure in five places is persuasive.

The cost isn't a waiting screen, because the recipe watches the deploy with gh run watch; nobody counts a fixed twelve minutes. The cost is in expectation. AGENTS.md says the article is live "~12 min after the merge to main"; to anyone reading that, a deploy finishing in three minutes is a surprise and one taking twelve is normal. On the day the pipeline really does slow down, nobody is left to notice.

There's a third number in the same family, but calling it a defect would be unfair. The recipe refers to the policy tests that run before publication as "seventy-nine plus". I ran them: ninety-three tests passed. The plus sign saves the sentence from being wrong, so this isn't an error — but the figure you're reading is fourteen tests out of date.

Let me close the tally here, because the reader should be able to count it. The seven broken claims are: the 250 reading-time divisor, the incomplete gate list, the sentence tying the title limit to the wrong enforcer, the loose ceiling shown for the description, the pace rule grounded in a dead cron, the 730-article count, and the 12-minute deploy. One claim is stale but not false: the test count. The remaining nine are exactly right.

What the nine surviving claims have in common

Looking at what they leaned on, there's a pattern, and that pattern turned out to be more useful to me than any style rule.

The sentence saying cover titles wrap to three lines was right, because it transcribes the wrapTitle(28, 3) call in generate-hero.ts verbatim. "At least three sources, at least two distinct domains" was right, because it matches the MIN_ARTICLE_REFERENCE_SOURCES and MIN_ARTICLE_REFERENCE_HOSTS constants in the source policy. The Mermaid component's path, the directory layout for cover images, the fact that the --staged flag is supported, that pushing to a branch doesn't deploy, that astro and @astrojs/node are pinned to exact versions, that the Cloudflare cache is purged wholesale — all in place. The ninth was the most satisfying: the document's claim that a new bundle is booted on a temporary port and never swapped in unless it passes a smoke test maps to a block in the deploy script that starts the server on port 3941, curls a health address, and only continues on success.

The pattern: claims that name a symbol survive; claims that copy a value rot. When I write wrapTitle(28, 3) I'm sending the reader to the single source of truth in the code; if the number changes, the sentence is visibly falsified, because a function call is easy to grep for. When I write "divide reading time by 250" I'm writing the constant a second time, in a place nobody audits. The second copy always goes stale. Diligence doesn't fix it; as long as the constant lives in two places, it recurs.

Four rules I'll apply to my own instructions from here

1. Write the symbol, not the value. Instead of repeating a limit's number, point at the file and constant that hold it: "for the description limit, see the check inside validate-en-post.ts". A copied number is debt.
2. Put the enforcing gate next to the rule. If a limit goes into a document, so does the name of the mechanism that rejects it. "The schema rejects it" won't do; which schema, which collection, which line.
3. Wire the gate list into automation. A validator that has to be remembered doesn't exist on the day it isn't remembered. validate-mdx-staged.ts runs because it's on a hook, unwritten in the recipe; the other one never runs because it's only written down.
4. Date the reason as well. If a threshold rests on a mechanism, keep the mechanism's name and the decision's date beside it. A dead reason makes a rule unquestionable; a date at least makes "how old is this sentence" a question you can ask.

This article doesn't apply the third rule either, and I know it. Concretely it would look like this: add a case to the policy test suite that reads the constant in reading-time.mjs, compares it with the number in the recipe, and breaks on the day they diverge. The figure in the document would then age along with the code, and nobody would need a late night of curiosity. You can do the same for the gate list: a case that counts the validators under scripts/ and compares them with the ones the recipe names forces the document to be updated whenever a new gate appears.

To be honest, this article has fixed none of the seven lines; all three files still carry their old numbers as I write this sentence. And this article's own reading time was computed by dividing by 200, not 250 — I departed from my recipe quietly, because running the gate left no other option. Measuring doesn't count as fixing; I've put the fixing in the next step.

Closing

What last night's measurement taught me is bigger than seven wrong lines. I've measured a moved repository's old name surviving in three scripts, and measured three findings slipping past me when I merged without waiting for a review round. In both the fault was in the process. This time the fault was in the text that describes the process.

If the difference between code and documentation needs a name, it's this: when code is wrong, something tells you. A test breaks, it won't compile, it explodes at runtime, or at worst a user complains. Nothing will tell you a document is wrong. And when the party reading that document is an agent that doesn't know how to object, the error spreads not as a bug but as a new norm. A divisor that declares four hundred words two minutes shorter starts becoming the correct number, because nobody shouts.

So I now read instruction files with two separate eyes. One on what they say — that's the easy part. The other on how they say it: does this sentence point somewhere in the code, or does it write a value from the code a second time? The first kind gets itself repaired. The second kind waits, six months on, for someone who sets out one night to measure their own recipe.

Official Sources

• Astro — Content Collections: how collection schemas validate frontmatter through Zod, and why a file matching no loader pattern isn't a collection entry.
• GitHub Docs — Events that trigger workflows: workflow_dispatch starts a workflow only manually, via API or CLI; schedule is the only time-based trigger.
• Node.js — Test runner: the built-in runner's multi-file discovery, reporters and exit-code behaviour.