Turn a Repeated Mistake Into a Check

A reminder lives for one session, a rule lives while it is read, a check lives in the tooling.

---

👋 I'm Anton - a software engineer working mostly in PHP/Symfony and Go, currently carving a live
PHP monolith into Go services. The earlier parts of this series were about how to write a task so
that nothing has to be looked up. This one is about the mistakes that survive all of that and keep
coming back. Notes: github.com/brilliant-almazov.

The thought I want to share is small and slightly uncomfortable: the only place a rule reliably
lives is in a tool that refuses. Everything above that line is a hope. Maybe you've already solved
this better than I have; maybe you read it the other way round. Either way, here's my version with
the numbers attached.

As before: these are my habits on one codebase, not advice for yours.

---

The mistake I kept paying for

The mistake is boring. Someone writes, by hand, a copy of something the tree already has as a
generic.

Not "similar functionality". A copy. Same shape, different type parameter.

I ran an audit of one service against the shared platform library - the plain question "what in
here already exists somewhere else" - and the single biggest finding wasn't an architectural sin.
It was arithmetic: 44 copies of four shapes of reading rows out of the database.

Read shape	Copies
loop over many rows (for rows.Next() → rows.Err() → error wrapper)	18
read exactly one row	6
COUNT(*) counter	8
existence probe (a single rows.Next())	12
total	44

Of those 44, 12 differ from another copy only in the text of the error wrapper. Not in the
query, not in the scan, not in the type - in the string.

Here is the part that stings. The four shapes differed only by result type and by which
dependency they held. That is the definition of a type parameter and a field. And the correct
shape was already in the tree - a Reader[In, Out] with executor, statement, scanner and one
method Query(ctx, in). It was just sitting inside a domain package instead of a shared one, so
nobody found it and everyone re-typed it.

// what already existed, one file, one declaration
type Reader[In, Out any] struct {
    executor  Executor
    statement Statement
    scanner   Scanner[Out]
}

func (r Reader[In, Out]) Query(ctx context.Context, in In) ([]Out, error)

// what got written 44 times instead, with the type and one string changing
rows, err := q.pool.Query(ctx, stmt, args...)
if err != nil {
    return nil, fmt.Errorf("query order list: %w", err)
}
defer rows.Close()

out := make([]Order, 0, 8)
for rows.Next() {
    var item Order
    if err := rows.Scan(&item.ID, &item.Key); err != nil {
        return nil, fmt.Errorf("scan order row: %w", err)
    }
    out = append(out, item)
}
if err := rows.Err(); err != nil {
    return nil, fmt.Errorf("iterate order rows: %w", err)
}

I want to be precise about whose fault this is, because the honest answer changed how I work. It's
not sloppiness. Every one of those 44 was written by someone - human or executor - who had the
rule available and didn't have the primitive in front of them. The rule said "don't copy a
generic". The tooling said nothing at all. Forty-four times.

This is the shape of every recurring mistake I've had: it isn't a lapse, it's an accumulation. The
cost doesn't arrive as one bad day. It arrives as a number you find during an audit.

The ladder

There are exactly three places the memory of a mistake can live, and they have wildly different
lifetimes:

CHECK      lives always            linter · forbidding test · structure test · blocking hook
  ↑
RULE       lives while it is read  the instruction file, the style doc, the convention
  ↑
REMINDER   lives one session       "hey, don't do that again"

A reminder lives one session. You say it in the thread, it works, the thread ends. Nothing
carried it forward. This is not a criticism of reminders - inside one session they're the fastest
thing available. They just have no persistence layer.

A rule lives while it is read. This is the rung that fools you, because it feels durable: the
rule is written down, it's in the file, the file is loaded. And it does work - sometimes for weeks.
Then the file gets longer, or the violation looks perfectly normal, and the rule quietly stops
being applied while remaining perfectly true. A rule has no failure signal. Its decay is silent,
which is why you find out by counting to 44.

A check lives always. It doesn't care who is writing, whether the instruction was read, or
whether anyone remembers the incident that caused it. There are four kinds I actually use:

• a linter rule,
• a forbidding test - a test whose job is to fail when a banned shape appears anywhere in the tree,
• a structure test - one that asserts the layout rather than the behaviour,
• a blocking hook in the tooling - the write itself is refused.

The rung you want isn't "the strictest". It's the lowest rung at which the mistake becomes
mechanically impossible, because every rung above that is a rung that needs a human to be paying
attention.

Five checks that used to be rules

Each of these started life as a sentence in an instruction file. Each of them failed as a sentence.
Here's what replaced it, and what happens now when someone gets it wrong.

mistake                              check that closes it
───────────────────────────────────  ─────────────────────────────────
a copy of an existing generic    ──▶ row loop in exactly one package
env catalog drifts               ──▶ regenerate in --check, build fails
metrics snapshot drifts          ──▶ regenerate in --check, build fails
coverage threshold lowered       ──▶ ratchet: may rise, never fall
empty context in a test          ──▶ write refused at the hook
  421 calls in 184 test files

1. A copy of an existing generic → a forbidding test

The rule was "if only the type parameter differs, it's one generic; a domain-local copy next to an
existing generic is forbidden". That rule is what the 44 copies were written under.

The replacement is a test with a checkable criterion, and the criterion is the interesting part:

for rows.Next(), rows.Err(), rows.Close() and QueryRow( occur in exactly one package
in the tree.

That's it. Not "reads should use the core". Not "prefer the generic". A grep over the tree with a
count, asserted. When someone hand-writes a 45th copy, the suite goes red on a test that has
nothing to do with their feature, with a message naming the package that's allowed to contain that
loop.

Two things I like about this form. First, it's verifiable - I can tell you today whether the
mistake is closed, which I could never do about the rule. Second, it converts the migration into a
finish line: the stage that moved 44 copies onto one core was 40 iterations, one file each, and the
definition of done was this test going green rather than my judgement that it looked done.

# what the check actually asserts
occurrences of  for rows.Next() | rows.Err() | rows.Close() | QueryRow(
across the tree, grouped by package  →  exactly 1 package

2. The environment-variable catalog drifts → a pipeline check

The service publishes a machine-generated catalog of every environment variable it reads: name,
source (platform or service config), whether it's required, whether it's secret, help text, and -
for service variables - the file and line where it's defined. 59 variables: 45 declared by the
platform, 14 by the service's own config.

The rule version was "keep the catalog up to date". You can guess how that went.

The check runs the same generator in --check mode in CI on any change to a .go file, the
service manifest, the module files, or the snapshot itself. If the generated catalog differs from
the committed one, the build fails. The important detail is that the generator is built at the
same platform version the service's modules pin, resolved out of the module file - otherwise the
check would be comparing your snapshot against somebody else's catalog and failing for no reason,
which is the worst thing a check can do.

3. The metrics snapshot drifts → the same check, different artefact

Second machine-generated snapshot, same mechanism: metric name, type, help, labels, histogram
buckets, source, where it's declared. 67 records: 59 from the platform, 6 from the service, plus
one <dynamic> entry for the dynamic-metric factory.

Drift fails the build exactly like the variable catalog. I mention it separately because it's
evidence about the form: once you have one "regenerate and compare" check, the second one costs
almost nothing, and that's the cheapest kind of check there is. The generator already existed for
documentation reasons; making it a gate was a --check flag and a CI step.

4. Someone lowers the coverage threshold → a ratchet

The rule was "don't lower coverage". The check is a ratchet: the threshold can be raised and cannot
be lowered. An attempt to lower it fails the check.

threshold(new) >= threshold(committed)   else fail

This one deserves its own honesty section, below - because the mechanism is in place and the bar is
not yet set.

5. A banned phrase in a task file → a blocked write

I write executor tasks as files, and there's a family of phrasings that guarantee a bad iteration:
"figure out how it's done here", "by analogy with the neighbouring domain", "walk the path and
fix every step", "if needed", "TBD". Every one of them is a hole where a fact should be.

That used to be a rule about my own writing. It's now a blocking hook: the write of the task file
is refused when one of those phrasings is in the text. Not flagged - refused. I fix the sentence
and write a fact in its place.

6. An empty context in a test → a blocked write

Worth its own section. Next one.

The rule that could not possibly hold

The rule read: in tests, the context comes from the test runtime. t.Context() in tests,
b.Context() in benchmarks, derived contexts built from those; a port stub stores the context
it received in a field so the test can assert on it; every transport, repository, manager and
consumer gets a cancellation test.

That rule existed. It was written down. It was correct. Here is the scale of the problem at the
moment it was replaced by a blocking hook:

421 calls with an empty context, across 184 test files.

Why did this one decay so completely? Because of a property that I now look for in every rule I
write: the violation looks normal.

// looks completely fine in a diff
func TestSomething(t *testing.T) {
    repo := newRepo(t)
    got, err := repo.Get(context.Background(), id)
    ...
}

Nothing about that line asks to be questioned. It compiles, it passes, it reads like every Go test
anyone has ever seen. In review, your eye slides straight over it, because review attention goes to
the assertion, not to the first argument. A rule can only be enforced by reading where the
violation is visible when read. This one isn't.

And it isn't cosmetic. A test built on an empty context doesn't get cancelled when the test that
owns it is cancelled - it hangs until the package timeout instead of failing. Worse, it asserts
nothing about context at all: if production code drops the incoming context and substitutes its
own, all 421 of those tests stay green. A suite that large and that quiet is not a safety net, it's
a decoration.

The blocking hook refuses the write. Empty context in a test file or in test scaffolding: rejected
at the point of writing, before it can become the 422nd.

An honest note about the ratchet

The coverage ratchet from check #4 is the weakest thing on this page, and I'd rather say so than
have it read as a win.

The threshold is currently 0. The mechanism is in place, it can only move upward, and the bar
has not been set. Actual coverage on the service is 86.7% - so the gap between what's enforced
and what's true is the entire 86.7 points.

That's a check that's wired but not tightened. It will catch the day someone deliberately lowers a
number that's already set; it catches nothing today. I'm including it because the interesting
failure mode of this whole approach is exactly this: a check that exists, looks green, and enforces
nothing. If you're building a ratchet, the mechanism is the easy half.

The unexpected part: checks tell you the task was written badly

I keep a small list of tells that a set of tasks was written expensively - the signals I calibrate
my own writing against:

Tell	What it actually means
the executor asked a clarifying question	a fact wasn't written down
a rule-block fired	the task didn't name the primitive that must be reused
two iterations edit the same file	the split was wrong
an iteration gets rewritten after acceptance	the acceptance criterion wasn't checkable
an iteration burned more context than its named files need	a defect in the task, full stop

Look at the second row, because it's the one this article changed for me. When a blocking check
fires, my first instinct used to be "good, it caught something". That's only half of it. If the
check had to fire, the task text failed to name the primitive that should have been reused. The
executor didn't go looking for the generic and ignore it - it never knew the generic existed,
because I didn't write the name down.

So a firing check is two pieces of information: a mistake was stopped, and an upstream defect
exists in my writing. The second one is the more valuable of the two, and I used to throw it away.

That reframes the checks as something other than a fence. They're instrumentation on the quality of
my own specifications - the only part of this system that has no automated check on it.

What it costs

A check is not free, and three of the costs are real enough to argue about.

A check is code. It gets written, reviewed, and maintained. The forbidding test on read shapes
has to be updated whenever the tree legitimately changes - a new package that genuinely needs its
own row loop means editing the assertion, and that edit needs a reason attached, or the check erodes
one exception at a time.

A false positive costs more than a missed mistake. This is the one I underestimated. When a
check fires wrongly, the executor doesn't shrug - it starts looking for the cause, and looking is
the single most expensive thing it does. That's context spent on my bug in the check, on top of the
work being blocked. A check that's wrong 5% of the time is worse than no check, because it teaches
everyone - people included - that red doesn't necessarily mean stop.

Some mistakes have no cheap check. "This abstraction is wrong for the domain" is not greppable.
The ladder doesn't have a top rung for everything, and pretending otherwise just produces
checks that assert the shape of the wrong thing very precisely.

Where it does pay: mechanical mistakes with a textual signature, in a codebase where the right
shape is already decided. All six of my checks are of that kind. None of them is clever. Every one
of them is a grep, a regenerate-and-compare, or a comparison of two numbers.

Why this matters more with an executor than with a team

One property changes the arithmetic completely: the executor has no memory between tasks.

There is no "we agreed on this last time", because there was no last time. Every task starts from
zero, with whatever is in its own text plus whatever the tooling enforces. The middle rung of the
ladder - the rule that lives while it's read - degrades from "usually holds" to "holds exactly as
far as this one document was read, this one time".

Which means the ladder isn't three equally-valid options anymore. On a codebase where most of the
mechanical work is delegated, a rule is a reminder with better formatting, and the only durable
place for the memory of a mistake is the tooling. Not because the executor is careless - because
remembering is not a thing it does, and I kept designing as if it were.

The one conclusion

Count the recurrences. If the same mistake has happened three times, the next thing you write
should not be a clearer version of the rule - it should be the smallest mechanical check that makes
the mistake impossible. And when that check fires, read it twice: once as a mistake caught, once as
a sentence you failed to write.

---

Now the part I'm actually curious about. If you do this better - a cleaner place for these
checks than a mix of tests and hooks, or a way to keep the false-positive rate low enough to trust -
I'd like to hear the shape of it. If you've been through this already, I'd like to know which
rung you ended up on and what you decided wasn't worth a check. And if you read this the other
way round - that codifying mistakes into tooling ossifies a codebase faster than it protects it -
that's the argument I'd most like to see made, because I can't rule it out from where I'm standing.

How is this solved on your side, and what broke?

---

Working with agents - Part 7.

Next: what one iteration actually costs - where the tokens go, which of the six sources dominates,
and the number that made me stop letting executors gather their own context.