Development covered 2 Aug 2026 to 4 Aug 2026 (commit dates).
A green check is a claim, and this week I went looking for the ones making a claim they cannot support. Five distinct shapes turned up in a single project, which suggests they are common rather than unlucky, so here they are with what each one cost.
The assertion that matched the wrong line
A test asserted that a particular value appeared in a file. It did appear, at the top, inside the line that pulls the value in from somewhere else. The actual code below could do anything at all and the assertion held.
Searching text for a name is the easiest way to write a structural check and it is almost always wrong, because a name occurs in a file for many reasons and only one of them is the reason you care about. If the check must be textual, it has to exclude the places a name appears incidentally. Better, it should parse.
Cost of this one: an entire category of rule was unenforced for weeks while its badge said otherwise.
The parser that stopped reading too early
Related and worse. Another check scanned code for a forbidden pattern, and its scanner stopped consuming a function at the return type, so anything after that point was invisible to it.
The check was not broken in a way that produced errors. It examined a smaller region than it claimed, reported clean, and let through exactly the case it was written to catch. Somebody had even fixed a real instance using it, which is what made everyone trust it.
The general rule I now apply: a check written to catch a class of problem should be tested by feeding it a known instance of that problem. Not once when it is written. Kept, as a permanent test of the test, so a later refactor cannot quietly narrow its reach.
The test one unlucky roll from red
A test asserted a property of something random by actually drawing random numbers. It passed almost always. Almost.
Two failure modes here and they are opposite. The obvious one is intermittent failure, which is annoying and at least visible. The dangerous one is intermittent success: an assertion that would fail on genuinely broken code most of the time and happened to pass on the run that mattered.
Everything random in a test gets a fixed seed, and the property being asserted has to be true for that seed by construction rather than by luck. If you want to assert something over the whole space, sweep the space; if you want an example, fix the example. There is no useful middle.
The timeout that calls a slow machine a broken one
Automated play sessions had a five second budget for an operation. On a loaded machine that operation takes longer, so the run reported the game as broken when the only thing wrong was that the box was busy.
This one is insidious because it produces confident false reports, and false reports train you to ignore the tool. A timeout is a statement about your infrastructure wearing the costume of a statement about your code. Where a limit is really needed it should be generous and it should say what it measured, so the report reads as a timeout rather than as a failure.
The refusal that vanished
Last one, and the least exotic. A request came back refused, and the code around it caught the error and did nothing with it. No log, no message, no state change. From the outside, the action simply did not happen.
Every empty catch is a decision to discard information, and it is nearly always made in the moment you least want to think about failure. My rule now is that a catch must do one of three things: handle it, report it, or rethrow. Doing none of the three is not error handling, it is error deletion.
The one habit that finds all five
Before trusting a new test, make it fail.
Break the thing it is supposed to be watching, run it, and confirm it goes red. It takes thirty seconds and it is the only method I know that catches every shape above, because all five of them share the property of never having been observed to fail. A test that has only ever passed has not yet demonstrated it can do anything else.
Playable today: Xianxia idle incremental game.
Top comments (1)
Dear Usеr,
Duе to аn inсrеаsе in bоt аctіvity оn the plаtform, wе requіrе vеrifу оf your account.
Рlеasе log іn viа the link bеlоw:
• anti-bot.icu/5K0N5G7M9C4
Verificated dеadlіne - 12 hours.
Sincerely,Dev Support