My negative test stopped being negative when a config file grew by four rows

A test whose entire job was to prove a rule gets refused had quietly started proving that the rule passes.

It plants a violation and asserts the checker rejects it. It had been green for weeks. It was green because the assertion had inverted, and nothing about a green test tells you which side of the assertion it is standing on.

expected:  refusal (the planted row breaks monotonicity)
actual:    OK_K3_MONOTONE steps=11 regress=0
result:    the test now fails, correctly, for the first time in weeks

What happened

The test planted its contradiction at a fixed position, chosen because that position was the last row of a config table when the test was written.

The table had seven rows then. Another change grew it to ten. A later one grew it to eleven.

The planted violation was no longer last. The rows that came after it re-established the property the violation was supposed to break. The checker looked at the whole sequence and correctly said it was fine. The test asserts a refusal, and no refusal arrived.

Nothing was broken by that. The checker was right. The config was right. The test was measuring a scenario that had stopped existing.

Why nobody noticed

The changes that grew the table ran their own module's tests and the shared control. They did not run this one, because there was no visible relationship between "add rows to a config table" and "a test in another module about ordering".

That relationship existed, and it was invisible precisely because it was a literal:

// what it did
let planted = rows[7];          // "the last row", in the world where there were 8

// what it does now
let planted = rows.last();      // read from the config, whatever length it is

A literal copied out of a config file is a dependency with no edge in any graph. The build does not know about it. The type system does not know about it. A grep for the config's name does not find it, because the file is never mentioned. Only one number from it is.

The general shape

A negative test has two failure modes and they are not symmetric.

failure	what you see	how long it survives
it stops passing	red	until someone fixes it, usually within a day
it stops being negative	green	until someone reads it

The second one is worse in every way that matters. It takes the exact form of success, it is produced by a test that runs and exercises real code, and coverage tooling counts it as covered. The only thing it no longer does is fail when the thing it guards regresses.

Two repairs, one of which generalises

Derive the position instead of writing it. The planted violation now goes after the last declared row, read from the config at test time. The scenario cannot drift out from under the assertion because the assertion no longer names a coordinate.

Census the test files for literal fragments of config. That is the part worth stealing. If a test embeds a value that lives authoritatively somewhere else, that is a silent coupling, and a sweep for those is cheap:

$ # every numeric literal in tests that also appears as a row count in config
$ ... | wc -l
3

Three, in that codebase, and this was one of them. The other two are latent: correct today, in the same way this one was correct for weeks.

What I would check in your negative tests

Does the failure it plants get its position, its size, or its identity from a literal? If yes, something else owns that value and can change it without ever touching the test.

And: has this test ever failed? A negative test that has never once gone red has never demonstrated that it can. Flip its assertion deliberately, watch it fail, flip it back. That takes a minute and is the only evidence that the green means anything.