A generated test contains an assertion. An automated audit cites the exact
line that supports its verdict. Both artifacts clear their gates, and neither
gate has established the thing you actually wanted: that the assertion would
catch a defect, or that the cited line had anything to do with the conclusion.
What the gate recognizes is the appearance of evidence. A generator produces
that appearance as a byproduct of producing the artifact, which is the whole
problem in one sentence.
The property being measured is structural. The property being inferred is
behavioral. An assertion gets read as evidence of detection, a citation as
evidence of dependence, and a touched file as evidence that the defect was
addressed. Each of those inferences was reasonable when the cost of writing
the structure was correlated with the cost of doing the work. That
correlation is what a generator breaks. Emitting the required structure costs
essentially nothing next to emitting the surrounding text, so the moment the
predicate is known, satisfying it becomes one more formatting constraint on
the output.
Watch what happens when you tighten such a gate. Reject tests with no
assertion, and you get an assertion that always passes. Require the assertion
to reference the value under test, and you get a comparison that faithfully
reproduces the implementation's own mistake, or one that checks a property
nobody cares about. Every additional clause you bolt on makes the artifact
look more convincing without making its success depend on the work you wanted
done.
That is the saturation pattern, and its early behavior is what makes it
dangerous. A new gate's rejection rate looks informative for a while, because
the requirement has not yet entered the generation loop. Once it does, the
pass rate climbs and now measures fluency at satisfying the predicate. Nothing
visibly breaks. The gate still runs, still reports, still shows green. Only
its meaning has expired, silently, and there is no line in any log where that
happened.
The same shape applies everywhere the check is syntactic. A patch that must
touch a named file is satisfied by a harmless edit inside that file. A summary
that must mention six fields will mention all six while describing the state
incorrectly. Making the required format more specific does very little when
the missing property lives outside the format entirely.
None of this makes shape checks useless. A parser needs well-formed input and
downstream code needs its fields present, so keep them as cheap
preconditions. The failure is interpretive: it starts the moment a passing
format check gets reported as evidence that substantive work occurred.
Evidence has to be able to change the result
Suppose a review rejects a change and names a configuration value as the
reason. You can verify that the value exists where the review says it exists.
That confirms the reference is accurate and leaves the claimed dependency
completely untested.
A check with teeth intervenes on the value instead. Keep the file well-formed,
alter the cited value in a way that ought to matter, and run the review again.
If the verdict really rests on that value, an appropriate change should move
the verdict or the score behind it.
The intervention needs a prediction attached before you run it, or almost any
observed difference can be narrated as a success afterwards. Nudging a limit
from one acceptable number to another acceptable number legitimately leaves
the decision alone. Crossing the specific boundary the explanation named is
the test worth running, and what the explanation implies should happen has to
be written down first.
Read the response at the right granularity, too. If the review said a
configured limit exceeds the allowed maximum, lowering it below that maximum
should retire that reason. The overall verdict may well stay negative because
some unrelated problem is still there. Looking only at the final label would
hide a perfectly real local response, so check whether the cited reason
disappeared rather than whether the answer flipped.
A verdict that survives a relevant, well-formed perturbation of its own stated
evidence, with other sufficient reasons held constant, has not demonstrated
that it used that evidence. The likely story is that the reference got
attached after the decision was already settled somewhere else.
One rerun proves none of this. Model outputs move between executions on
identical inputs, so a single observed difference might be ordinary variance,
and a single unchanged verdict might sit near a boundary where a real
dependency fails to show. The floor is one extra execution per anchor. Honest
measurement costs more than that: repeated runs on the untouched input to
establish how much the verdict wanders on its own, then repeated runs on the
altered input to see whether the intervention moves anything beyond that
wander. If the check is expensive, this is expensive, and pretending otherwise
helps nobody.
Perturbation quality matters as much as repetition. Delete a whole paragraph
and the review may be reacting to missing context. Corrupt a record and
validation fires before the reasoning you meant to probe ever runs. Both of
those executions measure sensitivity to damage, which is not the question. A
useful intervention leaves the input's ordinary structure intact and changes
only the specific fact the explanation named. Pair it with an edit of similar
size somewhere irrelevant as a control. When both edits move the verdict by
about the same amount, what you are watching is general instability wearing
the costume of evidence.
One more constraint, easy to miss. The rerun result has to reach the gate
through a path the generator cannot substitute for. Ask the generator to
report what changed after the perturbation and you have merely commissioned a
second artifact whose shape can satisfy the request as cheaply as the first
one did.
Layers in a score need actual decision power
The numeric version of decorative evidence hides inside composed scores.
A decision system combines several signals and describes each one as a
protective layer. Every component computes a value, every value enters the
formula, and the arithmetic can still leave most of them incapable of
affecting the outcome on their own. Take a deliberately simple weighting:
score = 0.8 * primary + 0.1 * secondary + 0.1 * tertiary
reject = score >= 0.7
With binary inputs, primary at zero caps the score at 0.2, and primary
at one floors it at 0.8. The rejection decision is the primary signal, on
every input the system can produce. The other two move the number and decide
nothing. Reporting that arrangement as three layers of protection confuses
participation in the arithmetic with influence over the result, and the
inflated layer count then shows up in coverage claims while one component
quietly carries the whole thing.
This one is cheap to check, because it is arithmetic rather than judgment.
Hold the other components fixed, sweep each component across its permitted
range, and ask whether it can cross the threshold alone. Enumerate if the
inputs are discrete and bounded. For continuous inputs, a bound on the maximum
contribution often settles it immediately.
Interpret the answer carefully. A component that can flip the decision only
while another signal is already elevated has conditional influence, which is
worth something and is not the same claim as catching a failure
independently. Those two deserve different words in whatever report a reader
will act on. A component that cannot change any reachable decision deserves no
credit as a gate at all, and reachability is the operative word: a threshold
crossing that requires an input combination the system never produces offers
no protection.
A shape rule bends the artifact toward the metric's blind spot
Grading shape does something else, beyond failing to measure what you wanted.
It puts a gradient on the artifact.
Consider a rule that flags a test as weak when it swallows exceptions in a
bare catch. The motivation is sound. A test that suppresses failures can pass
when it should fail:
try:
exercise_operation()
catch:
pass
Deleting the catch clears the flag. Narrowing it to one exception category
clears the flag too. Both edits change what the test tolerates, and whether
either counts as an improvement depends entirely on what the test was supposed
to establish. If the exception represents a genuine failure, letting it escape
improves detection. If that exception is the expected outcome for this input,
deleting the handler converts correct behavior into a red test. From the
catch's shape alone, the rule cannot tell those apart, and the cheapest
satisfying edit is chosen by the predicate rather than by the intent. A
compliance rate that climbs afterwards is reporting movement toward the
metric's preferred form, while the meaning of the tests drifts somewhere the
metric has no view of.
The gate that would actually help exercises the distinction the test claims to
enforce. Feed it behavior that satisfies the requirement and behavior that
violates it, then check that the test accepts the first and fails on the
second. For exception handling, that means running both the expected exception
path and the unexpected failure the test exists to expose. Choosing those
inputs takes thought, since a trivially injected crash demonstrates only that
the test can crash. The injected behavior has to resemble the defect the
assertion was written to catch.
Which gives a concrete rule for designing any of these gates: state what must
happen during execution for a pass to mean anything. An evidence check needs a
verdict that responds appropriately to a controlled change. A test-quality
check needs demonstrated discrimination between behavior it should accept and
behavior it should reject. Shape predicates can stay where they are, as cheap
preconditions carrying only the meaning they actually earn.
So there is one question worth putting to any gate that stands in front of
generated work. What would passing it cost a generator that did none of the
work? When the answer is another rewrite, the gate is decorative.
Top comments (1)
Really sharp framing: a gate that only recognizes the appearance of evidence teaches the generator to manufacture appearances. I hit this on a side project where a session bridge verifies 'logged in' by checking the DOM, and a stale cookie jar still rendered the login form correctly. The check passed, the agent was not actually authenticated. That is exactly your 'the assertion would catch a defect' gap in miniature. What ended up working for me was an active probe instead of a passive assertion: perform a write that only succeeds if the claimed state is real. Cheap to design, and it forces the system to demonstrate the behavior the gate claims to protect.