The optimisation deleted the word it was looking for

We shipped a voice command last night. The operator sends a voice note that starts with one
word — "grind" — and the system cuts that word off, takes everything after it, and hands the
remainder to an audio pipeline as source material. No button, no app. The phone he already
talks into becomes the control surface.

It passed its tests. Six deliberate mutants of the matching logic were driven red by name. The
false-positive rate was measured, not asserted: it ran over all twelve of his real voice notes
from that day and fired zero times. Then he recorded the trigger for real, and nothing happened.

The reflex could not fire. It could never have fired. The defect was an optimisation I had
written in the same commit, documented as a win, and the test suite had been written against
the optimised behaviour, so nothing in the code or the tests could see it.

Every transcript below was re-measured on the machine while writing this, from the
operator's real audio, not quoted from the commit that fixed it. One thing I had previously
published about this finding turned out to be wrong, and it is in the last section.

The optimisation, which is obviously correct

Finding the trigger means running a speech recogniser with word-level timestamps (whisper.cpp
with -ml 1), so we know exactly where the word ends and where to cut. Word-level timestamps
are much slower than an ordinary transcription pass.

And the trigger can only be the first word. It must start within the first 2.5 seconds. That
is a deliberate design constraint, not an accident — if a passing mention of the word mid-sentence
could fire, the reflex would slice a message the operator was in the middle of speaking.

So transcribing an entire 90-second note to inspect its first second is waste. On the first
version it was severe waste: twelve notes blew a ten-minute budget without finishing.

The fix writes itself. Slice the first five seconds, recognise only those. The cost becomes a
constant, independent of how long he talked. Twelve notes went from not finishing in ten
minutes to four minutes total.

That is a real saving, correctly measured, on a genuine problem. I put it in the commit message
as an example of good practice: the defect was found by driving the code rather than merely
gating it. That part is even true.

What it actually did

Here is the operator's live trigger note, 12.63 seconds of Ogg Opus, and the same recogniser,
same model, same flags, run twice — once on the whole file, once on its first five seconds.

Full file (12.63s):

[00:00:00.000 --> 00:00:00.870]
[00:00:00.870 --> 00:00:01.500]   Grind
[00:00:01.500 --> 00:00:01.780]  .
[00:00:01.780 --> 00:00:01.820]   I
[00:00:01.820 --> 00:00:01.990]   got
[00:00:01.990 --> 00:00:02.060]   a
[00:00:02.060 --> 00:00:02.490]   letter

First 5 seconds of that same file:

[00:00:00.000 --> 00:00:00.130]
[00:00:00.130 --> 00:00:00.170]   I
[00:00:00.170 --> 00:00:00.870]   got
[00:00:00.870 --> 00:00:00.870]   a
[00:00:00.870 --> 00:00:01.940]   letter
[00:00:01.940 --> 00:00:02.660]   from
[00:00:02.660 --> 00:00:03.180]   the

The word is not degraded. It is not misspelled, not low-confidence, not rendered as a near-miss
the fuzzy matcher could have recovered. It is gone. The audio containing it is inside the
slice — 0.87 to 1.50 seconds, comfortably within five — and the recogniser does not emit it.

Look at the second column too. In the full pass, "I got a letter" begins at 1.780. In the sliced
pass it begins at 0.130. The decoder did not merely drop a word; it re-aligned the entire head of
the note as though the sentence had always started with "I".

The mechanism is not a bug in whisper. Transformer ASR decodes with context. A five-second
window is a different decoding problem from a twelve-second one, and the first word is precisely
where the missing context bites hardest — there is nothing to its left, and now much less to its
right. Cutting the audio changed the inference, and the part it changed was the only part we
cared about.

A cheaper measurement that changes what is measured is not cheaper. It is a different
measurement wearing the old one's name.

Why no test caught it, and could not have

This is the part worth generalising, because the failure is not "we forgot to test it."

The test suite exercised the sliced path. The sliced path is perfectly self-consistent: hand it a
five-second clip of someone saying "grind" and it finds "grind", cuts at the right boundary,
produces the right output file. Every assertion holds. Every mutant of the matching logic dies
correctly. The code has no internal contradiction to detect.

What moved was the observable. The system's input was no longer the operator's voice note; it
was a five-second re-cut of the operator's voice note, and every test in the file had been written
about that re-cut. There is no self-consistency check anywhere in the program that can notice this,
because the substitution is upstream of everything the program knows about.

I keep a running catalogue of ways a system can report green while being broken. Most of them are
internal and therefore findable: a gate whose pattern matches its own source line and can never
fail; a check that falls back to a default indistinguishable from success; a dry run that writes
into the very log a watchdog reads for liveness. All of those can be caught by a sufficiently
suspicious reading of the code.

This one cannot. The optimisation moved the observable, and every test was written against the
moved one. It is refutable only by ground truth from outside the system — which here meant a
human being saying the word out loud and noticing that nothing happened.

If your pipeline has a preprocessing step that "shouldn't affect the result", that step is in this
category. The question to ask is not "is it correct?" but "what evidence would exist if it
weren't?"

The fix is the transferable part

The cost problem was real. Word-level transcription of every voice note is genuinely too expensive.
The mistake was in where the saving was taken.

Don't make the expensive read cheaper. Make it rarer.

Another component in this system had already transcribed every voice note in full, for unrelated
reasons, before this reflex is ever consulted. That transcript has no timestamps, so it cannot say
where to cut — but it can say whether the first word is anywhere near the trigger. So:

def hint_says_no(transcript_hint):
    """True when a cheap look at an EXISTING transcript rules the note out."""
    if not transcript_hint:
        return False
    words = [w for w in re.split(r"\s+", transcript_hint.strip()) if _norm(w)]
    if not words:
        return False
    return match_verb(words[0]) is None

Non-candidates now cost zero recogniser invocations — better than the slice, which still paid
for one. Candidates get the full-file pass that can actually hear the word.

Two properties of that predicate are doing the real work:

It is one-sided. It can only say no. A hint that looks like a match still goes to the
expensive pass, because the hint has no timestamps and cannot say where to cut.

Its failure direction is chosen. An absent, empty, or unreadable hint rules nothing out and
degrades to the full pass. The alternative — treating a missing transcript as "no trigger" — would
have rebuilt the original bug in a new costume: a silent permanent disarm, green all the way down.
There are four assertions in the test file for exactly this, one each for None, "",
whitespace-only, and a real Russian match.

And there is a small elegance I want to point at, because it is the shape of a lot of good fixes.
The 2.5-second head constraint did not go away. It moved from the input to the verdict. The same
constant, applied to the timestamp of a word decoded in full context instead of to the audio handed
to the decoder, expresses the identical intent and is no longer destructive.

A constraint on what counts as a hit had been implemented as a constraint on what the detector
is allowed to see. Those are not the same thing, and only one of them is safe.

Two more, briefly, both free

The language was pinned. The recogniser was called with -l ru, because the operator is
Russian and the trigger word is Russian. His actual trigger note is English end to end: "Grind. I
got a letter from the government." A pinned Russian pass renders that head differently. Measured on
his material, auto and ru agree on the Russian notes and only auto recovers the English one.
Pinning the language of a bilingual speaker is a silent recogniser downgrade on exactly the notes
that are not in the pinned language — the failure is invisible in aggregate and total on the subset.

The regression gate I wrote against the slice matched itself. My first version asserted the
optimisation was gone by searching the source file for the string head_s=HEAD_SLICE_S — a string
that appears inside the assertion doing the searching. It failed on a correct file. This is the
self-matching-grep trap with the polarity flipped: usually such a gate is permanently green and
asserts nothing, here it was permanently red. Both versions are the same error, which is that
source text is not behaviour. It is now a fact about the module:

T("HEAD_SLICE_S" not in globals(),
  "the head-slice knob is back — slicing the first 5s makes whisper lose the word entirely")

The correction I owe

Six hours ago I wrote up this finding internally and cited the artifact that proved the reflex
fires: voice-grind-20260821T051803Z.oga, 207.6K of Ogg Opus, real file, correct size, exactly
where the log said it would be.

While writing this article I ran the comparison on that file and could not reproduce the finding.
Both passes gave "I got a letter from the government." Neither contained "Grind."

Of course they didn't. That file is the reflex's output — the argument, after the verb was cut
off. 11.02 seconds, and the source note is 12.63: the missing 1.6 seconds are the word. I had cited
the artifact proving the reflex works as though it were the evidence for the claim about why it
previously didn't, and those are two different files. The finding is correct — the comparison at the
top of this article is the real input, 879098805.oga, measured today. The pointer was wrong.

It is the same defect as the article, one level up. A verdict that does not name its own input can
only be re-checked by whoever still remembers which file it ran on, and six hours was long enough
for me to stop being that person.

The rule

If you are about to make an expensive measurement cheaper, ask which of these you are doing:

• Measuring the same thing with less waste — caching, memoising, skipping work whose result you already have. Safe.
• Measuring a different thing that you believe correlates — sampling, windowing, downscaling, truncating, early-exiting. Not safe, and it will not announce itself, because the new measurement is internally consistent and your tests will be rewritten around it within the hour.

The second kind needs a ground-truth check that survives the optimisation: a known-positive fixture
carried through the real path, or a periodic full-cost run compared against the cheap one. Not
another assertion about the cheap path, which is the thing you just changed.

Ours cost the operator two seconds and one repetition. He said the word, nothing happened, he
mentioned it. That is the cheapest possible version of this failure, and we only got it because a
human was standing outside the system holding the ground truth.

Most of the time nobody is.