Knowing a word is not understanding it: the bug that made our app say safe

Here is a bug that shipped, that every test passed, that no error tracker would ever catch, and that told a lactose-intolerant user a bottle of kefir was fine.

It is my favourite kind of bug, because nothing was broken. Two correct systems grew at different speeds, and the gap between them turned into a confident wrong answer.

The setup

Munchable reads a food label and gives a verdict for your gut condition. Two separate things have to happen for each ingredient:

1. Naming. Turn the word on the label into a canonical id. "cukier" is sugar. "E471" is mono- and diglycerides of fatty acids. "comflour" is an OCR error for cornflour.
2. Scoring. Look that id up in the rule map for your condition and find out whether it means anything for you.

Both worked. They were fed by completely different processes, and that was the problem.

The naming taxonomy was grown by a nightly automated job. It got bigger every single night, without anybody doing anything.

The scoring rule maps were hand-checked data compiled into the app. They only got bigger when we shipped a release.

One grew continuously. The other grew in discrete jumps, when a human had time. Give that a few months.

The engine ended up able to name roughly 7,000 ingredients and score about 1,600 of them.

Why that gap was worse than useless

If the gap had merely made us unhelpful, this post would not exist. Being unable to score an ingredient is a fine thing to admit.

The problem was the confidence calculation. It worked like this, and read it slowly because it sounds right:

Confidence goes down when the engine finds words on a label it cannot place.

Which is perfectly sensible. An unrecognised word is genuinely a reason to be less sure. If half a label is gibberish, you should not be confident about anything.

Now run a label through it where every single ingredient was in the 7,000 and none were in the 1,600.

The engine names every word. Nothing is unplaceable. Confidence goes up. No rule fires, because none of those ingredients are in any rule map, so there is nothing to warn about. And the app renders a confident green badge.

Naming an ingredient raised our confidence that the product was safe, when all it actually proved was that we knew how to spell it.

The canonical case: kefir, for someone who is lactose intolerant. We knew exactly what kefir was. It was in the taxonomy, correctly placed under dairy, with a tidy canonical id. There was just no lactose rule on it. So the engine looked at a fermented milk drink, recognised every word, found nothing to complain about, and said: good fit.

The root cause is a conflated boolean

Strip away the domain and the bug is this: two genuinely different states were being represented by the same one.

• "We have not looked at this yet."
• "We looked at this and there is nothing to say."

In the data, both of them looked like absence. No row in the rule map. And the confidence code interpreted absence as the second one, which is the reassuring one, when in reality it was mostly the first.

Any time absence carries meaning in your data model, check whether it is carrying two meanings. This is the same reason a nullable boolean is a code smell and the same reason HTTP 200 with an empty body is a bad API. Silence is ambiguous, and systems reliably resolve ambiguity in whichever direction is least alarming.

Fix one: measure the two things separately

You cannot manage a number you do not have. So we started recording both coverages, as a side effect of the jobs that were already scanning the corpus:

export type CoverageKind = 'naming' | 'scoring';

export interface CoverageSample {
  /** ISO time the mining job took the sample. */
  at: string;
  /** Distinct tags on servable labels the job looked at. */
  seen: number;
  /** The gap: tags the engine cannot name (naming) or cannot score (scoring). */
  gap: number;
  /** 1 - gap / seen, rounded to four places. */
  coverage: number;
  /** Rows the job queued for curation. */
  queued: number;
}

Two kinds, one latest value each and a capped 52-sample history in Redis. No migration, nothing to prune, and the trend is visible instead of vanishing into a log line.

Before this, "coverage" was one number, and it was the flattering one. Splitting it in two made the actual state of the product impossible to look away from.

The failure handling is deliberate too:

export async function recordCoverage(kind: CoverageKind, sample: CoverageSample): Promise<void> {
  if (!redis) return;
  try {
    // ...
  } catch {
    // Observability must never fail the job that feeds it.
  }
}

Your metrics pipeline going down should not take out the pipeline it measures.

Fix two: make scoring grow on its own schedule

The structural fix was to stop the two systems growing at different rates. We built a second curation job whose only question is the one that decides a verdict.

The naming job asks: what is this word?

The scoring job asks: given that we know exactly what this is, does any condition rule have anything to say about it?

That job runs nightly too, against the same backlog, ranked by how often people actually scan the thing. The gap still exists, because it always will, but it now closes automatically instead of waiting for somebody to remember.

Fix three: say "reviewed and clear" out loud

The last piece is to stop absence from being ambiguous. Ingredients that a human has genuinely looked at and cleared are marked as cleared, explicitly, rather than just having no rule. Water, salt and sugar are in that set. The engine knows the difference between "no rule because nobody has looked" and "no rule because somebody looked and there was nothing to say", and only the second one is allowed to increase confidence.

The lesson worth taking away

Confidence must be computed from the thing you are actually claiming.

We were claiming "this product is a good fit for you", and computing confidence from "we recognise the words on this label". Those feel adjacent. They are not the same claim, and the distance between them was an entire class of false green.

If any part of your product presents a confidence score, a health indicator or a green badge, go and read what feeds it. Not the docstring, the code. Ask what it would take to make that number high while the underlying answer is wrong. If you can construct that input in your head, someone can construct it by accident, and a confident wrong answer is strictly worse than an honest "I do not know".

You can see where we landed at munchable.app, and the lactose condition page spells out exactly what the engine checks and what it will not claim. The free tier takes no card, and pointing it at something in your own fridge is a fairer test than anything we could stage.