DEV Community

Daniel Pertu
Daniel Pertu

Posted on

The quality gates we deleted from our user submitted data

#architecture #webdev #typescript #security

Munchable scans a barcode and tells you whether the product suits your gut condition. When we do not have the product, the app offers to read the label from a photo, and that capture becomes a row everybody else's scan will hit.

So the ingest path is the most hostile surface we own. It takes a photograph from a phone, runs it through optical character recognition, and writes the result into the catalog that answers other people's health questions. We spent a fair amount of effort putting quality gates on it.

Then we deleted two of them, and the product got better.

The gates we removed

A lexicon hit rate floor. The read had to be at least 80 percent words we recognised, or the capture was refused. The intent was obvious: a photo of a car park should not become a product.

An OCR confidence floor. Below 0.75 the capture was refused as "not clear enough".

Both are the kind of check that looks responsible in review and is quietly awful in the aisle. Picture the person who hit them. They stood in a shop, scanned something we did not have, agreed to help, framed the label, took the photo, and got told the photo was not good enough. They did everything right. They are now holding a phone that has just refused their contribution, and Munchable still does not have that product, so the next person to scan it also gets nothing.

Worse, the words are the thing the floor was measuring. A pan-European pack, a regional brand, an ingredient our lexicon has not learned yet: all of these read perfectly and score badly. The floor was not detecting bad photographs. It was detecting products we did not know about, which is precisely the population the capture feature exists to serve.

Why it was safe to delete them

Deleting a gate is only safe if nothing downstream was depending on it. This is the part worth copying, rather than the specific numbers.

Trust in that row never came from the floor. It came from three things that are still there:

  1. A capture enters as unverified. It is not treated as settled data on the strength of one person's photo.
  2. It needs a second independent read to agree before it counts as consensus. Agreement between two reads is evidence. One read plus a high confidence score is not.
  3. The engine fails closed on the phone. If the words in a row are ones the rules cannot interpret, the app says so and asks for a label photo rather than inventing a verdict. There is no path where "we could not read this" turns into a reassuring green tick.

Given all three, the confidence number was never load bearing. We still record it, because it is genuinely useful for deciding what to curate next, but it decides nothing at ingest. A read the reader itself doubted enters with its honest doubt attached.

The lesson generalises past label photos: a quality gate is only doing work if removing it would let something bad through. If the layers behind it already refuse to act on weak data, the gate is not protecting your users, it is rejecting your contributors.

What the gates are actually for now

The checks that remain are not about quality at all. They are about plausibility and poisoning, which are different jobs.

Plausibility is cheap and boring, and that is the point. Grams per 100 g have to be between 0 and 100. A serving is between 1 and 500 grams. An ingredient statement is a few thousand characters at most, tokenizes into a sane number of tokens, and is not a base64 image someone pasted into a text field. A missing number is completely fine and is stored as missing, because the engine already knows how to fail closed on a condition whose input is absent. A nonsense number is refused, because a nonsense number is worse than nothing: it silently answers a question it has no right to answer.

grams per 100 g      0 .. 100      absent is fine, out of range is not
serving size         1 .. 500 g
ingredient text      short, tokenizes, not an image blob
name and brand       plain label, no links, no markup, hard length cap
Enter fullscreen mode Exit fullscreen mode

Poisoning is the interesting one. Every later scanner of that barcode sees the product name the contributor typed, so a free-text field that reaches strangers gets the treatment any such field deserves: control characters stripped, whitespace collapsed, links and email addresses refused outright, angle brackets refused, hard length cap. A rejected name is dropped rather than fatal. The ingredients were the contribution; the name is a convenience.

The structural version of the same rule matters more. The ingredient tags every future scan reads are derived from the label text and from nothing else the client sends. A client can send us a photograph and a transcription. It cannot send us the tags. That single constraint is what keeps a crafted request from reaching into the catalog and making a product look safe for someone with inflammatory bowel disease.

The refusal that cannot be repaired

One gate is deliberately strict, and it is the allergen line.

If a label prints "Contains: milk, soya", that line is parsed into warnings that every later scanner of the product sees, and it is bound into the receipt the reader issues for the read. The client has to echo it back byte for byte.

That means the reader and the gate must agree exactly on what a valid line is. If the reader emits a line and the gate then refuses it, the product becomes uncontributable forever: trimming the line breaks the receipt, keeping it fails the gate, and retaking the photo reads the same words again. So both sides call the same function, and the gate refuses a bad line outright instead of quietly tidying it up. Anything that would pass after repair is a line whose repaired form was never on the label.

Allergens are the one place in Munchable where the code is allowed to be less helpful in exchange for being unambiguous.

See the pieces

If you run a user contribution path, the audit is short. List your gates, and for each one ask what would actually get through if you deleted it tomorrow. The ones with no answer are costing you contributions and buying you nothing.

Top comments (0)