60% of my LLM backlog was never a question for an LLM

We run a nightly job that takes words scraped off food labels and works out what ingredient each one actually is. The backlog is thousands of rows and the obvious implementation is a loop with a model call in it.

Then we measured what was actually in the backlog, and roughly 60 percent of it was never a question for a model in the first place.

Measured on the open backlog in September 2026:

• 13 percent was label boilerplate. "may contain", "best before", phone numbers, storage instructions.
• 17 percent was a word our own multilingual lexicon already knew, filed under the wrong language prefix. en:cukier is Polish for sugar. en:stabilisator is a stabiliser. We already had both.
• 4 percent was a known ingredient wearing a modifier. "madagascan vanilla extract", "kibbled onion".
• 21 percent was an English compound whose head noun we already knew. "roast chicken stock", "vimto flavouring".

None of those needed a model. They needed string handling and a lookup table we already had in memory.

The layer stack

Before anything reaches a model, a candidate word falls through a chain of pure functions. The first one that answers wins.

overlay      the engine already knows this word (canonical id, alias, or noise)
header       "ingredients: wheat flour" glued into one tag
boilerplate  curated multilingual patterns for text that is not an ingredient
e-number     "150d" -> E150d
lexicon      the bare word through the synonym table
singular     the same, after stripping a plural ending
modifiers    the same, after dropping "dried", "organic", "madagascan"
ocr          one letter off from exactly one known name: "sait", "comflour"
head-noun    English compounds: longest known suffix is the head

The whole thing is a pure module with no database and no network. It runs in microseconds, it is trivially unit testable, and it is free. The model is what happens to the remainder.

Two of these are worth expanding on.

The OCR layer

Our users photograph labels, so the text arrives via OCR, so the text arrives slightly wrong. sait for salt. comflour for cornflour. rapeseed oll for rapeseed oil.

The layer accepts a candidate that is edit distance one from exactly one known English ingredient name. The "exactly one" is the entire safety property. If a typo is one letter away from two different ingredients, we do not pick the more popular one, we give up and pass it on. A confident wrong guess on an ingredient name is how somebody with a dairy allergy gets told a product is fine.

The head-noun layer

English compounds put the head noun last. "chicken stock" is a stock. "carrot puree" is a puree. So take the longest known suffix as the head, then look at what is in front of it:

• If the prefix is itself a known ingredient, create a genuinely new id sitting under both. "carrot puree" becomes a child of carrot and puree. This is what makes a FODMAP rule on carrot reach carrot puree automatically, which is the whole point.
• If the prefix is only modifiers or unknown words, it is an alias to the head. That is the brand case: "Vimto flavouring" is a flavouring.

This one layer handled 21 percent of the backlog.

The decision we are proudest of: no hardcoded brand list

There are thousands of supermarket brands and retailer own-labels in the data. The obvious move is a list.

We deliberately did not write one.

There is deliberately NO hardcoded list of brands or retailers. A brand name
the layers cannot place goes to the model once, the model files it as noise,
and the overlay answers it for free from then on: the database learns the name
permanently, where a list in here only saved the first sighting and had to be
edited and deployed to grow.

Think about the economics. A hardcoded list saves you exactly one model call per brand, the first time you ever see it, and in exchange you own a list forever that requires a code change and a deploy to extend.

Letting the unknown word go to the model once and recording the answer costs you that same single call, and after that the top layer, the overlay, answers it for free. The data learns. The code does not grow. Coverage improves while you are asleep instead of when somebody remembers to open a pull request.

This generalises well beyond ingredient names: when you are tempted to hardcode a lookup table to avoid an expensive call, check whether you could cache the expensive call's answer instead. The cache version usually has the same cost profile and does not need a deploy to grow.

The guard that makes the fast path safe

Here is where it would be very easy to build something dangerous.

The deterministic layers are just string manipulation. They have no idea what food is. So they do not get to skip the safety checks. Every resolution they propose goes through the same validation as a model answer: the same lineage plausibility checks, the same health guards, the same engine-level overlay validation.

With one deliberate difference:

the proposal's `english_name` is the SOURCE phrase, so the hidden-ingredient
scan sees every word of it. "fully refined soya bean oil" can therefore never
become an alias to `en:oil`: the guard finds soya, and refuses to drop it out
of the rule graph.

That example is the whole reason the rule exists. The modifier layer looks at "fully refined soya bean oil", strips words it considers decoration, and lands on oil. As a piece of string processing that is completely reasonable. As a piece of food data it is a disaster, because you have just taken a soya product and filed it somewhere no soya rule will ever fire on it.

By keeping the full source phrase attached to the proposal, the hidden-ingredient scan reads the actual words, finds "soya", and refuses. The fast path proposes. It does not decide. (That is the same principle as the model itself: AI proposes, the engine disposes.)

There is one narrow exception. The boilerplate layer is allowed to override the hidden-ingredient guard, because "may contain nuts" contains the word "nuts" and is precisely not a declaration that the product contains nuts. A pattern has to be that unambiguous to earn the waiver, and they are curated by hand.

What this actually bought us

• Roughly 60 percent of the backlog resolved with no model call.
• Every one of those resolutions is deterministic, reproducible and unit tested.
• The model handles the genuinely ambiguous remainder, where it is worth paying for.
• The same layers run on the request path too, so a label photographed right now gets the free answers immediately rather than waiting for the nightly job.

The headline is the cost saving, but the part that matters more is reproducibility. When 60 percent of your pipeline is pure functions, 60 percent of your pipeline gives the same answer every time, can be tested without a network, and never needs a re-run because a model version changed underneath you.

Before you put a model in a loop, print a sample of the loop's input and read it. A surprising amount of what looks like a language problem is a string problem wearing a hat.

The results of all this are public: a few hundred ingredient and condition pages at munchable.app/answers, every verdict generated by the rules engine rather than written by hand. Or scan your own cupboard, which is a considerably better test of ingredient coverage than any benchmark we could write.