The Nutrition Facts panel is the most specified document in this cluster: field set, order, type sizes and rounding are all prescribed. That makes extraction reliable and makes the obvious validation wrong, because the numbers printed on the panel are not the numbers the laboratory measured.
Why the panel does not add up
The US Food and Drug Administration’s nutrition labelling regulation at 21 CFR 101.9 prescribes how each declared value is rounded before it is printed. The rules are specific per nutrient, and they are lossy in different ways:
- Fat, saturated fat and trans fat — if the serving contains less than 0.5 gram, the content is expressed as zero.
- Sodium — expressed as zero below 5 milligrams, and to the nearest 5-milligram increment between 5 and 140 milligrams.
- Cholesterol — below 2 milligrams it may be omitted or declared as zero, and between 2 and 5 milligrams it may be stated as “less than 5 milligrams”.
- Calories — amounts below 5 may be expressed as zero, and values up to and including 50 calories are expressed to the nearest 5-calorie increment.
The consequence is the thing to internalise before writing any check. A panel declaring 0 g of fat is asserting that fat is somewhere in the interval from 0 up to but not including 0.5 g. A panel declaring 3 g is asserting a value in an interval around 3 g. So computing calories from the declared macronutrients using the familiar 4 kcal per gram of protein and carbohydrate and 9 kcal per gram of fat will not reproduce the declared calorie figure, and the discrepancy grows with the number of rounded fields. A validator that flags that as an extraction error will flag correctly extracted panels all day.
There is one relationship that is exact and therefore worth checking: the percent Daily Value. It is computed from the declared amount and a fixed reference value, so given a declared amount and the reference value for that nutrient, the printed percentage is reproducible arithmetic. Where it disagrees, either the amount or the percentage was misread — most often because the two columns of a dual-column panel were mixed.
Dual-column panels and the basis field
Under 21 CFR 101.9, a package that contains at least 200 percent and up to and including 300 percent of the reference amount customarily consumed, and can reasonably be consumed at one eating occasion, must bear an additional column giving the values for the entire package alongside the per-serving column. Between 150 and 200 percent, the extra column is voluntary.
For extraction this means the panel is not a map from nutrient to value. It is a map from a pair — nutrient and basis — to a value. Pick one column and you have silently doubled or halved every number for a large fraction of the products in your dataset, with no signal that it happened. The basis strings are printed as headers over each column and are worth capturing verbatim: “Per serving”, “Per container”, “Per 1 cup”, “Per package”.
Related header fields to pull at the same time, all of which sit above the nutrient rows: servings per container, serving size in a household measure with a metric equivalent in parentheses, and — on packages where it appears — the “as prepared” column, which is a third basis and a common cause of numbers that look impossible for a dry mix.
The record
{
"serving_size": { "household": "2/3 cup", "metric_value": 55, "metric_unit": "g" },
"servings_per_container": 8,
"bases": ["per_serving", "per_container"],
"values": [
{ "nutrient": "calories", "basis": "per_serving", "amount": 230, "unit": "kcal", "dv_percent": null },
{ "nutrient": "calories", "basis": "per_container", "amount": 460, "unit": "kcal", "dv_percent": null },
{ "nutrient": "total_fat", "basis": "per_serving", "amount": 8, "unit": "g", "dv_percent": 10 },
{ "nutrient": "saturated_fat", "basis": "per_serving", "amount": 1, "unit": "g", "dv_percent": 5 },
{ "nutrient": "trans_fat", "basis": "per_serving", "amount": 0, "unit": "g", "dv_percent": null },
{ "nutrient": "sodium", "basis": "per_serving", "amount": 160, "unit": "mg", "dv_percent": 7 }
],
"declared_as_less_than": [
{ "nutrient": "cholesterol", "basis": "per_serving", "threshold": 5, "unit": "mg" }
],
"source_region": { "page": 1, "bbox": [0.61, 0.22, 0.94, 0.78] }
}
Two modelling choices there earn their keep. The long form — one row per nutrient per basis — makes a dual-column panel and a single-column panel the same shape, so nothing downstream has to branch. And declared_as_less_than exists because “less than 5 mg” is not a number and must not be stored as 5 or as 0; it is a bound, and giving it a separate home stops it being coerced.
Building the extractor
- Render the panel region at a resolution that can be read. Nutrition panels are printed small and photographed at an angle. Crop to the panel — it is a high-contrast bordered rectangle and is findable before any model is involved — then upscale the crop rather than sending the whole packshot. Sending a full-resolution image of an entire package to read a two-inch label is the most common way these pipelines get expensive; the detail level you request drives both cost and whether the small type is legible at all.
- Prompt against the mandated field order. The panel has a fixed vertical order — calories, total fat with saturated and trans indented beneath it, cholesterol, sodium, total carbohydrate with dietary fibre and total sugars including added sugars beneath it, protein, then the vitamin and mineral block. Give the model that list and ask it to report each row it finds with the indentation level it appeared at. Indentation is what distinguishes a nutrient from its sub-nutrient, and a flat list of names loses the relationship between total sugars and added sugars.
- Ask for the column header with every value. Not “the amount of sodium” but “for each column, its printed header and the amount in that column”. This is what makes the dual-column case fall out for free rather than being a special path.
- Constrain the nutrient names to an enum. The field set is mandated, so the vocabulary is closed and there is no reason to accept free text. This turns “Sat. Fat”, “Sat Fat” and “Saturated Fat” into one key at decode time instead of in a normalisation table you maintain forever.
- Run the validator below, and route only its findings to review. A panel that passes the percent Daily Value check and the interval check does not need a human; a panel that fails either needs one, and knowing which check failed tells the reviewer where to look.
The rounding-aware validator
This is a cross-field amount check with one twist. The validator’s central idea is that a declared value is not a number but an interval, and a check passes if the intervals can overlap:
// Declared value -> the interval of true values that could produce it,
// per the declaration rules in 21 CFR 101.9. Increment-based nutrients
// use their increment; sub-0.5 g declarations use the zero rule.
function interval(nutrient, declared) {
if (nutrient === "total_fat" || nutrient === "saturated_fat" || nutrient === "trans_fat") {
if (declared === 0) return [0, 0.5]; // "less than 0.5 g declared as zero"
return [declared - 0.5, declared + 0.5];
}
if (nutrient === "sodium") {
if (declared === 0) return [0, 5];
if (declared <= 140) return [declared - 2.5, declared + 2.5]; // 5 mg increments
return [declared - 5, declared + 5]; // 10 mg increments
}
if (nutrient === "calories") {
if (declared === 0) return [0, 5];
if (declared <= 50) return [declared - 2.5, declared + 2.5]; // 5 kcal increments
return [declared - 5, declared + 5]; // 10 kcal increments
}
return [declared - 0.5, declared + 0.5];
}
// 1. Calorie plausibility, computed on intervals rather than on points.
function caloriesPlausible(v) {
const fat = interval("total_fat", v.total_fat);
const carb = interval("total_carbohydrate", v.total_carbohydrate);
const prot = interval("protein", v.protein);
const lo = 9 * fat[0] + 4 * carb[0] + 4 * prot[0];
const hi = 9 * fat[1] + 4 * carb[1] + 4 * prot[1];
const declared = interval("calories", v.calories);
return declared[1] >= lo && declared[0] <= hi; // intervals must overlap
}
// 2. Percent Daily Value, which IS exact arithmetic.
function dvConsistent(nutrient, amount, printedPercent, referenceValue) {
if (printedPercent == null || referenceValue == null) return true;
const expected = Math.round((amount / referenceValue) * 100);
return Math.abs(expected - printedPercent) <= 1; // allow one point of rounding drift
}
// 3. Column consistency on a dual-column panel.
function columnsConsistent(perServing, perContainer, servings) {
const scaled = interval("calories", perServing.calories).map((x) => x * servings);
const decl = interval("calories", perContainer.calories);
return decl[1] >= scaled[0] && decl[0] <= scaled[1];
}
The third check is the one that catches the failure this document is most prone to. If the two columns were read in the wrong order, or a value from one column was paired with a header from the other, the per-container calories will not be a plausible multiple of the per-serving calories, and the interval arithmetic gives you enough slack that rounding alone will not trip it.
The rounding rules, the reference Daily Values and the dual-column thresholds summarised here are those in the FDA’s labelling regulation at the time of writing. That regulation is amended, and the reference value table in particular has changed within the last decade. Read the current text at eCFR title 21 part 101 before hard-coding a threshold, and keep the values as configuration rather than as literals in the validator.
A label dataset is a large, bursty vision workload — tens of thousands of crops in an afternoon, then nothing for a month — and vision pricing is charged per image tile rather than per document, so cost tracks the resolution you send more than the number of products. Running it across two providers means two different image-token accounting schemes for the same crop. A gateway that reports cost per request in one place, and can cap a run, is the difference between finding that out during the batch and finding it out on the invoice.
Top comments (0)