I built a rule-based sugar detector and a fine-tuned model to compete against it, ran them against my gold set, and the rule engine scored 100% accuracy. Every metric, perfect.
It took me longer than it should have to notice the actual problem: most of that gold set had been labeled by running the rule engine's own code against the data. I hadn't built an eval. I'd built a mirror.
The two systems
SugarShield catches hidden sugar in ingredient lists: things like "maltodextrin" or "evaporated cane juice" that don't read as sugar unless you already know they are. Two systems do the detecting:
- A deterministic rule engine: a lexicon of 145 aliases across 77 canonical terms, feeding a risk score. Detection and scoring are separate steps on purpose, so finding "milk" (a natural sugar context) never raises risk on its own.
- A fine-tuned model: a LoRA fine-tune of Qwen2.5-1.5B-Instruct, trained on 1,375 records sourced from GroceryDB, a licensed academic dataset of real grocery products.
The build environment couldn't reach Hugging Face Hub, Open Food Facts, Wikipedia, or Kaggle: all four returned a hard 403 from the network egress proxy, confirmed with curl and Python's requests before switching plans rather than assumed. That ruled out pulling a pretrained checkpoint for the original LoRA plan inside that environment, so the training script trains a byte-level BPE tokenizer and a small GPT-2-architecture model from scratch instead, full-parameter, no pretrained weights to adapt. That from-scratch run (5.87M parameters, 6 layers, CPU-only, 607 seconds) is a stand-in for the plan that couldn't execute there, not the model that mattered for the actual decision. The real fine-tune ran separately, on different hardware with real Hugging Face access: genuine LoRA weights on top of genuine pretrained Qwen.
Getting that real fine-tune working surfaced an actual bug worth naming: load_model() never called model.to(device). Training on Apple's MPS backend worked fine, but at inference time the model itself stayed off the GPU, so generation silently ran on CPU: a 25-plus-minute hang producing nothing, while the rest of the pipeline happily reported device=mps. The fix was small: resolve the device explicitly, move both model and input tensors onto it, wrap generation in torch.inference_mode(), cap max_new_tokens at 128. After the fix: 4 to 8 seconds per sample.
The eval that lied by construction
With both systems trained, I scored them against a 59-record gold set. Rule engine: 100% accuracy, 100% recall, zero false negatives.
Here's the part that should have been obvious immediately and wasn't: most of those 59 labels were "silver-labeled" by running the rule engine's own production code (lib/lexicon.ts / lib/riskEngine.ts) against the data, and marked "verified": false to say so, honestly, in the dataset itself. A system scored against labels it helped generate will score close to perfectly. That's not a property of the system being good. It's a property of the measurement being circular.
The tell was sitting right next to it: the from-scratch model, on the exact same 59 records, scored 69% accuracy and 21.05% hidden-sugar recall on a genuine blind test (it never saw those examples in training). One system looked perfect and the other looked mediocre, on the same eval, and the honest read was that neither number could be trusted for a production call.
Building an eval that can fail
The fix wasn't better labels for the existing set. It was a completely separate one, built specifically so neither system had a hand in producing it: 132 records, every label written by directly reading raw ingredient text the way a human reviewer would, never by calling risk_engine.ts or the model. A dedicated script, check_no_overlap.py, verified zero id or product-name overlap with anything either system had already seen in training, validation, or the original gold set.
Rerun on the independent set, the rule engine's self-graded 100% became an honest 90.9%. That drop is exactly what you'd expect once the circularity is gone, and it's the correct outcome: a decent system, no longer grading its own homework, still does fine.
Rule Engine Qwen2.5-1.5B LoRA Hybrid
Accuracy 90.9% 83.0% 89.4%
Precision 92.6% 96.5% 90.9%
Recall 96.2% 81.2% 96.2%
Hidden-sugar recall 50.0% 15.2% 56.3%
False negatives 4 19 4
Avg latency 0.9ms 4,552ms 4,553ms
15.2% hidden-sugar recall, against the rule engine's 50%, on a product whose entire job is catching sugar a shopper would otherwise miss. That's the number that ended the conversation about shipping the fine-tune alone.
The part that isn't a clean loss
The model beats the rule engine on precision (96.5% vs 92.6%) and throws fewer false positives (3 vs 8). Layered on top of the rule engine as a hybrid, with a hallucination guard dropping any model-claimed term with no textual support, it lifts hidden-sugar recall to 56.3% and trigger-match accuracy to 72.9%, both better than the rule engine alone. That's a real capability: naming novel sugar aliases the fixed lexicon was never written to catch.
It still didn't ship, because hybrid's overall accuracy (89.4%) is slightly below the rule engine alone (90.9%), the extra false positives from the model's own guesses cost more than the term-level gain was worth, and the app runs on Vercel serverless functions with no persistent process to hold a loaded PyTorch model between requests. Production still calls the deterministic engine.
Checking it again later
I went back afterward and found two real bugs in the rule engine's own matching logic: an apostrophe-stripping order bug that broke matching on terms like "confectioner's sugar," and a missing generic fallback for fruit-juice-concentrate phrasing. Fixing both moved the rule engine's own independent-set numbers: 90.9% to 91.7% accuracy, 4 to 2 false negatives.
With that fix in place and about 5x more training data, a second Qwen LoRA run (V2) now beats the rule engine on accuracy (93.1% vs 91.7%) and precision (97.0% vs 91.9%) on the same independent set. Hidden-sugar recall is still behind: 48.9% (6 false negatives) versus the rule engine's 54.2% (2 false negatives). Closer than the original 15.2%-vs-50%, not closed. Production hasn't moved. The independent benchmark is still the thing deciding that, not how far the model has come since.
The actual lesson
If your eval set was built, even partly, by the system you're testing, a good score on it isn't evidence. It's an artifact of the measurement. The fix isn't a smarter model or more careful prompting on the first eval. It's a second, independently constructed benchmark with zero shared provenance, built specifically to fail if the first one was lying to you, and the discipline to publish what it says even when it disqualifies the thing you spent the most effort building.
Top comments (0)