When I started building my side project, I thought the difficult part would be machine learning.
I was completely wrong.
The hard part turned out to be something much less glamorous:
making sure every single number could be trusted.
The project
For the past year I’ve been building Vinradar, a hobby project for Danish wine buyers.
Every day it scans around 17,500 wine listings from ten online retailers, collects professional critic ratings, tracks historical prices and tries to identify wines that are genuinely good value.
The stack is intentionally simple.
- Python
- SQLite
- Static site generation
- GitHub
- Cloudflare Pages
- Cloudflare Functions
No Kubernetes.
No distributed systems.
Just one machine doing scheduled jobs three times a day.
My first mistake
Like many developers today, I initially thought:
“I’ll let an LLM help enrich the data.”
That lasted about a week.
The model wrote perfectly convincing vintage descriptions.
Unfortunately, they occasionally contradicted the actual vintage ratings.
Later I found something even worse.
One parsing bug had accidentally shifted a “/100” denominator into the next reviewer’s score.
Instead of noticing the parser error, the LLM happily echoed the incorrect value.
A perfectly ordinary 95-point wine suddenly became a fictional 100-point masterpiece.
The language was beautiful.
The data was wrong.
That was enough.
I deleted every AI-generated numerical enrichment from the project.
The second surprise
I also assumed professional wine critics could be calibrated with one correction factor.
The data disagreed.
After analysing tens of thousands of ratings I found something unexpected.
Critic generosity isn’t fixed.
It depends on the combination of critic and wine region.
One critic might score Burgundy significantly higher than peers while being almost perfectly aligned in Piedmont.
That completely changed my scoring model.
Instead of correcting by critic…
I now correct by critic × region.
I also discovered that generosity increases near the top of the scale, so wines rated 96–100 require larger adjustments than wines in the low 90s.
The rules I eventually adopted
After a few painful lessons, I ended up with a set of principles that now drive the entire pipeline.
- Every numerical value must come from a verifiable source.
- LLMs may explain data, but they never invent or complete it.
- Missing information is preferable to guessed information.
- Every calculation should be reproducible.
- Known weaknesses should be documented rather than hidden.
These rules sound obvious.
I only wrote them after breaking every one of them.
An unexpected lesson
Building scrapers wasn’t difficult.
Building the website wasn’t particularly difficult.
Even the analytics weren’t the difficult part.
The difficult part was resisting the temptation to make the data look complete.
Users forgive missing information.
They rarely forgive incorrect information.
I’d rather display an empty field than a plausible lie.
I’m curious
Has building your own project fundamentally changed the way you think about data quality?
Did you end up creating rules you never expected to need?
I’d love to hear your experiences
Top comments (0)