Pulling an invoice number or a total out of text is the easy half of the job. The useful half is deciding, for every field of every document, whether it goes straight into the accounting system or lands on a person's desk. Here is how a small demo handles it. Everything below is test data from a public synthetic dataset, not a client result.
What the demo does
Input: the OCR text of an invoice. Output: invoice number, date, IBAN and gross total, each with a confidence. Fields with low confidence are routed to a human instead of being posted.
The data is mychen76/invoices-and-receipts_ocr_v1: OCR text plus ground-truth fields. The invoices are synthetic with a fixed layout, and only the invoice-type documents are used (484 of about 2,200; the rest are receipts with a different schema).
Candidates first, then a ranker
For each field the code generates candidates with patterns, then a small logistic regression picks the right one using context: the words around the candidate, its position, its format. This matters most for the total, because an invoice contains many amounts (net, tax, line items, gross) and "the biggest number" is not the answer.
Validation beats confidence for IBANs
For IBANs the demo does not trust a score at all. It validates the ISO 13616 checksum. If OCR confused one character (S for 5, O for 0, a dropped or doubled character) and exactly one repair makes the checksum pass, it applies the repair. If no unique valid value exists, the field goes to a human.
def iban_ok(s: str) -> bool:
s = s.replace(" ", "").upper()
rearranged = s[4:] + s[:4]
digits = "".join(str(int(c, 36)) for c in rearranged)
return int(digits) % 97 == 1
A checksum is a hard gate a model cannot talk its way past.
Results on 74 held-out documents (test data)
| Field | Docs scored | Accuracy when automated | Automated share |
|---|---|---|---|
| Invoice number | 74 | 100% | 97.3% |
| Invoice date | 74 | 100% | 100% |
| IBAN | 56 | 100% | 100% |
| Gross total | 74 | 98.6% | 100% |
96.4% of clean documents were fully automated (all four fields), and 98.2% of those had every field correct. Thresholds were tuned on a validation split for a 98% target; the held-out set was used for reporting only.
Honest caveats
- 74 documents with a fixed synthetic layout is a small, friendly test. Real, varied invoices will score lower, and a pilot starts by measuring on the client's own sample.
- In 18 of the 74 documents the dataset's IBAN label itself fails the checksum while the OCR text passes it, so the label is corrupted. Those are excluded from the IBAN score and counted separately in
results.json. Check your ground truth before blaming the model. - The input is OCR text. Turning scans and PDFs into that text is a separate, replaceable step.
Takeaways
- Decide per field what is allowed to auto-post, with a number as the bar.
- Use deterministic validation (checksums, date ranges, sum checks) wherever it exists.
- Route what you cannot validate to a human, and make that route cheap.
- Audit your labels before you measure.
Code and numbers: ai-invoice-extraction-demo. Portfolio and contacts: @Kotenev01_bot.
I write about practical, measurable AI adoption for business: demos on test data, no invented case studies. Telegram channel: @kotenev01_en
Top comments (0)