The Receipt Sorter That Shows Its Work

Submission draft for the Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle. Prize category: Best Use of Gemma. This project was built with AI assistance; the build notes below say what the AI got wrong, too.

The problem with receipt sorters isn't extraction

There are already good tools that read a receipt and hand you the numbers. The problem starts a month later, at tax time, when a number in your spreadsheet looks wrong and there is no way to answer the only question that matters: which document did this come from, and what actually read it?

So I built Receipt Sorter around the audit trail instead of around the extraction. It is a small command-line tool — Python, standard library only — that turns a folder of receipts and invoices into a ledger.csv. A local open-weight model (Gemma, served by Ollama) reads each document. Images go through vision; plain-text invoices are read directly. Five fields per document: vendor, date, amount, currency, category.

Then the part I actually trust: the validation layer.

Validation, not vibes

• Dates must be real ISO dates. Amounts must be positive. Currency must be a 3-letter code. Category must come from a fixed list of 12 — no free text.
• Anything missing or invalid is flagged needs_review, with the reason written next to it. The tool never guesses a value to make a row look complete.
• Duplicates are skipped, defined strictly: same vendor, date, amount, and currency.
• Every ledger row carries provenance: the source file, the model used, and the extraction timestamp.

Financial documents deserve that care. A receipt shows who was paid, how much, and when — which is also why the tool runs entirely on your own machine. Nothing is uploaded, there is no per-seat API cost, and it works offline once the model is downloaded.

What the demo actually showed — stated carefully

Status, precisely: built and fixture-tested. It is released publicly alongside this post, and it has never touched a real client's receipts.

On 2026-10-13 I ran it end to end against three sample fixtures created for testing — one rendered grocery receipt as an image, one software invoice as text, one cab receipt as text — and scored the output against a ground-truth file:

• 15 of 15 fields correct — 5 fields across 3 clean fixtures. Zero rows flagged for review.
• Model: gemma3:4b, fully local, on a small 2-CPU VM.
• Speed, honestly: the warm 3-file run took 2 minutes 58 seconds. A cold single-image call took about 175 seconds, including model load. Small local models are slow, and I would rather publish that than hide it.

Three clean fixtures prove the pipeline works. They do not prove real-world accuracy, and this post doesn't claim it. Photographed, crumpled, real receipts are the next test, and whatever accuracy shows up there is what gets published.

What broke while I built it

Two failures worth more than the demo:

1. The model downloads failed their own provenance test. The build machine's network intercepted DNS in a way that made ollama pull refuse the model files. The fix was to fetch each file directly and verify its SHA-256 against the model's manifest before assembling anything — all 9 files across both models, hash-verified. A tool that records where every value came from should hold its own downloads to the same standard.
2. The small model's name lied. gemma3:1b turned out to be text-only in its Ollama packaging — no vision component — so image receipts were rejected until the run moved to gemma3:4b. The box didn't say that. Testing did.

What's next

• Run it against messy, real-world photographed receipts and publish the accuracy as measured — good or bad.
• Make the needs_review queue easier to work through, because flagged rows are where a user's time actually goes.

Code: https://github.com/marsojuji-cmyk/receipt-sorter — the demo ledger, ground truth, and results are in the repo (demo/RESULTS.md).