I built a receipt sorter that never uploads a receipt

#python #ai #ollama #opensource

Every freelancer I know has the same monthly chore: a folder of screenshots, PDFs and receipt photos that gets sorted by hand before the accountant asks. I built receiptsort to do that sorting, and I wanted it to run with no cloud at all.

Live site: https://receiptsort.onrender.com/
Source: https://github.com/maltheshhh/receiptsort

What it does

Point it at a folder of receipts and invoices. A vision model extracts vendor, date, invoice number, currency, subtotal, tax, total and category. Each file is copied to YYYY-MM/Category/ with a searchable name like 2026-09-03_Figma_12.00.png, and ledger.csv gets one row per receipt.

Anything uncertain goes to _needs_review/ instead of being filed with a guess:

a missing vendor, date or total
subtotal + tax that doesn't match the total
the same vendor, date and total as an earlier receipt

Files are tracked by SHA-256, so re-running on the same folder only processes new ones.

Why open models

Privacy. Invoices carry client names, rates and bank details, and many freelancers are under NDAs. The model runs locally through Ollama (default qwen2.5vl:7b), so nothing is uploaded and it works offline. A hosted API would exclude the exact users this is for.

Cost. Hosted vision APIs bill per page. Local inference is free per receipt, so a flat price for the software still makes sense in a busy month.

Swappable model. Any Ollama vision model works with --model. I can try a newer one the week it ships or fine-tune for a niche, with no vendor in the way.

The interesting part: schema-constrained output

Ollama accepts a JSON schema in the format field, which constrains the model's reply. That makes a 7B model dependable for extraction:

body = {
    "model": model,
    "stream": False,
    "format": SCHEMA,              # JSON schema with a category enum
    "options": {"temperature": 0},
    "messages": [{"role": "user", "content": PROMPT, "images": [image_b64]}],
}

The schema limits category to a fixed list, and a validation step checks dates, parses numbers and cross-checks the totals. Whatever fails validation is routed to review.

Testing without a model

The test starts a stub Ollama server on localhost that returns canned answers, then runs the whole pipeline: filing, flags, ledger and the idempotent re-run. It caught a real bug where duplicates inside a single run weren't flagged, because in-memory rows held floats while rows loaded from the CSV held strings.

I haven't yet measured accuracy on a large set of real receipts. That's the next step.

Try it

ollama pull qwen2.5vl:7b
pip install pymupdf   # optional, for PDFs
python3 receiptsort.py ~/Downloads/receipts --out ~/receipts-sorted

The website is a static page on Render. The sorter itself runs on your own machine, which is the point.

Feedback welcome, especially on receipts from other countries and languages.

DEV Community