Receipt Sorter: a private, offline bookkeeping helper for freelancers, powered by Qwen2.5-VL

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

receipt-sorter is a command-line tool that turns a messy folder of receipts and invoices (PDFs and phone photos) into something an accountant can use. It renames each file (2026-10-14_Brew-Bean-Cafe_441.0.png), writes an accountant-ready receipts.csv (date, vendor, total, tax, currency, category), and produces a summary.md with totals per category.

Who would pay for it: freelancers, consultants and tiny agencies who dread tax season and handle other people's financial documents. Their problem is twofold: the sorting is tedious, and they don't want client invoices and bank details uploaded to a third-party cloud. A tool that runs entirely on their own laptop solves both.

Demo

Code

hv8499-svg / receipt-sorter

Private, offline receipt and invoice sorter powered by a local open-weight vision model

receipt-sorter

Drop a folder of receipts and invoices (PDFs and phone photos) in, get back renamed files an accountant-ready receipts.csv, and a category summary. A local open-weight vision model does the reading, so client invoices and bank details never leave your laptop.

sorted_output/
  sorted/          2026-03-04_Figma_15.0.pdf ...   (confident results)
  needs_review/    anything unreadable, low-confidence or possibly duplicated
  receipts.csv     date, vendor, total, tax, currency, category, notes ...
  summary.md       totals per currency and category + a review list

Originals are copied, never moved or modified.

Setup

# 1. Install Ollama (https://ollama.com) and pull an open vision model
ollama pull qwen2.5vl:7b        # ~6 GB; use gemma3:4b or qwen2.5vl:3b on smaller machines

# 2. Install deps
pip install -r requirements.txt

# 3. Run
python -m sorter ~/Documents/receipts -o ~/Documents/receipts_sorted
python -m sorter ./receipts --model gemma3:4b --dry-run   # preview, writes nothing

Add --verify to read every document twice (second pass at a different…

View on GitHub

How I Built It

• Model: Qwen2.5-VL 7B, an open-weight vision-language model, served locally through Ollama. A vision model reads tilted phone photos far better than classic OCR plus rules.
• Pipeline (Python): PyMuPDF renders PDFs, Pillow normalises photos, and the model is asked for one JSON object per document using Ollama's schema-constrained output. Parsing is tolerant, with up to two retries.
• Validation instead of trust: amounts like ₹1,234.50 and 1.234,50 are normalised, dates are converted to ISO, tax greater than total is flagged, and unknown categories are caught. Anything uncertain goes to a needs_review/ folder rather than being silently accepted. Exact duplicates are skipped by SHA-256 without spending any inference.
• Customisation: categories.json holds your own categories and vendor rules (which always override the model), and examples.json holds corrected examples used as few-shot guidance. --model swaps in any Ollama vision model.
• Tests: the whole pipeline is tested end to end against a fake model, so it runs with no GPU and no network.

Testing on synthetic receipts

I generated seven fake receipts to test with: a café bill in INR, a SaaS invoice in USD, a tilted Uber photo, a German hardware-store receipt with comma decimals, an internet bill, a deliberately unreadable one, and an exact duplicate. On my first run, the model got the first five right: vendor, date, total, currency and category, including the tilted photos and the German number format. The duplicate was skipped.

Where it failed (and what I changed)

The deliberately blurry receipt slipped through. The real image read 1?/09/2026 and 3?5.00, but the model confidently returned 2026-09-17 and 375.0, with no warning. For a bookkeeping tool, a quiet wrong number is worse than a flagged one.

I added two safeguards:

1. Evidence check: the model must also quote the date and total exactly as printed. A ?, an unparseable quote, or a quote that disagrees with the extracted value gets flagged.
2. --verify mode: each document is read twice at different resolutions, and any disagreement on date or total is flagged for review.

Performance: [YOUR LAPTOP: RAM, CPU/GPU] processed the 7 documents in about [X] minutes without --verify and [Y] minutes with it.

Why Does Open Innovation Matter?

• Privacy is the product. The only network call in the code is to localhost (Ollama). Client invoices never leave the laptop, which is something a cloud API can't honestly promise.
• The economics work. There is no per-document or per-token cost. Processing a thousand receipts costs the same as processing ten, so a one-time or flat yearly price is realistic. [Optional: say what price you'd charge.]
• No lock-in. The model is a one-flag swap: a smaller model for an old laptop, a bigger one for accuracy, or a fine-tune for a specific niche. A closed API can change its pricing, its behaviour or its availability whenever it wants.
• I could fix a failure at the right layer. When the model guessed on a blurry receipt, I could change the prompt, the schema and the validation, and add a second pass, because every layer is mine to modify.