Receipt Sorter: a private, offline bookkeeping helper for freelancers, powered by Qwen2.5-VL

Harshaa Vardhini R — Sat, 17 Oct 2026 11:55:59 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

receipt-sorter is a command-line tool that turns a messy folder of receipts and invoices (PDFs and phone photos) into something an accountant can use. It renames each file (2026-10-14_Brew-Bean-Cafe_441.0.png), writes an accountant-ready receipts.csv (date, vendor, total, tax, currency, category), and produces a summary.md with totals per category.

Who would pay for it: freelancers, consultants and tiny agencies who dread tax season and handle other people's financial documents. Their problem is twofold: the sorting is tedious, and they don't want client invoices and bank details uploaded to a third-party cloud. A tool that runs entirely on their own laptop solves both.

Demo

Code

hv8499-svg / receipt-sorter

Private, offline receipt and invoice sorter powered by a local open-weight vision model

receipt-sorter

Drop a folder of receipts and invoices (PDFs and phone photos) in, get back renamed files an accountant-ready receipts.csv, and a category summary. A local open-weight vision model does the reading, so client invoices and bank details never leave your laptop.

sorted_output/
  sorted/          2026-03-04_Figma_15.0.pdf ...   (confident results)
  needs_review/    anything unreadable, low-confidence or possibly duplicated
  receipts.csv     date, vendor, total, tax, currency, category, notes ...
  summary.md       totals per currency and category + a review list

Originals are copied, never moved or modified.

Setup

# 1. Install Ollama (https://ollama.com) and pull an open vision model
ollama pull qwen2.5vl:7b        # ~6 GB; use gemma3:4b or qwen2.5vl:3b on smaller machines

# 2. Install deps
pip install -r requirements.txt

# 3. Run
python -m sorter ~/Documents/receipts -o ~/Documents/receipts_sorted
python -m sorter ./receipts --model gemma3:4b --dry-run   # preview, writes nothing

Add --verify to read every document twice (second pass at a different…

View on GitHub

How I Built It

Model: Qwen2.5-VL 7B, an open-weight vision-language model, served locally through Ollama. A vision model reads tilted phone photos far better than classic OCR plus rules.
Pipeline (Python): PyMuPDF renders PDFs, Pillow normalises photos, and the model is asked for one JSON object per document using Ollama's schema-constrained output. Parsing is tolerant, with up to two retries.
Validation instead of trust: amounts like ₹1,234.50 and 1.234,50 are normalised, dates are converted to ISO, tax greater than total is flagged, and unknown categories are caught. Anything uncertain goes to a needs_review/ folder rather than being silently accepted. Exact duplicates are skipped by SHA-256 without spending any inference.
Customisation: categories.json holds your own categories and vendor rules (which always override the model), and examples.json holds corrected examples used as few-shot guidance. --model swaps in any Ollama vision model.
Tests: the whole pipeline is tested end to end against a fake model, so it runs with no GPU and no network.

Testing on synthetic receipts

I generated seven fake receipts to test with: a café bill in INR, a SaaS invoice in USD, a tilted Uber photo, a German hardware-store receipt with comma decimals, an internet bill, a deliberately unreadable one, and an exact duplicate. On my first run, the model got the first five right: vendor, date, total, currency and category, including the tilted photos and the German number format. The duplicate was skipped.

Where it failed (and what I changed)

The deliberately blurry receipt slipped through. The real image read 1?/09/2026 and 3?5.00, but the model confidently returned 2026-09-17 and 375.0, with no warning. For a bookkeeping tool, a quiet wrong number is worse than a flagged one.

I added two safeguards:

Evidence check: the model must also quote the date and total exactly as printed. A ?, an unparseable quote, or a quote that disagrees with the extracted value gets flagged.
--verify mode: each document is read twice at different resolutions, and any disagreement on date or total is flagged for review.

Performance: [YOUR LAPTOP: RAM, CPU/GPU] processed the 7 documents in about [X] minutes without --verify and [Y] minutes with it.

Why Does Open Innovation Matter?

Privacy is the product. The only network call in the code is to localhost (Ollama). Client invoices never leave the laptop, which is something a cloud API can't honestly promise.
The economics work. There is no per-document or per-token cost. Processing a thousand receipts costs the same as processing ten, so a one-time or flat yearly price is realistic. [Optional: say what price you'd charge.]
No lock-in. The model is a one-flag swap: a smaller model for an old laptop, a bigger one for accuracy, or a fine-tune for a specific niche. A closed API can change its pricing, its behaviour or its availability whenever it wants.
I could fix a failure at the right layer. When the model guessed on a blurry receipt, I could change the prompt, the schema and the validation, and add a second pass, because every layer is mine to modify.

DEV Community: Harshaa Vardhini R