This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
ReceiptBox is a local-first invoice and receipt sorter for freelancers.
Every freelancer knows the end-of-quarter scramble: a downloads folder full of IMG_4821.jpg, invoice (3).pdf and screenshots of payment confirmations, and an accountant waiting for a tidy spreadsheet. ReceiptBox fixes that.
You drop a folder of messy receipts and invoices onto it. It then:
- Reads each file (photo, scan or PDF) with a local vision-language model
- Extracts vendor, date, total, tax, currency and a likely expense category
- Renames and files each document, e.g.
2026-09-14_Figma_USD-15.00_software.pdf - Exports a clean CSV that an accountant (or a tax tool) can open straight away
- Flags anything it is unsure about for a quick human check, instead of guessing
Who would pay for it? Freelance designers, developers, writers and consultants who handle their own books, and the small bookkeepers who serve them. The pitch is a one-time license or a small monthly fee (think $5-10/month) for something that saves a few hours every quarter and never uploads client paperwork anywhere.
Demo
No hosted link: the whole point is that it runs on your own machine. Here is a typical run from the terminal:
$ receiptbox sort ./inbox --out ./sorted
Scanning 14 files...
[ok] IMG_4821.jpg -> 2026-09-14_Figma_USD-15.00_software.jpg
[ok] invoice (3).pdf -> 2026-09-02_Hetzner_EUR-4.51_hosting.pdf
[ok] scan0007.png -> 2026-08-29_Uber_INR-342.00_travel.png
[check] receipt_blurry.jpg -> confidence 0.41, moved to ./sorted/_needs_review
...
Done: 13 sorted, 1 needs review
Wrote ./sorted/expenses_2026-Q3.csv
And the resulting CSV:
date,vendor,total,currency,tax,category,confidence,file
2026-09-14,Figma,15.00,USD,0.00,software,0.97,2026-09-14_Figma_USD-15.00_software.jpg
2026-09-02,Hetzner,4.51,EUR,0.72,hosting,0.95,2026-09-02_Hetzner_EUR-4.51_hosting.pdf
2026-08-29,Uber,342.00,INR,16.29,travel,0.93,2026-08-29_Uber_INR-342.00_travel.png
Code
I'm not linking a repo for this post, so here are the core pieces inline.
Schema for extraction (Pydantic):
from pydantic import BaseModel, Field
from datetime import date
from typing import Literal
Category = Literal[
"software", "hosting", "travel", "meals", "equipment",
"office", "marketing", "education", "other",
]
class Receipt(BaseModel):
vendor: str
date: date
total: float
currency: str = Field(min_length=3, max_length=3) # ISO 4217
tax: float = 0.0
category: Category
confidence: float = Field(ge=0, le=1)
Extraction with a local VLM through Ollama:
import ollama
from pathlib import Path
PROMPT = """You are an accounting assistant. Read this receipt or invoice
and return ONLY JSON matching the schema. If a field is unreadable, lower
your confidence instead of guessing."""
def extract(path: Path) -> Receipt:
resp = ollama.chat(
model="qwen2.5vl:7b",
messages=[{
"role": "user",
"content": PROMPT,
"images": [str(path)],
}],
format=Receipt.model_json_schema(), # constrained JSON output
options={"temperature": 0},
)
return Receipt.model_validate_json(resp["message"]["content"])
Sorting, renaming and the review lane:
import shutil, csv
THRESHOLD = 0.75
def process(path: Path, out: Path, rows: list[dict]):
try:
r = extract(path)
except Exception:
shutil.move(path, out / "_needs_review" / path.name)
return
if r.confidence < THRESHOLD:
shutil.move(path, out / "_needs_review" / path.name)
return
new = f"{r.date}_{r.vendor.replace(' ', '')}_{r.currency}-{r.total:.2f}_{r.category}{path.suffix}"
shutil.move(path, out / new)
rows.append({**r.model_dump(), "file": new})
PDFs are rasterized page by page (first page for receipts, all pages for multi-page invoices) with pdf2image before being passed to the model.
How I Built It
- Model: an open-weight vision-language model, Qwen2.5-VL 7B, served locally through Ollama. It reads photos, scans and screenshots directly, so there is no separate OCR step to tune.
- Structured output: Ollama's JSON-schema constrained decoding plus Pydantic validation. The model cannot return a malformed record, and bad values get caught before they reach the CSV.
-
Confidence-gated workflow: the model reports a confidence score and anything below the threshold goes to a
_needs_reviewfolder. For money data, "I'm not sure" is a feature. -
Plumbing: Python,
pdf2imagefor PDFs, SQLite for a local index (so re-running never duplicates files), and a small Typer CLI. A drag-and-drop desktop wrapper is the next step for non-technical users. - Hardware: it runs on a laptop with 16 GB RAM. A quantized build keeps it usable without a dedicated GPU, just slower.
Why Does Open Innovation Matter?
Privacy is the product. Receipts and invoices reveal who your clients are, what you charge and how you live. Many freelancers are under NDAs, and a lot of them simply do not want their financial paperwork sitting on someone else's server. With an open-weight model, ReceiptBox works with the Wi-Fi off. That is a real selling point, and a closed API can't offer it.
The economics work. A closed vision API charges per image, which punishes exactly the heavy, bursty usage of tax season, and it makes a $5/month price point risky. Local inference costs the user electricity, so my marginal cost per customer is effectively zero and the business can stay small and sustainable.
No lock-in, and room to specialize. I can swap in a newer or smaller model as soon as one is better, pin a version so extraction doesn't silently change mid-quarter, and fine-tune on region-specific formats (GST invoices in India, VAT receipts in the EU, handwritten rickshaw slips) that general-purpose APIs handle unevenly. If a vendor changes pricing or deprecates a model, my product doesn't break.
Where closed would have been better: a frontier hosted model probably has higher raw accuracy on very bad photos. The review lane and confidence threshold are how I make up for that, and for this use case, "private and cheap with a human check on the hard 5%" beats "slightly more accurate but uploads everything."
Top comments (0)