This is a submission for the Side Hustle challenge: build something with open-source AI that someone would actually pay for.
Shoebox is a desktop tool for freelancers. You drop in a messy folder of receipts, invoices, and screenshots: PDFs, phone photos, whatever you have. Shoebox reads each one with an open-weight vision model running on your own laptop, pulls out the vendor, date, amount, tax, and client, sorts everything into categories, and exports a clean spreadsheet for your accountant.
Your invoices contain your clients' names, your rates, and your income. None of that is uploaded anywhere.
And because the AI runs on the customer's machine, each document costs me nothing to process. That's what lets me sell it for a one-time price instead of yet another monthly subscription.
The problem: the shoebox
Every freelancer I know has a version of the same folder. It's called receipts, or tax stuff, or 2026 DO NOT DELETE. It contains:
- PDF invoices sent to clients
- Invoices from software subscriptions, in three currencies
- Phone photos of paper receipts, some of them sideways
- Screenshots of payment confirmations
- At least one file named
IMG_4821.jpgthat nobody can explain
Once a quarter, or once a year if they're honest, they spend a whole weekend opening every file and typing numbers into a spreadsheet. It's tedious, error-prone, and exactly the kind of work that makes people put off their taxes until it's urgent.
There are cloud tools that solve this. Most of them work the same way: upload all your documents to their servers, then pay a monthly fee. For many freelancers, both parts are a problem:
- Their invoices aren't only their data. They contain client names, project details, and rates. Some clients have contracts that limit where that information can go.
- Another subscription is a hard sell for a task you do four times a year.
I wanted something that works like a cloud service but runs like a desktop app: buy it once, and it works on your own laptop.
What I built
Shoebox has one main screen and one job.
1. Drop a folder in
Point Shoebox at a folder. It finds every PDF and image inside it, including subfolders, and processes them one by one. You can go make coffee.
2. Review only what needs reviewing
Most documents come through cleanly and go straight into the ledger. Anything Shoebox isn't sure about lands in a short review queue, with the reason it was flagged:
⚠ NEEDS REVIEW (3 of 87)
IMG_4821.jpg
Total doesn't match the line items (items add up to 412.50, total reads 421.50)
→ Check the total
hosting-invoice-march.pdf
Possible duplicate of hosting-invoice-march (1).pdf
→ Keep one
photo_2026-08-03.jpg
New vendor, category unclear (Office supplies? Equipment?)
→ Pick a category; Shoebox will remember it
You fix three things instead of typing in eighty-seven.
3. Export for your accountant
Shoebox exports a spreadsheet with one row per document and a summary tab: income by client, expenses by category, and tax totals per quarter. Every row links back to the original file, so your accountant can check anything.
How it works
┌──────────────┐
│ Your folder │ PDFs, photos, screenshots
└──────┬───────┘
▼
┌─────────────────────────────┐
│ 1. Text PDF? → extract text │ fast path, no AI needed
│ Image/scan? → vision LLM │ local model via Ollama
└──────┬──────────────────────┘
▼
┌─────────────────────────────┐
│ 2. Structured output │ JSON schema: vendor, date, total, tax…
└──────┬──────────────────────┘
▼
┌─────────────────────────────┐
│ 3. Checks │ arithmetic, dates, duplicates
└──────┬──────────────────────┘
▼
┌─────────────────────────────┐
│ 4. Categories │ your rules first, model only for unknowns
└──────┬──────────────────────┘
▼
Ledger ──or── Review queue
▼
Spreadsheet export
The same principle runs through the whole design: the model reads, but code checks. A vision model is very good at finding the total on a crumpled receipt. It's not something I want to trust blindly with numbers that end up on a tax return.
Step 1: Don't use AI when you don't need to
Many invoices are PDFs that already contain real text. Running a vision model on them would be slow and wasteful. So Shoebox tries plain text extraction first:
import pdfplumber
from pathlib import Path
def extract_text(path: Path) -> str | None:
if path.suffix.lower() != ".pdf":
return None
with pdfplumber.open(path) as pdf:
text = "\n".join(page.extract_text() or "" for page in pdf.pages)
return text if len(text.strip()) > 50 else None # too little text = probably a scan
Text PDFs go to a small text-only model. Images and scanned PDFs go to the vision model. In a typical freelancer's folder, a good share of documents take the fast path.
Step 2: Read the document with a local vision model
For images, Shoebox uses an open-weight vision-language model through Ollama. The response must match a JSON schema, so there's no free text to parse:
import ollama
from pydantic import BaseModel
from typing import Literal
class LineItem(BaseModel):
description: str
amount: float
class Document(BaseModel):
doc_type: Literal["invoice_sent", "invoice_received", "receipt", "other"]
vendor_or_client: str
date: str # ISO format, YYYY-MM-DD
currency: str # ISO code, e.g. USD, EUR, VND
line_items: list[LineItem]
tax: float | None
total: float
invoice_number: str | None
PROMPT = """Read this financial document and extract its details.
Use only what is visible in the document. If a field is not present, use null.
Write dates as YYYY-MM-DD. Use the currency shown on the document."""
def read_image(path: Path) -> Document:
resp = ollama.chat(
model="qwen2.5vl:7b",
messages=[{"role": "user", "content": PROMPT, "images": [str(path)]}],
format=Document.model_json_schema(),
options={"temperature": 0},
)
return Document.model_validate_json(resp.message.content)
Step 3: Check the numbers with plain code
This is where most of the reliability comes from. A model can misread a 7 as a 1. Arithmetic can catch that:
from datetime import date
def check(doc: Document) -> list[str]:
problems = []
if doc.line_items:
items_sum = round(sum(i.amount for i in doc.line_items), 2)
expected = round(items_sum + (doc.tax or 0), 2)
if abs(expected - doc.total) > 0.05 and abs(items_sum - doc.total) > 0.05:
problems.append(
f"Total doesn't match the line items "
f"(items add up to {items_sum}, total reads {doc.total})"
)
if doc.tax is not None and doc.tax > doc.total:
problems.append("Tax is larger than the total")
try:
d = date.fromisoformat(doc.date)
if d > date.today():
problems.append("Date is in the future")
except ValueError:
problems.append(f"Date couldn't be read ({doc.date})")
if doc.total <= 0:
problems.append("Total is zero or negative")
return problems
Duplicates are caught two ways: an exact file hash (the same file saved twice), and a match on vendor + date + total (the same invoice downloaded twice under different names).
import hashlib
def file_hash(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def duplicate_key(doc: Document) -> tuple:
return (doc.vendor_or_client.strip().lower(), doc.date, round(doc.total, 2))
Any document with a problem goes to the review queue instead of the ledger. Shoebox would rather ask you than be wrong on a tax document.
Step 4: Categories that learn from you
Categorizing is personal. One designer's "equipment" is another's "office supplies." So Shoebox doesn't let the model decide categories on its own. It checks a local rules file first, and only asks the model about vendors it has never seen:
import json
RULES = Path("~/.shoebox/rules.json").expanduser()
def load_rules() -> dict[str, str]:
return json.loads(RULES.read_text()) if RULES.exists() else {}
def categorize(doc: Document, categories: list[str]) -> tuple[str, bool]:
rules = load_rules()
key = doc.vendor_or_client.strip().lower()
if key in rules:
return rules[key], True # known vendor: no AI needed
suggestion = suggest_category(doc, categories) # small local text model
return suggestion, False # unknown vendor: goes to review
def remember(vendor: str, category: str) -> None:
rules = load_rules()
rules[vendor.strip().lower()] = category
RULES.parent.mkdir(exist_ok=True)
RULES.write_text(json.dumps(rules, indent=2))
Every time you confirm a category in the review queue, Shoebox remembers it. The first quarter needs a few corrections. After that, most vendors are already known, and the review queue gets shorter every time.
Those learned rules live in a plain JSON file on your laptop. You can read it, edit it, or back it up.
Step 5: Export
The ledger exports to .xlsx with pandas and openpyxl: one sheet with every document, one summary sheet by quarter, category, and client. Each row includes the original file path, so nothing is a black box.
The business: why someone would pay for this
The pitch to a freelancer is simple:
- Saves a weekend every quarter. That's the whole value.
- Your client data stays on your laptop. That makes it easy to use even under strict client contracts.
- One-time purchase. No subscription for a task you do four times a year.
That last point only works because of how the costs are structured. If every document went through a paid API, each new customer would add an ongoing cost for me, and I'd have to charge a subscription to cover it. With an open-weight model running on the customer's machine, my cost per document is zero. Selling it once doesn't put me at risk if a customer processes thousands of documents.
My cost is development time and a small download server. That's a business a single person can run on the side.
Why open innovation matters for this project
1. Client data never leaves the laptop
This is the main selling point, and it only exists because the model is open. Invoices contain client names, project descriptions, rates, and income. With a local model, there is no server to trust, no data processing agreement to read, and nothing to explain to a cautious client. I can honestly say that Shoebox cannot leak your documents to me, because they never reach me.
2. Zero cost per document makes a one-time price possible
With a paid API, my costs would grow with every receipt every customer processes. That pushes the product toward a subscription, and toward cutting corners (smaller images, fewer checks) to save money. Running open weights on the customer's hardware takes that pressure away. I can process every page at full resolution and run every check, because none of it costs me anything.
3. No lock-in to someone else's pricing or policies
A side hustle can't survive a supplier who doubles prices or discontinues a model. With open weights, the model Shoebox ships with today will keep working forever. If a better small vision model appears, I can test it and switch. That's a one-line change, and the decision is mine.
4. I can tune it for the documents freelancers actually have
General-purpose models are trained mostly on clean, English-language documents. Freelancers have crumpled receipts, invoices in several languages, and payment screenshots from local apps. With open weights, the path to improving that is clear: collect anonymized examples from users who opt in, and fine-tune a small adapter for the document types the base model gets wrong. With a closed API, I'd just have to hope the provider improves on my niche.
5. It works offline
Tax season sometimes happens on a train, in a café with bad Wi-Fi, or on a long flight. Shoebox doesn't care: after the first model download, everything runs offline.
Where closed would have been easier (to be fair)
The largest hosted vision models are more accurate on very messy documents, and they don't need a capable laptop. A 7B vision model wants a reasonably modern machine with enough memory. Older laptops will be slow, and some won't run it at all.
I accept that trade-off, because the checks in Step 3 catch most reading errors, and the review queue handles the rest. For a freelancer, "slightly slower, and I review 3 out of 87 documents" is a much better deal than "upload all my client invoices to a server and pay monthly."
My first user
[Replace this section with what really happened when you put Shoebox in front of a real freelancer: who they are (e.g. "a freelance illustrator friend"), how many documents they ran through it, what broke, whether they paid or said they would, and what they asked for next. Real details here matter more than anything else in the post.]
Limitations
- It needs a decent laptop. The vision model needs a fair amount of memory. On older machines, processing is slow.
- Handwritten receipts are hit-or-miss. Printed text works well. Handwriting is unreliable and usually ends up in the review queue.
- It doesn't do your taxes. Shoebox organizes documents and adds them up. It doesn't know your country's tax rules and it doesn't replace an accountant.
- Currency conversion is basic. Amounts are stored in their original currency. Conversion uses an exchange-rate file you update yourself, so totals in a home currency are only as current as that file.
What's next
- [ ] Watch a folder and process new documents automatically as they arrive
- [ ] Export formats that common accounting software can import directly
- [ ] An opt-in, anonymized way to share hard examples for fine-tuning
- [ ] A smaller model option for older laptops
Try it
pip install ollama pydantic pdfplumber pandas openpyxl
ollama pull qwen2.5vl:7b
python shoebox.py scan ~/Documents/receipts --out ledger.xlsx
If you're a freelancer with a folder you've been avoiding, I'd love for you to try it and tell me what breaks.
Then close the laptop. Your weekend is free. 🧾
Built with: Ollama, Qwen2.5-VL (open weights), pdfplumber, Pydantic, pandas, openpyxl, Python.
Top comments (0)