Receipt Box: a private receipt sorter for freelancers, powered by Gemma

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Receipt Box is a receipt and invoice sorter for freelancers that runs entirely on your own laptop.

Every freelancer knows the end-of-quarter ritual: a folder of crumpled photos, PDF invoices in six currencies, and an accountant asking "which of these have VAT?" Receipt Box turns that folder into a clean expense dashboard in a few minutes. Drop in photos or PDFs and a vision model reads each one: vendor, date, total, VAT, currency, and a category such as Software, Travel or Meals. You get:

• A dashboard: total spend, VAT to reclaim, spend by category and by month.
• A VAT-by-quarter table, because that is the number a freelancer actually has to hand over.
• An editable table. A 4B model will sometimes be wrong, so every cell is fixable. Change a vendor's category once and Receipt Box remembers it: that vendor is sorted the same way from then on.
• Duplicate detection, so the same invoice uploaded twice gets flagged instead of counted twice.
• Multi-currency handling, with a currency switch instead of silently adding euros to forints.
• Exports to CSV and to an Excel workbook (receipts, spend per category, VAT per quarter) that goes straight to the accountant.
• Optional line-item extraction for when you want to see what was bought, not just the total.

Who would pay? Freelancers and tiny agencies who want bookkeeping help but cannot (or should not) upload client invoices, travel documents and personal receipts to a cloud service. The same goes for the accountants who serve them: with many clients' paperwork on their desk, "nothing leaves this machine" is a feature they can promise. I see it as a one-time licence for freelancers, or a small monthly fee per accountant seat.

Demo

Live app: https://receipt-box-69v1.onrender.com (hosted on Render; click Load demo data to explore the dashboard)

The repo includes three fictional sample documents in samples/ (a PDF invoice and two receipt images in different currencies). Drop them in and the dashboard fills in as each document is read.

Code

banana0000 / receipt-box

Private receipt & invoice sorter for freelancers: Gemma 3 reads your receipts locally via Ollama. Editable table, learned categories, VAT by quarter, multi-currency, CSV/Excel export.

Receipt Box

A receipt and invoice sorter for freelancers that runs entirely on your laptop Photos and PDFs go to an open-weight vision model (Gemma 3 via Ollama), and the result is a categorised expense dashboard plus a CSV or Excel file for your accountant Nothing is uploaded anywhere.

Features

• Reads receipt photos (JPG / PNG) and PDF invoices (text layer or scanned) with Gemma 3.
• Dashboard: total spend, VAT to reclaim, spend by category and by month, VAT by quarter.
• Editable table: fix any cell, delete rows. Changing a category teaches the app: that vendor is sorted the same way from now on (saved in rules.json).
• Possible duplicates are highlighted.
• Several currencies: pick one, or "All currencies" to see them side by side. Amounts are never converted.
• Export to CSV or Excel (receipts, spend per category, VAT per quarter).
• Optional line-item extraction (slower).
• Files are read one at…

View on GitHub

How I Built It

The whole project is built around one open-weight model: Gemma 3 (4B, vision), running locally through Ollama. The only network call the app makes goes to localhost.

• Structured output. Each image is sent to Gemma with a JSON schema (vendor, date, total, vat, currency, category), and the category is constrained to a fixed list. The reply always parses, with no regex and no retry loop.
• Two paths for PDFs. If a PDF has a text layer, the text goes to the model directly, which is faster. Scanned PDFs are rendered to an image (PyMuPDF) and go through the vision path.
• Queue with visible progress. On a CPU a receipt takes roughly 15-20 seconds, so files are processed one at a time and the page shows "Reading 2 of 5: name.png" instead of freezing.
• The model is allowed to be wrong. The prompt tells it to use only what is printed and not to guess. After that, the human-in-the-loop table and the learned vendor rules do the rest. In my tests the three sample documents were read correctly, and for real-world crumpled receipts the correction table is what makes the tool usable.
• Dashboard. A Dash app with Plotly charts and pandas for the numbers (quarterly VAT, duplicates, per-currency totals). Excel export uses openpyxl.
• Tests. The data layer (duplicates, quarters, rules, exports) is covered with pytest.

Hosting. The same app also deploys to Render from a Blueprint (render.yaml): a private Ollama service with Gemma 3 baked into its Docker image, and the web app talking to it over Render's private network, so the model is never exposed to the internet. Gemma 3 4B needs about 4-5 GB of RAM, so the Ollama service runs on a large instance. On a laptop none of this is needed.

Honest limits: a 4B model on a CPU is slow, and line-item extraction is slower still, so it is off by default. A bigger Gemma or a GPU is a one-line change (RECEIPT_MODEL).

Why Does Open Innovation Matter?

Receipts are about the most sensitive "boring" data a freelancer has: who you work for, where you travel, what you pay for. A closed API would force a trade-off between convenience and privacy, and that is exactly where I would lose the customers who care about it most.

An open-weight model removes the trade-off. Because Gemma runs on the user's own machine:

• the documents never leave the laptop, so there is nothing to leak, retain or train on;
• there is no per-receipt API bill, so the price of the product is not tied to someone else's pricing;
• it keeps working offline, on a train or in a client's office with no Wi-Fi;
• the user can swap the model by changing one environment variable as better open models arrive.

A small open model that runs on a laptop is what makes "private by design" a real promise here, not a marketing line.

Prize Categories

• Gemma: the entire product is built on Gemma 3 running locally.
• Render: the web app and a private Gemma/Ollama service are deployed together from one Blueprint.