BillBuddy AI: Private Expense Tracking for Freelancers, Powered by Gemma and Ollama

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

Every freelancer knows the pile: photos of receipts, PDF invoices, and a spreadsheet that never gets updated until tax time. I built BillBuddy AI to turn that pile into organized expenses, without uploading a single receipt to someone else's server.

What I Built

BillBuddy AI is a receipt and invoice organizer that runs entirely on your own computer.

1. You drop in a receipt (JPG, PNG, or PDF).
2. OCR reads the text.
3. An open-weight model, Gemma, suggests the vendor, date, and total.
4. You check and correct the fields, then save them.
5. The app keeps a searchable expense list with totals per currency, a category breakdown, and a CSV export.

Who would pay, and why. Freelancers and small shop owners spend hours each month typing receipts into spreadsheets. Many of them are also uneasy about sending financial documents to a cloud AI service, because those documents hold client names, addresses, and purchase history. A tool that does the typing locally removes both problems. The side hustle angle I'm exploring is offering receipt and expense organizing to small local businesses: use BillBuddy to cut the data entry, and charge per batch or per month.

I want to be upfront: this is a working prototype. Nobody has paid for it yet, and I haven't tested whether people would.

Demo

BillBuddy runs on your own computer (that is the whole point), so there is no hosted link. Here is a 2-minute walkthrough of the full flow, from reading a receipt to saving, searching, and deleting expenses:

▶ Watch the 2-minute demo

[Drag your screenshots into the DEV editor here: the receipt-style OCR text panel, the AI-suggested fields with a warning visible, and the saved expenses with totals in two currencies. Fake receipts only.]

Code

kaushanikoner20-hub / BillBuddyAI

BillBuddy AI

An open-source, local-first receipt and invoice organizer for freelancers and small businesses Read a receipt with OCR, let an open-weight AI model (Gemma, running on your own computer through Ollama) suggest the expense details, correct them, and save them to a local SQLite database.

Screenshots

Demo

Watch the 2-minute demo

Features

• Upload JPG, JPEG, PNG, or PDF receipts (10 MB limit, first 5 PDF pages are read)
• Read the text with Tesseract OCR, including scanned PDFs
• Gemma suggests the vendor, date, and total. The currency is detected from the receipt text itself.
• Warnings when a suggested value does not appear in the receipt text
• Review and correct every field, or fill in the form manually
• Save expenses (vendor, date, total, currency, category, description) in SQLite
• Search by vendor or description, filter by category and date, edit, and delete with confirmation
• Dashboard: number of receipts, totals per currency…

View on GitHub

The repository has Windows setup instructions, a .env.example, automated tests, a GitHub Actions workflow, and an MIT license.

How I Built It

Receipt (JPG / PNG / PDF)
   -> Tesseract OCR (PDF pages rendered with PyMuPDF first)
   -> Gemma through Ollama, running locally, suggests fields
   -> validation and warnings
   -> you review and correct
   -> saved in SQLite (reviewed fields only)

• Open-weight model: Gemma 3 (gemma3:4b) served locally by Ollama. The backend calls Ollama's local API with temperature 0 and JSON output.
• OCR: Tesseract with Pillow. Scanned PDFs work too, because each page is rendered to an image first.
• Backend: Python and FastAPI. Storage: SQLite, with only the fields you reviewed, not the receipt file or the full OCR text.
• Frontend: React and Vite.

I built it in stages with help from an AI coding assistant, testing each stage on my own machine before moving on.

The lesson: a small model fills in blanks

The part that taught me the most was the currency field. I tested an Indian bill, and the app labeled the amount in USD. Nothing on the bill said dollars. The 4B model simply guessed, because a blank is uncomfortable for a language model. Tesseract's default settings can also misread the ₹ sign, so the model often never saw it.

So I stopped asking the model about currency. BillBuddy now detects it from the receipt text itself: a currency code, ₹, Rs, or Indian tax terms like GSTIN and CGST. When the text doesn't say, the field stays empty. A plain $ isn't enough either, because it doesn't tell you the country. An empty field you fill in yourself is better than a confident wrong one.

That one bug shaped how the whole app treats the model:

• Never trust the output. Dates must be real dates, amounts must be non-negative numbers, and anything else becomes empty.
• Warn when the numbers don't appear in the receipt. If the model suggests a total that isn't in the OCR text, the app says so.
• Treat receipt text as untrusted. A receipt can contain instructions meant to steer the model, so the prompt tells it to ignore them, and the answer is validated anyway.
• Never mix currencies. Totals are grouped by currency, so rupees and dollars are never added together.
• Export safely. CSV cells that start like a spreadsheet formula are neutralized.

Because everything is local, testing was easy too: the tests replace the model with a stand-in, so the suite runs without Ollama, and a GitHub Actions workflow runs the tests and the frontend build on every push.

Limits

The AI can be wrong, so a person reviews every field before saving. I haven't measured extraction accuracy on real receipts, so I won't claim a number. There is no login or database encryption, so it is meant for one local user, and I developed and tested it on Windows.

Why Does Open Innovation Matter?

A receipt is exactly the kind of document you don't want to hand to a third party: it has names, addresses, and what you bought and where. With an open-weight model running locally, the receipt text never leaves the computer. A closed cloud API can promise to be careful with it, but it still has to receive it.

Open also changed the economics. There are no API keys and no per-receipt fee, so it costs nothing extra whether a shop processes ten receipts or ten thousand. For a business that might run this every day, that is the difference between a tool that works and one that eats the margin. After the one-time downloads, the pipeline doesn't need the internet.

And the model is one setting. If a larger Gemma, or a different model, reads receipts better, I change a single line instead of rewriting the app or being locked into someone's pricing.

I'll be honest about the trade-off: a 4B model on a laptop is less accurate than a large hosted model, and that is why the review step and the validation exist. For this use case, I think a reviewed, private, free-to-run answer beats a faster one that needs your documents uploaded.

Prize Categories

• Best Use of Gemma: BillBuddy runs Gemma 3 (4B) locally through Ollama to read OCR text and suggest expense fields, with validation and warnings built around its answers.