ReceiptOwl — stop losing receipts, start keeping your margin

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

ReceiptOwl is a local-first desktop app that turns a messy folder of receipts — PDFs, phone photos, screenshots — into tax-ready records, without any of it ever leaving your computer.

It's built for the "side hustle" crowd this challenge is about: freelancers, small shop owners, anyone running a side business out of email attachments and a camera roll. You drop receipts into the app, it OCRs and extracts the vendor/amount/date/tax with a local AI model (falling back to a deterministic heuristic if no model is available), sorts them into your categories, flags anything that looks off (missing tax, duplicate charge, low-confidence read), and gives you a CSV, a tax-summary PDF, and an anomaly report — plus the organized folder underneath all of it.

Five screens, one pipeline:

Nest → Hunt → Perch → Treasure → Settings
drop    process   review/edit   totals & export   configure

Honest disclaimer: this isn't a finished v1. It's not perfect, and it still has flaws — some interface rough edges, limited test coverage, and also messy ui not responsive :]

Demo

(Attach a short screen recording or GIF of Nest → Hunt → Perch → Treasure here before publishing.)

Screens, for now:

Nest (upload)	Hunt (live processing)

Perch (review & edit)	Treasure (totals & reports)

Code

Repo: https://github.com/Malvin555/ReceiptOwl

How I Built It

The app is Python + Gradio for the UI, with a custom CSS theme layered on top so it doesn't look like a default Gradio demo. Everything runs locally:

• OCR: pypdf reads a PDF's text layer directly when there is one (fast, no model needed); pytesseract (Tesseract) handles images and scanned PDFs as a fallback.
• Field extraction (the "AI" step): a local Ollama server, defaulting to Gemma 2 (2B), pulls vendor, amount, date, and tax out of the OCR'd text. If Ollama isn't running — no install, server down, whatever — extraction falls back automatically to a deterministic regex/heuristic extractor that still returns a usable result with a lower confidence score. The app never hard-fails because a model isn't available.
• Categorization is a keyword match against categories the user actually configured, not a fixed taxonomy.
• Storage is SQLite plus content-addressed copies of the original files on disk, all under one folder the user can see and open in a file explorer.

The trickiest part wasn't the AI pipeline — it was making the UI actually look and behave like a real app rather than a generic Gradio form: custom navigation between the five screens, a styled file-drop zone, editable tables, and (the bug that ate the most time) Gradio's theme auto-switching ~70 CSS variables based on the OS's light/dark setting, which made icons and text invisible on certain systems until every one of those variables got pinned to the app's own light design.

GitHub Copilot was part of the day-to-day workflow throughout — inline completions while writing the Python pipeline and CSS, plus Copilot Chat for quick debugging and "why is this not working" questions alongside the editor.

Why Does Open Innovation Matter?

Because the people this app is for are the ones least served by "send your documents to our cloud and we'll process them" SaaS tools — a freelancer's receipts are their financial life, and a lot of side-hustle tooling asks you to hand that over to a third party by default. Open-weight local models like Gemma make it possible to put real extraction intelligence on the user's own machine, with no API key, no per-document cost, and no data leaving the laptop. That's the whole thesis of this build: the AI should come to your data, not the other way around. Open-source OCR and open-weight LLMs are what make that actually achievable for a solo developer in a few weeks, instead of something only a funded company could build.

My Agent Session

Built with two AI tools working different parts of the job. GitHub Copilot sat in the editor the whole time — inline completions and Copilot Chat for the moment-to-moment "write this function," "why does this throw" work. Copilot also handled the bigger, multi-step chunks: scaffolding the Gradio screens and routing, writing the OCR/extraction/categorization pipeline, debugging the dark-mode CSS variable issue across the whole UI (tracing a theme bug across ~70 CSS custom properties by diffing computed styles in a real browser context), generating fixture receipts and screenshots for this write-up, and drafting this very post.

My First User

Maybe Me if it counted:] — I've been using it while building it, which is how the "missing tax" and duplicate-detection flags ended up in Perch; I kept making the exact mistakes the app now catches.

Prize Categories

• Best Use of Gemma — gemma2:2b is ReceiptOwl's default local extraction model (config.py: DEFAULT_MODEL = MODEL_OPTIONS[0]), run through a local Ollama server. It's the first thing the pipeline tries before falling back to the heuristic extractor.
• GitHub Copilot — used throughout development for inline code completion and Copilot Chat debugging, alongside project

---

Still rough in places — see the repo's README for known limitations — but the core idea works, and I wanted to put it out there now rather than wait.