DEV Community

Awej Pathan
Awej Pathan

Posted on

ReceiptSage: A Local-First AI Receipt Organizer for Freelancers & Creators

#devchallenge #hf26challenge

What I Built

ReceiptSage is a local-first receipt and invoice organizer engineered specifically for freelancers, independent creators, side-hustlers, and small business owners.

Freelancers frequently handle a chaotic avalanche of PDF receipts, phone photos, and digital invoices for cloud infrastructure, domain renewals, coffee client meetings, and office equipment. Existing accounting software creates an unnecessary dilemma:

  1. Manual Entry: Hours spent doing tedious, error-prone data entry in spreadsheets.
  2. Cloud SaaS Platforms: High recurring fees ($15–$35/month) combined with serious privacy hazards: uploading sensitive financial documents—containing private billing addresses, tax IDs, and bank transactions—to closed cloud AI providers.

ReceiptSage solves this with open innovation:

  • Ingestion: Ingests pasted text, plain text files, digital PDFs (pypdf), and scanned images (pytesseract).
  • Open-Weight Gemma AI: Extracts structured candidate fields (Merchant, Date, Currency, Subtotal, Tax, Total, and Category) using Google's open-weight Gemma model (gemma3:1b) run locally via Ollama.
  • Deterministic Rule Fallback: Automatically runs a regex extraction engine if Ollama is offline or on resource-constrained devices, ensuring zero application downtime.
  • Math Reconciliation & Human Verification: Checks Subtotal + Tax == Total and requires user confirmation before marking records verified.
  • Local SQLite Persistence: Provides persistent storage in data/receiptsage.db with dual-mode support for ephemeral in-memory sessions during public demos.
  • Duplicate Detection: Prevents accidental double-entries using SHA-256 document hashing and content matching (merchant + date + amount).
  • Visual Analytics & Filters: Real-time filtering by currency, category, and month, paired with visual category spending charts.
  • AI Monthly Expense Summary: Generates an executive plain-language spending audit in Markdown using Gemma or statistical analytics.
  • Safe Exports & Backups: Exports RFC 4180 CSV with formula injection protection and creates full JSON backups with one-click restore.

Demo

Here is the end-to-end workflow:

  1. Privacy Notice & Presets: The app opens with a clear local-first privacy notice and 4 realistic synthetic presets (Indian GST, US Cloud SaaS, European Cafe, UK Cab).
  2. Extraction & Math Check: Click "Extract candidate fields". Gemma extracts candidate fields in seconds; an automated check validates whether Subtotal + Tax == Total.
  3. Duplicate Detection: If the document or amount was already registered, ReceiptSage alerts the user immediately with the matching record ID.
  4. Human Review: The user inspects and edits candidate fields, adds optional project notes, checks the confirmation box, and saves to SQLite.
  5. Multi-Currency Register & Charts: The register displays live metric totals grouped by currency (INR, USD, EUR, GBP) and renders an expense breakdown bar chart.
  6. AI Monthly Summary: Clicking "✨ Generate AI Expense Summary" produces a plain-language executive spending review.
  7. Safe Exports: One-click download of spreadsheet-safe CSV (utf-8-sig with formula escaping) and JSON backup export/restore.

(All public demos use synthetic receipts; no personal or client data is ever exposed.)

Code

The complete source code is publicly available on GitHub:

👉 GitHub Repository: https://github.com/awejofficial/ReceiptSage_Starter_Repo

  • License: Permissive MIT License (see LICENSE).
  • AI Model Terms: Governed by Google's Gemma Terms of Use (see MODEL_TERMS.md).
  • Dev Environment: Preconfigured for GitHub Codespaces (.devcontainer/devcontainer.json).
  • CI/CD: Automated testing on every push and pull request via GitHub Actions (.github/workflows/tests.yml).
  • Cloud Deployment: Preconfigured Render Blueprint (render.yaml).

How I Built It

ReceiptSage was built in Python 3.12 with modular architecture:

1. Ingestion & Preprocessing

  • Digital PDF extraction via pypdf without network calls.
  • Image OCR via Pillow and pytesseract with graceful error handling when Tesseract is missing.
  • Input security: 10 MB file ceiling and 10,000-character text window.

2. Open-Weight Gemma Integration (via Ollama)

  • Connects to Ollama's local REST API (http://127.0.0.1:11434/api/generate) with candidate model gemma3:1b (or gemma2:2b).
  • Prompt injection defense: document content is enclosed inside strict boundary delimiters (--- BEGIN UNTRUSTED RECEIPT TEXT ---) with system directives ordering the model to treat text strictly as passive data.
  • Structured JSON output with zero temperature for deterministic reliability.

3. Resilient Deterministic Fallback

  • Catches network/timeout errors and seamlessly executes regular-expression rules.
  • Detects merchants, dates (YYYY-MM-DD, DD/MM/YYYY, DD Mon YYYY), currencies (INR, USD, EUR, GBP), subtotals, split CGST + SGST tax aggregation, and totals.
  • UI explicitly displays whether extraction used Gemma + rules or Rules-only fallback.

4. Pydantic Schemas & Local SQLite Persistence

  • Schema validation via Pydantic v2 (ReceiptFields, ExpenseRecord).
  • Dedicated SQLite database (data/receiptsage.db) with indices on dates, categories, and hashes.
  • Real-time duplicate detection based on SHA-256 source hashing and content matching.
  • JSON export and import for full backup/restore.

5. Automated Testing (44 Tests, 100% Pass)

A comprehensive 44-test suite verifies all features in under 3 seconds:

python -m pytest -v
# 44 passed in 2.94s
python -m compileall -q app.py src tests
# Clean compilation (Exit code 0)
Enter fullscreen mode Exit fullscreen mode

Why Does Open Innovation Matter?

Open innovation—specifically open-weight AI—is transformative for solo entrepreneurs and side-hustlers:

  1. Uncompromised Financial Privacy: Financial invoices contain banking details, tax numbers, and private residential addresses. Running open-weight Gemma on local hardware guarantees that private documents never leave the user's laptop.
  2. Zero Marginal Operating Cost: Solo creators operating on thin margins cannot afford $20/month AI subscriptions or volatile pay-per-token API fees. Open-weight inference runs on local CPU cycles for free.
  3. Resilience & Autonomy: Freelancers work from trains, flights, and cafes with spotty connectivity. Local open-weight AI with deterministic fallback ensures 100% uptime regardless of internet availability.
  4. Compliance & Trust: Adheres strictly to Google's Gemma Terms of Use (April 2026), treating AI output as untrusted candidates requiring human sign-off.

Target Audience & Commercial Potential

While ReceiptSage is released as a free, open-source tool, it addresses a validated commercial market:

  • Target Persona: Freelancers, digital nomads, and independent consultants who submit monthly expense reports to clients or accountants.
  • Commercial Opportunity: A one-time purchase desktop application ($29–$49) or a packaged desktop installer (Tauri/Electron bundling the local engine) for non-technical users seeking privacy without command-line setup.

Known Limitations

  • Scanned PDFs: Digital PDFs with embedded text parse natively via pypdf. Image-only / scanned PDFs require conversion to PNG/JPG or local Tesseract OCR.
  • Hardware Latency: Local inference speed with gemma3:1b depends on the host CPU/GPU (typically 2 to 5 seconds per receipt).
  • Productivity Aid: ReceiptSage is an organizational productivity tool, not certified accounting software or automated tax filing.

Prize Categories

  • Best Use of Gemma: ReceiptSage implements Google's open-weight gemma3:1b model running locally through Ollama to parse unstructured receipts into schema-validated JSON, featuring untrusted prompt isolation, resilient fallback, and compliance with the official Gemma Terms of Use.
  • Best Use of GitHub Copilot / Actions: The repository features automated continuous integration via GitHub Actions (.github/workflows/tests.yml), testing dependency installation, executing the 44-test pytest suite on Ubuntu with Tesseract, and running module compilation checks on every commit.
  • Best Use of Render: Preconfigured with an infrastructure-as-code Render Blueprint (render.yaml) for one-click deployment of the headless Streamlit web application on Python 3.12.

![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/n3mbcj

z1ajs3bljyyx29.png)

Top comments (1)

Collapse
 
keelsheet profile image
Keelsheet •

Requiring a human confirm before a record counts as verified is a nice touch. Since you're aiming at freelancers, here are a few things from the US tax side that would make the export a lot more useful at filing time (other countries differ, so treat these as US-only):

  1. Map categories to Schedule C lines. US self-employed people report expenses on Schedule C, Part II, which has fixed lines: advertising (8), car and truck (9), contract labor (11), legal and professional (17), office expense (18), supplies (22), travel (24a), meals (24b), utilities (25) and other expenses (27a). If your categories roll up to those lines, the CSV totals drop straight into the form or go to an accountant without re-sorting. Cloud hosting and domains don't have their own line, so letting users choose where they land helps.

  2. Prompt for business purpose on meals and travel. IRS Publication 463 says to record the amount, date, place and business purpose, and for meals who you met with. A receipt alone doesn't show the coffee was a client meeting. Your optional project notes field is close; asking "who and why" when the category is meals or travel would cover it.

  3. Show meals at 50%. Business meals are generally only 50% deductible, so a meals total presented as fully deductible overstates the deduction. Keep the full amount and show the deductible amount next to it.

  4. Keep the original file, not just the extracted fields. The IRS generally expects records kept for at least 3 years after you file, and the source document is what holds up if anyone asks. Since you already hash files for duplicate detection, storing the original next to the hash turns the JSON backup into a real audit trail.

One small thing on the math check: US restaurant receipts often add the tip after the tax line, so Subtotal + Tax won't equal Total. A tip field would cut a lot of false mismatches.