This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
Every quarter, my brother, who runs a freelance graphic design studio in Indore loses an entire weekend to a shoe box of crumpled paper slips, WhatsApp photos, and email PDF invoices. They type every line into a spreadsheet by hand. They tried commercial expense apps, but deleted them immediately: those apps upload receipts to third-party cloud servers. Receipts carry home addresses, personal bank details, client names, and tax numbers. For privacy-conscious freelancers and small businesses, the cloud is a non-starter.
I built SlipSense to solve this. SlipSense is an open-source, local receipt sorter. You point it at a folder on your computer, and it extracts the data, validates the math, flags questionable rows, and organizes files into clean accounting categories. It runs entirely on your local machine using an open vision model in Ollama. Your files never touch an external server, and it works with Wi-Fi switched off.
The SlipSense dashboard: point to an input folder of receipts, confirm local Ollama status, and process in one click.
Who would pay, and how
The target audience is solo freelancers, tax consultants, and small shop owners who track business expenses for deductions. Cloud tools charge \$10 to \$30 every month for features a solo operator rarely uses.
The product model is a simple one-time desktop license or an annual "tax season pack" priced at INR 499 for standalone offline use.
To be completely clear: I have not charged anyone yet, and SlipSense currently has zero paying customers.
How it outputs data
SlipSense normalizes every document into nine standard columns:
Status | Vendor | Sub Total | Tax | Total | Currency | Category | Date | Reason
Here is a real example from my test runs on receipt_01_amazon.jpg:
| Status | Vendor | Sub Total | Tax | Total | Currency | Category | Date | Reason |
|---|---|---|---|---|---|---|---|---|
| OK | Amazon India | 1101.69 | 197.31 | 1299.00 | INR | Software | 2024-09-12 |
On this receipt, the tax was split into 9% CGST (₹98.66) and 9% SGST (₹98.66). SlipSense extracted both tax lines, summed them into a single tax figure (₹197.31), verified that 1101.69 + 197.31 = 1299.00, matched the category from categories.yaml, and passed the validation check.
The review table showing extracted fields, currency tags, and automatic Needs-review flags.
Catching errors with human-in-the-loop review
A small local model will make mistakes. SlipSense assumes the model will fail on difficult images. Instead of blindly trusting model outputs, rule-based verification checks the numbers. If a vendor is missing, if total is zero or negative, if subtotal plus tax differs from the total by more than 1%, or if the model required a retry, the row is flagged Needs review with a plain-English reason.
Flagged rows automatically sort to the top of the table. Clicking any row opens an inspector showing the original receipt side-by-side with editable fields.
Side-by-side inspector card: examine the document while confirming or editing flagged fields.
When you export, SlipSense generates expenses.csv, copies each receipt into output/<Category>/YYYY-MM-DD_Vendor.ext without ever modifying your original files, and creates a category_summary.csv. The summary groups totals strictly by category and currency, never adding US Dollars to Indian Rupees.
Summary table aggregating expenses by category and currency, with grand totals.
Demo
The video shows SlipSense running on a laptop with Wi-Fi physically disconnected: selecting a folder of raw receipts, extracting structured data in real time, reviewing a flagged item in the inspector, and exporting the final spreadsheet and categorized folders.
Code
SlipSense is licensed under the open-source MIT License (Copyright (c) 2026 SlipSense Contributors).
How I Built It
Architecture
The entire pipeline runs locally in Python and Streamlit:
[Receipt Images & PDFs]
│
▼
[image_prep.py] ── In-memory rendering (PyMuPDF at 110 DPI),
EXIF rotation fix (ImageOps.exif_transpose),
RGB conversion, in-memory resize (max_side <= 1000px),
JPEG quality 85 in io.BytesIO
│
▼ (in-memory base64 string; zero disk temporary files)
[Ollama] ── Model: qwen2.5vl:3b
JSON schema constraint via "format" parameter
│
▼ (raw JSON response)
[Pydantic Validation] ── Coerce numeric strings, currency symbols, nullable fields
│
▼
[validation.py] ── Deterministic checks: 1% arithmetic tolerance, future dates,
category matching, duplicate detection, tax derivation
│
▼
[Streamlit UI (app.py)] ── Interactive review table, inspector card, inline edits
│
▼
[exporter.py] ── expenses.csv, category_summary.csv,
safe copy into output/<Category>/YYYY-MM-DD_Vendor.ext
The stack
- Language & UI: Python 3.13, Streamlit
-
Local Vision Model:
qwen2.5vl:3brunning in Ollama on localhost:11434 -
Image & Document Processing: Pillow (PIL), PyMuPDF (
fitz) - Data Validation & Parsing: Pydantic v2, PyYAML, Pandas
Image preparation settings
All image pre-processing is configured in config.yaml with zero hardcoded parameters:
-
max_side: 1000 pixels (longest dimension downscaled in memory; smaller images are never enlarged) -
jpeg_quality: 85 (compressed in-memory viaio.BytesIO) -
pdf_dpi: 110 (rendered directly from vector page to raster image)
User originals are never touched, and no temporary resized images are written to disk.
Measured benchmark results
I wrote a standalone benchmark script (benchmark_sizes.py) that evaluates qwen2.5vl:3b on the same test receipts across three different resolutions (max_side = 1400, 1000, 800).
Here are the real measured numbers from benchmark_report.md:
| max_side | Average seconds per file | Files flagged "Needs review" | Total-field accuracy |
|---|---|---|---|
| 1400 | 36.50s | 0 / 5 | 100.0% (5/5) |
| 1000 | 8.57s | 0 / 5 | 100.0% (5/5) |
| 800 | 9.28s | 0 / 5 | 100.0% (5/5) |
A failure I hit
When testing resolution scaling, I initially tried sending larger images (max_side = 1400) believing higher resolution would yield better field extraction.
Instead, I hit a massive latency cliff. Vision transformers tokenize images into fixed-size visual patches. At 1400px, complex receipts like receipt_03_invoice_consulting.jpg and receipt_04_aws.jpg caused token counts to swell, spiking latency from 8.5 seconds up to 56.21s and 57.37s per document on my machine. Worse, accuracy did not improve by a single percent: max_side = 1000 achieved identical 100% total-field accuracy at 8.57s average—over 4.25x faster. Downscaling to 1000px proved essential for running locally without memory exhaustion.
Four design decisions I defend
-
Rule-based code decides trust, not model confidence scores: Vision models frequently hallucinate high confidence on wrong numbers. My validation code checks arithmetic (
subtotal + tax == total), verifies date boundaries, and flags discrepancies deterministically. -
The model never guesses: If a field is illegible, the prompt instructs the model to output
null. SlipSense then flags the row for human review instead of fabricating numbers. - Original files are sacred: SlipSense only reads originals. Copies are placed into category folders with sanitized names, and collision counters prevent overwrites.
- Needs-review rows sort to the top: In a folder of 50 receipts, a user should not have to hunt for the three that need human attention.
Why Does Open Innovation Matter?
- Absolute privacy and offline reliability: Receipt data contains bank details, vendor contacts, and private purchases. With open models and local runtimes like Ollama, data never leaves the device. The application functions in a remote cabin or on an airplane with Wi-Fi switched off.
- Zero marginal cost: Closed vision APIs charge per page or per API call ([CHECK CURRENT PRICING, e.g. $0.01 to $0.05 per document]). For an irregular freelancer processing 30 receipts one month and none the next, subscription fees and API meters add friction. An open model running locally costs zero cents per file.
-
Model independence and longevity: Closed cloud APIs deprecate endpoints, adjust system prompts, and change rate limits without warning. In SlipSense, switching to another model requires changing one line in
config.yaml(model_name: "qwen2.5vl:3b"). The software remains operational ten years from now on local hardware. -
Inspectable, customizable taxonomy: Business categories vary across countries and industries. Instead of opaque cloud categories, SlipSense uses an open
categories.yamlandexamples.yamlthat any user can adapt to their chart of accounts.
The honest counterweight
A closed frontier model (like GPT-4o or Gemini 1.5 Pro) running on cloud TPU clusters would be faster in raw throughput and better at deciphering crumpled, handwritten cursive or faded thermal receipts. A 3B local model requires decent CPU/RAM, takes about 8.5 seconds per document on a laptop, and cannot reliably read illegible handwriting. I chose the local model anyway because for accounting documents, absolute data confidentiality outweighs cloud speed.
My Agent Session
I built SlipSense pairing with Google Antigravity, Google AI Studio an agentic AI coding environment. Antigravity handled file generation, wrote comprehensive pytest suites, ran background CLI commands to measure model latency, and helped refine image preparation logic.
My First User
To be transparent: SlipSense currently has no paying customers, and I have not charged anyone.
I have reached out to several local shopkeepers and independent freelancers in my neighborhood to get early feedback on the offline workflow.
Prize Categories
I’m a backend developer, so ML was new ground for me. Tinker let me write a fine-tuning loop in plain Python without wrestling with GPUs, which meant I could actually focus on learning. I fine-tuned [MODEL] on [N] synthetic receipts and compared it with the base model: [REAL RESULT]. I used synthetic data only, since SlipSense exists to keep real receipts on the user’s laptop, and the final model still runs locally through Ollama.
Top comments (0)