LedgerLens: A Privacy-First Receipt Sorter That Never Uploads Your Client's Data

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built
LedgerLens is a receipt and invoice sorter for freelancers that runs the expensive part of the pipeline — reading the document — entirely on your own laptop.

Here's the problem it solves. If you're a freelancer, you already know the drill: every quarter you sit down with a shoebox of receipts and a spreadsheet, and you retype vendor names, dates, amounts, and categories by hand. It's an hour or two of pure tedium, and it's the exact kind of task an LLM should be doing for you.

But freelancers have a specific reason to be nervous about that. Receipts contain client names, project details, rates, and sometimes partial card numbers. Handing that to a hosted vision API means uploading your clients' financial information to a server you don't control, under a privacy policy you didn't write, on a per-image bill you have to absorb.

So LedgerLens splits the job:

The document never leaves your machine. A local Gemma model, running through Ollama, reads the image and extracts vendor, date, amount, and a short neutral summary. No network call.

Only the anonymized summary goes to the backend. The vendor name stays local; the summary is something like "monthly subscription for design software, billed once." No client names, no card numbers, no images.

TabPFN classifies it. A backend on Render takes that summary plus the amount and predicts a tax category — Software, Office Supplies, Travel, Meals, Equipment — using a model fitted on the freelancer's own past categorizations.

You get a CSV. Categorized, ready to hand to your accountant.

Someone would pay for this because the alternative is either an hour of their Sunday or uploading client data to a company they've never heard of. LedgerLens costs essentially nothing to run, which means it can be sold for $5/month instead of $30/month — the pricing actually works as a side hustle.

Demo
Deployed frontend: https://ledgerlens.onrender.com

The demo walks through: dropping in a photo of a receipt, watching the local extraction step (you can see the network tab stays quiet during it), then the classification coming back from Render a second later.

Code
GitHub repo: https://github.com/vaibhav7549/ledgerlens

text
ledgerlens/
├── local/
│ ├── extract.py # Gemma via Ollama — runs on user's machine
│ └── requirements.txt
├── backend/
│ ├── main.py # FastAPI + TabPFN classifier
│ ├── embed.py # sentence-transformers for text → vector
│ ├── seed_invoices.csv # example historical categorizations
│ └── requirements.txt
├── frontend/
│ └── index.html
└── render.yaml
How I Built It
The local layer: Gemma via Ollama.

The user installs Ollama and pulls a Gemma model:

bash
ollama pull gemma2:2b
local/extract.py takes an image path and asks Gemma for structured output:

python
import ollama, json, base64

PROMPT = """Read this receipt. Return ONLY JSON with these keys:
vendor, date (YYYY-MM-DD), amount (float), summary.
The summary must describe WHAT was purchased in neutral terms.
Do NOT include any client names, project names, or card numbers in the summary."""

def extract(image_path):
with open(image_path, "rb") as f:
img = base64.b64encode(f.read()).decode()

resp = ollama.chat(
    model="gemma2:2b",
    messages=[{"role": "user", "content": PROMPT, "images": [img]}],
    format="json",
)
return json.loads(resp["message"]["content"])

That Do NOT include line in the prompt is doing real work. It's the boundary between the private half of the pipeline and the public half, and I test it explicitly — see the sanitization test in backend/tests/test_sanitize.py.

The classification layer: TabPFN on Render.

The interesting part is that TabPFN is a tabular foundation model, and a receipt summary is text. So the backend converts the summary to a vector first with a small local sentence-transformer, then feeds [embedding..., amount] to TabPFN as a feature row. What TabPFN sees is a table; what the user typed is English.

python
from fastapi import FastAPI
from tabpfn_client import TabPFNClassifier, set_access_token
from sentence_transformers import SentenceTransformer
import numpy as np, pandas as pd

set_access_token(os.environ["TABPFN_API_KEY"])
embedder = SentenceTransformer("all-MiniLM-L6-v2")

app = FastAPI()

def featurize(summary: str, amount: float) -> list[float]:
vec = embedder.encode(summary).tolist()
return vec + [amount]

def train():
df = pd.read_csv("seed_invoices.csv")
X = np.array([featurize(s, a) for s, a in zip(df.summary, df.amount)])
y = df.category.values
clf = TabPFNClassifier(model_path="v3.5_default")
clf.fit(X, y)
return clf

classifier = train()

@app.post("/classify")
def classify(payload: dict):
x = np.array([featurize(payload["summary"], payload["amount"])])
return {"category": classifier.predict(x)[0]}
There's a real design decision here worth naming: TabPFN is fitted on the individual freelancer's history, not on a global dataset. Twenty of your own past receipts is enough to get useful predictions. That's a property a general-purpose LLM API can't match, because you can't fine-tune it on twenty rows without paying for a training run.

The deployment layer: Render.

The FastAPI service is deployed as a Render Web Service, defined in render.yaml:

yaml
services:

• type: web name: ledgerlens-api runtime: python buildCommand: pip install -r backend/requirements.txt startCommand: uvicorn backend.main:app --host 0.0.0.0 --port $PORT envVars:
  • key: TABPFN_API_KEY sync: false Render handles the build, the TLS, and the restarts. The TabPFN API key lives in Render's environment variables and never touches the client.

The connection between the two halves.

The local script is what orchestrates everything. It extracts locally, strips the vendor name, POSTs the sanitized summary to Render, and appends the returned category to a local CSV. The user's document store and the classification service never share a file.

python
def process(image_path):
data = extract(image_path) # local only
safe = {"summary": data["summary"], "amount": data["amount"]}
r = requests.post(f"{API_URL}/classify", json=safe)
row = {**data, "category": r.json()["category"]}
append_to_csv("ledger.csv", row)
Why Does Open Innovation Matter?
Three things about this project are only possible because the models are open.

Privacy is architectural, not contractual. With a hosted vision API, "we don't train on your data" is a promise in a terms-of-service document. With a local Gemma model, it's a fact about where the bytes went. The receipt images never cross a network boundary because there is no code path that sends them. A freelancer can hand this to a client and say your invoice data stayed on my laptop and mean it literally. No closed API can offer that, because the whole product is the API call.

The unit economics actually work. Per-image pricing on a hosted vision model means every receipt the user sorts costs me money. At $5/month, a heavy user could put me underwater. Running Gemma locally moves that cost to the user's own hardware, where it's zero. The only paid call in the whole pipeline is the TabPFN classification, which is one small request per receipt instead of one vision request per receipt. This is the difference between a side hustle that can be priced to sell and one that has to be priced to survive.

The classifier belongs to the user. TabPFN gets fitted on the freelancer's own categorization history, which means their weird personal convention — that they file their phone bill under "Utilities" and their domain renewals under "Software" — becomes part of the model. You can't do that with a closed API without a fine-tuning bill, and you can't do it per-user at all at $5/month. And because the local model is swappable, a user with a beefier laptop can drop in a larger Gemma and get better extraction without asking anyone's permission.

The last one matters most to me. The open pieces aren't decoration on top of a closed product — remove any of them and the design collapses.

Prize Categories
Best Use of Gemma — the extraction layer runs Gemma locally via Ollama, and it's what makes the privacy guarantee real.

Best Use of TabPFN — TabPFN-3.5 is the classifier, fitted per-user on embedded receipt summaries plus amount, inside a wider prediction flow.

Best Use of Render — the FastAPI service and frontend are deployed as a Render Web Service via render.yaml.