This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
KhataLens turns a photo of a handwritten udhaar notebook page into a clean digital ledger and one-tap WhatsApp payment reminders. It runs entirely on the shopkeeper's own computer.
Walk into a neighbourhood kirana, dairy or medical shop and you will find the same thing behind the counter: a ruled notebook where the owner writes who took what on credit (udhaar) and who paid it back (jama). It is how a lot of small-shop credit works, and it has three problems:
- Nobody can answer "who owes me, and how much?" without adding up pages by hand.
- Money gets forgotten because chasing it is awkward and nothing prompts it.
- The notebook holds your neighbours' names and what they owe. That is not something an owner wants to upload to a cloud service.
KhataLens fixes this without changing the owner's habit. They keep writing in the notebook, photograph the page, and the app:
Reads the handwriting (Hindi and other Devanagari, mixed date formats, Devanagari digits).
Shows every line in an editable table, with suspicious lines highlighted and a reason next to each.
Saves the confirmed lines to a local ledger.
Shows who owes me, filterable by how long since the last entry, and a needs a check view for customers who appear to have paid more than they owed.
Opens WhatsApp with a ready-written reminder in Hindi, Marathi, Tamil or English. Nothing is sent automatically; the owner reviews and sends it.
Who would pay, and how it is a side hustle. The customer is the small shop owner who has years of notebooks and no time to type them in. The service I would sell is setup and digitization: install it on the shop's existing laptop, scan the backlog of pages, set up reminders, and be on call afterwards. My planned pricing is a one-time setup fee of ₹999 and ₹149 per month for support. These are planned prices that I have not yet validated with a shop owner. Because inference is local, there is no per-scan cloud bill eating the margin, and the shop never needs a subscription to an AI provider.
Demo
Code
KhataLens
Photograph a handwritten udhaar (credit) notebook page. Get a clean ledger and one-tap WhatsApp payment reminders. Everything runs on the shopkeeper's own computer using an open-weight vision-language model served by Ollama. No customer data leaves the machine.
The app is a plain HTML/CSS/JavaScript web page (no build step, no CDN, works offline)
served by a small local Python server that wraps the khatalens package.
Quick start
pip install -r requirements.txt
ollama pull qwen2.5vl:7b # or any vision model your hardware can run
python scripts/synth_ledger.py --n 20 --out data/synth # sample pages for the demo
python server.py # opens http://127.0.0.1:8000
Click Try a sample page to test the review -> save -> remind flow with no model, or
upload / photograph a real page and click Read this page. Use another model with
KHATALENS_MODEL=<name> or in Settings (this is where your fine-tuned model plugs in).
Server options:
python server.py…Quick start (from the README):
pip install -r requirements.txt
ollama pull gemma3:4b
KHATALENS_MODEL=gemma3:4b python server.py # opens http://127.0.0.1:8000
# (Windows PowerShell: $env:KHATALENS_MODEL="gemma3:4b"; python server.py)
How I Built It
The open-weight AI: Gemma 3 4B, Google's multimodal open-weight model, served locally by Ollama. It reads the page image directly and returns the entries as structured JSON. At 4B parameters it is small enough to run on an ordinary laptop, which matters because that is what a shop owner has. I ran it on a Ryzen 5 laptop with 8 GB RAM, about 100-120 seconds per page. The model name is a setting (KHATALENS_MODEL or the Settings drawer), so any vision model Ollama can serve, including a fine-tuned Gemma, plugs in without code changes.
The pipeline around it:
photo -> preprocess -> local VLM -> validate -> human review -> SQLite ledger -> WhatsApp link
Preprocess (OpenCV): resize to a fixed maximum side, even out phone-shadow lighting with CLAHE, and deskew using the notebook's ruled lines. I kept this light on purpose, because heavy binarisation tends to hurt vision models more than it helps.
Structured output: the model is asked for JSON that matches a schema, so the output is data, not prose.
Validation layer: converts Devanagari digits, normalises dates (including missing years), and flags suspicious rows, such as an unreadable amount, instead of silently trusting the model.
Human review before saving: a small 4B model will misread handwriting sometimes, especially smudged amounts and unusual letter shapes. Rather than hide that, the product is built around it: every line is editable, and flagged lines are highlighted.
SQLite ledger: balance is total udhaar minus total payments. Customers are Pending (owes), Settled (exactly zero) or Needs a check (paid more than recorded). Deleting an entry is only allowed in that last view, and the server enforces it.
Reminders: plain
wa.melinks with message text in four languages. There is no WhatsApp API, no account and no cost.UI: plain HTML, CSS and JavaScript (no build step, no CDN, works offline) served by a small Python standard-library server on
127.0.0.1. Three themes: Clean, Ledger and Midnight.Training and evaluation tooling: a synthetic handwritten-page generator that also writes a LLaMA-Factory training file, and an evaluation script that scores field-level accuracy for the base model, a fine-tuned model and a closed API on the same labelled pages.
Results so far. On my own handwritten page (Hindi, mixed date formats), the base model got 6 of 10 lines fully right before review. I have not fine-tuned yet; that is the next step (see below).
A real-world bug that tests missed. My sample pages worked, but the first real notebook photo crashed the server with a NumPy error. The cause was OpenCV's line detector returning a different array shape on my installed version than the one I had coded against. My sample-page mode skips the model and the image preprocessing entirely, so that code path never ran until I tried a real notebook photo. It is a small thing, but it is why the evaluation set has to be real handwriting.
What I would do next:
- Fine-tune a LoRA on synthetic pages plus pages in my own handwriting, and test only on handwriting the model has never seen.
- Compare base, fine-tuned and a closed API on the same test pages.
- Put it in front of a real shop owner.
Building it: I wrote the code with an AI coding assistant (Claude) in a chat session. The shipped product's AI is only the open-weight model running locally; nothing in the running app calls a hosted API.
Why Does Open Innovation Matter?
Three things would not be possible, or would not make sense, with a closed API.
Privacy is the product, not a feature. A udhaar notebook is a list of neighbours and the money they owe. A shop owner will not photograph that and send it to a third party, and I would not want to be the one holding it. With an open-weight model running through Ollama, the photo is never sent anywhere. "Your customers' names and amounts never leave this computer" is a statement I can actually make true.
The economics only work offline. A small shop may scan a few pages a week. A per-call API fee, a subscription, or a dependence on a stable internet connection all break that model. Local inference means the cost of the hundredth page is zero, and the app works when the shop's connection does not. A model as small as Gemma 3 4B is what makes that realistic: it runs on hardware the shop already owns.
I can adapt the model to the handwriting. Handwritten Devanagari, regional shorthand and every shopkeeper's own style are exactly where a general model is weakest. Open weights mean I can fine-tune on pages like the ones I am trying to read, and measure the gain against a closed baseline. With a closed API I could only rewrite the prompt.
Prize Categories
Overall winner, and Gemma (featured category).
Gemma: KhataLens uses Gemma 3 4B, running locally through Ollama, to read handwritten Devanagari ledger pages straight from a photo and return structured entries. Gemma is the only AI in the running app. Its small size is what lets the whole thing run offline on a shop's existing laptop.
Top comments (1)
Nice approach on keeping credit data strictly local for small merchants! How are you handling handwritten receipt noise?