Udhaar-Khata: Voice AI Digital Ledger for My Father Using 100% Local Open-Weights (Gemma 2 + Whisper)

#devchallenge #weekendchallenge #hf26challenge #ai

Hacktoberfest Weekend Challenge: Build for a Friend Submission 🤝

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built Udhaar Khata (उधार खाता) — a voice-powered, 100% local open-weights digital ledger and credit book built specifically for my father.

My father manages a neighbourhood shop in India. Like millions of small business owners and community elders, managing daily customer credit (udhaar) is a constant challenge. Customers regularly take essential items (rations, groceries, milk) on credit and settle their balances later.

For decades, my father has tracked everything in dog-eared paper notebooks (bahi khata). Whenever I suggested commercial bookkeeping apps, he found them frustrating:

Smartphone touch keyboards are awkward and slow when your hands are busy handling merchandise and customers are waiting in line.
The menus are cluttered with English-first business jargon and ads.
Most importantly, he hated the idea of uploading his customers' personal phone numbers and debt history to unknown third-party cloud servers.

Udhaar Khata solves this completely through voice-first open-weights AI.

My father simply taps the microphone and speaks naturally in colloquial Hindi or Hinglish:

"Ramesh ne 200 udhaar liya" (Ramesh took ₹200 on credit)
"Sunita ne 500 wapas kiye" (Sunita repaid ₹500)
"Sharma ji ko 1200 ka ration udhaar diya" (Sharma ji took ₹1200 worth of ration on credit)
"Mukesh ne 450 jama karwaye" (Mukesh deposited ₹450)

The application transcribes the speech locally using OpenAI Whisper, extracts the customer name, rupee amount, credit/debit direction, and item notes using Google Gemma 2 (2B), displays a clean confirmation card with audio playback, updates the customer's balance sheet, and even crafts polite, culturally respectful Hindi payment reminders with 1-click WhatsApp messaging!

Demo

📸 Dashboard Overview

🎥 End-to-End Walkthrough (Voice Input, Confirmation, Reminders & Settlements)

Voice & Speech Input Hub: Pulsating mic button with audio waveform animation, bilingual speech recognition, and instant sample chips.
AI Extraction & Confirmation Card: Displays detected customer name, amount, credit/debit toggle, and audio confirmation feedback before committing to the ledger.
Customer Directory & Balance Sheet: Instant breakdown of Kispar kitna baaki hai (Who owes what), color-coded red for Lena hai (To collect) and green for Hisaab chukta (Settled).
Culturally Respectful Payment Reminders (तगादा): Generates reminders in 3 distinct tones (विनम्र / Polite, मित्रतापूर्ण / Friendly, व्यावसायिक / Formal) plus custom AI drafting, with a direct 1-click WhatsApp Share button.
1-Click Settlement ("Hisaab Chukta"): One click settles debts with confetti celebration.
100% Offline & Private: Zero external cloud APIs, zero subscription fees.

Code

The entire source code is structured with a Python FastAPI backend and a modern React Vite frontend:

GitHub Repository: https://github.com/ish1416/Udhaar-Khata

How I Built It

Udhaar Khata is built from the ground up around open-weights AI models running locally on consumer hardware:

1. Information Extraction & Semantic Parsing (Google Gemma 2 2B)

We run Google's Gemma 2 2B (gemma2:2b) locally via Ollama with Metal GPU acceleration on Apple Silicon.
Gemma 2 is prompted to understand colloquial Indian financial terms (udhaar, jama, chukta, baaki, rokda, saman) and outputs strict structured JSON containing customer_name, amount, type, and note.
To guarantee 100% uptime and sub-millisecond responsiveness even under cold-start conditions, we paired Gemma 2 with a deterministic Indic-NLP bilingual rule engine.

2. Speech-to-Text (Open-Weight OpenAI Whisper)

Audio transcription runs through open-weight faster-whisper (ctranslate2 + int8 quantization) and the browser Web Speech API, allowing my father to speak naturally in Hindi (hi-IN) and Hinglish without typing a single character.

3. Polite Hindi Reminder Generator

In Indian business culture, asking customers to repay credit can feel awkward or socially sensitive.
Using Gemma 2 2B, the app drafts culturally nuanced payment reminders in three predefined tones (Polite, Friendly, Formal) as well as custom prompts (e.g. "Tell them distributor payment is due tomorrow"), and encodes them into instant WhatsApp click-to-chat links (https://wa.me/...).

4. Local Database & Privacy Architecture

All customer accounts, contact numbers, and transaction ledgers are stored in a local SQLite database on my father's laptop. Not a single byte of financial data ever touches an external cloud server.

Why Does Open Innovation Matter?

This project is a living example of where an open-weights approach worked infinitely better than closed commercial APIs:

Financial Sovereignty & Absolute Privacy: Small shopkeeper credit books contain confidential debt records, personal trust networks, and customer phone numbers. Sending this sensitive financial ledger to third-party proprietary APIs (like OpenAI or Claude) is an unnecessary privacy and trust risk. With Gemma 2 and Whisper running locally, my father's records stay 100% on his machine.
Works in Marketplaces with No Internet: Indian local markets and small towns frequently experience broadband cuts and spotty mobile reception. Closed APIs fail the moment the Wi-Fi drops. By running open-weights inference locally, Udhaar Khata works seamlessly 24/7 without internet.
Zero Operating Cost (Forever ₹0): A small neighbourhood shopkeeper operating on tight margins cannot pay metered token bills or $20/month SaaS subscription fees. Open weights allow this application to run indefinitely on consumer hardware for free.
Colloquial Linguistic Freedom: Open models gave us the flexibility to fine-tune system prompts for colloquial Hinglish dialects without hitting arbitrary API censorship or rigid guardrails around financial language.

My Agent Session

I built this project with pair programming and browser automation. You can explore the full session transcript here:

Building Udhaar-Khata: Voice AI Digital Ledger for Father using Local Gemma 2 and Whisper

You

I am participating in Hacktoberfest Weekend Challenge: Build for a Friend. Building Udhaar Khata for my father using local open-weight AI models (no closed APIs). He speaks in Hindi/Hinglish like 'Ramesh ne 200 udhaar liya' or 'Sunita ne 500 wapas kiye'.

Agent

Architecting Udhaar-Khata with 100% local open-weights AI: Google Gemma 2 (2B) running via Ollama on Apple Silicon for information extraction and polite reminder drafting, OpenAI Whisper for local speech-to-text, and a bilingual React interface with SQLite local storage.

Agent

Installed and configured Ollama with gemma2:2b locally. Verified local inference with sub-second Hindi/Hinglish transaction parsing and entity extraction.

Agent

Implemented polite payment reminder generator with multiple culturally respectful Hindi tones (Vinamra, Mitratapoorna, Vyavsayik) and 1-click WhatsApp deep integration. Tested end-to-end via browser automation.