DEV Community: Amrit Kang

I Built a Voice-First AI Khata for a Shopkeeper Who Shouldn’t Have to Type

Amrit Kang — Mon, 05 Oct 2026 06:21:29 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built Khata Ledger, a voice-first AI credit ledger for Family Friend, who runs Baba gernal store in Ludhiana.

Small neighbourhood shops often give groceries and other essentials on credit. These transactions are usually written in a notebook, saved in scattered messages, or simply remembered.

That works until the shop becomes busy.

A customer’s name gets forgotten, a payment is recorded incorrectly, or someone promises to pay on Friday without there being a useful reminder when Friday arrives.

Most accounting applications expect the shopkeeper to stop serving customers, open a form, type several fields, and understand formal bookkeeping terminology.

I wanted the interaction to feel as natural as speaking across the counter:

“Ramesh bhai ko 850 rupaye ka ration diya, Friday tak dega.”

Khata Ledger extracts:

Customer: Ramesh
Amount: ₹850
Transaction: credit given
Context: ration
Due date: Friday

It then safely adds the transaction to the shopkeeper’s private ledger.

The shopkeeper can:

create a private shop account;
record a voice note directly in the browser;
see a real waveform and recording duration;
discard and re-record an incorrect note;
convert natural Hindi or Hinglish into structured ledger data;
review uncertain entries before they affect a balance;
see every customer’s outstanding amount;
record repayments;
search and filter the complete passbook;
receive reminders only when payment is due;
generate a message containing the customer’s dated account history.

The application does not blindly trust its AI model. If an amount is missing or extraction confidence is below 0.6, the note is placed in an owner review queue instead of being added to the financial ledger.

AI can assist with bookkeeping, but uncertainty should never quietly become financial truth.

Demo

Live application: [https://khata-ledger-3deb.onrender.com]

GitHub repository:

https://github.com/amritkang165/khata-ledger

The main flow is:

Shopkeeper speaks
        ↓
Browser records temporary audio
        ↓
ElevenLabs Scribe transcribes it
        ↓
FastAPI sends the transcript to local Gemma
        ↓
Gemma extracts validated ledger fields
        ↓
Low confidence? → Owner review queue
Confident?      → Private SQLite ledger
        ↓
Balances, passbook and payment reminders

Example notes:

Ramesh bhai ko 850 rupaye ka ration diya, Friday tak dega
Sunita owes 420 rupees for school books, payment by Monday
Vanshika ne 500 rupaye wapas de diye
Deepak ko aaj 333 rupaye ka school samaan diya

Credit given increases the customer’s outstanding balance. A received payment reduces it.

Code

amritkang165 / khata-ledger

Voice-first, local-first AI credit ledger for small shopkeepers

Khata Ledger

Bol ke likho. Hisaab simple rakho.

Khata Ledger is a voice-first credit ledger for kirana stores and other small shops. A shopkeeper can speak a natural note such as:

“Ramesh bhai ko 850 rupaye ka ration diya, Friday tak dega.”

The app transcribes the audio, extracts the customer, amount, transaction type and due date, then safely adds the entry to that shopkeeper's private ledger.

Why this exists

Many neighbourhood shopkeepers still manage credit in notebooks, memory, or scattered chat messages. Traditional accounting products often expect careful typing, formal bookkeeping language, and constant connectivity.

Khata Ledger is designed around the way a shopkeeper already works:

speak naturally in Hindi, Hinglish, Punjabi-accented English, or everyday shop language;
review uncertain entries instead of silently saving bad financial data;
keep customer balances separated by shop account;
see exactly who owes money and when a reminder is actually due;
retain the real financial…

View on GitHub

The project is built with:

FastAPI
SQLite
Gemma 3 4B
Ollama
ElevenLabs Scribe v2
MongoDB Atlas Vector Search
Sentry Agent Tracing
Tinker
Vanilla HTML, CSS, and JavaScript
Docker

The repository contains the complete application, tests, synthetic datasets, Atlas scripts, Tinker evaluation assets, Docker configuration, and Render Blueprint configuration.

To run it locally:

git clone https://github.com/amritkang165/khata-ledger.git
cd khata-ledger

cp .env.example .env
uv sync --extra dev

ollama pull gemma3:4b
./start.sh

Then open:

http://localhost:8000

How I Built It

Voice transcription with ElevenLabs

The browser records a temporary audio file and sends it to the FastAPI backend.

The backend forwards that audio to ElevenLabs Scribe v2. Khata Ledger uses the returned transcript but does not save the uploaded recording.

This gives the project accurate speech transcription while keeping its financial system of record separate.

Local extraction with Gemma and Ollama

The transcript is processed by Gemma 3 4B running locally through Ollama.

Gemma converts everyday shop language into validated JSON:

{
  "customer_name": "Ramesh",
  "amount_rupees": 850,
  "due_day": "Friday",
  "context": "ration",
  "transaction_type": "credit_given",
  "confidence": 0.95
}

Natural shop language can be ambiguous.

Ramesh ko 500 rupaye ka ration diya

This means the shopkeeper gave goods on credit.

Ramesh ne 500 rupaye de diye

This means the customer repaid money.

A wrong interpretation would reverse the customer’s balance.

Along with the model prompt, I added deterministic transaction-direction safeguards for important phrases such as:

ration diya
udhaar diya
Friday tak dega
payment kar diya
jama kar diya
paise wapas diye

This combines the flexibility of an open model with predictable financial safeguards.

A review gate for uncertain entries

The model is not allowed to write every answer directly into the ledger.

An entry requires manual review when:

the amount is missing; or
model confidence is below 0.6.

The shopkeeper can correct the customer, amount, context, due date, or transaction type before saving it.

Duplicate-entry protection

People double-tap buttons, mobile connections fail, and browsers retry requests.

In a credit ledger, one repeated request could create a false debt.

Every submission receives an idempotency key. SQLite enforces that the same key cannot create two transactions for the same shop owner.

The button also locks while a request is processing, but correctness does not depend only on the frontend.

Local authentication and isolated ledgers

I deliberately kept authentication beside the ledger instead of requiring a hosted identity provider.

Accounts, sessions, customers, and transactions live in the same local SQLite database.

Passwords use salted PBKDF2-SHA256 hashes. Raw session tokens are not stored, sessions expire on the server, and every query is scoped to the authenticated owner.

Each registered shop therefore receives an isolated ledger.

Useful reminders rather than instant pressure

My first implementation immediately displayed a collection message for every outstanding balance.

That felt wrong.

If someone received groceries today, the app should not immediately pressure the shopkeeper to contact them.

The updated rule shows a reminder only when:

the promised due date has arrived; or
the credit has remained open for at least three days.

The application never sends a message automatically.

The shopkeeper taps Send message, reviews the draft, and can copy or share it. The message contains the current balance and the complete dated history of credits and payments.

Synthetic-only MongoDB Atlas search

I used MongoDB Atlas Vector Search to retrieve similar repayment patterns.

However, Atlas only contains explicitly synthetic customer profiles. Real customers, real transactions, and the SQLite ledger are never uploaded.

This allowed me to explore vector retrieval without breaking the project’s privacy boundary.

Sentry Agent Tracing

I added Sentry spans around the extraction pipeline.

The traces record:

model name;
extraction latency;
token usage;
fallback reason;
parsing failure;
whether manual review was required.

Parsed financial output is redacted by default.

This helped me confirm that the local model call dominated request time while FastAPI and SQLite work remained small.

Tinker fine-tuning experiment

I also ran a separate extraction experiment using Tinker and Qwen 3.5 4B.

The Tinker model catalog available during development did not expose a Whisper or audio fine-tuning path, so I measured transcript-to-ledger extraction instead.

The reproducible synthetic dataset contains:

160 training examples
40 fixed held-out examples

Metric	Baseline	Fine-tuned
Valid JSON	100%	100%
Customer exact match	100%	100%
Amount exact match	100%	100%
Transaction direction	80%	100%

This result measures text extraction accuracy, not speech-recognition word error rate. The application’s normal runtime continues to use Gemma through Ollama.

Why Does Open Innovation Matter?

Open innovation is central to Khata Ledger because this application handles personal financial relationships.

The ledger stays under the shopkeeper’s control

The real financial ledger remains in a local SQLite database. Gemma runs on the shopkeeper’s machine through Ollama.

Customer names, balances, and transaction history do not need to be stored by a hosted LLM provider.

The model can be inspected and replaced

The extraction prompt, validation schema, language rules, and fallback logic are all visible in the repository.

I can improve the prompt, change the model, add another language, or run a different open-weight model without rebuilding the product around another provider.

Local inference reduces recurring cost

After the model has been downloaded, extraction can run locally without paying for every ledger entry.

That matters for a product intended for small shops, where a recurring AI fee could cost more than the value provided by the application.

The application can degrade safely

If Ollama becomes unavailable, Khata Ledger uses a deterministic local fallback and reports that fallback in its response.

It does not silently pretend the model succeeded.

Privacy affected the architecture

“Local-first” was not added as marketing text after development.

It changed where I placed:

authentication;
model inference;
financial storage;
observability;
vector retrieval;
review and validation.

The application is not completely offline because high-quality voice transcription currently uses ElevenLabs. However, its financial system of record and structured AI extraction remain local.

My Agent Session

I did not record a DevRelay session for this build.

Instead, I kept the Git history intentionally divided into small, timestamped commits so that each backend, authentication, safety, and UI decision can be traced independently:

https://github.com/amritkang165/khata-ledger/commits/main/

Prize Categories

I am entering Khata Ledger in the following applicable categories:

Best Use of Gemma

Gemma 3 4B is the open-weight model at the centre of the transaction-extraction pipeline. It runs locally through Ollama and converts natural Hinglish shop notes into validated ledger data.

Best Use of ElevenLabs

ElevenLabs Scribe v2 transcribes voice notes before the transcript is processed by the local Gemma model. Audio is not persisted by Khata Ledger.

Best Use of Tinker

I used Tinker to fine-tune and evaluate transaction extraction on a reproducible synthetic dataset. Transaction-direction accuracy improved from 80% to 100% on the fixed held-out split.

Best Use of MongoDB Atlas

Atlas Vector Search retrieves similar repayment profiles from an explicitly synthetic-only collection while real financial data remains in local SQLite.

Best Use of Sentry Agent Tracing

Sentry traces show the complete extraction request, including model latency, token usage, parsing status, review requirements, and downstream HTTP work, while financial output remains redacted.

Building for one person

After trying the project, Hardeep Singh said:

“He'll most definately use this”

Their feedback directly changed UX had to make it more simple and easily readable.

Building for one real person made every decision more concrete.

The project began with one question:

What if maintaining a credit ledger felt as natural as speaking across the counter?

Open-weight local AI made that idea practical without asking the shopkeeper to surrender control of their financial records.

From Study Tracker to Study Companion: My Finish-Up-A-Thon Journey

Amrit Kang — Sun, 07 Jun 2026 22:15:57 +0000

This is a submission for the GitHub Finish-Up-A-Thon Challenge

From Prototype to Product: Why I Built Study Companion

As a student, I found myself constantly switching between apps while trying to study.

★One app for tasks.
★One for focus sessions.
★One for tracking progress.
★One for getting help when I was stuck.

Juggling multiple tools wasn't just annoying—it was a distraction machine. I didn't want another productivity app, I wanted a complete study system that students could genuinely rely on every single day.

That is why I built Study Companion

What is Study Companion?
Study Companion is a full-featured study productivity platform available on both web and mobile. It combines task management, focus tools, AI assistance, and progress tracking into a single, seamless experience.

Here is a look under the hood at what the platform includes:

• Subject & Topic Management
• Smart Task Management
• Pomodoro Timer & Focus Sounds
• AI Study Buddy
• Progress Analytics
• Consistency Heatmaps
• Cloud-Backed Storage
• Cross-Platform

The goal wasn't to build another productivity app.
The goal was to build a study system that students could genuinely rely on every day.'

At the start of the challenge, Study Companion already worked.
But it wasn't finished.

The GitHub Finish-Up-A-Thon gave me the perfect excuse to revisit the project and finally transform it from a working prototype into a complete product :)

Demo

Live Application: https://mystudy-companion.vercel.app/

Repository: https://github.com/amritkang165/study-companion

Before vs After

Data Storage

✿ Before

• Subjects stored in localStorage
• Tasks stored in localStorage
• Revisions stored in localStorage
• Data tied to a single browser
• No real backend

✿ After

• Full Supabase integration
• Dedicated database tables
• Row Level Security (RLS)
• Automatic migration of existing user data
• Cross device synchronization
• Persistent cloud-backed storage

This was the biggest architectural upgrade in the project.
What started as a browser-based prototype became a real cloud-backed application.

Key Improvements

⋆ Advanced Analytics : streak tracking, consistency heatmaps, weekly comparisons, progress visualizations, and priority insights generated entirely from existing task data.
⋆ Focus Mode : transformed a basic Pomodoro timer into a distraction-free study workspace with fullscreen mode, task panel, and focus sounds.
⋆ AI Study Buddy : upgraded to Groq + Llama 3.3 70B with summaries, question generation, and dedicated learning views.

⋆ Profile Management : editable profiles, synced user data, instant dashboard updates, and account statistics.
⋆ Mobile Support : fully responsive experience with mobile-optimized navigation and layouts.
⋆ UI Redesign : complete neo-brutalist design system with light/dark mode, improved accessibility, cleaner navigation, and a more polished dashboard experience.

The Comeback Story

I thought Study Companion was already finished. The tasks worked, the Pomodoro timer worked, and the AI worked.

• To truly finish it, I rebuilt major parts of the app—adding a proper backend with Supabase, real authentication, analytics, profile management, mobile support, and a complete UI redesign.

• I also refactored components, improved responsiveness, and polished the overall user experience.

• The result is a far more complete, scalable, and polished study platform than the version that originally entered the challenge.

Technical Highlights

Frontend

• React
• Context API
• Responsive Design
• Web Audio API

Backend

• Supabase Auth
• Supabase Database
• Row Level Security

AI

• Groq API
• Llama 3.3 70B Versatile

Features

• Optimistic UI updates
• Automatic data migration
• Cloud persistence
• Analytics engine
• AI-powered learning support
• Streak tracking
• Consistency monitoring
• Mobile and web support

My Experience with GitHub Copilot

GitHub Copilot served as a coding companion, turning ideas into implementations more efficiently.

I used Copilot extensively while:

Refactoring

• Large portions of the project were rewritten during the migration from localStorage to Supabase.
• Copilot helped accelerate repetitive CRUD patterns and state management logic.

Backend Integration

It was particularly useful while working with:
• Authentication flows
• Database queries
• Validation logic
• Error handling

UI Development

• Copilot helped speed up:
• Component creation
• Responsive layouts
• Dashboard improvements
• Styling iterations

Feature Development

• Analytics, profile management, Focus Mode, and AI-related features were much faster to iterate on thanks to Copilot's suggestions and scaffolding.

The result wasn't code that Copilot built for me.

It was code I could build faster because Copilot handled much of the repetitive work, allowing me to focus on architecture and user experience.