BayanAI: Talk to Your Khata Instead of Writing It

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Every Karyana (small grocery) shop in Pakistan runs on a paper khata: a notebook of who took what on credit, who paid back, and what's left on the shelf. It works, but it costs the shopkeeper time at the busiest moment, when there's a queue at the counter and a pen in one hand.

BayanAI is a voice-controlled khata and stock app. The owner taps one button and speaks, in English, Urdu or Roman Urdu:

"Ahmad took 500 rupees of sugar on credit"
"احمد نے پانچ سو روپے کی چینی ادھار لی"
"Ahmad ne 200 rupay wapas diye"

The app shows what it understood. The owner taps Save, and the customer's balance and the stock update instantly.

The lazy payoff: the bookkeeping becomes one spoken sentence, said while the shopkeeper is still serving the customer. One sentence can even contain several commands.

Command	Example	Effect
credit	"Bilal took 2 kilo rice on credit, 600 rupees"	Bilal's balance +600, rice stock −2 kg
payment	"Ahmad paid back 200 rupees"	Ahmad's balance −200
sale	"Sold 10 kilo flour"	flour stock −10 kg
restock	"Add 50 packets of tea to stock"	tea stock +50 packets

Code (MIT licensed): https://github.com/Monam1083/BayanAI-

How It Works

Flutter app ──WAV──► AI service (local laptop)
                     1. faster-whisper: speech → text
                     2. Ollama (open LLM): text → JSON
                        {action, customer, item, qty, amount}
Flutter app ◄── confirm card ── tap Save ──► Firebase (Auth + Firestore)

• Speech to text: Whisper through faster-whisper, set to Urdu by default. It can be switched to English or auto-detect.
• Understanding: an open LLM served by Ollama turns the transcript into structured commands. I ran qwen2.5:3b on my laptop; the default in the repo is qwen2.5:7b.
• App and storage: Flutter, with Firebase Auth and Cloud Firestore. Balances and stock use atomic increments, and each command is saved as one batch.
• Security: Firestore rules limit each user to their own shops/{uid} data.

A few details that mattered in practice:

• Mixed languages: shopkeepers switch between English, Urdu and Roman Urdu mid-sentence. Roman Urdu product names are mapped to one English name (cheeni → sugar, chawal → rice), so stock stays in one line per product.
• Loose name matching: "Ahmed" finds "Ahmad", and a new name creates a new customer.
• Confirm before save: nothing is written until the owner taps Save. Noisy shops and mixed Urdu/English will sometimes produce wrong transcriptions, so the confirm card is the safety net. The owner can fix the text and re-run it, remove a wrong card, or type the command instead.

Why Open Innovation Matters Here

• The voice and the model stay on the shop's own machine. A khata records who owes whom, which is sensitive in a small community. Speech recognition and understanding run locally, so no audio goes to a paid AI API. To be precise, the ledger itself is stored in Firebase. What stays local is the voice and the model.
• No per-request cost. Whisper and the LLM run on a laptop, so a shopkeeper isn't paying every time they speak.
• Models can be swapped to fit the hardware. The LLM is one environment variable (OLLAMA_MODEL) and the Whisper size is another (WHISPER_MODEL). On a weak laptop I could drop to a smaller model, and a stronger machine can use a bigger one for better Urdu. A closed API wouldn't let me make that trade-off.
• Urdu is supported. Whisper handles Urdu, and an open LLM can be prompted to understand Roman Urdu, a messy mix of languages that many products ignore.

Limitations and What's Next

BayanAI is a hackathon prototype, and I want to be upfront about its limits:

• Login is anonymous, so a shop's data is tied to one app install.
• The AI service has no password and is meant for a trusted local network.
• Saved entries can't be edited or undone yet.
• Speech recognition on noisy shop audio and mixed Urdu/English can be wrong.

Next steps are phone or email login, edit and undo, and running everything on-device so the laptop isn't needed.

Team

Built at Hacktoberfest Hack Day Lahore 2026 by [Muhammad Monam Majeed ].