Orderly: turning WhatsApp voice notes into priced orders, on the shop's own computer

This is my submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

The side hustle

Every evening between 6:30 and 8:30 PM, Ramesh's neighborhood grocery store gets bombarded with WhatsApp messages and 15-second voice notes:

"bhaiya, do kilo cheeni, ek tel, aur adha kilo chai patti, Patil ke ghar bhej do."

Between bagging goods and attending to counter customers, Ramesh has to replay each audio note multiple times, scribble items on a paper slip, calculate the prices by hand, and type out an invoice reply.

Orderly takes that voice note or typed text and instantly produces an itemized, priced order and a ready-to-send WhatsApp reply. Would the shop pay? Ramesh confirmed he would readily pay ₹150–₹200 a month: "If it calculates my voice orders with zero math errors right from my counter laptop, it easily saves me an hour every single evening."

Demo

Watch the demo on YouTube

Code

GitHub Repository

Why open source mattered here

1. The data belongs to the customers. Voice notes carry names, home addresses and what a family buys. A small shop should not have to send that to a foreign cloud service to read its own orders. Orderly's speech-to-text (faster-whisper) and its language model (Gemma, via Ollama) both run on the shop's own computer.
2. The cost per order is zero. A shop that handles 30 orders a day cannot absorb a per-message fee. With open weights, the 10,000th order costs the same as the first.
3. The language gap is real. Customers rarely order in dictionary Hindi or Marathi. They mix Marathi words (tandul, sakhar, varan, pith), Hindi (cheeni, atta, bhej do), number words (don, ek, char) and fractions (aadha kilo). Because the models are open and local, I could test these real edge cases directly, choose the Whisper size, and tune the prompt for this exact kind of message. faster-whisper transcribes locally on the shop's computer, and Gemma extracts items and units as strict JSON at temperature 0. ## How it works

voice note -> faster-whisper -> Gemma (JSON, temperature 0) -> validator
           -> prices from the shop's own catalogue.json -> invoice + reply

The decision I'm most sure about: the model reads, the code prices. Gemma extracts items and quantities. It never sees a price. Prices come from the shop's catalogue, so the model cannot invent a total. Anything unclear (an item not in the catalogue, "2 litre" of something sold per packet, a quantity it cannot read) is shown in a "Check these" list instead of being guessed.

If Ollama is not running, a rule-based parser takes over and the page says which one was used.

Does it work? (real numbers)

I ran python eval.py --model gemma3:1b on 30 [[CONFIRM: "real customer messages" if they came from real customers, otherwise "realistic messages I wrote from the orders Ramesh gets"]] in Hindi, Marathi, Hinglish and English (eval/cases.jsonl):

• Whole orders exactly right: 30 of 30
• Item precision: 100%
• Item recall: 100% I tuned the prompt and parser using the mistakes on these 30, so I also wrote 10 fresh messages without looking at the code. On those: [[CONFIRM: X of 10 whole orders exactly right]].

One honest failure, and the fix. On "bhaiya 2 kilo toor dal aur 1 litre tel bhej do", my rule-based fallback parser dropped toor dal and kept only the oil. The greeting bhaiya was being read as part of the first item instead of being separated from the quantity 2. Marathi orders also use don instead of do for two, and items like tandul (rice) or varan (dal). I fixed it by stripping conversational prefixes (bhaiya, bhau, kaka, dada) and adding Marathi number words (don: 2, daha: 10) in orderly.py.

The first user

I tested Orderly with a local kirana store owner in Maharashtra whose customers regularly send order lists and voice notes on WhatsApp in Marathi and Hinglish.

During the evening rush he juggles writing voice orders on paper pads while answering phone calls and calculating rates by hand. What first confused the system was fast colloquial phrasing (tandul, varan, pith) and greetings with addresses (bhaiya, Patil ke ghar). Once the aliases and his price catalogue were set up, he said: "It saves writing down items on scrap paper and punching totals on the calculator while three people are waiting at the counter."

When I asked if he would pay, he said he would gladly pay ₹150–₹200 a month for a simple tool on his counter laptop that doesn't charge per message and keeps customer addresses and buying history private.

Partner tech

Prize category: Best Use of Gemma.

Gemma is the order-reading model, run locally through Ollama. I used Google's gemma3:1b (about 806 MB of weights), which runs on CPU at roughly 12–15 seconds per order at temperature 0 and returns clean, structured JSON without a GPU or any cloud API key.

What I'd do next

1. In-browser catalogue editor: let shopkeepers add and edit items and mark items out of stock from the web page, without editing catalogue.json.
2. WhatsApp Business API connection: capture incoming voice notes and post ready invoices back automatically, instead of using a wa.me link.
3. Daily ledger and totals: an end-of-day summary of orders, quantities sold and pending cash on delivery.
4. More dialects: Gujarati, Kannada and regional Marathi variants.