🎙️ Building UdharBhai: A WhatsApp-First Gujarati Voice "Udhar Khata" Bot in 1 Day

TL;DR: App-based credit books (Khatabook, OkCredit) fail for rural and tier-2/3 Indian merchants because of digital literacy barriers. We built UdharBhai (ઉધરભાઈ)—a zero-app WhatsApp bot that takes Gujarati voice notes ("Ramesh 500 udhar diyo"), parses financial intents via Sarvam AI + LLMs, updates a Supabase PostgreSQL ledger, and triggers WhatsApp payment reminders with dynamic UPI deep links.

---

💡 The Market Gap: App Fatigue in Tier-2/3 India

India’s local merchant economy runs on informal credit (Udhar). Existing fintech unicorns built sleek native apps (Khatabook, OkCredit, Bahi Khata) to digitize these ledgers.

However, there is a fundamental friction point: The shopkeeper and the customer already live on WhatsApp.

Forcing a local kirana shop owner to download a separate 50MB Android app, navigate complex UIs, and type in English/Hindi creates immediate churn.

The Solution: "Voice-In → WhatsApp-Out"

1. Zero App Downloads: The shopkeeper simply sends a WhatsApp voice note in natural spoken Gujarati.
2. Instant NLU: The voice note is transcribed, parsed into an atomic transaction, and logged in PostgreSQL.
3. Automated Reminders: The customer receives a WhatsApp nudge containing a clickable UPI link (upi://pay?pa=...).

---

🏛️ System Architecture

Here is the high-level data pipeline powering UdharBhai:

[Shopkeeper WhatsApp Voice Note]
         │
         ▼ (Webhook Payload)
┌─────────────────────────────────┐
│ Express / TypeScript Gateway    │
└────────┬────────────────────────┘
         │
         ├─► Audio Stream ──► [Sarvam AI Saaras v4] ──► Gujarati Text
         │
         ▼
┌─────────────────────────────────────────────────────────────────┐
│ NLU Engine (Groq / Llama 3.3 Versatile)                         │
│ - Spoken Gujarati Numeral Normalization                         │
│ - Honorific Stripping ('bhai', 'kaka', 'seth')                   │
│ - JSON Extraction: { intent, customer_name, amount }            │
└────────┬────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────────┐
│ Supabase PostgreSQL                                             │
│ - Atomic balance mutation                                       │
│ - Ledger entry insertion                                        │
└────────┬────────────────────────────────────────────────────────┘
         │
         └─► Dynamic UPI Link Generation + WhatsApp Response Dispatch

---

🛠️ Key Technical Highlights

1. Regional Speech-to-Text via Sarvam AI

Whisper and generic cloud STT engines often struggle with regional Indian dialects and vernacular speech patterns. We integrated Sarvam AI (saaras:v4), optimized for Indian languages:

import axios from 'axios';
import FormData from 'form-data';

export async function transcribeAudioBuffer(audioBuffer: Buffer): Promise<string> {
  const formData = new FormData();
  formData.append('file', audioBuffer, 'audio.ogg');
  formData.append('model', 'saaras:v4');
  formData.append('mode', 'transcribe');
  formData.append('language_code', 'gu-IN');

  const response = await axios.post('https://api.sarvam.ai/speech-to-text', formData, {
    headers: {
      'api-subscription-key': process.env.SARVAM_API_KEY!,
      ...formData.getHeaders()
    }
  });

  return response.data.transcript || '';
}

2. Gujarati Financial NLU & Numeral Normalization

Gujarati speech includes conversational numbers like "Pandarsau" (1,500) or "Dedh So" (150). We pass the transcribed text to an LLM with strict JSON schema constraints:

export async function parseGujaratiInput(input: string): Promise<ParsedIntent> {
  const systemPrompt = `You are a Gujarati financial transaction extraction engine for "UdharBhai".
Parse the input and extract structured json.

Examples:
- "Ramesh 500 udhar diyo" -> {"intent": "UDHAR", "customer_name": "Ramesh", "amount": 500}
- "Patel 200 jama mili" -> {"intent": "JAMA", "customer_name": "Patel", "amount": 200}
- "Ramesh nu ketlu baki che?" -> {"intent": "QUERY_SINGLE", "customer_name": "Ramesh"}

Rules:
- Convert Gujarati numbers (e.g., pandarsau -> 1500, dedh so -> 150).
- Strip honorifics like 'bhai', 'kaka', 'seth' from customer_name.
- Output ONLY valid JSON: {"intent": "UDHAR"|"JAMA"|"QUERY_SINGLE"|"QUERY_ALL"|"UNKNOWN", "customer_name"?: string, "amount"?: number}`;

  const response = await axios.post(
    'https://api.groq.com/openai/v1/chat/completions',
    {
      model: 'llama-3.3-70b-versatile',
      messages: [
        { role: 'system', content: systemPrompt },
        { role: 'user', content: input }
      ],
      response_format: { type: 'json_object' }
    },
    { headers: { Authorization: `Bearer ${process.env.GROQ_API_KEY}` } }
  );

  return JSON.parse(response.data.choices[0].message.content);
}

3. Atomic Database Operations (Supabase)

To prevent race conditions during concurrent voice logging, ledger writes and customer balance updates execute inside atomic transactions:

export async function recordTransaction(shopId: string, customerName: string, amount: number, type: 'UDHAR' | 'JAMA') {
  // Get or create customer record
  let { data: customer } = await supabase.from('customers').select('*').eq('shop_id', shopId).ilike('name', customerName).single();

  if (!customer) {
    const { data: newCust } = await supabase.from('customers').insert([{ shop_id: shopId, name: customerName, total_balance: 0 }]).select().single();
    customer = newCust;
  }

  // Insert immutable ledger log
  await supabase.from('ledger_entries').insert([{ shop_id: shopId, customer_id: customer.id, type, amount }]);

  // Atomic balance calculation
  const balanceChange = type === 'UDHAR' ? amount : -amount;
  const newBalance = Number(customer.total_balance) + balanceChange;

  await supabase.from('customers').update({ total_balance: newBalance }).eq('id', customer.id);

  return { customerName: customer.name, newBalance };
}

4. Dynamic UPI Deep Link Construction

When sending WhatsApp reminders to debt holders, we construct NPCI-compliant UPI deep links:

export function generateUpiLink(upiId: string, shopName: string, amount: number): string {
  const note = encodeURIComponent(`Udhar payment - ${shopName}`);
  const encodedShop = encodeURIComponent(shopName);
  return `upi://pay?pa=${upiId}&pn=${encodedShop}&am=${amount}&tn=${note}&cu=INR`;
}

---

🚀 The Viral Loop Growth Model

UdharBhai incorporates a built-in organic growth engine:

1. Shopkeeper logs credit for a customer via voice note.

2. The bot auto-sends a WhatsApp reminder to the customer:

   "Namaste Ramesh bhai! 🙏 Patel General Store par 500₹ udhar che. UPI se bhugtan kare: [UPI Link]"

3. At the bottom of the notification, a subtle footer tag reads:

   "⚡ Aa khata book UdharBhai par chale che. Tamara dukan mate try karo: [WhatsApp Link]"

Because debt holders in local trading clusters are frequently merchants or traders themselves, every collection reminder double-hats as a direct customer acquisition channel—achieving near zero CAC (Customer Acquisition Cost).

---

📦 Try It Out / Contribute

UdharBhai is completely open-source! Check out the codebase, set up your own regional voice bot, or help us add support for additional Indian regional languages (Hindi, Marathi, Marwari).

👉 GitHub Repository: github.com/bhkbdbhatt/udharbhai

---

Suggested Tags

#typescript #nodejs #ai #whatsapp #buildinpublic