Namma Kadai Voice Assistant

Building Namma Kadai: A Voice AI Assistant for Indian Local Commerce

Introduction to the Agent

Meet Namma Kadai (Our Shop), an intelligent, multilingual voice assistant designed specifically for Indian local commerce, street vendors (such as PM SVANidhi beneficiaries), self-help groups (SHGs), and micro-entrepreneurs.

• The Problem: Many local vendors, artisans, and small shop owners face digital friction when managing online ONDC catalogues, tracking orders, navigating microloan schemes, or checking market logistics due to complex UI interfaces and language barriers.

• The Track: Indian Local Commerce & Grassroots Empowerment.

• Who It Is For: Street vendors, handicraft artisans, small retail shop owners, and local buyers.

• Why Voice Matters: Voice eliminates the literacy and digital hurdles of traditional text-heavy apps. Vendors can speak naturally in their preferred language while managing inventory, checking real-time weather for outdoor market setups, or looking up financial support schemes hands-free while running their shops.

---

Important Features Built (The 10-Day Journey)

The project brings together a comprehensive suite of tools and agentic workflows tailored for grassroots commerce:

1. Core Voice AI Pipeline & Architecture: Established the foundation using LiveKit, Deepgram for multi-language speech-to-text (STT), Murf AI for text-to-speech (TTS), and Google Gemini as the core language model.

2. Persistent Database Memory: Configured an SQLite backend to manage returning caller profiles, language preferences, past order history, and automated call-outcome analytics.

3. Dynamic Multilingual Processing: Implemented real-time language detection and voice switching to seamlessly navigate English, Tamil, Tanglish, and Hindi.

4. Live Weather Integration: Integrated the Open-Meteo API to give local vendors real-time temperature, wind, and rain conditions for market setups and delivery logistics.

5. Specialized Security & Handoffs: Built a dedicated Cyber Safety & Fraud Prevention Specialist agent for instant handoffs during cybercrime or scam reports, alongside compliant human-help escalation workflows.

6. Linphone VoIP Integration: Configured open-source SIP client and softphone connectivity via Liblinphone to route traditional telephony and audio streams cleanly into the agent infrastructure.

7. Digital Storefront & Cataloguing: Developed ONDC-aligned digital catalogue management to help local artisans and MSMEs structure product inventory and display items seamlessly.

8. Microloans & Financial Schemes: Integrated dedicated conversational flows and guidance for accessing government support programs like PM SVANidhi, PM Vishwakarma, Udyam Registration, and MUDRA loans.

9. Outbound Workflows: Implemented automated outbound calling capabilities for proactive vendor coordination, delivery confirmations, and market updates.

10. Polish & Feedback Collection: Finalized strict turn-detection controls, VAD tuning, and a graceful sign-off feedback collection system to ensure an intuitive user experience.

---

Overcoming Development Challenges

One of the trickiest hurdles faced during development was managing dynamic language detection and real-time TTS voice switching without clipping user interruptions or breaking the conversation flow.

• The Problem: When callers seamlessly mixed Tamil keywords with English (Tanglish), the default text-to-speech model occasionally mispronounced words or defaulted back to a rigid accent, making the interaction feel robotic.
• What Was Tried: Initially, static language configurations were tested, but they failed during code-switched dialogues. Next, custom keyword sets and Unicode character range checks (Baloo Thambi 2 script tracking) were added to the transcription event listener.
• The Solution: By catching user transcripts mid-stream via LiveKit's user_input_transcribed event, the agent evaluates incoming text against a Tamil/Tanglish keyword dictionary and dynamically invokes session.tts.update_options(voice="ta-IN-anisha") on the fly. This resulted in smooth, natural code-switching audio responses.

---

Guide: How to Build Your Own Voice Agent

To build and run a similar real-time voice agent, you need four foundational components:

1. Speech-to-Text (STT): Transcribes audio streams into text in real time (e.g., Deepgram Nova-3).

2. Large Language Model (LLM): Handles reasoning, tool-calling, and conversational logic (e.g., Google Gemini 2.5/3.5 Flash).

3. Text-to-Speech (TTS): Converts the LLM text output back into natural spoken voice (e.g., Murf AI Falcon).

4. Real-Time Transport & Orchestration: Manages audio tracks, WebRTC connections, and VAD (Voice Activity Detection) using LiveKit and Silero VAD.

Setup Instructions

1. Clone and Install Dependencies:

git clone https://github.com/kvk1999/murf-livekit-starter.git
cd murf-livekit-starter
pip install -r requirements.txt

1. Configure Environment Variables: Create a .env.local file in your root directory. Never commit this file or expose your API keys publicly.

LIVEKIT_URL=wss://voicebharat-ak7frth8.livekit.cloud
LIVEKIT_API_KEY=API8e2v252xYx47
LIVEKIT_API_SECRET=1yHXG66xvIQofEUUccsKbv67fXCMHlsBzllZ2aeYNnFA
MURF_API_KEY=ap2_e5b802b4-231e-4d24-b7f9-150fb7fdd204
DEEPGRAM_API_KEY=db3ac822d4044ea4f737047a9f9114975294fea8
GOOGLE_API_KEY=AQ.Ab8RN6KJojfAGacYzk5rqvfnCHOeMjkhzvlxJyqq8c8ZeOLOpA

# Linphone & SIP Trunking (Task 6)
LINPHONE_SIP_URI=sip:[e-mail]
SIP_OUTBOUND_HOST=sip.linphone.org
LIVEKIT_SIP_TRUNK_ID=ST_fxqmyQZCSwkB
LIVEKIT_SIP_OUTBOUND_TRUNK_ID=ST_fxqmyQZCSwkB
SIP_TRUNK_ID=ST_fxqmyQZCSwkB
OTEL_SDK_DISABLED=true

1. Run the Agent:

python agent.py dev

1. Test the Conversation: Connect via the LiveKit web sandbox or configure a Linphone softphone client using your SIP credentials (sip:[e-mail]) to dial into your agent room and test live voice interactions.