DEV Community: Gautam

Mitra Local: A Private, Local-First AI Voice Memory & Task Assistant Built for My Friend Ketan

Gautam — Sat, 03 Oct 2026 17:03:15 +0000

This article is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

Meet My Friend Ketan

My childhood friend Ketan runs a textile and garment trading firm in Ahmedabad, Gujarat. If you spend even one morning with him at his shop or warehouse, you will notice three things immediately:

His phone rings unceasingly. Between 9 AM and 7 PM, he takes over 50 phone calls from yarn spinners, dye mills, courier agencies, brokers, and retail store owners.
He switches languages three times in a single sentence. He will greet in Gujarati, negotiate quantities in Hindi, and discuss dispatch timelines or bank transactions in English.
He is constantly making verbal commitments on the go: > "હા Maheshbhai, કાલે સવારે તમને sample swatches મોકલાવી દઉં છું." (Yes Maheshbhai, sending you the sample swatches tomorrow morning.) > "Sureshji, aapka 45,000 ka payment RTGS kar diya hai, shaam tak credit ho jayega." (Sureshji, your 45,000 payment was sent via RTGS, will credit by evening.) > "Priya, please make sure the GST invoice for bill #1042 is cleared before Friday 4 PM."

By evening, half of those verbal promises were forgotten or scribbled hastily on scraps of paper, receipts, or scattered WhatsApp self-chats.

Why Off-the-Shelf Tools Failed Ketan

Ketan tried Siri, Google Keep, Notion, and standard Todoist apps. None of them worked for him:

Language Inflexibility: Siri and Google Assistant choked on code-mixed Gujarati-English sentences ("Maheshbhai ne call kari de" became unintelligible gibberish).
High Friction: Notion and Todoist demanded manual typing with date pickers—impossible while inspecting fabric rolls or driving between warehouses.
Privacy & Business Secrets: Ketan strictly refused to upload his customer contact books, invoice figures, and verbal business memos to public corporate AI clouds.

The Solution: Mitra Local

So I promised him: "I will build an assistant designed specifically for how you speak and how you work—running completely on your laptop, 100% private, with zero subscription fees."

That project became Mitra Local (મિત્ર / मित्र — "Friend").

Mitra Local is an open-source, local-first voice memory and task companion. Ketan simply hits a hotkey or clicks the microphone, talks naturally in whatever mix of Gujarati, Hindi, or English comes out of his mouth, and lets Mitra handle the rest:

🗣️ Native Code-Mixed Understanding: Seamlessly parses Gujarati (ગુજરાતી), Hindi (हिंदी), English, and hybrid mixtures without requiring manual language toggling.
🎯 Automatic Intent & Action Extraction: Distinguishes between actionable tasks (calls, deliveries, payments, follow-ups) and knowledge memories (client preferences, phone numbers, notes).
👤 Entity & Person Graph: Mentions of "Maheshbhai" or "Sureshji" automatically link to contact records with conversation histories and pending to-dos.
⚡ Local Voice Feedback: Mitra speaks back a concise confirmation in natural audio so Ketan knows his commitment was captured without staring at the screen.
🔒 100% Offline & Private: Powered by open-source AI (Ollama + Gemma / Llama) and local speech-to-text. Zero cloud telemetry.

Demo

Here is how Mitra Local works in real life when Ketan speaks to it:

1. Real-World Voice Interaction Scenarios

Language Input	Spoken Command	Extracted Action & Intelligence
Gujarati + Eng	"કાલે Maheshbhai ને call કરી દે, shipment વિશે પૂછવાનું છે."	Call Task linked to contact `Maheshbhai`, scheduled for tomorrow, categorized under shipments.
Hindi + Eng	"Sureshji ko bol dena ki 45,000 ka payment RTGS kar diya hai."	Payment Memory & Verification Task linked to contact `Sureshji` with amount `₹45,000`.
English	"Remind me to file GST tax documents by Friday 3 PM."	High-Priority Task with absolute deadline set to Friday 15:00.
Pure Gujarati	"યાદ રાખજે કે રમેશભાઈની દુકાને નવા સેમ્પલ પહોંચાડવાના છે."	Delivery Task linked to `Rameshbhai`'s shop record.
Pure Hindi	"याद रखना कि राजेश का जन्मदिन 15 तारीख को है."	Personal Memory tagged under `Rajesh` in the local memory bank.

2. The Real-Time Workflow

Speak naturally: Ketan clicks the central microphone button (or triggers the local hotkey) and speaks in Gujarati, Hindi, English, or a mix.
Instant Structured Extraction: Within 400 milliseconds, the dashboard dynamically updates with:
- Extracted task title with original code-mixed transcript preserved
- Categorized action type (Call, Payment, Meeting, Delivery, Note)
- Identified person / contact entity
- Normalized due date & priority level
Verbal Confirmation: The local Web Speech API generates an instant natural voice response: > "Call task for Maheshbhai regarding shipment scheduled for tomorrow."
Relational Organization: The task is instantly filed in the Task Matrix and linked to the Person's profile in the SQLite database.

3. Ketan's Reaction

When I handed Ketan the first build on his laptop, he laughed and immediately tested it with his fastest Kathiyawadi Gujarati:

"કાલે સવારે 11 વાગ્યે શર્માજી સાથે પેમેન્ટનું સેટલમેન્ટ કરવાનું છે, યાદ રાખજે!"

Within less than half a second, the card appeared on his screen:

Title: Payment settlement with Sharmaji
Action: Payment / Financial
Person: Sharmaji
Due: Tomorrow at 11:00 AM
Audio Response: "Payment task created for Sharmaji tomorrow at 11:00 AM."

He looked up and said:

"Gautam, this is the first app that actually talks like an Indian businessman. No complex forms, no English-only barriers, and no fear of my data leaving my laptop."

Code

Mitra Local is 100% open source under the permissive MIT License.

GitHub Repository: https://github.com/gunmasterg9/mitra-local

Quickstart: Running Locally in 3 Minutes

1. Clone the Repository

git clone https://github.com/gunmasterg9/mitra-local.git
cd mitra-local

2. Start the Backend

cd backend
python -m venv .venv
# On Windows:
.venv\Scripts\activate
# On macOS/Linux:
# source .venv/bin/activate

pip install -r requirements.txt
python -m uvicorn app.main:app --reload --port 8000

Backend runs on http://localhost:8000 (Interactive API docs at http://localhost:8000/docs).

3. Start the Frontend

cd ../frontend
npm install
npm run dev

Frontend runs on http://localhost:5173.

4. Automated Verification Suite

We built a comprehensive test suite covering multilingual entity extraction, relative date parsing, contact deduplication, and database transactions:

cd backend
python -m pytest tests/

======================= 36 passed, 16 warnings in 8.02s =======================

All 36 unit and integration tests pass, ensuring zero-regression reliability.

How I Built It

Mitra Local is built from the ground up as a resilient, local-first system designed for zero latency and complete data sovereignty.

Architecture Overview

┌────────────────────────────────────────────────────────┐
│             Modern React 19 + Vite Frontend            │
│  - One-click Voice Mic (Web Audio API / MediaRecorder) │
│  - Real-time Task Matrix & Priority Filters            │
│  - Memory Bank & Person Profiles                       │
│  - Local Browser Speech Synthesis (TTS)                │
└───────────────────────────┬────────────────────────────┘
                            │ REST API
                            ▼
┌────────────────────────────────────────────────────────┐
│                FastAPI Asynchronous Engine             │
│  - Audio Transcription Pipeline (Local Whisper / API)  │
│  - Structured Multilingual Extraction Pipeline         │
│  - Heuristic Fallback Engine (Zero-Downtime Guarantee) │
└───────────────┬────────────────────────┬───────────────┘
                │                        │
                ▼                        ▼
┌──────────────────────────────┐ ┌───────────────────────┐
│     Local AI with Ollama     │ │  Local SQLite Engine  │
│  - Gemma 4 12B / Llama 3.2   │ │  - aiosqlite Async DB │
│  - Zero Cloud Telemetry      │ │  - Full-Text Search   │
│  - Structured JSON Schema    │ │  - People & Memories  │
└──────────────────────────────┘ └───────────────────────┘

Technical Stack

Frontend: React 19, Vite, Lucide Icons, Vanilla CSS design tokens with sleek dark mode, Web Speech Synthesis API.
Backend: FastAPI (Python 3.12), Pydantic v2 schemas, SQLAlchemy with aiosqlite for asynchronous local database transactions.
Local AI Inference: Ollama running open-weight models (gemma4:12b, llama3.2, or mistral).
Storage: Local SQLite database stored securely in backend/data/mitra.db on Ketan's machine.

Solving the Multilingual Code-Switching Challenge

Indian business conversations freely mix Gujarati, Hindi, and English with honorifics ("-bhai", "-ji", "-ben"). Conventional prompts often fail by either translating names into English words (e.g. converting "Sureshji" into "Sir Suresh") or dropping dates.

To achieve bulletproof parsing, Mitra combines two layers:

Structured Few-Shot Prompting with Open LLMs: We prompt local models with strict JSON output schemas that explicitly understand Indic code-mixing:

   {
     "intent": "create_task",
     "task": {
       "title": "Call Maheshbhai regarding shipment",
       "title_original": "કાલે Maheshbhai ને call કરી દે, shipment વિશે પૂછવાનું છે.",
       "action_type": "call",
       "person_name": "Maheshbhai",
       "due_date": "2026-10-04",
       "priority": "medium",
       "language": "gu-en"
     },
     "speech_response": "Call task for Maheshbhai regarding shipment scheduled for tomorrow."
   }

Heuristic Linguistic Fallback Engine: Laptops in the field don't always have GPU acceleration, and background tasks might temporarily tie up resources. Mitra includes an integrated regex- and rule-based Indic NLP engine that recognizes common action verbs in Gujarati ("call kari de", "moklavano chhe", "poochhvanu chhe"), Hindi ("bol dena", "bhej do", "yaad rakhna"), and English ("remind me", "pay"), along with relative temporal expressions ("kaale", "parso", "kal"). Even with Ollama paused, Mitra processes inputs flawlessly in under 5 milliseconds.

Why Does Open Innovation Matter?

When starting this project for Ketan, the easiest route would have been wiring up a quick commercial cloud API (OpenAI or Claude) with a credit card. But for Ketan, and millions of independent business owners like him, closed cloud AI is fundamentally the wrong answer.

Here is why open innovation and open-source AI were strictly essential to solving this problem:

1. Absolute Privacy and Data Sovereignty

In the textile trading world, commercial relationships and pricing are hard-won trade secrets. A typical memo contains:

Supplier identities and credit balances
GST numbers and pending payment amounts
Private personal numbers and commitments

Sending audio snippets or text logs of these conversations to third-party public clouds exposes small business owners to data harvesting, telemetry profiling, and compliance vulnerabilities. With open-weight models running on Ollama and local SQLite storage, Ketan has 100% mathematical certainty that not a single byte of his business data ever leaves his hardware.

2. Zero Recurring Overhead for Everyday People

Proprietary AI SaaS platforms charge $20 to $50 per user per month, plus metered API token fees. For an independent merchant, recurring software subscriptions add up quickly and become a financial burden. Open-source models (like Gemma and Llama) democratize state-of-the-art intelligence: once downloaded, they run indefinitely with zero subscription costs, zero token meters, and zero paywalls.

3. Freedom to Adapt for Underrepresented Languages

Commercial AI models are optimized primarily for formal Western corporate English. They frequently misinterpret code-mixed regional Indian dialects like Gujarati-English or Hinglish, treating common honorifics as foreign errors or hallucinations.
Because open models and open-source code can be inspected, steered, and fine-tuned locally, we were able to build customized prompt pipelines and hybrid heuristic engines tailored specifically to Gujarati merchants. Open innovation puts the power to build AI in the hands of the communities that need it most, rather than waiting for Silicon Valley giants to prioritize regional dialects.

4. True Offline Autonomy

Wholesalers and traders frequently operate in basement warehouses, noisy textile markets, and transport hubs with spotty cellular reception. A tool that depends on roundtrips to an external cloud server will fail right when it is needed most. Open-source local inference delivers instant, reliable performance anywhere—on a flight, in a godown, or during an internet outage.

Conclusion & What's Next

Building Mitra Local for Ketan showed me that the most impactful AI tools aren't generic chatbots—they are personalized, empathetic utilities built for the specific rhythms of real people's lives.

Our upcoming roadmap includes:

Fine-tuning a quantized Whisper model on regional Gujarati and Kathiyawadi accents
Local WhatsApp Desktop webhook bridge for automated reminder dispatch
Offline vector search using DuckDB / ChromaDB for multi-year business archives

Thank you to the DEV Community and Hacktoberfest for championing open innovation that makes technology truly personal and accessible.

Links:

GitHub Repository: https://github.com/gunmasterg9/mitra-local
License: MIT License

devchallenge #hacktoberfest #hf26challenge

Building Bharat Voice AI: My 10-Day Journey from a Voice Agent to a Multilingual Multi-Agent System

Gautam — Sat, 15 Aug 2026 08:29:00 +0000

# Building Bharat Voice AI: My 10-Day Journey from a Voice Agent to a Multilingual Multi-Agent System

Over the last 10 days, I took a simple idea, a voice assistant, and gradually turned it into a more complete multilingual voice-agent system.

This project is called Bharat Voice AI.

The goal was to build a voice agent that can communicate naturally with users in India, understand different languages and code-mixed speech, remember returning users, use real-world tools, make outbound calls, escalate to humans, measure its performance, and hand specialized tasks to another agent.

This journey was part of:

10 Days of Voice Agents — VoiceForBharat Edition

The project uses Murf Falcon for voice generation together with LiveKit, Gemini, Deepgram, Python, SQLite, and real external tools.

What is Bharat Voice AI?

Bharat Voice AI is a multilingual voice assistant designed around the idea that voice interfaces should feel natural for Indian users.

Instead of forcing users to interact only through a text interface, the system allows them to speak naturally.

It supports:

English
Hindi
Gujarati
Hinglish and code-mixed conversations
Persistent user memory
Real-time tools
Human escalation
Outbound voice calls
Call analytics
Specialist-agent handoff

The system is designed so that the main agent does not have to do everything itself.

When a task requires specialized knowledge, it can hand the conversation to a specialist agent.

Why Voice?

India has a huge diversity of languages, communication styles, and levels of digital literacy.

A voice interface can make technology easier to access because users do not have to type everything.

For Bharat Voice AI, I wanted the interaction to feel closer to a normal conversation.

For example:

"Veraval mein aaj weather kaisa hai?"

The system should understand the intent even though the sentence mixes Hindi and English.

It should then respond naturally in the appropriate language.

Architecture

The basic architecture looks like this:

                    Bharat Voice AI
                           |
                     LiveKit Agents
                           |
              +------------+------------+
              |                         |
           Deepgram                   Gemini
             STT                       LLM
              |                         |
              +------------+------------+
                           |
                         Tools
              +------------+------------+
              |            |            |
           Weather      Memory      Escalation
              |            |            |
          Open-Meteo     SQLite       SQLite
                           |
                    Specialist Agents
                           |
                     Murf Falcon TTS
                           |
                         User

Day 1 - Building the Voice Agent
The first step was getting the basic voice pipeline working.
The core pipeline became:

User Speech
     |
     v
Deepgram STT
     |
     v
Gemini
     |
     v
Murf Falcon
     |
     v
User Voice

The agent could listen to the user and respond using voice.
The main technologies were:
LiveKit Agents
Deepgram
Gemini
Murf Falcon
Day 2 - Persona and Guardrails

The next step was defining who the agent actually is.

The agent became:

Bharat Voice AI

I added clear instructions covering:

Identity
Objectives
Knowledge boundaries
Language behavior
Guardrails
Escalation
Conversation style

The agent should not pretend to know something it does not know.

It should also know when a request is outside its role.

This became especially important later when I added human escalation and specialist agents.
Multilingual Conversations

One of the important goals was supporting Indian languages.

The agent supports:

English

"What's the weather today?"

Hindi

"आज वेरावल में मौसम कैसा है?"

Gujarati

"આજે વેરાવળમાં હવામાન કેવું છે?"

I also tested code-mixed speech such as:

"Veraval mein aaj weather kaisa hai?"

The important part is not simply detecting the language.

The agent should also respond in the appropriate script.

For example, Hindi should be written and spoken naturally rather than being forced into Romanized Hindi.
Day 3 - Frontend

After the voice pipeline worked, I customized the frontend.

The frontend clearly represents the state of the agent.

The main states are:

Ready
  |
  v
Connecting
  |
  v
Listening
  |
  v
Speaking
  |
  v
Call Ended

t
Day 4 - Persistent Memory

A voice agent that forgets everything after every call isn't very useful for returning users.

So I added persistent memory using SQLite.

The profile stores information such as:

user_id
name
language_preference
facts
last_interaction

The important design decision was that memory is handled by backend functions rather than being written into the LLM prompt.

The agent can:

Look up a caller.
Ask permission before saving information.
Save approved information.
Retrieve the profile during a later conversation.

For example:

First conversation:

"My name is Gautam."

The agent asks whether it should remember the name.

After permission is granted, the information is stored.

During a later conversation, the agent can recognize the returning user.

Day 5 - Real Tools

The agent needed to do more than generate answers from the LLM.

I added a real weather tool.

The weather flow is:

User
 |
 | "What is the weather in Veraval today?"
 v
Gemini
 |
 | tool call
 v
get_weather()
 |
 v
Weather API
 |
 v
Real weather data
 |
 v
Gemini
 |
 v
Murf Falcon
 |
 v
User

The important rule is:

The agent must never invent current weather information.

If the weather service is unavailable, the agent should say that it could not retrieve the latest information.

It should not guess.

One Real Debugging Lesson

One of the problems I encountered during development was a Deepgram connection failure.

The voice pipeline produced an error similar to:

APIConnectionError:
failed to connect to deepgram

Instead of assuming the code was broken, I tested the network connection from Windows PowerShell.

I used:

Test-NetConnection api.deepgram.com -Port 443

Initially the connection failed.

After investigating the network/DNS path, I eventually got:

TcpTestSucceeded : True

This was a good reminder that voice-agent problems are not always LLM problems.

The failure can be caused by:

Network connectivity
DNS
Firewall
API availability
WebSocket connections
Authentication
Tool execution
Frontend state

Debugging the complete pipeline is essential.

Day 6 - Outbound Voice Calls

The next step was making the agent call a user instead of waiting for the user to open the browser.

I integrated outbound calling using LiveKit telephony/SIP.

I also tested the Linphone route.

The basic flow became:

Bharat Voice AI
      |
      v
LiveKit SIP
      |
      v
Linphone
      |
      v
    Phone

The outbound conversation needs a different opening from a browser conversation.

The agent must immediately explain:

Who is calling
Why it is calling
How the user can end the call

I also tested the SIP connection and call lifecycle.

Day 7 - Human Escalation

A voice agent should not try to solve everything.

I added a human escalation workflow.

When a user says:

"I want to talk to a human."

the agent can start the escalation process.

But it does not automatically share information.

It first asks for permission.

For example:

"I can create a request for human assistance. Before I do, I would share your name, the issue you described, what I checked, your preferred language, and the urgency. Would you like me to create the request?"

If the user says yes, the backend creates a real escalation record.

A reference ID is generated by the backend.

The important rule is:

Never tell the user that an escalation was created unless the database operation actually succeeded.

This avoids fake success messages.

Day 8 - Call Analytics

After adding many features, I needed a way to measure what was happening.

So I built a Call Analytics Dashboard.

The dashboard tracks:

The data comes from real calls stored in SQLite.

It is not hardcoded.

A call is considered successful when the user's intended task is successfully completed.

For example:

User asks for weather
        |
Weather tool succeeds
        |
Weather information delivered
        |
     SUCCESS

If the required tool fails or the task is incomplete, the call can be recorded as failed or incomplete.

The dashboard can therefore show actual agent performance.

Day 9 - Specialist Agent Handoff

This was one of the most interesting parts of the project.

Instead of forcing the main agent to handle every type of request, I created a specialist:

Bharat Weather Specialist

The architecture became:

                   User
                     |
                     v
              Bharat Voice AI
                     |
              Weather request?
                 /       \
               No         Yes
               |           |
               v           v
          Main Agent   Weather Specialist
                           |
                           v
                      get_weather()
                           |
                           v
                       Real Data

python
For example:

User:

"What is the weather today in Veraval?"

The main agent says:

"For detailed weather information, I'll connect you with our weather specialist."

The specialist then takes over.

The important part is that the user should not have to explain the entire question again.

The conversation context is passed to the specialist.

Native Agent Handoff

The LiveKit agent-handoff pattern allows the main agent to return a specialist agent together with an announcement.

Conceptually:

@function_tool
async def handoff_to_weather_specialist(
    self,
    context: RunContext,
) -> tuple[Agent, str]:
    specialist = BharatWeatherSpecialist(
        chat_ctx=self.chat_ctx.copy(
            exclude_instructions=True
        )
    )

    return (
        specialist,
        "I'll connect you with our weather specialist."
    )

The actual implementation in the repository follows the project's LiveKit version and architecture.

The important concept is that the specialist receives the previous conversation context.

A Day 9 Bug That Taught Me Something

During testing, the handoff itself worked.

However, I encountered a problem where the internal function-call information could appear as visible text instead of the actual weather response.

For example, the user could see something similar to:

The function called is get_weather
location = Veraval
forecast_days = 1

That is not acceptable for a real voice assistant.

The user should hear:

"The latest weather in Veraval is..."

not internal tool-call information.

This led me to investigate the difference between:

Function registration
Tool execution
LLM output
LiveKit agent handoff
Frontend message rendering

It was another reminder that a production voice agent is a complete system, not simply an LLM with speech.

The Complete SystemAfter the nine days, Bharat Voice AI looks roughly like this:

                         USER
                          |
                    Voice / Browser
                          |
                          v
                    LiveKit Agents
                          |
                    Bharat Voice AI
                          |
        +-----------------+------------------+
        |                 |                  |
      Memory           Tools              Routing
        |                 |                  |
      SQLite       +------+-------+          |
                   |              |          |
                Weather      Other Tools     |
                   |                         |
              Open-Meteo                     |
                                             |
                                      Specialist Agent
                                             |
                                      Weather Specialist
                                             |
                                      get_weather()
                                             |
        +------------------------------------+
        |
   Human Escalation
        |
      SQLite
        |
    Murf Falcon
        |
        v
       USER

Outbound calls extend the system through SIP/telephony.

Analytics records the outcome of calls and makes it visible through the dashboard.

Technology Stack

The main technologies used in the project include:

Python
LiveKit Agents
Gemini
Deepgram
Murf Falcon
SQLite
Open-Meteo
SIP / Linphone
Browser frontend
GitHub

Each component has a different responsibility.

Deepgram
Speech → Text

Gemini
Reasoning + Conversation

Tools
Real-world data and actions

SQLite
Persistent state

LiveKit
Real-time voice transport and agent orchestration

Murf Falcon
Text → Natural Voice

Security

One of the most important lessons is to never put API credentials directly into the source code.

Use environment variables.

For example:

DEEPGRAM_API_KEY=your_key_here
MURF_API_KEY=your_key_here
LIVEKIT_API_KEY=your_key_here
LIVEKIT_API_SECRET=your_secret_here

The real .env file should never be committed to GitHub.

Use:

.env

locally and:

.env.example

for documentation.

Never publish:

API keys
SIP credentials
passwords
OTPs
PINs
caller information
private database records

Running the Project

The complete project is available on GitHub:

https://github.com/gunmasterg9/bharat-voice-ai

A typical setup is:

git clone https://github.com/gunmasterg9/bharat-voice-ai.git

cd bharat-voice-ai

cd backend

uv sync

Create your environment configuration:

.env

Add the required API credentials.

Then start the backend using the project's configured startup command.

Start the frontend and open the browser interface.

Allow microphone access.

Then start a conversation.

For example:

"Hello Bharat Voice AI."

Then test:

"What is the weather today in Veraval?"

Then test multilingual interaction:

"આજે વેરાવળમાં હવામાન કેવું છે?"

Then test memory:

"My name is Gautam."

Then restart the application and verify that the saved profile can be retrieved.

Testing

I created tests covering important parts of the system.

The project includes testing for:

Agent behavior
Memory
Persistent memory
Language switching
Weather tools
Escalation
Outbound calls
Linphone
Analytics
Specialist handoff

The Day 9 development test suite included specialist handoff tests alongside the previous functionality.

The important lesson was that every new feature should be tested without breaking the previous days' work.

What I Learned

The biggest lesson from this challenge is that building a voice agent is much more than connecting an LLM to a TTS API.

A reliable voice agent needs:

Voice
+
Reasoning
+
Memory
+
Tools
+
State
+
Error Handling
+
Security
+
Human Handoff
+
Observability

A model can generate a great response, but the surrounding system determines whether the product is actually reliable.

I also learned to debug the entire pipeline instead of assuming every problem is caused by the LLM.

What I Would Build Next

There is still a lot I would like to add.

Future improvements could include:

More specialist agents
Better interruption handling
More Indian languages
Better low-bandwidth support
Advanced call analytics
Conversation quality scoring
Better tool observability
More real-world Indian datasets
Improved outbound-call workflows
Specialist-to-specialist routing
More sophisticated RAG
Production deployment and monitoring

The long-term goal would be to turn Bharat Voice AI into a platform where different specialized voice agents can work together.

Final Thoughts

The most interesting part of the challenge wasn't building the first voice conversation.

It was everything that came afterward.

Making the agent remember.

Making it use real data.

Making it call a phone.

Making it know when to ask a human.

Measuring whether conversations actually succeeded.

And finally, teaching one agent when another agent is better suited to help.

That progression changed how I think about voice AI.

A voice agent isn't just a chatbot that speaks.

It can become a complete software system with memory, tools, workflows, specialized agents, and real-world actions.

That is what I wanted to explore with Bharat Voice AI.

Project Links
GitHub

https://github.com/gunmasterg9/bharat-voice-ai
All Videos in Linkedin
https://www.linkedin.com/in/gautam-vandar-71a75632b/

Challenge

10 Days of Voice Agents — VoiceForBharat Edition

Voice Technology

Murf Falcon

Thank You

Thank you to Murf AI for organizing the 10 Days of Voice Agents challenge and providing the opportunity to build, experiment, debug, and learn through a real voice-agent project.

Building Bharat Voice AI over these 10 days was a great experience, and I hope this project helps someone else start building their own voice agent.