Jan Sahayak: Building a Voice AI Assistant for Bharat 🇮🇳

Thirumala Reddy — Sat, 15 Aug 2026 16:12:22 +0000

Yes — now I understand exactly what you mean.

The written content you provided is your friend's blog. The screenshots are from your actual Jan Sahay implementation. So I will use your friend's article only as a reference for structure/storytelling, but the blog below will be written around your actual UI and features shown in the screenshots.

I also won't claim things like "sub-500ms" or specific backend implementation details unless they are actually part of your project.

Jan Sahay: Building a Voice AI Assistant for Bharat 🇮🇳

10 Days of Voice Agents — VoiceForBharat Edition

What started as a simple voice agent on Day 1 slowly became something much bigger.

For the 10 Days of Voice Agents — VoiceForBharat Edition, I decided to build Jan Sahay (जन सहायक), a voice-first AI assistant focused on financial services, government schemes, digital fraud awareness, and citizen support.

The idea was simple:

Make important financial and government information easier to access through a natural voice conversation.

Instead of expecting users to search through multiple websites or understand complicated financial terminology, Jan Sahay lets them simply talk to an AI assistant.

Over the ten days, I gradually added more capabilities — from basic voice conversations to memory, real-world tools, outbound calling, human escalation, call analytics, and specialist agents.

This blog is a summary of that journey.

🏛️ What is Jan Sahay?

Jan Sahay is designed as a citizen-focused AI voice assistant for the Financial Services track.

It can help users explore areas such as:

🏛️ Government schemes
🛡️ Digital fraud prevention
💰 Financial literacy
📞 Complaint and escalation guidance
🎙️ Voice-based conversations
🤖 Specialist-agent assistance

The main goal isn't to replace government officials, banks, or financial experts.

It is to provide a simple first point of assistance — helping users understand their options and guiding them toward the appropriate next step.

🎙️ Why Voice?

One of the biggest reasons I chose voice is accessibility.

Not everyone wants to type a long question into a search box.

Sometimes, asking:

"Can I get help under this government scheme?"

is much more natural than trying to figure out which website, menu, or form to open.

For Jan Sahay, the interaction is designed around:

Speak → Understand → Guide → Take Action

The assistant can listen to the user, understand the request, retrieve relevant information when required, and respond naturally through voice.

The voice experience is powered by Murf Falcon, which I used for the text-to-speech side of the project.

🧩 The Main Features

🏛️ Government Scheme Guidance

Jan Sahay provides a dedicated area for government schemes and welfare information.

Users can ask questions about schemes and receive guidance around eligibility, benefits, and the information required to proceed.

The interface also has a dedicated Schemes Search section so users can explore scheme-related information beyond the voice conversation.

🛡️ Fraud Prevention

Digital fraud is another major focus of Jan Sahay.

The assistant is designed to help users understand common situations such as:

UPI scams
OTP-related fraud
Phishing messages
Suspicious links
Digital payment safety

The goal is not simply to explain what a scam is.

It is to help users understand what they should do next.

💰 Financial Literacy

Jan Sahay also focuses on making financial concepts easier to understand.

Instead of responding with complicated terminology, the assistant is designed to provide simple conversational guidance around financial topics.

This is particularly important when the user may not have a strong financial background.

📞 Complaint & Escalation Guidance

Sometimes an AI assistant should not try to solve everything itself.

Jan Sahay includes complaint and escalation capabilities so users can move toward appropriate human or official support when necessary.

The interface includes an Open Escalations section where escalation-related information can be managed.

This became one of the most important lessons I learned during the challenge:

A useful AI agent should know when it needs help.

🤖 From One Agent to Multiple Specialists

One of the features I am particularly happy with is the specialist-agent approach.

Instead of making one agent responsible for everything, Jan Sahay can work with specialized agents.

The current interface includes specialist options such as:

🛡️ Cyber Fraud Specialist
📈 Pension Specialist
💼 Business Loan Specialist
🌱 Crop Specialist

This allows the main assistant to act as the first point of contact and route a conversation toward a more focused specialist when required.

For example, a general question can stay with Jan Sahay.

But if a user asks a detailed question about a particular financial domain, the conversation can be handed to the appropriate specialist.

The user shouldn't have to restart the entire conversation just because a specialist is needed.

🧠 Building a More Context-Aware Agent

Another important part of the journey was adding memory.

A conversational agent becomes much more useful when it can retain appropriate context instead of treating every interaction as completely new.

This allows Jan Sahay to move beyond:

"Ask a question → receive an answer."

toward:

"Understand the user → remember useful context → continue the conversation."

At the same time, financial applications require careful handling of sensitive information, so memory and escalation flows need to be designed with privacy in mind.

📊 Measuring the Voice Agent

By Day 8, I realized that simply saying "the agent works" isn't enough.

I needed a way to understand what was actually happening during conversations.

So I built a Call Performance Dashboard.

The dashboard tracks information such as:

Total calls
Successful calls
Failed calls
Call duration
Language
Channel
Outcome
Failure reason
Recent call history

For example, during testing, my dashboard showed actual call records rather than hardcoded numbers.

This was useful because failed calls became something I could investigate instead of simply ignoring.

A voice agent shouldn't only be able to talk. It should be measurable and improvable.

🌐 A Frontend Designed Around the Conversation

I also spent time improving the frontend experience.

The main voice interface clearly communicates the current state of the assistant:

Connecting → Listening → Speaking → Call Ended

The interface also includes a live transcript so the user can see the conversation happening.

I added language-selection support and dedicated sections for:

Home
Schemes Search
Fraud Prevention
Complaint Helpline
Open Escalations
Call Dashboard

The idea was to make Jan Sahay feel like a complete citizen-assistance platform rather than just a microphone button connected to an AI model.

🏗️ How the System Works

At a high level, the voice interaction follows this flow:

                    👤 USER
                      │
                      ▼
                🎙️ Voice Input
                      │
                      ▼
              Speech-to-Text
                      │
                      ▼
              🤖 Main AI Agent
                      │
            ┌─────────┼─────────┐
            │         │         │
            ▼         ▼         ▼
         Memory     Tools   Specialist
            │         │      Agents
            │         │         │
            └─────────┼─────────┘
                      │
                      ▼
               Murf Falcon
                      │
                      ▼
                🔊 Voice Output

The important idea is that the LLM is not expected to do everything on its own.

The agent can use supporting components when the task requires them.

📅 My 10-Day Journey

Day 1 — Getting Jan Sahay Talking

The first goal was simple: establish a working voice conversation.

I connected the voice pipeline and integrated Murf Falcon for speech generation.

Hearing the agent respond for the first time was a small moment, but it made the whole project feel real.

Day 2 — Giving the Agent a Purpose

I moved from a generic voice assistant toward the Financial Services track.

Jan Sahay received a clear role focused on government schemes, financial literacy, fraud awareness, and citizen support.

Day 3 — Personalising the Frontend

I worked on the frontend and made the experience more suitable for the users Jan Sahay is intended to serve.

The interface began to clearly communicate whether the agent was connecting, listening, or speaking.

Day 4 — Adding Memory

I introduced memory so Jan Sahay could retain useful context and provide a more continuous experience across conversations.

Day 5 — Connecting Real-World Data

The agent learned to use tools instead of relying only on what the language model already knew.

This was an important change:

Don't just generate an answer. Get the right information first.

Day 6 — Making Outbound Calls

Jan Sahay gained outbound calling capability.

This opened up the possibility of an assistant that doesn't always wait for the citizen to initiate the conversation.

Day 7 — Knowing When Humans Are Needed

I added human escalation.

The agent can recognize situations where human assistance is more appropriate and move the user toward the next step.

Day 8 — Measuring Conversations

I built the call-performance dashboard and started looking at real conversation outcomes, failures, durations, and call history.

Day 9 — Adding Specialist Agents

The main agent gained the ability to work with specialist agents for focused areas such as cyber fraud, pensions, business loans, and crop-related assistance.

This was one of the biggest steps toward making Jan Sahay a multi-agent voice system.

Day 10 — Documenting the Journey

And now Day 10.

Instead of adding another feature, I am documenting what I learned and sharing the project so that someone else can understand how a voice agent can be built step by step.

🧪 What Was Difficult?

The most challenging part wasn't creating a single AI response.

It was getting all the pieces to work together.

A voice agent involves several moving parts:

Real-time audio
Speech recognition
LLM reasoning
Text-to-speech
Frontend state
Memory
Tools
Telephony
Specialist agents
Error handling

A problem in one part can affect the entire conversation.

I also spent time dealing with API configuration, model compatibility, voice integration, call flows, and making sure the different components continued working as the project became more complex.

The biggest lesson for me was:

Building an AI agent is less about one model and more about designing the entire system around it.

🔐 Safety Matters

Because Jan Sahay deals with financial information, safety cannot be treated as an optional feature.

The agent should not encourage users to share sensitive information such as:

Passwords
OTPs
PINs
Card details
Other confidential credentials

The system also needs to distinguish between general guidance and situations where the user should be directed toward official or human assistance.

That is especially important when building AI systems intended for real-world users.

🚀 What I Learned From These 10 Days

Before starting this challenge, I mainly thought about voice AI as:

Speech → AI → Speech

After ten days, my understanding is very different.

A useful voice agent needs much more:

Voice + Context + Memory + Tools + Safety + Human Escalation + Specialist Agents + Analytics

The biggest lesson I am taking from this challenge is:

A good AI agent isn't one that tries to do everything. It's one that knows what to do, what not to do, and when to ask for help.

🔮 What's Next for Jan Sahay?

There is still a lot I would like to improve.

Some of the ideas I want to explore next include:

More Indian language support
Better regional-language conversations
More verified government-scheme data
More specialist agents
Improved outbound calling
Better call analytics
Stronger privacy controls
More robust production deployment
Better accessibility for users with limited digital experience

The ten-day challenge may be over, but Jan Sahay doesn't have to be.

📂 Explore the Project

The complete project is available on GitHub:

Jan Sahay — GitHub Repository

For the final DEV.to article, place your actual deployed/demo link here as well.

🙏 Final Thoughts

Ten days ago, I started with the simple goal of making an AI voice agent talk.

Today, I have a system that can have voice conversations, remember context, use tools, make outbound calls, escalate to humans, track call outcomes, and hand conversations to specialist agents.

There were plenty of bugs, configuration problems, failed calls, and moments where something that looked simple turned out to be much harder than expected.

But that's probably the most valuable part of the experience.

I didn't just learn how to build a voice agent.

I learned how to design, debug, measure, and improve an AI system one step at a time.

Thank you to Murf AI for creating the 10 Days of Voice Agents — VoiceForBharat Edition and giving me the opportunity to build and learn in public.

**Built with Murf Falcon for the voice experience

DEV Community: Thirumala Reddy