# FINORA AI: Building a Voice-First Financial Assistant for Bharat with Murf Falcon 🎙️🇮🇳

Sujay H S — Sat, 15 Aug 2026 09:53:52 +0000

FINORA AI: Building a Voice-First Financial Assistant for Bharat with Murf Falcon 🎙️🇮🇳

What if getting help with financial services was as simple as having a conversation?

Introduction

When I started the 10 Days of Voice Agents — VoiceForBharat Edition, I wanted to build something more than a basic voice chatbot.

I wanted to build an assistant that could actually help users with financial questions, remember useful context, communicate naturally, know when it needed human support, and hand specialized questions to another agent.

That idea became FINORA AI.

FINORA AI is a voice-first financial services assistant designed for Bharat. The main voice agent, Dia, interacts with users through voice and supports English and Hindi.

As the challenge progressed, FINORA AI evolved from a simple voice agent into a complete voice-AI system with memory, voice workflows, outbound calling, human escalation, call analytics, and a dedicated Government Scheme Specialist named Krishna.

The goal was simple:

Make financial assistance less complicated and more conversational — just a voice away.

In this blog, I'll walk through what I built, how the system works, the challenges I faced, and what I learned during the ten-day journey.

🌱 The Problem

Financial services can often feel complicated.

Users may have to:

Search through different websites
Understand financial terminology
Navigate multiple pages
Find the right government scheme
Understand eligibility requirements
Figure out where to get help

The information may already exist, but finding and understanding the right information can be the difficult part.

I wanted to make that experience more natural.

Instead of:

Search → Navigate → Read → Understand

I wanted:

Ask → Talk → Understand → Get Help

That's where voice becomes useful.

🎙️ Meet FINORA AI

FINORA AI is a voice-first financial services assistant.

The main agent is Dia.

Dia handles general financial conversations and acts as the primary entry point for the user.

FINORA AI can:

Handle voice conversations
Support English and Hindi
Remember relevant conversational context
Handle financial-service related questions
Make outbound calls
Escalate sensitive situations to humans
Hand government-scheme questions to Krishna
Track call outcomes
Track agent handoffs
Display analytics through a dashboard

The idea wasn't to make Dia sound like a robot reading information.

I wanted the interaction to feel more like talking to an assistant:

"Sure, let me help you with that."

rather than:

"According to the available information..."

Caption: FINORA AI's minimalistic interface, designed to keep voice interaction simple and easy to understand.

The website follows the same philosophy as the voice experience:

Minimalistic. Clean. Easy to understand.

🎯 Why Voice?

Voice removes a layer of complexity.

Users don't have to figure out exactly what to type or where to click.

They can simply say:

"What documents do I need?"

or:

"Can you explain this government scheme?"

They can also speak naturally in Hindi.

For FINORA AI, voice isn't just another feature.

Voice is the primary interface.

Caption: A live voice conversation with Dia through the FINORA AI interface.

🏗️ How FINORA AI Works

The overall architecture looks like this:

flowchart TD
    U[User] --> L[LiveKit]

    L --> STT[Speech to Text]

    STT --> D[Dia - Main Agent]

    D --> LLM[LLM Reasoning]
    LLM --> M[Memory / Relevant Context]

    D --> K[Krishna - Government Scheme Specialist]

    D --> H[Human Escalation]

    K --> GS[Government Scheme Guidance]

    D --> TTS[Murf Falcon]
    K --> TTS

    TTS --> L
    L --> U

    L --> A[Call Analytics]
    A --> DB[(Analytics Database)]
    DB --> DASH[FINORA Dashboard]

At a high level, the voice flow is:

User Speech
     ↓
LiveKit
     ↓
Speech-to-Text
     ↓
Dia
     ↓
LLM Reasoning
     ↓
Response / Specialist / Human Escalation
     ↓
Murf Falcon
     ↓
User

LiveKit handles the real-time communication layer.

The agents handle the conversation and decision-making.

Murf Falcon provides the voice output.

The analytics system records call-level information so the dashboard can show how the system is performing.

Caption: High-level architecture of FINORA AI, showing the voice pipeline, Dia, Krishna, human escalation, Murf Falcon, and call analytics.

🔊 Murf Falcon — Giving FINORA AI a Voice

A voice assistant isn't only about what it says.

How it sounds matters too.

For FINORA AI, I used Murf Falcon, the fastest TTS API highlighted in the challenge.

The goal was to make the voice experience:

Natural
Clear
Fast
Friendly
Conversational

Instead of feeling like a traditional IVR system, I wanted the interaction to feel like a real conversation.

This was one of the most interesting parts of the project because voice changes how you design the entire agent.

A text chatbot can give a long answer.

A voice assistant needs to think about:

What should I say, and how should I say it?

🇮🇳 English + Hindi Voice Support

FINORA AI supports English and Hindi across both agents — Dia and Krishna.

The language doesn't stop when the conversation is handed from one agent to another.

If a user starts speaking in English, Dia and Krishna can continue in English.

If the user starts speaking in Hindi, the conversation can continue in Hindi even after the handoff.

English

User:

"What documents do I need for PM-KISAN?"

Dia:

"This is a government-scheme question, so I'll connect you with Krishna, our government-scheme specialist."

Krishna:

"Hi, I understand you're asking about the documents required for PM-KISAN. Let me help you with that."

Hindi

User:

"PM-KISAN के लिए कौन-कौन से documents चाहिए?"

Dia:

"यह सरकारी योजना से जुड़ा सवाल है, इसलिए मैं आपको कृष्ण, हमारे सरकारी योजना विशेषज्ञ से जोड़ता हूँ।"

Krishna:

"नमस्ते, मुझे समझ आया कि आप PM-KISAN के लिए जरूरी दस्तावेजों के बारे में पूछ रहे हैं। मैं आपकी मदद करता हूँ।"

This was important to me because building for Bharat isn't just about adding Indian languages — the entire conversation should continue naturally in the user's preferred language, even when another specialist agent takes over.

🧠 Memory and Context

Another important part of FINORA AI is memory.

A useful assistant shouldn't force returning users to repeat relevant information unnecessarily.

FINORA AI can retain useful conversational context where appropriate, helping the interaction feel more continuous.

However, because FINORA is related to financial services, I designed the system with clear privacy boundaries.

Sensitive information such as:

OTPs
PINs
Passwords
CVVs
Authentication secrets

should not be stored as ordinary conversational memory.

The goal is to remember useful context, not sensitive credentials.

🛠️ Agent Capabilities and Workflows

Rather than trying to make one agent responsible for everything, FINORA AI separates responsibilities.

Dia is the main financial assistant.

Krishna focuses specifically on government schemes.

And when something requires a human, the system can use the existing human escalation workflow.

This creates a simple separation:

General Financial Assistance
            ↓
           DIA

Government Scheme Questions
            ↓
         KRISHNA

Sensitive / Human-Only Issues
            ↓
      HUMAN SUPPORT

This separation became one of the most important architectural decisions in the project.

📞 Outbound Calling

FINORA AI also supports outbound calling workflows.

This changes the interaction from:

"Ask me something."

to:

"Let me help you with something you need."

Building outbound calls made me think more carefully about how an agent starts a conversation.

An outbound voice agent needs to:

Introduce itself
Explain why it is calling
Establish context quickly
Avoid sounding overly scripted
Give the user a clear next step

🧑‍💼 When AI Should Ask for Human Help

One of the most important things I learned during the challenge was that AI shouldn't try to solve everything.

For financial services, this becomes even more important.

For example, imagine a user says:

"I see a transaction that I didn't make."

Dia shouldn't pretend it can investigate or resolve the issue itself.

Instead, the system recognizes that human assistance may be required.

The flow becomes:

User reports possible fraud
          ↓
Dia identifies the situation
          ↓
Dia explains why human help is needed
          ↓
Ask for user permission
          ↓
User agrees
          ↓
Create escalation
          ↓
Generate reference ID
          ↓
Send request to human support

The user receives a reference ID and a clear explanation of what happens next.

Caption: FINORA AI creating a human-support request after receiving the user's permission.

🔐 Privacy by Design

Since FINORA AI is related to financial services, privacy cannot be an afterthought.

The escalation workflow is designed to avoid sharing unnecessary sensitive information.

It should never send:

Passwords
OTPs
PINs
CVVs
Full account numbers
Authentication secrets

Instead, the human receives a short summary containing only useful information.

For example:

Reference: FIN-2026-0001

Reason:
Possible Fraud

Summary:
Caller reports an unauthorized transaction.

Urgency:
High

Language:
Hindi

Preferred Follow-up:
Phone

The principle is simple:

Give the human enough information to help without giving them information they don't need.

🤝 Meet Krishna — Government Scheme Specialist

As FINORA AI grew, I realized that Dia shouldn't have to be an expert in everything.

So I created a separate specialist agent:

Krishna

Krishna is FINORA AI's Government Scheme Specialist.

His job is focused specifically on government financial and welfare schemes.

For example:

PM-KISAN
PM Jan Dhan Yojana
PM SVANidhi
Mudra-related schemes
Pension schemes
Insurance schemes
Scholarship schemes
Other supported government schemes

The key idea is that Krishna has a smaller and more focused responsibility than Dia.

Caption: Krishna, FINORA AI's dedicated Government Scheme Specialist.

🔄 Real Agent-to-Agent Handoff

This is one of the features I am most excited about.

The system doesn't simply ask Dia to generate a response about government schemes.

Instead, Dia can hand the conversation over to Krishna, a separate specialist agent.

For example:

User:

"What documents are required for PM-KISAN?"

Dia responds:

"This is a government-scheme question, so I'll connect you with Krishna, our government-scheme specialist. You won't need to repeat your question."

Then Krishna takes over:

"Hi, I'm Krishna, the government-scheme specialist. I understand you're asking about the documents required for PM-KISAN. Let me help you with that."

The important part is that the user doesn't have to explain the entire problem again.

The relevant context is passed during the handoff.

flowchart LR
    U[User] --> D[Dia]

    D -->|Government Scheme Question| K[Krishna]

    K -->|Continues Same Conversation| U

    D -->|Sensitive Issue| H[Human Support]

This makes FINORA AI a multi-agent voice system, rather than just a single chatbot with many prompts.

Caption: Dia handing the conversation to Krishna without requiring the user to repeat their question.

📊 Building the Call Analytics Dashboard

After adding multiple capabilities, I wanted to answer a simple question:

Is the agent actually working?

That's why I built a call analytics dashboard.

The dashboard tracks real calls and displays:

Total Calls

The total number of calls handled.

Successful Calls

Calls that reached the defined success condition.

Failed Calls

Calls that ended without reaching the intended outcome.

I also added agent handoff information.

For example:

Total Calls: 25

Successful Calls: 21

Failed Calls: 4

Agent Handoffs: 7

Handoffs to Krishna: 7

Replace these example numbers with your actual dashboard values before publishing.

These numbers come from actual call activity rather than being hardcoded.

[INSERT IMAGE: FINORA AI CALL ANALYTICS DASHBOARD]

Caption: FINORA AI's call analytics dashboard showing real call outcomes and specialist handoffs.

📈 Handoff Analytics

An important design decision was that a specialist handoff should not create a second call.

For example:

CALL-001
   │
   ├── Dia
   │
   └── Krishna

This is still:

ONE CALL

with the handoff recorded as part of that call.

This keeps the analytics accurate.

The dashboard can therefore show the complete journey of a call without artificially inflating the number of calls.

Caption: Call analytics recording a Dia-to-Krishna handoff as part of the same conversation and call.

🎨 Designing a Simple User Experience

While building all these features, I didn't want the website to become complicated.

The FINORA AI interface is intentionally:

Minimalistic
Clean
Easy to understand
Voice-focused

The main navigation includes:

Voice Assistant
Finance Dashboard
Call Analytics
Settings
Help

The user should be able to open the website and immediately understand what to do.

Talk to Dia. Get help.

The technology stays in the background.

🧩 The Challenges I Faced

Building FINORA AI over ten days wasn't a simple process where everything worked on the first attempt.

There were several challenges.

1. Building a Natural Voice Experience

Connecting real-time voice communication, the agent, and TTS required understanding how all the components interact.

A small configuration problem could mean the agent connected successfully but didn't respond correctly.

I had to debug the voice pipeline step by step instead of assuming the issue was always with the LLM.

2. Deciding When AI Should Stop

One of the biggest design questions was:

When should AI answer, and when should AI stop?

Too many escalations make the experience frustrating.

Too few can make the system unsafe.

So I created clear conditions for human escalation and specialist handoff.

3. Keeping Analytics Real

Displaying numbers on a dashboard is easy.

Making those numbers represent actual calls is much harder.

The system needs to record the call outcome and handoff information from the actual voice session.

This helped me understand the difference between building a UI and building a system that produces meaningful data.

4. Multi-Agent Handoff

The biggest challenge with Krishna was making the handoff feel like one continuous conversation.

Krishna needs to understand:

What the user asked
Why Dia handed the conversation over
What language the user is using
What context is relevant

The user shouldn't have to start again with:

"Hello, I was asking..."

The handoff should feel natural.

🚀 How You Can Build Your Own Voice Agent

If you want to build a voice agent yourself, there are four major pieces you need.

1. Speech-to-Text

Converts the user's speech into text.

User Voice
   ↓
Speech-to-Text
   ↓
Text

2. LLM / Agent

The agent understands the request and decides what to do.

Text
 ↓
LLM
 ↓
Decision

3. Text-to-Speech

The response is converted back into voice.

For FINORA AI, I use Murf Falcon.

Agent Response
      ↓
Murf Falcon
      ↓
Natural Voice

4. Real-Time Transport

A real-time communication layer connects the user and the voice agent.

FINORA AI uses LiveKit.

The simplified pipeline is:

User
 ↓
Microphone
 ↓
LiveKit
 ↓
Speech-to-Text
 ↓
Dia / Krishna
 ↓
LLM
 ↓
Murf Falcon
 ↓
LiveKit
 ↓
User

⚙️ Running the Project

Start by cloning the project:

git clone https://github.com/Sujayhs07/murf-livekit-starter/tree/DAY10
cd murf-livekit-starter

Create a .env file for your API keys and configuration.

For example:

MURF_API_KEY=your_murf_api_key

LIVEKIT_URL=your_livekit_url
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

Add any additional credentials required by your implementation.

Never commit your .env file to GitHub.

Add it to .gitignore:

.env

Then install the dependencies and start the LiveKit agent according to the project's setup instructions.

Once the browser client connects, you can start a conversation with Dia.

🧪 Testing FINORA AI

I tested different conversation paths.

Test 1 — Normal Question

User:

"What is compound interest?"

Expected:

User → Dia

No specialist handoff.

Test 2 — Government Scheme

User:

"What documents do I need for PM-KISAN?"

Expected:

User
 ↓
Dia
 ↓
Krishna

The user should not have to repeat the question.

Test 3 — Hindi Government Scheme

User:

"PM-KISAN के लिए कौन-कौन से documents चाहिए?"

Expected:

User
 ↓
Dia — Hindi
 ↓
Krishna — Hindi

The language should remain consistent across the handoff.

Test 4 — Possible Fraud

User:

"Someone made a transaction from my account."

Expected:

User
 ↓
Dia
 ↓
Permission
 ↓
Human Escalation

It should not be routed to Krishna.

Test 5 — Analytics

After the call ends, the dashboard should reflect the real result:

Total Calls ↑

Successful Calls ↑

Failed Calls ↑

Handoffs to Krishna ↑

Caption: A snapshot of FINORA AI's end-to-end voice-agent workflow.

📚 What I Learned

The biggest lesson I learned from this challenge is that building a voice agent isn't really about making an AI talk.

The interesting engineering begins after that.

You have to think about:

What should the agent know?
What should it remember?
When should it use a tool?
When should it stop?
When should a human take over?
When should another agent take over?
How should context move between agents?
How do we measure success?
How do we protect user information?

These questions completely changed how I think about voice AI.

🔮 What I'd Build Next

FINORA AI still has a lot of room to grow.

Some things I'd like to work on next are:

More Indian languages
More specialist agents
Better multilingual conversations
Improved personalization
More advanced analytics
More government-scheme coverage
Better accessibility
Smarter follow-up workflows
A richer human-support dashboard
More robust real-world integrations

I'd especially like to expand the multi-agent architecture so FINORA AI can route users to different specialists depending on what they need.

❤️ What 10 Days Taught Me

When I started the challenge, I thought the hardest part would be getting an AI agent to speak.

By Day 10, I realized:

Voice is only the beginning.

The real challenge is building an agent that knows how to behave.

An agent that can say:

"I can help with that."

But also knows when to say:

"Let me connect you with Krishna, our specialist."

And sometimes:

"This needs human assistance."

That's what I wanted FINORA AI to become.

Not just an AI that talks.

An AI that knows what to do next.

🔗 Project Links

GitHub Repository

https://github.com/Sujayhs07/murf-livekit-starter/tree/DAY10

Challenge

10 Days of Voice Agents — VoiceForBharat Edition

🙌 Final Thoughts

Building FINORA AI during the 10 Days of Voice Agents — VoiceForBharat Edition has been one of the most hands-on AI projects I've worked on.

Over these ten days, I explored real-time voice communication, TTS, agent design, memory, outbound calls, human escalation, call analytics, and multi-agent handoffs.

More importantly, I learned that building a useful AI system isn't only about adding more features.

It's about knowing when each feature should be used.

Dia doesn't need to know everything.

Krishna doesn't need to handle everything.

And AI doesn't need to handle everything either.

Sometimes the best decision an AI can make is to ask another specialist or a human for help.

A good voice agent isn't the one that tries to do everything. It's the one that knows what it can do, knows what it can't, and knows what to do next.

Thanks to Murf AI for organizing the 10 Days of Voice Agents — VoiceForBharat Edition and giving me the opportunity to build, experiment, break things, fix them, and learn along the way.

🛠️ Built With

🎙️ Murf Falcon
⚡ LiveKit
🧠 LLM-powered Agents
🗣️ English + Hindi
🤝 Human Escalation
🔄 Multi-Agent Handoff
📊 Call Analytics
🎨 Minimalistic Web Interface

#VoiceForBharat #10DaysOfVoiceAgents #MurfAI #VoiceAI #GenerativeAI #AI #FinTech@MURFAI

DEV Community: Sujay H S

# FINORA AI: Building a Voice-First Financial Assistant for Bharat with Murf Falcon 🎙️🇮🇳

FINORA AI: Building a Voice-First Financial Assistant for Bharat with Murf Falcon 🎙️🇮🇳

Introduction

🌱 The Problem

🎙️ Meet FINORA AI

🎯 Why Voice?

🏗️ How FINORA AI Works

🔊 Murf Falcon — Giving FINORA AI a Voice

🇮🇳 English + Hindi Voice Support

English

Hindi

🧠 Memory and Context

🛠️ Agent Capabilities and Workflows

📞 Outbound Calling

🧑‍💼 When AI Should Ask for Human Help

🔐 Privacy by Design

🤝 Meet Krishna — Government Scheme Specialist

Krishna

🔄 Real Agent-to-Agent Handoff

📊 Building the Call Analytics Dashboard

Total Calls

Successful Calls

Failed Calls

📈 Handoff Analytics

🎨 Designing a Simple User Experience

🧩 The Challenges I Faced

1. Building a Natural Voice Experience

2. Deciding When AI Should Stop

3. Keeping Analytics Real

4. Multi-Agent Handoff

🚀 How You Can Build Your Own Voice Agent

1. Speech-to-Text

2. LLM / Agent

3. Text-to-Speech

4. Real-Time Transport

⚙️ Running the Project

🧪 Testing FINORA AI

Test 1 — Normal Question

Test 2 — Government Scheme

Test 3 — Hindi Government Scheme

Test 4 — Possible Fraud

Test 5 — Analytics

📚 What I Learned

🔮 What I'd Build Next

❤️ What 10 Days Taught Me

🔗 Project Links

GitHub Repository

Challenge

🙌 Final Thoughts

🛠️ Built With