DEV Community

Arham Sajid
Arham Sajid

Posted on

Building Suraksha Sathi: My 10-Day Journey into Voice AI for Disaster Response

#agents #ai #devchallenge #llm

🌪️ Building Suraksha Sathi: A Hindi/Hinglish Voice Agent for Responsible Disaster Response

Building a Hindi/Hinglish voice agent designed to provide responsible disaster-response assistance through real-time voice conversations.

What if, during a disaster, someone could simply speak to an AI assistant instead of navigating complicated interfaces?

That question became the starting point for my project, Suraksha Sathi, built during the 10 Days of Voice Agents — VoiceForBharat Edition challenge.

Over 10 days, I gradually evolved the project from a basic voice conversation into a more complete disaster-response voice system with:

  • 🎙️ Real-time voice conversations
  • 🇮🇳 Hindi, English, and Hinglish interaction
  • 🧠 Persistent user memory
  • 🔧 Disaster-response tools
  • 📞 Outbound welfare calls
  • 👤 Human escalation
  • 📊 Call analytics
  • 🤝 Specialist-agent handoffs
  • 🛡️ Safety-focused disaster-response guardrails

This article covers the problem, architecture, challenges, implementation approach, and lessons I learned while building Suraksha Sathi.

🌪️ Why Disaster Response?

During a disaster, information can be just as important as physical resources.

People may need answers to questions such as:

  • What should I do right now?
  • What safety precautions should I follow?
  • What verified information is available?
  • Where can I find shelter information?
  • When should I seek human assistance?
  • Can someone check on a family member?

In stressful situations, typing through menus or searching through multiple websites may not always be convenient.

Voice provides another interface.

Instead of navigating:

Open website
      ↓
   Search
      ↓
Read information
      ↓
Figure out what applies
Enter fullscreen mode Exit fullscreen mode

the interaction becomes:

Speak
  ↓
Suraksha Sathi understands
  ↓
Verified information / guidance
  ↓
Voice response
Enter fullscreen mode Exit fullscreen mode

The goal was not to create an AI that could replace emergency services.

The goal was to create a responsible conversational layer for disaster-response information and assistance.

🎙️ Why Voice Matters in Emergencies

Voice can make interaction more immediate.

A user may be:

  • Stressed
  • In a hurry
  • Unable to type comfortably
  • More comfortable speaking Hindi or Hinglish
  • Looking for a quick explanation

A voice agent can provide a conversational interface without requiring the user to navigate a complex UI.

But disaster response introduces an important constraint:

🛡️ A voice agent must be especially careful about what it claims.

That became one of the central design principles of Suraksha Sathi.

🛡️ Safety First: The Most Important Guardrail

Disaster-response AI is not an area where an agent should confidently make things up.

Suraksha Sathi therefore has strict boundaries.

It must never:

  • ❌ Issue an evacuation order
  • ❌ Declare an area safe
  • ❌ Issue an all-clear
  • ❌ Claim that a disaster has ended
  • ❌ Invent emergency alerts
  • ❌ Invent shelter availability
  • ❌ Invent shelter capacity
  • ❌ Claim that someone has been rescued
  • ❌ Claim that emergency responders have been dispatched unless actually confirmed

Instead, the agent should distinguish between:

Verified information

and

General safety guidance.

For example, if current information is unavailable, the agent should say that it cannot verify the information rather than inventing an answer.

This became one of the biggest lessons of the challenge:

In safety-critical systems, knowing what not to say is just as important as knowing what to say.

🇮🇳 Hindi, English and Hinglish

Disaster-response assistance needs to be accessible to users with different language preferences.

A user might say:

"Flood aa raha hai, mujhe abhi kya precautions lene chahiye?"

Or:

"Mere area ka current alert status kya hai?"

Or:

"What should I do if water level is increasing?"

The system should understand these different conversational styles without forcing users to communicate in one particular language.

The goal was simple:

Let people communicate naturally.

🏗️ The Suraksha Sathi Architecture

The project gradually evolved into a multi-component voice-agent system.

At a high level:

                         ┌───────────────┐
                         │     User      │
                         └───────┬───────┘
                                 │
                                 ▼
                         ┌───────────────┐
                         │ Voice         │
                         │ Interface     │
                         └───────┬───────┘
                                 │
                                 ▼
                     ┌──────────────────────┐
                     │ Real-Time            │
                     │ Communication        │
                     └──────────┬───────────┘
                                │
                                ▼
                     ┌──────────────────────┐
                     │   Suraksha Sathi     │
                     │     Main Agent       │
                     └──────────┬───────────┘
                                │
              ┌─────────────────┼─────────────────┐
              │                 │                 │
              ▼                 ▼                 ▼
          ┌────────┐       ┌────────┐      ┌──────────────┐
          │ Memory │       │ Tools  │      │    Human     │
          │        │       │        │      │ Escalation   │
          └────────┘       └───┬────┘      └──────────────┘
                               │
                               ▼
                      Verified / Available
                            Data
                                │
                                └──────────┐
                                           │
                                           ▼
                                ┌────────────────────┐
                                │ Specialist Agent   │
                                │    ShelterSathi    │
                                └─────────┬──────────┘
                                          │
                                          ▼
                                ┌────────────────────┐
                                │    Murf Falcon     │
                                │   Text-to-Speech    │
                                └─────────┬──────────┘
                                          │
                                          ▼
                                        User
Enter fullscreen mode Exit fullscreen mode

📞 Outbound Call Architecture

For outbound calls, the architecture additionally involves SIP:

Frontend
   ↓
Outbound Call Request
   ↓
Backend
   ↓
LiveKit Room
   ↓
SIP Trunk
   ↓
Phone
   ↓
Suraksha Sathi
Enter fullscreen mode Exit fullscreen mode

This architecture helped me understand that a voice agent is much more than an LLM.

It is a complete system involving communication, reasoning, tools, memory, safety, telephony, and monitoring.

⚡ Why Murf Falcon?

A voice agent needs to respond quickly enough that the interaction still feels natural.

For this project, I used Murf Falcon as the text-to-speech layer.

The objective wasn't simply to convert text into audio.

It was to make the system feel conversational.

The difference is between:

"The system generated a response."

and:

"The agent is actually talking to me."

For a real-time disaster-response assistant, that conversational experience matters because the user should be able to ask follow-up questions naturally instead of interacting with a static information system.

🧠 Giving Suraksha Sathi Memory

On Day 4, I introduced persistent memory.

Without memory, every interaction starts from zero.

With memory, the agent can retain useful conversational context across interactions.

A simplified structure can look like:

{
  "user_id": "user_123",
  "name": "Example User",
  "language_preference": "Hinglish",
  "facts": {
    "preferred_language": "Hinglish"
  },
  "last_interaction": "timestamp"
}
Enter fullscreen mode Exit fullscreen mode

The important principle is that memory should be intentional.

The system shouldn't store everything simply because it can.

It should store information that is actually useful for future interactions and appropriate to retain.

This led to an important design rule:

Memory should improve the conversation without becoming unnecessary data collection.

🔧 Giving the Agent Disaster-Response Tools

A language model alone should not be expected to know current disaster-response information.

That is where tools become important.

The general flow is:

User Question
      ↓
Suraksha Sathi
      ↓
Does this require available external information?
      ↓
     Yes
      ↓
   Tool Call
      ↓
Verified / Available Data
      ↓
Suraksha Sathi explains the result
Enter fullscreen mode Exit fullscreen mode

Tools can provide structured information that the model should not simply guess.

This is particularly important in disaster response.

If information is unavailable, the system should communicate that limitation rather than hallucinating.

For example:

"I couldn't verify the latest shelter information right now."

is much safer than inventing a shelter location.

📞 Outbound Welfare Calls

Day 6 introduced outbound calling.

Instead of waiting for a user to start a conversation, Suraksha Sathi could proactively initiate a call.

The simplified architecture became:

Outbound Request
      ↓
Backend
      ↓
LiveKit
      ↓
SIP
      ↓
Phone
      ↓
Suraksha Sathi
Enter fullscreen mode Exit fullscreen mode

The disaster-response context makes outbound calling particularly interesting.

Possible applications include:

  • Welfare checks
  • Follow-up calls
  • Status checks
  • Information reminders
  • Proactive assistance

However, outbound calls also require transparency.

The agent should clearly identify itself and explain why it is calling.

It should not pretend to be a human emergency responder.

👤 Human Escalation

One of the most important capabilities of Suraksha Sathi is knowing when AI is not enough.

Disaster situations can involve circumstances where human assistance is required.

For example:

  • A person may need urgent human support.
  • A situation may require intervention.
  • The AI may not have enough verified information.
  • The user may explicitly request human assistance.

In such cases, the system should follow the existing human-escalation workflow rather than pretending to solve the problem.

The key principle is:

AI assistance should complement human response, not pretend to replace it.

The escalation workflow can be represented as:

User needs human help
        ↓
Suraksha Sathi recognizes the situation
        ↓
Explains the escalation
        ↓
Requests permission where required
        ↓
Creates escalation request
        ↓
Provides reference information
Enter fullscreen mode Exit fullscreen mode

The system should also avoid passing unnecessary sensitive information.

📊 Measuring the Calls

On Day 8, I introduced call analytics.

Instead of simply saying:

"The agent works."

I wanted to understand what actually happened during calls.

The system tracks outcomes such as:

  • Total Calls
  • Successful Calls
  • Failed Calls
  • Success Rate

The data comes from actual call outcomes.

The architecture becomes:

Voice Call
    ↓
Call Outcome
    ↓
Database
    ↓
Analytics
    ↓
Dashboard
Enter fullscreen mode Exit fullscreen mode

This was an important shift.

I wasn't just building a voice demo anymore.

I was building a system that could be observed and evaluated.

🤝 Introducing ShelterSathi

Day 9 introduced one of the biggest architectural changes.

Suraksha Sathi shouldn't try to become an expert in every possible disaster-response topic.

So I introduced a specialist agent:

🏠 ShelterSathi

ShelterSathi is focused specifically on shelter-related information.

The architecture becomes:

                    Suraksha Sathi
                         │
               ┌─────────┴─────────┐
               │                   │
        Normal Question      Shelter Question
               │                   │
               ▼                   ▼
       Suraksha Sathi          ShelterSathi
Enter fullscreen mode Exit fullscreen mode

Normal Question

"Flood ke time ghar mein kya precautions lene chahiye?"

Suraksha Sathi handles it directly.

Specialist Question

"Mere area ke paas available shelter kahan hai aur capacity kitni hai?"

Suraksha Sathi recognizes that the request requires shelter-specific assistance.

It announces the handoff.

ShelterSathi then takes over.

🔄 Context-Preserving Specialist Handoff

A specialist handoff is only useful if the user doesn't have to start over.

A bad experience would be:

User:
"Mere area ke paas shelter kahan hai?"

        ↓

Suraksha Sathi:
"I'll connect you to a specialist."

        ↓

ShelterSathi:
"Hello. How can I help?"
Enter fullscreen mode Exit fullscreen mode

The user now has to repeat the question.

Instead, the desired flow is:

User:
"Mere area ke paas shelter kahan hai?"

        ↓

Suraksha Sathi:
"I'll connect you to the Shelter Information Specialist."

        ↓

ShelterSathi:
"Aapne nearby shelter ke baare mein poocha tha.
Main wahi se continue karta hoon."
Enter fullscreen mode Exit fullscreen mode

The specialist receives the relevant conversation context.

This makes the handoff feel like one continuous conversation rather than two separate chatbots.

🚨 A Critical Disaster-Response Guardrail

One of the strongest safety rules in the project is that the agent must never issue an all-clear or evacuation instruction.

This distinction is extremely important.

The agent can provide:

  • General safety guidance
  • Verified information
  • Available shelter information
  • Guidance on seeking appropriate human or official assistance

But it should not independently declare:

"You are safe now."

or:

"You should evacuate immediately."

or:

"The emergency is over."

Those decisions belong to appropriate authorities and verified emergency-response channels.

This limitation is intentional.

A responsible disaster-response agent should inform and assist without pretending to have authority it does not possess.

🧩 What Building Suraksha Sathi Taught Me

At the beginning of the challenge, it was easy to think of a voice agent as:

LLM + TTS
Enter fullscreen mode Exit fullscreen mode

After building Suraksha Sathi, the architecture looks much more like:

Voice
  +
Real-Time Communication
  +
LLM
  +
Prompting
  +
Tools
  +
Memory
  +
Safety Guardrails
  +
Error Handling
  +
Telephony
  +
Human Escalation
  +
Analytics
  +
Specialist Routing
Enter fullscreen mode Exit fullscreen mode

Each layer solves a different problem.

Component Purpose
🧠 LLM Reasoning and conversation
🔧 Tools External / available information
🧠 Memory Conversational continuity
📞 Telephony Reach beyond the browser
📊 Analytics Measurement and evaluation
👤 Human escalation Support beyond AI authority
🤝 Specialist agents Focused expertise
🛡️ Guardrails Safety boundaries

And that taught me to think much more like a systems engineer.

🧯 The Difficult Parts

Building the system wasn't completely smooth.

And those difficulties were some of the most valuable parts of the experience.

1. Outbound Calling

Outbound calling involved several interconnected components:

Frontend
   ↓
Backend
   ↓
LiveKit
   ↓
SIP Trunk
   ↓
Telephony Client
   ↓
Agent
Enter fullscreen mode Exit fullscreen mode

A problem anywhere in that chain could affect the final call.

This taught me to debug from the infrastructure layer upward instead of immediately blaming the AI model.

2. Tool Reliability

A tool can fail even when the agent correctly understands the user's request.

That means the agent needs explicit fallback behavior.

Instead of inventing an answer, it should explain that the required information could not be verified.

This is particularly important in disaster response because inaccurate information can have serious consequences.

3. Specialist Context

Creating a second agent is easy compared with making the handoff feel natural.

The specialist needs to understand:

  • Why it was called
  • What the user asked
  • What context is relevant
  • What it should do next

Without context transfer, the user has to repeat themselves.

That defeats the purpose of delegation.

🛠️ How to Build Your Own Voice Agent

If you want to build a similar system, don't start by implementing every feature at once.

Start with the smallest possible voice loop.

1. Understand the Core Architecture

At minimum:

Speech-to-Text
      ↓
     LLM
      ↓
Text-to-Speech
      ↓
Real-Time Audio Transport
Enter fullscreen mode Exit fullscreen mode

Once this works, add:

Memory
Tools
Guardrails
Telephony
Human Escalation
Analytics
Specialist Agents
Enter fullscreen mode Exit fullscreen mode

Build incrementally.

This makes debugging much easier.

2. Create the Project Structure

A simple structure can look like:

disaster-response-agent/
│
├── backend/
│   └── src/
│       ├── agent.py
│       ├── prompt.py
│       ├── tools/
│       ├── database/
│       └── outbound_call.py
│
├── frontend/
│   ├── app/
│   └── components/
│
├── .env.local
├── .gitignore
└── README.md
Enter fullscreen mode Exit fullscreen mode

Keep the agent logic separate from the frontend.

This makes the project easier to maintain and extend.

🔐 3. Protect API Keys and Private Data

Never hardcode credentials inside your application.

Use environment variables:

MURF_API_KEY=your_murf_api_key
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret
Enter fullscreen mode Exit fullscreen mode

Add environment files to .gitignore:

.env
.env.local
.env.*.local
Enter fullscreen mode Exit fullscreen mode

Commit only a safe example:

MURF_API_KEY=your_api_key_here
LIVEKIT_API_KEY=your_api_key_here
LIVEKIT_API_SECRET=your_api_secret_here
Enter fullscreen mode Exit fullscreen mode

Never publish:

  • 🔑 API keys
  • 🔐 SIP credentials
  • 📞 Phone numbers
  • 👤 User information
  • 🔒 Private call data
  • 🗄️ Database credentials

4. Get the Basic Voice Loop Working

Before adding advanced features:

  1. Start the backend.
  2. Start the frontend.
  3. Connect the real-time communication layer.
  4. Open the voice interface.
  5. Grant microphone permission.
  6. Speak to the agent.
  7. Verify speech recognition.
  8. Verify that the agent responds with audio.

The first milestone should simply be:

Speak → Understand → Respond

Only move forward once that works reliably.

5. Add Memory

Once the basic voice interaction works, add persistent memory.

Store only information that genuinely improves future conversations.

Do not treat memory as permission to collect everything.

A good rule is:

Store useful context, not unnecessary personal data.

6. Add Disaster-Response Tools

Tools should be used when the agent needs information that must be retrieved or verified.

For example:

User asks for information
        ↓
Agent determines that verification is required
        ↓
Tool call
        ↓
Available / verified information
        ↓
Agent explains result
Enter fullscreen mode Exit fullscreen mode

If the tool fails:

Don't guess.

Tell the user that the information could not currently be verified.

7. Add Disaster-Response Guardrails

Before adding advanced capabilities, define the agent's boundaries.

The Agent CAN:

  • Provide general safety guidance
  • Explain verified information
  • Use available disaster-response tools
  • Provide available shelter information
  • Ask clarifying questions
  • Escalate to humans

The Agent CANNOT:

  • ❌ Issue an evacuation order
  • ❌ Issue an all-clear
  • ❌ Declare an area safe
  • ❌ Invent alerts
  • ❌ Invent shelter information
  • ❌ Claim rescue has occurred
  • ❌ Pretend to be an emergency authority

These boundaries should be part of the architecture from the beginning.

8. Add Outbound Calling

Once browser-based conversations work, add telephony.

A simplified architecture is:

Phone
  ↓
SIP
  ↓
Real-Time Communication
  ↓
Voice Agent
  ↓
TTS
  ↓
Phone
Enter fullscreen mode Exit fullscreen mode

Start with a test number you control.

First verify:

Call → Answer → Agent speaks → Conversation → End

Only then add more complicated outbound workflows.

9. Add Human Escalation

Teach the agent when it should stop trying to solve the problem itself.

Human escalation can be appropriate when:

  • The user needs human assistance
  • The situation requires intervention
  • The AI lacks sufficient verified information
  • The user explicitly requests human support

The key principle is:

The AI should complement human emergency response, not pretend to replace it.

10. Add Specialist Agents

Once the main agent works, create focused specialists.

For a disaster-response system, examples could include:

                    Main Agent
                        │
          ┌─────────────┼─────────────┐
          │             │             │
          ▼             ▼             ▼
      Shelter       Information     Support
      Specialist     Specialist    Specialist
Enter fullscreen mode Exit fullscreen mode

Each specialist should have:

  • A focused role
  • Separate instructions
  • Clear boundaries
  • A defined handoff trigger

The main agent should only delegate when specialist expertise is genuinely required.

11. Add Analytics

Finally, measure what happens.

For example:

Total Calls
Successful Calls
Failed Calls
Success Rate
Enter fullscreen mode Exit fullscreen mode

Use real call events:

Call
 ↓
Outcome
 ↓
Database
 ↓
Analytics Dashboard
Enter fullscreen mode Exit fullscreen mode

This turns the project from a demonstration into a measurable system.

🔮 What's Next for Suraksha Sathi?

The challenge may be complete, but the project can continue evolving.

Some possible future directions include:

  • 🌪️ More disaster types
  • 🇮🇳 Better multilingual support
  • 🔎 More verified data sources
  • 🏠 Stronger shelter-information workflows
  • 👤 Improved human escalation
  • 🚨 Better real-time alerts integration
  • 🧪 More robust evaluation
  • 📊 Better observability
  • 🔐 Stronger privacy controls
  • 🚀 Production deployment
  • 🤝 More specialized disaster-response agents

The goal isn't to make Suraksha Sathi replace emergency services.

The goal is to make it a useful, accessible, and responsible conversational layer for disaster-response assistance.

🎓 What I Learned

1. Voice Is More Than Speech

Latency, turn-taking, interruptions, pronunciation, and conversational flow all matter.

2. Safety Is More Important Than Confidence

In disaster response, an uncertain but honest answer is better than a confident false one.

3. Tools Reduce Guessing

If information needs verification, the agent should use available tools rather than inventing information.

4. Memory Needs Boundaries

Remembering useful context can improve conversations, but unnecessary data collection should be avoided.

5. Human Escalation Is a Strength

An AI agent asking for human help is not necessarily failing.

Sometimes that is exactly the correct behavior.

6. Specialist Agents Make Systems More Focused

Instead of making one agent responsible for everything, specialized agents can handle narrower tasks.

7. AI Engineering Is Systems Engineering

The hardest part isn't just making the model respond.

It is connecting:

Voice
  +
LLM
  +
Tools
  +
Memory
  +
Safety
  +
Telephony
  +
Human Support
  +
Analytics
  +
Specialist Agents
Enter fullscreen mode Exit fullscreen mode

into one reliable system.

🚀 Final Thoughts

Ten days ago, this project started as an idea for a disaster-response voice agent.

Today, Suraksha Sathi has evolved into a much broader voice-agent system.

It can:

  • 🎙️ Hold real-time voice conversations
  • 🇮🇳 Support Hindi, English, and Hinglish
  • 🧠 Remember useful user context
  • 🔧 Use disaster-response tools
  • 📞 Make outbound welfare calls
  • 👤 Escalate to human support
  • 📊 Track call outcomes
  • 🤝 Hand shelter-related requests to a specialist agent
  • 🛡️ Follow strict disaster-response safety boundaries

But the biggest lesson isn't the number of features.

It is understanding that responsible AI is also about knowing what the system should never claim to know or do.

Suraksha Sathi should provide assistance.

It should provide verified information when available.

It should clearly communicate uncertainty.

It should escalate when human intervention is needed.

And most importantly, it should never pretend to have emergency authority that it does not have.

In disaster response, being helpful isn't about sounding confident. It's about being accurate, responsible, and honest about your limits.

🇮🇳 10 Days of Voice Agents — VoiceForBharat Edition

This challenge gave me the opportunity to explore voice AI beyond a basic conversational demo.

Across 10 days, I worked through voice interaction, memory, tools, outbound calling, human escalation, analytics, and specialist-agent handoffs.

Each day added another layer to the system.

More importantly, each layer introduced another engineering and safety consideration.

The experience pushed me to think not only about how an AI agent talks, but also about:

  • How it uses information
  • How it handles uncertainty
  • How it remembers users
  • How it communicates limitations
  • How it interacts with humans
  • How it delegates work
  • How its performance can be measured

A huge thanks to the organizers and Murf AI for creating the 10 Days of Voice Agents — VoiceForBharat Edition challenge.

This challenge was a great opportunity to learn by building, testing, debugging, and iterating.

The 10 days may be over.

The project doesn't have to be. 🚀

🙌 Thanks for Reading!

If you're building voice agents or working on AI for emergency-response use cases, I'd love to hear what you're building.

Feel free to share your project or experience in the comments.

Let's build AI systems that are not only capable, but also responsible and useful for Bharat. 🇮🇳🎙️

#VoiceForBharat #10DaysOfVoiceAgents #VoiceAI #AI #DisasterResponse #AgenticAI #GenerativeAI #MurfAI #LiveKit

💻 GitHub Repository

[https://github.com/arhamsajid2812-cell/Sonix-AI/tree/day10]

🎥 Demo Video

Day 4 [https://www.linkedin.com/posts/arham-sajid-737195337_voiceforbharat-activity-7492142441911885824-FWFa?utm_source=social_share_send&utm_medium=member_desktop_web&rcm=ACoAAFSQ-tsB8DdAVgB8wZgleh9-ljEKiOwHPII]

Day 6 [https://www.linkedin.com/posts/arham-sajid-737195337_voiceforbharat-10daysofvoiceagents-murfai-activity-7492965530518065152-6suZ?utm_source=social_share_send&utm_medium=member_desktop_web&rcm=ACoAAFSQ-tsB8DdAVgB8wZgleh9-ljEKiOwHPII]

Top comments (0)