Building a Lifeline: How I Engineered an AI Disaster Response Voice Agent in 10 Days

Ishant — Sat, 15 Aug 2026 12:58:38 +0000

When floodwaters rise or infrastructure fails, screens become useless. Your hands might be full, your phone screen might be wet, or you might simply be in a state of panic. In these critical seconds, typing is a barrier. Talking is an instinct.

For the past 10 days, I participated in the 10 Days of Voice Agents — VoiceForBharat Edition challenge. My goal? To build Raksha—an AI-powered emergency voice assistant that doesn't just chat, but actively triages disasters, fetches live hazard data, and dispatches human rescue teams.

Here is the story of how Raksha evolved from a simple text prompt into a multi-agent, SIP-calling rescue coordinator, the engineering roadblocks I hit, and how you can build one yourself.

🧠 The Anatomy of an Emergency Agent
To build a voice agent that feels human, latency must be virtually zero. If a caller in danger has to wait 3 seconds for a response, trust is lost immediately. I orchestrated four core technologies using the LiveKit framework to achieve sub-second response times:

The Ears (STT): Deepgram Nova-3. Configured to language="multi", it flawlessly captures the code-mixed "Hinglish" spoken across India, even when the caller is speaking frantically.

The Brain (LLM): Google Gemini Flash. It processes the text, enforces strict safety guardrails, and triggers backend Python functions.

The Voice (TTS): Murf Falcon. This was the game-changer. Falcon's speed is unmatched, and using native Indian voices ("Pooja" for our main agent and "Samar" for our specialist) made the system sound empathetic and incredibly natural.

The Nervous System: LiveKit's WebRTC network seamlessly streams the audio back and forth, handling echo cancellation and interruption detection automatically.

🚀 Beyond Chat: The Features That Matter
I didn’t want to build a glorified FAQ bot. Raksha needed to take action.

Outbound SIP Calling & Persistent Memory
Emergency systems shouldn't just wait for the phone to ring. Raksha is integrated with SIP trunking, allowing her to proactively dial out to residents in high-risk zones. Backed by a local SQLite database, she remembers returning callers, their household size, and specific mobility needs.
Human-in-the-Loop Dispatch
AI shouldn't handle life-or-death situations unsupervised. If a caller is trapped, Raksha explicitly asks for consent. If granted, she generates a reference ID and fires a webhook to a live Discord command center, alerting human NDRF rescue teams instantly.
Live Analytics Dashboard
You can't improve what you don't measure. I built a lightweight, auto-refreshing Python dashboard. Through LiveKit's room events, it tracks active calls and only logs a call as "Successful" if Raksha actually gathered triage data or dispatched a rescue team.
True Two-Way Multi-Agent Handoffs
Disaster response requires different experts. Raksha is a triage specialist. But what if a caller is safe and just needs to find a food camp?

I built a Shelter Specialist named Vikram. If a caller needs food, Raksha says, "I will transfer you to Vikram." Raksha's AI engine dynamically shuts down, securely passes the caller's context to a new process, and Vikram's AI engine boots up—taking over the microphone with a distinct, male voice. If the caller suddenly reports a medical emergency, Vikram can seamlessly transfer the call back to Raksha.

🧗 The Crucible: Engineering for Voice
Building a text chatbot is easy; you have time to compute. Building a real-time voice agent is a completely different beast. Voice UX is unforgiving. Here were my biggest architectural hurdles and how I solved them:

Eliminating "Dead Air" During Tool Calling

The Problem: Raksha needs to hit external APIs (like Open-Meteo for flood data) and write to a database. These network requests take about 800ms to 1.5 seconds. Initially, when a user asked for weather data, Raksha would go dead silent for 2 seconds while waiting for the tool to return data, making callers think the line disconnected.

The Fix: I engineered the LLM to generate conversational fillers before executing the tool. By instructing the model to say, "Hold on, let me check the live hazard map for your district," the user stays engaged while the backend asynchronously fetches the JSON data. This completely masked the API latency.

The STT-to-TTS "Hinglish" Pipeline

The Problem: Disaster victims in India don't speak pure English or pure Hindi. At first, if the LLM generated Hindi words using the English alphabet (e.g., "Aap safe ho?"), the TTS engine would pronounce it with a heavy, robotic American accent, breaking the immersion entirely.

The Fix: I added a strict guardrail in Gemini's system prompt: "Always write Hindi words in Devanagari script." Because the Murf Falcon API processes Devanagari natively, the TTS instantly switched to a flawless, empathetic Indian accent, seamlessly blending English technical terms with native Hindi grammar.

Context Leakage in the Multi-Agent Handoff

The Problem: When building the specialist handoff, simply swapping the TTS voice to a male voice wasn't enough. Initially, Vikram would answer the phone and ask the user for their location all over again, which is infuriating in an emergency.

The Fix: I implemented a Stateful Context Transfer. When Raksha triggers the handoff tool, she generates a compressed JSON summary of the caller's location and needs. This payload is dynamically injected into Vikram's initialization prompt. When Vikram boots up, he says, "Namaste, I see you are safe in Patna with three family members. Let's find you food," creating a flawless multi-agent transition.

🛠️ Build Your Own: Ground Zero
Want to build your own voice agent? Here is your starting line.

Clone the Foundation
You can inspect my full code and architecture here:
👉 https://github.com/ic-heisenberg/murf-livekit-starter.git
Protect Your Keys
Voice AI requires API keys. Never push them to public repositories. Create a .env.local file in your root folder:

Code snippet
LIVEKIT_URL=wss://your-url.livekit.cloud LIVEKIT_API_KEY=your_key LIVEKIT_API_SECRET=your_secret MURF_API_KEY=your_murf_key DEEPGRAM_API_KEY=your_deepgram_key

Boot the Matrix Open your terminal and run the following commands to install dependencies and start the agent:

Bash
python -m venv .venv
# Activate it (Windows): .\.venv\Scripts\activate
# Activate it (Mac/Linux): source .venv/bin/activate
pip install livekit-agents livekit-plugins-murf livekit-plugins-google livekit-plugins-deepgram python-dotenv aiohttp
python src/agent.py dev

From there, you can connect via the LiveKit Hosted Playground and start talking to your agent immediately!

🔮 The Road Ahead
This 10-day sprint proved that building ultra-low-latency, context-aware voice AI is highly accessible. In the future, I plan to add Twilio SMS integration so callers receive their NDRF Reference IDs via text automatically after the call drops.

A massive thank you to Murf AI for hosting this challenge. The Murf Falcon API blew my expectations away regarding how fast and human AI can truly sound.

Let’s keep building tech that matters.

link to one of the demo-https://www.linkedin.com/safety/go/?url=https%3A%2F%2Flnkd.in%2FgFWU4-SZ&urlhash=9n0g&mt=s0Jp9AKrCGcEnEHU24WSzo-_UT2FpckxHwZ3vbK0t5HxVsuT7_OXgPMzTtPtpS1iQP_9FNA8KjrlwIA70KPS9htLBxXA0HJYOFXeYusAGnloCdIZTDf8QT93IOU&isSdui=true&lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_creator_profile_all_content_view%3BtMgp1nwpSkyGCFIiscV6pA%3D%3D

DEV Community: Ishant

Building a Lifeline: How I Engineered an AI Disaster Response Voice Agent in 10 Days