DEV Community

Cover image for When the Water Rises: Building Sentinel AI Disaster Response in 10 Days
Subhangi Dutta
Subhangi Dutta

Posted on

When the Water Rises: Building Sentinel AI Disaster Response in 10 Days

#ai #voiceai #python #learning

It was during the peak monsoon season when I read a distress report from a family stranded on their rooftop. Power was flickering, rain was hammering against the walls, and mobile data was down to a crawling 2G connection. They had a phone with 14% battery, but navigating complex portals or downloading 20-page government advisory PDFs was impossible.

All they could do was make a direct voice call.

When water is rising inside your living room, you cannot type or navigate dropdowns. You need an immediate, natural voice that provides real-time flood advisories, locates relief shelter beds, and dispatches rescue teams without delay.

That became the driving mission behind Sentinel — an autonomous, real-time Voice AI emergency dispatcher built over 10 days during the #VoiceForBharat challenge.

1. The Problem I Wanted to Solve

Emergency management during Indian monsoons faces three critical bottlenecks:

  1. Information Fragmentation: Emergency guidelines, live rainfall alerts, and shelter capacities exist across disconnected departments. Callers need instant answers (e.g., "Where is the nearest shelter in Patna with medical support?").
  2. The Friction of Touch Interfaces: Wet screens, low digital literacy, and high adrenaline make text UIs fail. Voice is the most accessible lifeline.
  3. Context Drop During Escalation: Traditional hotlines often disconnect or force victims to repeat their distress details to multiple operators.

2. System Architecture & Data Flow

To ensure sub-second response times, Sentinel connects audio streaming, AI reasoning, and low-latency speech synthesis into a unified WebRTC pipeline powered by LiveKit, Deepgram Nova-3, Google Gemini, Murf AI, and Open-Meteo:

Disaster Response Sentinel System Architecture

 Caller Voice (Browser WebRTC / Inbound SIP)
            │
            ▼
 ┌─────────────────────────────────────────────────────────┐
 │               LiveKit WebRTC Transport                  │
 │      (Audio Ingestion & Silero VAD Turn Detection)       │
 └──────────────────────────┬──────────────────────────────┘
                            │
                            ▼
 ┌─────────────────────────────────────────────────────────┐
 │               Deepgram Nova-3 (STT)                     │
 │          (Continuous Multilingual Streaming)           │
 └──────────────────────────┬──────────────────────────────┘
                            │ Real-time Transcript
                            ▼
 ┌─────────────────────────────────────────────────────────┐
 │          Google Gemini 3.5 Flash-Lite (LLM)             │
 │                                                         │
 │  • Emergency Triage & Safety Guardrails                 │
 │  • SQLite Caller Memory & State Tracking                │
 │  • Open-Meteo Weather Telemetry Tools                   │
 │  • Dynamic Specialist Agent Handoff                     │
 └──────────────────────────┬──────────────────────────────┘
                            │ Streaming Text Tokens
                            ▼
 ┌─────────────────────────────────────────────────────────┐
 │               Murf Falcon 2 (TTS)                       │
 │  (Ultra-Fast Indian Voice Stream: Anisha/hi-IN)         │
 └──────────────────────────┬──────────────────────────────┘
                            │
                            ▼
 Direct Audio Stream back to Caller (<500ms Time-to-First-Byte)
Enter fullscreen mode Exit fullscreen mode

3. The Sentinel Voice Interface

Sentinel is designed with a lightweight, accessible interface featuring real-time Indian helpline integrations (112, NDRF 1078, Medical 108) and live ticker alerts for flood and drought monitoring across India.

Sentinel in ready state with live disaster telemetry feeds:


# Low-Latency Voice Pipeline Configuration
session = AgentSession(
    stt=deepgram.STT(model="nova-3", language="multi"),
    llm=google.LLM(model="gemini-3.5-flash-lite"),
    tts=murf.TTS(
        voice="Anisha",
        locale="hi-IN",
        style="Conversation",
        tokenizer=tokenize.basic.SentenceTokenizer(min_sentence_len=1),
        text_pacing=True,
    ),
    turn_detection=MultilingualModel(),
    vad=ctx.proc.userdata["vad"],
    preemptive_generation=False,
)
Enter fullscreen mode Exit fullscreen mode

4. Real-Time Telemetry & Dispatch Dashboards

To make Sentinel production-ready for disaster management cells, I built two specialized real-time monitoring consoles:

A. Call Analytics & Success Engine

The Call Metrics Dashboard tracks total inbound WebRTC and SIP telephony calls, analyzing whether each interaction successfully delivered verified telemetry or required human intervention.

  • Outcome Verification: Automatically parses call logs and session duration.
  • Success Criteria Standard: Flags calls as successful only when verified relief shelter coordinates are provided or an escalation ticket is generated.

B. Live Emergency Dispatcher & PII Protection

When callers report being trapped or injured, Sentinel flags the situation as EMERGENCY urgency and generates a structured ticket.

  • Automated PII Redaction: Phone numbers, sensitive identification, and private contact info are masked automatically ([REDACTED_PHONE]) to protect caller privacy while giving rescue units necessary triage notes.
  • Status Lifecycle: Live sync tracks requests across Open, In Progress, and Resolved states with callback dispatch verification.

5. Specialist Handoffs: Shelter Information Agent

When a caller specifically requests shelter availability or medical resources, Sentinel delegates the session to a Shelter Information Specialist using dynamic agent handoffs:

class ShelterInformationSpecialist(Agent):
    def __init__(self, instructions: str, call_context: dict, location: str):
        super().__init__(instructions=instructions)
        self.location = location

    async def on_enter(self) -> None:
        shelter_data = disaster_data.compute_nearest_shelters(location=self.location)
        greeting = (
            f"Hello, I am the Shelter Information Specialist. "
            f"I have live capacity for {self.location}. {shelter_data} "
            f"Would you like details on pet policies, medical care, or check-in rules?"
        )
        await self.session.say(greeting, allow_interruptions=True)
Enter fullscreen mode Exit fullscreen mode

6. Challenges & Solutions: The "Silent Handoff"

  • The Problem: When delegating from Sentinel Command to the Shelter Specialist, the previous turn concluded and the audio stream went silent, leaving the user waiting without feedback.
  • The Solution: Implemented the on_enter() lifecycle method in LiveKit Agents to trigger an immediate audio synthesis turn via session.say() using Murf Falcon the instant the handoff occurs, eliminating dead air.

7. How to Run Sentinel

You need Python 3.10 through 3.13, Node.js 18+, and pnpm 9. Clone the public repository then create local environment files:

cp backend/.env.example backend/.env.local
cp frontend/.env.example frontend/.env.local
Enter fullscreen mode Exit fullscreen mode

Add these required backend values to backend/.env.local:

LIVEKIT_URL=your_livekit_url
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret
MURF_API_KEY=your_murf_api_key
GOOGLE_API_KEY=your_gemini_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
Enter fullscreen mode Exit fullscreen mode

Start the Services 

# 1. Clone the repository
git clone [https://github.com/subhangidutta23-ship-it/murf-livekit-starter.git](https://github.com/subhangidutta23-ship-it/murf-livekit-starter.git)
cd murf-livekit-starter

# 2. Start the Backend Worker
cd backend
.\.venv\Scripts\python.exe -m src.agent dev

# 3. Start the Frontend Console
cd ../frontend
pnpm install
pnpm dev
Enter fullscreen mode Exit fullscreen mode

Open http://localhost:3000 in your browser to begin an emergency call session.

Thank you to Murf AI and the VoiceForBharat organizers for this 10-day challenge!

Top comments (0)