DEV Community: Pratyush Dutta

From a Voice Agent to a Disaster Response System: Building Pooja with LiveKit and Murf Falcon

Pratyush Dutta — Sat, 15 Aug 2026 17:58:35 +0000

How I built a multilingual AI disaster-response voice agent with memory, real-time weather data, telephony, human escalation, analytics, and specialist handoffs during the 10 Days of Voice Agents — VoiceForBharat Edition.

Introduction

What started as a simple voice agent quickly turned into something much more ambitious.

During the 10 Days of Voice Agents — VoiceForBharat Edition, I built Pooja, a real-time AI-powered disaster-response voice agent designed to help people during situations where information needs to be delivered quickly, naturally, and reliably.

The core idea was simple:

What if someone could call an AI assistant during a disaster, speak naturally in an Indian language, receive verified information, get connected to a human when necessary, and have the system remember important information across conversations?

That question became the foundation for Pooja.

Over the challenge, I progressively evolved the system from a basic conversational voice agent into a more complete architecture involving:

Real-time voice communication
Indian English and Hindi code-switching
Murf Falcon for low-latency text-to-speech
Gemini as the reasoning layer
Deepgram for speech recognition
LiveKit for real-time communication
Persistent caller memory
Live weather data
Outbound phone calls
Human escalation
Slack notifications
Call analytics
A WebGL-powered frontend
Multi-agent specialist handoffs

This article explains what I built, why I built it, the problems I encountered, and how the different components fit together.

1. The Problem I Wanted to Solve

During a disaster, people don't necessarily need another complicated application.

They need answers.

Questions such as:

"Is there a flood warning in my district?"
"What should I do right now?"
"Where is the nearest shelter?"
"I'm trapped. Can someone help me?"

These questions become particularly challenging when users are under stress, using mobile devices, have limited digital literacy, or prefer communicating in their native language.

That's where voice becomes valuable.

Instead of requiring someone to:

Open a website
Navigate through menus
Read alerts
Search for a shelter
Fill out a form

they can simply speak to the system.

The goal of Pooja was therefore not to create a generic chatbot.

It was to create a voice-first disaster-response interface.

2. Why Voice?

Voice changes the interaction model completely.

A traditional application expects:

User
↓
Read
↓
Understand
↓
Navigate
↓
Click
↓
Submit

A voice agent can reduce that to:

User
↓
Speak
↓
AI understands
↓
AI responds

This becomes especially useful when the user may be:

stressed
moving
unable to type
unfamiliar with technology
more comfortable speaking Hindi or another Indian language

For this reason, multilingual and code-mixed conversations became an important part of Pooja.

The Core Architecture

At a high level, Pooja uses the following voice pipeline:

              USER
                │
                ▼
          ┌───────────┐
          │  LiveKit  │
          │ Real-Time │
          │ Transport │
          └─────┬─────┘
                │
                ▼
          ┌───────────┐
          │ Deepgram  │
          │    STT    │
          └─────┬─────┘
                │
                ▼
          ┌───────────┐
          │  Gemini   │
          │    LLM    │
          └─────┬─────┘
                │
    ┌───────────┼────────────┐
    │           │            │
    ▼           ▼            ▼
 Memory      Weather     Escalation
  Tools        Tool         Tool
    │           │            │
    ▼           ▼            ▼
 SQLite     Open-Meteo     SQLite
                              │
                              ▼
                            Slack
                │
                ▼
          ┌───────────┐
          │   Murf    │
          │  Falcon   │
          │    TTS    │
          └─────┬─────┘
                │
                ▼
              USER

For real phone calls, I extended the architecture further:

Pooja
↓
LiveKit Cloud
↓
SIP
↓
Twilio
↓
Real Phone Number

This allowed the project to move beyond a browser-only demonstration.

4. Giving the Agent a Personality

One of the first lessons I learned was that connecting an LLM to speech does not automatically create a good voice agent.

The agent needs a clear identity, objectives, boundaries, and communication style.

I structured Pooja's system prompt into operational sections:

IDENTITY
OBJECTIVES
KNOWLEDGE
LANGUAGE
GUARDRAILS
STYLE

For example:

Identity

Pooja is a disaster-response voice assistant.

Objectives

Her job is to:

provide verified disaster-related information
understand the caller's situation
provide safe guidance
escalate serious emergencies when required

Guardrails

Pooja must never:

invent emergency information
claim an evacuation order without verification
fabricate rescue ETAs
pretend a shelter is available without reliable data
provide unsupported medical advice

This separation made the system prompt much easier to reason about and debug.

5. Multilingual and Code-Mixed Conversations

For an India-focused voice agent, supporting only English wasn't enough.

I wanted Pooja to naturally handle conversations such as:

"Mere area mein flood warning hai kya?"

or:

"Can you tell me ki nearest shelter kahan hai?"

The objective wasn't simply translation.

The agent needed to understand code-mixed speech and respond naturally in the appropriate language/register.

The pipeline therefore combines:

Deepgram STT → Gemini → Murf Falcon TTS

with instructions to preserve the user's language preference and conversational style.

This allowed Pooja to move between Indian English and Hindi naturally rather than forcing the user into a rigid language mode.

6. Murf Falcon for Real-Time Voice

The voice is a major part of the experience.

For this project, I used Murf Falcon as the text-to-speech layer through the LiveKit Murf integration.

The objective was low-latency, conversational speech rather than waiting for an entire response to be generated before playback began.

This made interactions feel much closer to a real conversation.

The voice configuration was also aligned with the project's India-focused use case.

7. Building Persistent Memory

A voice assistant that forgets everything after every call quickly becomes frustrating.

So I introduced persistent caller memory using SQLite.

The system stores information such as:

user_id
name
language_preference
location
household information
last interaction

The important architectural decision was:

The LLM doesn't directly manipulate the database.

Instead, it uses tools.

For example:

lookup_caller()
save_caller_info()

The flow becomes:

Caller
↓
Pooja
↓
Needs caller information?
↓
lookup_caller()
↓
SQLite
↓
Information returned
↓
Pooja continues conversation

This keeps database access inside controlled application code.

8. Consent and Safety Around Memory

Memory introduces another problem:

What information should an AI actually remember?

I added explicit guardrails around persistent information.

For sensitive information, the agent should not casually save everything it hears.

The system is designed to explain when information is being retained and why it is needed for the disaster-response workflow.

This was an important lesson for me:

Adding memory isn't just a database problem. It's also a trust and privacy problem.

9. Connecting Pooja to Live Data

A disaster-response agent cannot rely entirely on static knowledge.

For Day 5, I connected Pooja to the Open-Meteo API to retrieve live weather information.

I implemented a custom asynchronous tool:

get_district_alert()

The tool retrieves weather metrics such as:

precipitation
wind speed
short-term forecast information
timestamps associated with the data

The LLM decides when this tool is necessary based on the user's request.

For example:

"Is there a flood warning in my district?"

can trigger the weather lookup.

The important part is that the returned API data is transformed into a natural spoken response rather than being read out as raw JSON.

10. Handling API Failure Gracefully

One of the most important lessons from the project was:

A tool failing should never cause the voice agent to silently break or hallucinate an answer.

External APIs can fail.

They can:

timeout
return errors
become unavailable
hit rate limits
return incomplete data

So I implemented explicit fallback handling.

If the weather API becomes unavailable, Pooja can say something like:

"I'm currently unable to reach the meteorological live feed, so I can't verify the latest information right now."

Instead of:

"There is no flood warning."

The difference is critical.

The agent must distinguish between:

"I don't know"

and:

"I verified that there isn't one."

Solving an LLM Tool Loop

One particularly interesting debugging problem happened when the weather tool was introduced.

Under certain ambiguous user inputs, the LLM could repeatedly attempt to call the same tool.

That created an LLM tool loop, eventually contributing to API quota exhaustion.

The root problem wasn't simply the API.

It was the interaction between:

tool descriptions
system instructions
ambiguous user input
LLM decision-making

I refined the tool descriptions and system prompt to make the tool's intended usage much more explicit.

The lesson:

A function tool's description is part of the agent's control logic.

If the model doesn't understand when a tool should be used, the function implementation itself may be perfectly correct and the system can still behave incorrectly.

*12. Taking Pooja Beyond the Browser
*
A major milestone came on Day 6.

I migrated the system from a local LiveKit server to LiveKit Cloud and connected it to real telephony infrastructure.

The pipeline became:

Pooja
↓
LiveKit Cloud
↓
SIP Outbound Trunk
↓
Twilio
↓
Real Phone

I built an outbound call orchestration flow using the LiveKit Server API.

The system could:

Create a unique LiveKit room
Dispatch the agent
Configure the outbound SIP participant
Initiate the call
Connect Pooja to a real phone number

This was the moment when the project stopped being only a browser demo.

Pooja could now actually make a phone call.

13. Context-Aware Outbound Calls

Outbound calls require a slightly different conversational strategy.

A user calling Pooja might start with:

"Hello, what can you help me with?"

But when Pooja initiates the call, she needs to proactively explain why she's calling.

I therefore introduced outbound/inbound context detection and separate greeting behavior.

This allowed the same agent architecture to support both:

Inbound:
Caller → Pooja

and:

Outbound:
Pooja → Caller

I also added handling for explicit "stop" requests so the agent would respect the caller's decision to end the interaction.

14. Knowing When AI Should Stop

This became one of the most important features of Pooja.

An AI disaster-response system should not try to solve every emergency itself.

For example:

"I'm trapped inside my house and the water is rising."

is fundamentally different from:

"What's the weather forecast?"

So I built a human escalation mechanism.

The architecture became:

Caller
↓
Pooja
↓
Emergency detected
↓
Ask permission
↓
Create escalation ticket
↓
Generate reference ID
↓
Notify human operator

15. Human-in-the-Loop Escalation

I created an escalation tool that generates a persistent ticket in SQLite.

The ticket captures useful operational information such as:

who needs help
what happened
urgency
relevant location information
what Pooja already checked
caller language/follow-up preference The caller receives a reference ID.

This means that when a human responder takes over, the caller doesn't necessarily have to repeat their entire situation.

16. Slack Integration for Human Operators

I also connected the escalation system to Slack using a webhook.

When an escalation is created:

Pooja
↓
SQLite Ticket
↓
Slack Webhook
↓
Human Operator

The operator receives a structured emergency alert containing the relevant information and reference ID.

This created a bridge between the AI system and an actual human workflow.

The objective was never:

"AI replaces the emergency operator."

It was:

"AI handles the first interaction and knows when to bring a human into the loop."

17. Measuring Whether Calls Actually Succeed

After building all these capabilities, another question became important:

How do I know whether Pooja is actually doing its job?

That's what led to the Day 8 analytics layer.

I introduced persistent call outcome tracking.

A call could be considered successful if, for example:

Pooja successfully delivered verified information
Pooja successfully created a human-help request

Otherwise, the call could be recorded as unsuccessful depending on whether the defined objective was achieved.

The backend now records metrics such as:

Total Calls
Successful Calls
Failed Calls

These values are derived from real interactions rather than hardcoded dashboard numbers.

18. Building the Analytics Dashboard

I built a lightweight Python analytics server exposing:

/api/stats

This acts as the bridge between SQLite and the Next.js dashboard.

The frontend then retrieves the actual call statistics dynamically.

I also spent time making the dashboard feel like a proper product rather than a plain admin page.

The dashboard includes:

WebGL shader background
Ruby Red visual identity
Glassmorphism cards
GSAP skeleton loading
Staggered entrance animations
Responsive layout

I also encountered an interesting frontend bug where GSAP cleanup was inadvertently removing React's inline styling.

Fixing that required separating the animation properties from the visual styling properties rather than allowing the animation lifecycle to manipulate both.

19. Introducing a Specialist Agent

By Day 9, Pooja had accumulated quite a lot of responsibilities.

Instead of making the main agent even larger, I introduced a specialist agent.

For the Disaster Response track, I created a:

Shelter Information Specialist

Its job is intentionally narrow:

Find and communicate verified shelter information relevant to the caller.

It can handle questions such as:

"Where is the nearest shelter?"

"Is there a shelter near me?"

"Does the shelter have capacity?"

But it doesn't need to understand everything Pooja understands.

20. Multi-Agent Handoff

The architecture now looks like:

                     POOJA
                Main Agent
                     │
                     │
          Shelter-related request
                     │
                     ▼
           ┌─────────────────┐
           │ Shelter         │
           │ Specialist      │
           └────────┬────────┘
                    │
                    ▼
             Shelter Lookup

Pooja uses a dedicated handoff tool when the user's request requires specialist knowledge.

The transition is explicit:

"I'll connect you to our shelter information specialist."

The specialist then introduces itself and continues the conversation.

Most importantly, the conversation context is preserved.

The caller doesn't have to explain everything again.

21. The Final Architecture

After the challenge, Pooja's architecture looks approximately like this:

                     ┌─────────────────────┐
                     │       Caller        │
                     └──────────┬──────────┘
                                │
                                ▼
                     ┌─────────────────────┐
                     │      LiveKit        │
                     │  Real-Time Transport│
                     └──────────┬──────────┘
                                │
                          Speech Input
                                │
                                ▼
                     ┌─────────────────────┐
                     │      Deepgram       │
                     │        STT          │
                     └──────────┬──────────┘
                                │
                                ▼
                     ┌─────────────────────┐
                     │       Gemini        │
                     │        LLM          │
                     └──────────┬──────────┘
                                │
           ┌────────────────────┼────────────────────┐
           │                    │                    │
           ▼                    ▼                    ▼
       Memory Tool         Weather Tool        Escalation
           │                    │                    │
           ▼                    ▼                    ▼
        SQLite             Open-Meteo            SQLite
                                                    │
                                                    ▼
                                                  Slack
                                │
                                ▼
                     ┌─────────────────────┐
                     │ Shelter Specialist  │
                     └─────────────────────┘
                                │
                                ▼
                          Shelter Data

                                │
                                ▼
                     ┌─────────────────────┐
                     │     Murf Falcon     │
                     │        TTS          │
                     └──────────┬──────────┘
                                │
                                ▼
                              Caller

And for real phone calls:

Pooja
↓
LiveKit Cloud
↓
SIP
↓
Twilio
↓
Real Phone

22. The Hardest Problems I Encountered

Building this wasn't a straight line from idea to working application.

Some of the most valuable lessons came from things that didn't work immediately.

22.1 Localhost vs Real Telephony

Initially, the agent worked in a local development environment.

Moving to real phone calls introduced an entirely different set of considerations:

LiveKit Cloud
SIP
Twilio
outbound trunks
authentication
room orchestration
latency

The lesson was that a voice agent is not just an LLM + TTS.

The transport layer matters just as much.

22.2 Tool Loops and LLM Behavior

The weather tool taught me that a correctly implemented function can still cause problems if the LLM doesn't understand when it should call it.

Repeated tool calls resulted in quota exhaustion.

Improving the tool description and system-level instructions helped constrain the behavior.

The lesson:

Prompt engineering and tool engineering are interconnected.

22.3 External API Failures

Real APIs fail.

Instead of allowing failures to propagate into the conversation, I built explicit fallback paths.

That made Pooja behave more like a reliable system and less like a demo that assumes everything is always available.

22.4 Context Preservation During Agent Handoffs

Introducing a specialist created another challenge.

A new agent normally needs its own context.

But the caller shouldn't have to repeat:

"I'm in X district and I need a shelter because..."

every time control changes.

Passing the existing conversation context during the handoff allowed the specialist to continue naturally.

22.5 Frontend Animation and React State

The analytics dashboard also produced a subtle issue.

GSAP cleanup was interacting with React's inline styles.

The animations worked, but the cleanup phase could remove styling that the React component still depended on.

The solution was to separate:

Animation state

from:

Component visual state

This was a good reminder that polished UI work requires understanding the lifecycle of both libraries involved.

23. How to Build Your Own Voice Agent

If you want to build a similar system, the basic architecture is surprisingly approachable.

A real-time voice agent generally needs four major pieces:

Speech-to-Text
↓
LLM
↓
Text-to-Speech
↓
Real-Time Transport

For my project:

Deepgram → Gemini → Murf Falcon
│
▼
LiveKit

LiveKit handles the real-time communication layer while the AI services handle speech recognition, reasoning, and speech generation.

Once this basic pipeline works, you can progressively add:

Memory
Tools
APIs
Database
Telephony
Human Escalation
Analytics
Specialist Agents

The important thing is not to build everything at once.

Build the smallest working voice loop first.

Then add capabilities one at a time.

24. Basic Project Setup

Clone the project:

git clone https://github.com/pratyush06-aec/murf-livekit-starter.git

Create a Python virtual environment:

python -m venv .venv

Activate it on Windows:

.venv\Scripts\Activate.ps1

Install dependencies:

pip install -r requirements.txt

Then configure your environment variables.

For example:

`LIVEKIT_URL=...
LIVEKIT_API_KEY=...
LIVEKIT_API_SECRET=...

DEEPGRAM_API_KEY=...
GOOGLE_API_KEY=...
MURF_API_KEY=...

TWILIO_ACCOUNT_SID=...
TWILIO_AUTH_TOKEN=...`

Never commit these values to GitHub.

Add your environment file to .gitignore:

.env .env.* !.env.example

You can safely commit an example configuration:

LIVEKIT_URL= LIVEKIT_API_KEY= LIVEKIT_API_SECRET= DEEPGRAM_API_KEY= GOOGLE_API_KEY= MURF_API_KEY=

25. Keeping Secrets Out of the Repository

This became especially important once the project involved:

LiveKit credentials
Gemini credentials
Deepgram credentials
Murf credentials
Twilio credentials
Slack webhooks

Never place these directly inside:

API_KEY = "your-secret-key"

Instead:

import os

API_KEY = os.getenv("MURF_API_KEY")

Keep the real values in your local environment or secure deployment environment.

If a credential is accidentally committed, rotating/revoking it is necessary. Simply deleting it from the latest commit is not sufficient because Git history may still contain it.

26. Running and Testing the Agent

Once the environment variables are configured, start the LiveKit agent according to the project's development command.

Then connect through the frontend or configured LiveKit client.

A useful first test is simply:

"Hello Pooja, can you hear me?"

Then progressively test:

Test 1 — General conversation

"What can you help me with?"

Test 2 — Weather tool

"What's the current weather situation in my district?"

Test 3 — Memory

"My name is Ramesh and I'm in Asansol."

Then reconnect and verify that the information can be retrieved appropriately.

Test 4 — Escalation

"I'm trapped and need urgent help."

Verify that Pooja asks for permission before creating the escalation.

Test 5 — Shelter specialist

"Where is the nearest emergency shelter?"

Verify the handoff.

Test 6 — Outbound call

Trigger the outbound flow and verify the call reaches the configured phone number.

Test 7 — Analytics

Complete a call and verify that the dashboard metrics change based on the actual interaction.

27. What I Learned During the Challenge

The biggest lesson from these 10 days wasn't a particular API or framework.

It was understanding that a useful voice agent is a system, not just a prompt.

A production-oriented architecture needs to think about:

Conversation
+
Latency
+
Memory
+
Tools
+
Data
+
Failures
+
Security
+
Human escalation
+
Observability
+
Specialization

Each layer solves a different problem.

And each layer introduces new failure modes.

28. From "Chatbot" to "Voice System"

At the beginning of the challenge, I was mainly thinking about:

How do I make an AI talk?

By the end, the question had completely changed.

I was thinking about:

How do I build a voice system that can understand, remember, act, fail safely, escalate, communicate with humans, make phone calls, and measure its own outcomes?

That's a much more interesting engineering problem.

*29. What's Next for Pooja?
*
The 10-day challenge gave me the foundation, but there is a lot more I want to explore.

Some potential next steps include:

Integrating official government disaster datasets
More accurate geolocation and nearest-shelter calculations
Multi-region shelter capacity tracking
SMS/WhatsApp-based emergency follow-ups
More Indian language support
Improved emergency classification
Stronger authentication for operator dashboards
Better observability and tracing
More specialist agents
More robust evaluation of voice-agent quality
Production-grade deployment and monitoring

The ultimate goal would be to move Pooja from a challenge project toward a reliable voice-first disaster-response platform.

30. Final Thoughts

Building Pooja over these 10 days completely changed the way I think about voice AI.

What started as:

Speech → AI → Speech

eventually became:

Speech
↓
Real-Time Transport
↓
Speech Recognition
↓
LLM Reasoning
↓
Tools + Memory + Live Data
↓
Human Escalation / Specialist Handoff
↓
Analytics
↓
Natural Voice Response

And that's what made this challenge so valuable.

I didn't just learn how to connect an LLM to a TTS API.

I learned how to engineer around an AI model.

Link of Day 1 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_voiceforbharat-murfai-voiceai-ugcPost-7491217998469595138-EDNF/
Link of Day 2 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_voiceforbharat-murfai-livekit-ugcPost-7491575388628967424-Cg3l/
Link of Day 3 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_voiceforbharat-murfai-livekit-ugcPost-7491983933178331136-L4qs/
Link of Day 4 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_voiceforbharat-murfai-gemini-ugcPost-7492308868140703744-t7wA/
Link of Day 5 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_voiceforbharat-murfai-livekit-ugcPost-7492616307574620160-OYKx/
Link of Day 6 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_10daysofvoiceagentschallenge-voiceforbharat-ugcPost-7493022074911879169-c6Df/
Link of Day 7 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_voiceforbharat-murfai-livekit-ugcPost-7493372541093322753-B_Lw/
Link of Day 8 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_voiceforbharat-murfai-livekit-ugcPost-7493765730602065920-_PuB/
Link of Day 9 Demo: https://www.linkedin.com/posts/pratyush-dutta-221b94302_10daysofvoiceagentschallenge-voiceforbharat-ugcPost-7494137786820730881-j4Wj/

This is a submission for the [GitHub Finish-Up-A-Thon Challenge]

Pratyush Dutta — Wed, 03 Jun 2026 18:39:20 +0000

What I Built:
The Context-Preserving Assistant is a modern, full-stack application designed to showcase how to maintain user context (like preferences, location, or customized settings) securely across a distributed system.

Often, personal assistants need to communicate with various third-party services (like weather, news, or calendar APIs) on behalf of the user. Doing this securely means the frontend should never hold sensitive third-party API keys. Instead, the frontend authenticates the user via Auth0 and receives a secure JWT (JSON Web Token). This token is then passed to our Express Backend, which acts as a secure proxy. The backend validates the user's identity, extracts any custom context (such as a preferred city injected into the token via Auth0 Actions), and securely communicates with third-party APIs using server-side secrets.

This architecture ensures that user context is preserved seamlessly while maintaining enterprise-grade security for API credentials.

This is a starter/mentorship scaffold — follow the steps below to configure Auth0 and run the demo.

It started in MLH GHW: GenAI conducted in the month of May.

Demo:

pratyush06-aec_Context-Preserving-Assistant_ Full-Stack Identity & API Access and 7 more pages - Personal - Microsoft Edge 2026-06-03 23-51-33.mp4 - Google Drive

drive.google.com

Github repo- https://github.com/pratyush06-aec/Context-Preserving-Assistant

The Comeback Story:
The project began as a basic starter scaffold with Auth0 authentication and a simple frontend-backend connection, but it lacked a complete workflow to demonstrate secure context preservation across services.

The main challenge was ensuring that user-specific context could travel securely from authentication to API consumption without exposing sensitive credentials. To solve this, we implemented Auth0 Universal Login, enriched JWTs with user preferences through Auth0 Actions, and built an Express backend that securely validates tokens and acts as a proxy for third-party services.

We then integrated the OpenWeather API, allowing the assistant to personalize responses using authenticated user context while keeping API keys protected on the server. Finally, we resolved deployment and configuration challenges across Auth0, Render, and Vercel, including audience validation, environment management, CORS, and production routing.

The end result is a production-style Context-Preserving Assistant that authenticates users once, securely preserves their preferences throughout the session, and performs personalized actions through third-party APIs while maintaining enterprise-grade security and identity propagation.

My Experience with GitHub Copilot:
This project demonstrates how GitHub Copilot accelerated the development of a secure, full-stack Context-Preserving Assistant powered by Auth0, React, Node.js, and OpenWeatherMap.

Throughout the development process, GitHub Copilot assisted in scaffolding the application architecture, generating frontend and backend starter code, creating protected API routes, and implementing Auth0 authentication flows. It helped establish a React frontend integrated with Auth0 Universal Login and an Express backend secured through JWT validation using Auth0-issued access tokens.

Copilot also streamlined the implementation of key features, including user profile retrieval, context preservation through custom Auth0 claims, and secure integration with the OpenWeatherMap API. By suggesting boilerplate code, middleware configurations, API handlers, and environment variable structures, it significantly reduced development time while maintaining best practices.

Beyond code generation, Copilot played an important role in debugging and refinement. It assisted in diagnosing backend startup issues, validating authentication flows, troubleshooting token verification, and ensuring seamless communication between the frontend, backend, Auth0, and third-party services.

Security improvements were another major contribution. Copilot helped identify an exposed API key, guided secret remediation, supported Git history cleanup, and reinforced secure credential management by ensuring that third-party API keys remained protected on the server side.

The final outcome is a production-style demonstration of secure identity propagation and context preservation. Users authenticate through Auth0, access protected backend resources, retrieve personalized context, and perform weather lookups through a third-party API—all while maintaining secure authentication and authorization practices. The project showcases how GitHub Copilot can act as an effective AI development partner, accelerating implementation, debugging, security hardening, and documentation throughout the software development lifecycle.

DEV Community: Pratyush Dutta

From a Voice Agent to a Disaster Response System: Building Pooja with LiveKit and Murf Falcon

This is a submission for the [GitHub Finish-Up-A-Thon Challenge]

pratyush06-aec_Context-Preserving-Assistant_ Full-Stack Identity & API Access and 7 more pages - Personal - Microsoft​ Edge 2026-06-03 23-51-33.mp4 - Google Drive

pratyush06-aec_Context-Preserving-Assistant_ Full-Stack Identity & API Access and 7 more pages - Personal - Microsoft Edge 2026-06-03 23-51-33.mp4 - Google Drive