๐ ๐ซ๐จ๐ฆ ๐ ๐๐๐ฌ๐ข๐ ๐ฏ๐จ๐ข๐๐ ๐๐ฌ๐ฌ๐ข๐ฌ๐ญ๐๐ง๐ญ ๐ญ๐จ ๐ก๐ฎ๐ฆ๐๐ง ๐๐ฌ๐๐๐ฅ๐๐ญ๐ข๐จ๐ง, ๐๐๐ฅ๐ฅ ๐๐ง๐๐ฅ๐ฒ๐ญ๐ข๐๐ฌ, ๐ฆ๐ฎ๐ฅ๐ญ๐ข๐ฅ๐ข๐ง๐ ๐ฎ๐๐ฅ ๐๐จ๐ง๐ฏ๐๐ซ๐ฌ๐๐ญ๐ข๐จ๐ง๐ฌ, ๐๐ง๐ ๐ฌ๐ฉ๐๐๐ข๐๐ฅ๐ข๐ฌ๐ญ ๐ก๐๐ง๐๐จ๐๐๐ฌ.
๐. ๐๐ก๐ ๐๐ซ๐จ๐๐ฅ๐๐ฆ ๐๐ง๐ ๐ญ๐ก๐ ๐๐ฌ๐๐ซ๐ฌ
Healthcare can be difficult to navigate, especially when users are not sure whether they need general health guidance, a clinic appointment, or support from a real person.
For this project, I chose the Health Access track and built "๐๐๐ซ๐จ๐ ๐ฒ๐ ๐๐๐ก๐๐ฒ๐๐ค", a voice-based health and wellness assistant designed to make basic health conversations easier and more accessible.
The idea was not to build an AI doctor or replace medical professionals. Instead, "๐๐๐ซ๐จ๐ ๐ฒ๐ ๐๐๐ก๐๐ฒ๐๐ค" is designed to help users with general health and wellness questions, guide them toward appropriate next steps, and connect them with human support when a situation requires it.
I chose voice because users can speak naturally instead of typing everything into a chat box. This makes the interaction feel more like a real conversation and can be especially useful for people who prefer speaking over typing.
The project also focuses on safety. When a conversation involves diagnosis requests, emergency or red-flag situations, or needs human intervention, the agent follows dedicated safety and escalation flows instead of trying to handle everything itself.
The assistant also supports scheduled reminder calls, allowing users to request reminders naturally and receive a follow-up call at the requested time. This extends the experience beyond a single conversation while keeping the existing safety and consent rules in place.
๐. ๐๐ก๐๐ญ ๐ญ๐ก๐ ๐๐จ๐ข๐๐ ๐๐ ๐๐ง๐ญ ๐๐จ๐๐ฌ
๐๐๐ซ๐จ๐ ๐ฒ๐ ๐๐๐ก๐๐ฒ๐๐ค is built as a conversational voice assistant rather than a simple question-and-answer bot.
A user can start a natural conversation about general health and wellness. The main agent can provide basic guidance while following clear safety boundaries. It does not try to diagnose medical conditions or act as a replacement for a healthcare professional.
- ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐ฆ๐ฒ๐น๐ฒ๐ฐ๐๐ถ๐ผ๐ป
One of the important improvements I added was a language-selection flow.
Before starting a conversation, the user can choose between "๐๐ป๐ด๐น๐ถ๐๐ต" ๐ฎ๐ป๐ฑ "๐๐ถ๐ป๐ฑ๐ถ". Once selected, the preferred language is carried through the agent session so the conversation stays consistent instead of randomly switching languages.
For Hindi conversations, the agent is instructed to respond using "๐๐ฒ๐๐ฎ๐ป๐ฎ๐ด๐ฎ๐ฟ๐ถ ๐๐ฐ๐ฟ๐ถ๐ฝ๐" rather than Romanized Hindi. This makes the conversation more natural for users who prefer reading and speaking Hindi.
- ๐๐๐บ๐ฎ๐ป ๐ฆ๐๐ฝ๐ฝ๐ผ๐ฟ๐
When a situation needs human assistance, ๐๐๐ซ๐จ๐ ๐ฒ๐ ๐๐๐ก๐๐ฒ๐๐ค can create a human support request and provide a unique reference ID.
The user can later use that reference ID on the Human Support page to check the status of the request.
The public support page is intentionally privacy-focused. It displays only safe request metadata such as the reference ID, status, timestamps, urgency, and follow-up information. Conversation content and sensitive health details are not exposed on the public page.
- ๐ฆ๐ฝ๐ฒ๐ฐ๐ถ๐ฎ๐น๐ถ๐๐ ๐๐ด๐ฒ๐ป๐ ๐๐ฎ๐ป๐ฑ๐ผ๐ณ๐ณ
Another important feature is the "๐๐น๐ถ๐ป๐ถ๐ฐ & ๐๐ฝ๐ฝ๐ผ๐ถ๐ป๐๐บ๐ฒ๐ป๐ ๐ฆ๐ฝ๐ฒ๐ฐ๐ถ๐ฎ๐น๐ถ๐๐".
When a user clearly wants to book or prepare for a clinic appointment, the main agent can hand the conversation to this dedicated specialist instead of trying to handle every task itself.
The specialist can help with appointment-related questions, preparation, preferred appointment timing, and general visit guidance.
The user does not have to repeat the entire problem because relevant context is transferred during the handoff.
- ๐ฆ๐ฝ๐ฒ๐ฐ๐ถ๐ฎ๐น๐ถ๐๐ ๐๐ฎ๐ป๐ฑ๐ฏ๐ฎ๐ฐ๐ธ
The specialist can also hand the conversation back to the main health assistant when the appointment-related task is complete or when the user wants to discuss a general health topic.
This creates a simple multi-agent flow where each agent has a focused responsibility.
- ๐๐ฎ๐น๐น ๐๐ป๐ฎ๐น๐๐๐ถ๐ฐ๐
I also built a Call Analytics Dashboard to make the voice agent measurable instead of relying only on how the conversation feels.
The dashboard tracks real call outcomes such as:
- Total calls
- Successful calls
- Failed calls
- Call outcome trends
The success and failure logic is connected to actual call outcomes rather than hardcoded numbers.
This helped me understand whether the agent was actually completing useful conversations.
- ๐ฆ๐ฐ๐ต๐ฒ๐ฑ๐๐น๐ฒ๐ฑ ๐ฅ๐ฒ๐บ๐ถ๐ป๐ฑ๐ฒ๐ฟ ๐๐ฎ๐น๐น๐
A new feature I added is scheduled reminder calling.
Users can ask the agent to schedule a reminder naturally, such as "Remind me in 5 minutes to drink water." The agent understands relative or specific times, validates the requested timezone when needed, and creates a reminder with a unique reference ID.
The reminder is processed by the existing outbound calling system instead of using a separate dialing implementation. An exactly-once mechanism prevents the same reminder from being dialed multiple times.
Users can also open the Scheduled Reminders page to check the reminder status and cancel a reminder while it is still pending.
- ๐ช๐ฒ๐น๐น๐ป๐ฒ๐๐ ๐ง๐ถ๐ฝ๐
I also added a dedicated Wellness Tips section for users who want simple general wellness guidance outside the live conversation.
It includes five areas:
- Hydration
- Sleep & Rest
- Healthy Eating
- Daily Activity
- Stress & Relaxation
The section is designed for general wellness information and includes a clear disclaimer that it is not medical advice.
๐.๐๐จ๐ฐ ๐ญ๐ก๐ ๐๐ฒ๐ฌ๐ญ๐๐ฆ ๐๐จ๐ซ๐ค๐ฌ
๐๐๐ซ๐จ๐ ๐ฒ๐ ๐๐๐ก๐๐ฒ๐๐ค is built around a real-time voice conversation pipeline. The goal was to keep the interaction simple for the user while allowing different parts of the system to handle different responsibilities.
At a high level, the flow looks like this:
- ๐ญ.๐จ๐๐ฒ๐ฟ ๐ฎ๐ป๐ฑ ๐๐ฟ๐ผ๐ป๐๐ฒ๐ป๐ฑ The user starts the conversation from the web interface and can choose their preferred language before the voice session begins.
The frontend handles the conversation interface, voice controls, language selection, agent status, and other user-facing states.
- ๐ฎ.๐ฅ๐ฒ๐ฎ๐น-๐ง๐ถ๐บ๐ฒ ๐ฉ๐ผ๐ถ๐ฐ๐ฒ ๐ง๐ฟ๐ฎ๐ป๐๐ฝ๐ผ๐ฟ๐ The voice conversation runs through "๐๐ถ๐๐ฒ๐๐ถ๐", which provides the real-time communication layer between the user and the agent.
This allows the agent to receive the user's audio, process it, and return spoken responses during the same conversation.
- ๐ฏ. ๐ฆ๐ฝ๐ฒ๐ฒ๐ฐ๐ต-๐๐ผ-๐ง๐ฒ๐ ๐ When the user speaks, their audio is converted into text so the language model can understand the request.
This text becomes the input for the agent's reasoning and routing logic.
- ๐ฐ.๐ ๐ฎ๐ถ๐ป ๐๐ด๐ฒ๐ป๐ The main ๐๐๐ซ๐จ๐ ๐ฒ๐ ๐๐๐ก๐๐ฒ๐๐ค agent is responsible for general health and wellness conversations.
It follows a system prompt containing its role, objectives, safety rules, language instructions, and routing decisions.
The main agent does not try to handle every request itself. Depending on the conversation, it can use tools or hand the conversation to another part of the system.
- ๐ฑ. ๐ง๐ผ๐ผ๐น๐ ๐ฎ๐ป๐ฑ ๐ฅ๐ผ๐๐๐ถ๐ป๐ด The agent can use dedicated tools when a task requires an action rather than a simple response.
For example, a clear clinic or appointment request can trigger a handoff to the "Clinic & Appointment Specialist".
Emergency or red-flag situations follow the existing escalation flow instead of being sent to the appointment specialist.
This separation keeps the responsibilities of each agent clear.
- ๐ฒ.๐ฆ๐ฐ๐ต๐ฒ๐ฑ๐๐น๐ฒ๐ฑ ๐ฅ๐ฒ๐บ๐ถ๐ป๐ฑ๐ฒ๐ฟ ๐๐น๐ผ๐
The agent can also create scheduled reminders from natural-language requests.
When a user asks for a reminder, the request is parsed and stored with a unique reference ID. The scheduler checks for due reminders and triggers the existing outbound calling system at the scheduled time.
The reminder system uses an exactly-once claiming mechanism to prevent duplicate calls. Users can also check the reminder status or cancel a reminder while it is still pending.
This flow reuses the existing outbound dialer instead of introducing a separate calling system.
- ๐ณ.๐ฆ๐ฝ๐ฒ๐ฐ๐ถ๐ฎ๐น๐ถ๐๐ ๐๐ด๐ฒ๐ป๐ The Clinic & Appointment Specialist is a separate agent with its own instructions and responsibilities.
During a handoff, only relevant context is transferred instead of exposing the entire conversation unnecessarily.
The specialist can continue the conversation without asking the user to repeat the original request.
When its task is complete or the user changes back to a general health topic, the specialist can hand the conversation back to the main agent.
- ๐ด.๐ง๐ฒ๐ ๐-๐๐ผ-๐ฆ๐ฝ๐ฒ๐ฒ๐ฐ๐ต After the agent generates a response, it is converted back into speech so the user can hear the answer naturally.
For this project, I used "๐ ๐๐ฟ๐ณ ๐๐ฎ๐น๐ฐ๐ผ๐ป" for the voice experience.
One of the things I wanted to achieve was a conversational experience that feels closer to talking to an assistant rather than interacting with a traditional text chatbot.
Architecture Overview
The overall architecture can be represented as:
๐. ๐๐ก๐ ๐๐จ๐ฌ๐ญ ๐๐ฆ๐ฉ๐จ๐ซ๐ญ๐๐ง๐ญ ๐ ๐๐๐ญ๐ฎ๐ซ๐๐ฌ
Over the course of the challenge, I gradually added features that made "Aarogya Sahayak" more useful, safer, and easier to evaluate.
- ๐ฆ๐ฎ๐ณ๐ฒ๐๐ ๐๐๐ฎ๐ฟ๐ฑ๐ฟ๐ฎ๐ถ๐น๐ The agent has clear boundaries around what it can and cannot do.
It is designed for general health and wellness guidance, not medical diagnosis. Emergency or red-flag situations follow a separate escalation flow instead of being routed to the appointment specialist.
The agent also avoids asking users for sensitive information such as passwords, OTPs, PINs, or card details.
๐ญ.๐๐๐บ๐ฎ๐ป ๐๐๐ฐ๐ฎ๐น๐ฎ๐๐ถ๐ผ๐ป
When a conversation needs human intervention, the agent can create a support request and generate a unique reference ID.
The user can use this ID to check the request status later through the Human Support page.
The public status page is intentionally privacy-focused and does not expose the user's conversation or sensitive health information.
๐ฎ.๐๐น๐ถ๐ป๐ถ๐ฐ ๐ฎ๐ป๐ฑ ๐๐ฝ๐ฝ๐ผ๐ถ๐ป๐๐บ๐ฒ๐ป๐ ๐ฆ๐ฝ๐ฒ๐ฐ๐ถ๐ฎ๐น๐ถ๐๐
Instead of making the main agent responsible for everything, I created a dedicated "Clinic & Appointment Specialist".
The specialist focuses on appointment-related tasks such as:
- Understanding appointment requests
- Helping with clinic visit preparation
- Discussing preferred appointment timing
- Providing general appointment guidance
The main agent announces the handoff before switching the conversation, and the specialist introduces itself after taking over.
๐ฏ.๐ฆ๐ฝ๐ฒ๐ฐ๐ถ๐ฎ๐น๐ถ๐๐ ๐๐ฎ๐ป๐ฑ๐ฏ๐ฎ๐ฐ๐ธ
The specialist can also return the conversation to the main health assistant.
For example, if the user finishes discussing an appointment and asks for general wellness advice, the specialist can hand the conversation back instead of continuing outside its scope.
Relevant context is preserved during both handoff and handback so the user does not have to repeat everything.
๐ฐ.๐ ๐๐น๐๐ถ๐น๐ถ๐ป๐ด๐๐ฎ๐น ๐๐ผ๐ป๐๐ฒ๐ฟ๐๐ฎ๐๐ถ๐ผ๐ป๐
The agent supports both English and Hindi through a language-selection flow.
The user chooses the preferred language before starting the conversation. The selected language is then carried through the agent session.
For Hindi conversations, the agent is instructed to use "๐๐ฒ๐๐ฎ๐ป๐ฎ๐ด๐ฎ๐ฟ๐ถ ๐๐ฐ๐ฟ๐ถ๐ฝ๐" rather than randomly switching to Romanized Hindi.
๐ฑ.๐๐ฎ๐น๐น ๐๐ป๐ฎ๐น๐๐๐ถ๐ฐ๐
I built a Call Analytics Dashboard that records real call outcomes.
It tracks metrics such as:
- Total calls
- Successful calls
- Failed calls
- Call outcome trends
The success condition is based on actual conversation outcomes, such as successful health guidance or successful human escalation, rather than simply counting every completed call as successful.
This made the system measurable and helped me evaluate whether the agent was actually completing useful conversations.
๐ฒ.๐ฆ๐ฐ๐ต๐ฒ๐ฑ๐๐น๐ฒ๐ฑ ๐ฅ๐ฒ๐บ๐ถ๐ป๐ฑ๐ฒ๐ฟ ๐๐ฎ๐น๐น๐
One of the major features I added was scheduled reminder calling.
Users can ask "Aarogya Sahayak" to schedule a reminder using natural language, for example:
"Remind me in 5 minutes to drink water."
The agent understands the requested time, handles timezone and AM/PM clarification when required, and creates a reminder with a unique reference ID.
The reminder is then stored and monitored by a background scheduler. When the scheduled time arrives, the system reuses the existing outbound calling infrastructure to place the reminder call.
A key reliability feature is the exactly-once mechanism, which prevents the same reminder from being triggered more than once.
Users can also check the status of a reminder using its reference ID and cancel a reminder while it is still pending.
This turned reminders from a simple notification concept into an actual scheduled voice-call workflow connected to the existing agent infrastructure.
๐ณ.๐ช๐ฒ๐น๐น๐ป๐ฒ๐๐ ๐ง๐ถ๐ฝ๐
I also added a dedicated Wellness Tips section for users who want simple health and wellness information without starting a full voice conversation.
The section provides practical, general wellness tips across five areas:
- Hydration
- Sleep & Rest
- Healthy Eating
- Daily Activity
- Stress & Relaxation
Each category contains short and easy-to-follow suggestions so users can quickly find relevant information.
The feature is intentionally focused on general wellness rather than diagnosis or treatment. A clear disclaimer reminds users that the information is not medical advice.
This gives users another way to interact with Aarogya Sahayak when they simply want quick wellness guidance.
๐. ๐๐ก๐๐ฅ๐ฅ๐๐ง๐ ๐๐ฌ ๐๐ง๐ ๐๐จ๐ฐ ๐ ๐๐ฏ๐๐ซ๐๐๐ฆ๐ ๐๐ก๐๐ฆ
Building a voice agent over multiple days was not always straightforward. A few problems taught me more than the features themselves.
- ๐๐ฒ๐ฒ๐ฝ๐ถ๐ป๐ด ๐๐ด๐ฒ๐ป๐ ๐ฅ๐ฒ๐๐ฝ๐ผ๐ป๐๐ถ๐ฏ๐ถ๐น๐ถ๐๐ถ๐ฒ๐ ๐ฆ๐ฒ๐ฝ๐ฎ๐ฟ๐ฎ๐๐ฒ Initially, it would have been easy to let one agent handle everything.
Instead, I separated responsibilities between the main health assistant and the Clinic & Appointment Specialist.
The main agent handles general health conversations and safety routing, while the specialist focuses only on clinic and appointment-related tasks.
This made the routing logic clearer and reduced the chance of the specialist answering questions outside its intended role.
- ๐๐ผ๐ป๐๐ฒ๐ ๐ ๐๐๐ฟ๐ถ๐ป๐ด ๐๐ฎ๐ป๐ฑ๐ผ๐ณ๐ณ๐ Another challenge was making sure the specialist understood what the user had already asked.
I did not want the user to repeat the entire conversation after every handoff.
The solution was to transfer only the relevant context, including a short request summary and the user's latest request, instead of transferring the entire conversation.
The same principle is used when handing the conversation back to the main agent.
- ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐๐ผ๐ป๐๐ถ๐๐๐ฒ๐ป๐ฐ๐ Another issue appeared when the agent sometimes mixed English and Hindi or used Romanized Hindi.
I added an explicit language-selection flow and made the selected language part of the agent session context.
For Hindi, the instructions explicitly require "Devanagari script" and prevent the agent from switching back to Romanized Hindi.
- ๐ฆ๐ฐ๐ต๐ฒ๐ฑ๐๐น๐ฒ๐ฑ ๐ฅ๐ฒ๐บ๐ถ๐ป๐ฑ๐ฒ๐ฟ ๐๐ฎ๐น๐น๐ Adding scheduled reminder calls introduced another engineering challenge.
The system needed to understand natural-language time requests such as "in 5 minutes" while handling timezone and AM/PM ambiguity safely.
Another important requirement was preventing the same reminder from triggering more than once. Since the scheduler checks for due reminders repeatedly, simply checking the reminder status was not enough.
I implemented an atomic claiming mechanism so that each reminder is claimed exactly once before the outbound call is triggered. Reminders then move through states such as pending, triggered, completed, failed, or cancelled.
I also reused the existing outbound calling system instead of creating a separate dialing implementation. This kept the reminder feature integrated with the project's existing calling infrastructure.
- ๐ง๐ฒ๐๐๐ถ๐ป๐ด ๐๐ถ๐๐ต ๐๐ถ๐บ๐ถ๐๐ฒ๐ฑ ๐๐๐ ๐ค๐๐ผ๐๐ฎ Some behavioral tests depend on LiveKit hosted inference and could not always be executed because the available free inference quota was exhausted.
Instead of treating those failures as application failures, I separated deterministic tests from LLM-dependent tests.
The deterministic test suite, linting, formatting, type checking, and frontend build could still be verified locally.
This was an important lesson: an external service failure and an application bug are not necessarily the same thing.
๐. ๐๐จ๐ฐ ๐๐๐๐๐๐ซ๐ฌ ๐๐๐ง ๐๐ฎ๐ข๐ฅ๐ ๐๐ง๐ ๐๐ฎ๐ง ๐๐ญ:
If you want to build a similar voice agent, the architecture can be broken into four main parts:
๐ญ. ๐ฆ๐ฝ๐ฒ๐ฒ๐ฐ๐ต-๐๐ผ-๐ง๐ฒ๐ ๐ (๐ฆ๐ง๐ง) ๐๐ผ ๐๐ป๐ฑ๐ฒ๐ฟ๐๐๐ฎ๐ป๐ฑ ๐๐ต๐ฎ๐ ๐๐ต๐ฒ ๐๐๐ฒ๐ฟ ๐๐ฎ๐๐
๐ฎ. ๐๐๐ ๐๐ผ ๐๐ป๐ฑ๐ฒ๐ฟ๐๐๐ฎ๐ป๐ฑ ๐๐ต๐ฒ ๐ฟ๐ฒ๐พ๐๐ฒ๐๐ ๐ฎ๐ป๐ฑ ๐ฑ๐ฒ๐ฐ๐ถ๐ฑ๐ฒ ๐๐ต๐ฎ๐ ๐๐ผ ๐ฑ๐ผ
๐ฏ. ๐ง๐ฒ๐ ๐-๐๐ผ-๐ฆ๐ฝ๐ฒ๐ฒ๐ฐ๐ต (๐ง๐ง๐ฆ) ๐๐ผ ๐๐ฝ๐ฒ๐ฎ๐ธ ๐๐ต๐ฒ ๐ฟ๐ฒ๐๐ฝ๐ผ๐ป๐๐ฒ
๐ฐ. ๐ฅ๐ฒ๐ฎ๐น-๐๐ถ๐บ๐ฒ ๐๐ฟ๐ฎ๐ป๐๐ฝ๐ผ๐ฟ๐ ๐๐ผ ๐ฐ๐ผ๐ป๐ป๐ฒ๐ฐ๐ ๐๐ต๐ฒ ๐๐๐ฒ๐ฟ ๐ฎ๐ป๐ฑ ๐ฎ๐ด๐ฒ๐ป๐ ๐ฑ๐๐ฟ๐ถ๐ป๐ด ๐ฎ ๐น๐ถ๐๐ฒ ๐ฐ๐ผ๐ป๐๐ฒ๐ฟ๐๐ฎ๐๐ถ๐ผ๐ป
For Aarogya Sahayak, I used LiveKit for the real-time agent session and Murf Falcon for the voice experience.
- ๐๐ฅ๐จ๐ง๐ ๐ญ๐ก๐ ๐๐ซ๐จ๐ฃ๐๐๐ญ Start by cloning the repository:
Bash:--
git clone https://github.com/codewithvishuuu/murf-livekit-starter.git
cd murf-livekit-starter
- ๐๐ผ๐ป๐ณ๐ถ๐ด๐๐ฟ๐ฒ ๐๐ฃ๐ ๐๐ฒ๐๐ API keys should never be hardcoded into the source code or committed to GitHub.
Store them in the project's local environment file, such as .env.local, according to the variables required by the project.
- ๐๐ผ๐ฟ ๐ฒ๐ ๐ฎ๐บ๐ฝ๐น๐ฒ: GOOGLE_API_KEY=your_key_here
Use your actual keys only in your local environment.
Make sure .env.local is included in .gitignore and never publish the file containing real credentials.
๐ฆ๐๐ฎ๐ฟ๐ ๐๐ต๐ฒ ๐๐ผ๐ฐ๐ฎ๐น ๐๐ถ๐๐ฒ๐๐ถ๐ ๐ฆ๐ฒ๐ฟ๐๐ฒ๐ฟ
If you are running LiveKit locally, start the local LiveKit server first. The backend agent connects to this server for the real-time voice session.๐๐ผ๐ฟ ๐ฒ๐ ๐ฎ๐บ๐ฝ๐น๐ฒ:
cd backend
uv run python src/agent.py dev
Then start the frontend...
๐ฆ๐๐ฎ๐ฟ๐ ๐๐ต๐ฒ ๐๐ฎ๐ฐ๐ธ๐ฒ๐ป๐ฑ ๐๐ด๐ฒ๐ป๐
cd backend
uv run python src/agent.py dev๐ฆ๐๐ฎ๐ฟ๐ ๐๐ต๐ฒ ๐๐ฟ๐ผ๐ป๐๐ฒ๐ป๐ฑ
Then start the frontend using the project's frontend development command.
Open the local application in your browser and start a conversation.
๐๐๐ฌ๐ญ๐ข๐ง๐ ๐ญ๐ก๐ ๐๐จ๐ง๐ฏ๐๐ซ๐ฌ๐๐ญ๐ข๐จ๐ง:
A simple test flow is:
Start the application.
Select English or Hindi.
Ask a normal health or wellness question.
Ask for a clinic appointment.
Verify that the main agent announces the specialist handoff.
Continue the appointment conversation with the specialist.
Ask for a general wellness topic and verify the specialist can hand the conversation back.
Test the Human Support flow and verify that a reference ID is generated.
Check the Call Analytics Dashboard after completing calls.
Schedule a reminder using a natural-language request, such as "Remind me in 5 minutes to drink water."
Verify that a unique reminder reference ID is generated.
Open the Scheduled Reminders page and check the reminder status.
Create a pending reminder and verify that it can be cancelled.
Leave a reminder active and verify that the scheduled outbound call is triggered at the requested time.
Verify that the same reminder is not triggered more than once.
This type of testing helped me verify the actual user journey across voice conversations, specialist handoffs, human support, scheduled reminders, and call analytics instead of relying only on automated tests.
๐บ๐ฎ๐ฟ๐ธ๐ฑ๐ผ๐๐ป
๐. ๐๐ก๐๐ญ ๐ ๐๐จ๐ฎ๐ฅ๐ ๐๐ฆ๐ฉ๐ซ๐จ๐ฏ๐ ๐๐๐ฑ๐ญ:
There are still several things I would like to improve.
๐๐ฒ๐๐๐ฒ๐ฟ ๐๐ฝ๐ฝ๐ผ๐ถ๐ป๐๐บ๐ฒ๐ป๐ ๐๐ป๐๐ฒ๐ด๐ฟ๐ฎ๐๐ถ๐ผ๐ป
The current specialist focuses on appointment guidance and preparation. A future version could integrate real clinic availability and appointment booking instead of only guiding the user through the process.๐ ๐ผ๐ฟ๐ฒ ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐ฆ๐๐ฝ๐ฝ๐ผ๐ฟ๐
The current language-selection flow focuses on English and Hindi. More Indian languages could be added while keeping the same strict native-script approach.๐๐ฒ๐๐๐ฒ๐ฟ ๐ข๐ฏ๐๐ฒ๐ฟ๐๐ฎ๐ฏ๐ถ๐น๐ถ๐๐
The analytics dashboard could be expanded with more operational metrics such as latency, handoff rates, tool usage, and failure reasons.๐ ๐ผ๐ฟ๐ฒ ๐ฅ๐ผ๐ฏ๐๐๐ ๐ง๐ฒ๐๐๐ถ๐ป๐ด
I would also like to expand automated behavioral testing with more real-world conversation scenarios and improve testing around external inference-service failures.๐ ๐ผ๐ฟ๐ฒ ๐๐ฑ๐๐ฎ๐ป๐ฐ๐ฒ๐ฑ ๐ฅ๐ฒ๐บ๐ถ๐ป๐ฑ๐ฒ๐ฟ ๐ช๐ผ๐ฟ๐ธ๐ณ๐น๐ผ๐๐
The current reminder system supports natural-language scheduling, status tracking, cancellation, and outbound reminder calls. A future version could support recurring reminders, richer scheduling options, user-configurable reminder preferences, and more advanced delivery controls.๐ ๐ผ๐ฟ๐ฒ ๐ฆ๐ฝ๐ฒ๐ฐ๐ถ๐ฎ๐น๐ถ๐๐ ๐๐ด๐ฒ๐ป๐๐
The same architecture could support additional focused agents, such as a pharmacy information specialist or a health-services navigation specialist, while keeping the main agent responsible for routing.
๐. ๐๐ข๐ง๐ค๐ฌ ๐ญ๐จ ๐ญ๐ก๐ ๐๐จ๐๐ ๐๐ง๐ ๐๐๐ฆ๐จ:
The project is open for others to explore and learn from.
๐๐ถ๐๐๐๐ฏ ๐ฅ๐ฒ๐ฝ๐ผ๐๐ถ๐๐ผ๐ฟ๐:
https://github.com/codewithvishuuu/murf-livekit-starter
The repository contains the agent implementation, prompts, specialist handoff logic, scheduled reminder calls, tests, frontend, analytics, wellness tips, and support flows.
I also recommend looking at the project structure and tests if you are building your own voice agent. The tests were especially useful for checking routing, safety rules, handoffs, handbacks, language behavior, and privacy boundaries.
๐ ๐ข๐ง๐๐ฅ ๐๐ก๐จ๐ฎ๐ ๐ก๐ญ๐ฌ:
Building Aarogya Sahayak over these 10 days taught me that a useful voice agent is much more than connecting an LLM to speech.
The difficult part is designing the system around the model: defining clear responsibilities, creating safety boundaries, routing conversations to the right agent, preserving context, protecting user information, and measuring whether the system actually works.
As the project evolved, I also learned how features such as scheduled reminder calls and accessible wellness guidance can extend a voice agent beyond a single conversation while still keeping safety and privacy at the center.
The biggest lesson I learned is that a good voice agent should know not only "how to answer", but also "when not to answer itself".
That is where tools, specialist agents, analytics, scheduled workflows, and human support become important.
I'm grateful to "Murf AI" for organizing the "10 Days of Voice Agents โ #VoiceForBharat Edition" and giving me the opportunity to build, test, break, fix, and improve a real voice-agent project.
I'm building voice agents using the fastest TTS API โ "Murf Falcon".
Have you experimented with real-time voice pipelines or LiveKitintegrations yet?๐ ๏ธ Whatโs the biggest latency hurdle youโve run into? Drop your thoughts in the comments below!๐๐ฌ
๐๐๐๐ฃ๐ ๐ฎ๐ค๐ช ๐๐ค๐ง ๐ง๐๐๐๐๐ฃ๐! ๐๐๐๐ก ๐๐ง๐๐ ๐ฉ๐ค ๐ง๐๐๐๐ ๐ค๐ช๐ฉ ๐ค๐ง ๐๐ค๐ฃ๐ฃ๐๐๐ฉ ๐ค๐ฃ ๐๐๐ฃ๐ ๐๐๐๐ฃ / ๐๐๐ฉ๐๐ช๐๐ค
๐ฝ๐๐๐๐๐ ๐ฒ๐๐๐๐๐
Top comments (0)