============================================================ -->
My grandmother does not type. If she wants to know whether the pain in her chest is worth a trip to the hospital, she asks a person. Millions of people in India navigate healthcare exactly that way — by voice, in their own language, usually through whoever is nearest.
So for Murf AI's 10 Days of Voice Agents — VoiceForBharat Edition, I built the thing that fits that behaviour instead of fighting it: Saathi Swasthya, a voice-first health navigation assistant. You speak to it in Hindi, Gujarati or English. It asks follow-up questions, tells you how urgent your situation sounds, finds real health facilities near you, and — if you say yes — passes a short summary to a human.
Let me be precise about what it is not, because in health the boundaries are the product. Saathi does not diagnose. It does not prescribe. It does not rank hospitals by quality, and it does not book appointments. Every one of those is a refusal written into the system prompt and tested, not a feature I ran out of time for.
Repo: https://github.com/manthansingh26/murf-livekit-starter
Here is how the ten days actually went.
The whole system on one screen
A deterministic layer inside on_user_turn_completed decides the language, the consent state, and whether a red-flag symptom fired — in Python, on every turn — then hands those decisions to the model as non-negotiable context. Anything I could decide in code, I decided in code.
Day 1: I spent about ten hours on a thirty-minute task
The Day 1 brief was to get a starter voice agent talking. I read it, decided it meant "design your entire project now", and disappeared into architecture for most of a day. I wired things I did not need yet. I rewrote prompts for a product that had no working audio path. By the time I got the actual required piece working, I had spent something like ten hours on what the task needed thirty minutes for.
The useful lesson was not "read the instructions". It was narrower than that: in a timeboxed build, the task tells you the smallest thing that must exist by tonight, and everything else is a bet you are making with tomorrow's hours. I got much faster once I started asking "what does today's task actually require?" before opening the editor.
The pipeline that came out of it stayed for all ten days: LiveKit Agents for the real-time transport, Deepgram Nova-3 for speech-to-text, Gemini 3.5 Flash Lite for reasoning, and Murf Falcon — the fastest TTS API — for the voice, with Silero VAD and LiveKit's multilingual turn detector deciding when the caller has actually finished speaking.
session = AgentSession(
stt=MultilingualDeepgramSTT(),
llm=google.LLM(model="gemini-3.5-flash-lite"),
tts=murf.TTS(
voice="Anisha",
style="Conversation",
tokenizer=tokenize.basic.SentenceTokenizer(min_sentence_len=2),
text_pacing=True,
),
turn_detection=MultilingualModel(),
vad=ctx.proc.userdata["vad"],
preemptive_generation=True,
)
Two of those arguments are the difference between a demo and something you would let a worried person use. min_sentence_len=2 with text_pacing=True means Falcon starts speaking after a couple of sentences' worth of text instead of waiting for the full completion, and preemptive_generation=True lets the LLM start working before the turn is formally closed. Voice quality is not only the voice. It is when the voice starts.
Days 2 and 3: the prompt is a product surface, and a useState(true) that mugged me
Day 2 was guardrails. Saathi refuses to diagnose, refuses to prescribe, refuses to say which hospital is "best", and if someone describes chest pain or unconsciousness it points at 112 and 108 before anything else. Those refusals are written as scripts in the prompt, because an agent that improvises its own refusal wording will eventually improvise its way around it.
Day 3 was the frontend: voice states you can read at a glance, a transcript that shows Devanagari and Gujarati script as spoken rather than transliterated, and a clear message when the browser denies microphone permission — which is the single most common way a voice app fails for a real user.
It also gave me my dumbest bug of the ten days. The transcript panel opened itself on every session. I went looking for an event handler, a race condition, something interesting. It was this:
const [transcriptOpen, setTranscriptOpen] = useState(true);
One character of intent, wrong. The fix was false. I lost real time to it, and the reason I lost that time is that I assumed a UI behaviour must have a UI cause — some listener firing — instead of checking the initial state first. Now when something happens "by itself", the initial value is the first place I look.
Day 4: memory only when someone says yes
Saathi remembers you between calls: your name, your preferred language, a few facts. Two pieces make that work. The frontend mints a saathi_<uuid> caller ID once, keeps it in localStorage with a cookie fallback, and reuses it as the LiveKit participant identity, so call one and call two are the same person. The backend keeps profiles in PostgreSQL through asyncpg.
The part I care about is the consent rule. Detecting that someone just told you their name is easy. Not saving it is the hard part, because the model wants to be helpful. So consent is not a suggestion in the prompt — it is a per-turn instruction injected into the conversation, and the tool that writes to the database fails closed:
if consent_confirmed is not True:
return _no_consent_result()
That strict is not True is deliberate. A truthy string like "false" must never count as consent. Credentials, OTPs, PINs and transcripts are never stored at all.
The hardest problem was not health. It was language.
Nobody in my family speaks one language at a time. Real sentences sound like "મને fever છે અને body બહુ weak લાગે છે" — Gujarati grammar, English medical nouns, in one breath. A voice agent that handles Gujarati and English separately handles neither.
Two things broke. First, transcription: a single Deepgram stream in multilingual mode reads Gujarati unreliably, and a Gujarati-locked stream mangles the English words. So Saathi runs two Nova-3 streams over the same audio — one with language="multi", one pinned to language="gu" — and arbitrates. The multilingual stream wins whenever it produces real Latin words or Devanagari with Hindi markers, and the Gujarati stream is only allowed through when the multilingual one has gone quiet for more than two seconds. It costs a second stream to fix an error that no amount of prompting can.
Second, and worse: language stability. My script-based detector looked at "મને fever છે અને headache" and confidently said English, because it counted more Latin words than Gujarati characters. Saathi would then answer a Gujarati speaker in English mid-conversation, which is exactly the failure that makes someone hang up. The fix was to stop treating each turn as a fresh guess and add a continuity layer on top of detection:
def language_continuity(current: str, text: str) -> str:
detected = detect_language(text)
if detected == current:
return current
gujarati_chars = len(re.findall(r"[-૿]", text))
hindi_chars = len(re.findall(r"[ऀ-ॿ]", text))
latin_words = len(re.findall(r"\b[a-zA-Z]+\b", text))
total_words = len(text.split()) or 1
if detected == "Gujarati" and gujarati_chars >= 2 and gujarati_chars > hindi_chars:
return "Gujarati"
if detected == "Hindi" and hindi_chars >= 2 and hindi_chars > gujarati_chars:
return "Hindi"
if detected == "English" and latin_words >= 2 and latin_words >= total_words * 0.5:
return "English"
# Ambiguous or mixed-script turn — stay with the established language.
return current
The last line is the whole idea. An ambiguous turn is not evidence of a language switch; it is evidence that you should not switch. A deliberate switch — two or more characters of a script, dominant over the alternative — is still honoured immediately, because people do genuinely switch.
The result then gets injected into that turn as a hard instruction rather than left to the model's judgement:
new_message.content.append(lang_inst + consent_inst + escalation_inst)
Three deterministic decisions — which language, what consent state, whether a red flag fired — computed in Python and handed to the LLM as non-negotiable context for that single turn. Anything I could decide in code, I decided in code.
Day 5: real facilities, or nothing
Saathi finds actual health facilities from live OpenStreetMap data — Nominatim to geocode the city or district, Overpass to pull hospitals, clinics, PHCs and pharmacies within five kilometres, with name, type, address and approximate distance.
Then the primary Overpass endpoint started returning 504s while I was testing, and I learned what "graceful" has to mean for a health tool. It cannot mean the model improvises a clinic. It cannot mean silence either. So the tool now walks three public endpoints with bounded failover, caches geocode results for ten minutes so the same town is never re-queried inside one conversation, and — the detail I am most pleased with — only a successful lookup carries a retrieved_at timestamp. Error results deliberately have none, so there is no path by which the agent can claim it fetched fresh data after a failure.
Distances are straight-line haversine, and Saathi says "approximately" out loud, because a five-kilometre straight line can be a twelve-kilometre drive and someone deciding whether to walk deserves to know which one they are being told.
Day 6: an agent that calls you
Day 6 added an outbound worker (saathi-outbound) that dials a softphone or PSTN endpoint through a LiveKit SIP trunk, introduces itself, states why it is calling, and tells the person they can ask it to stop at any time. Destination handling turned out to be the fiddly part — bare usernames, E.164 numbers and full SIP URIs all have to be normalised into what sip_call_to expects — and SIP failures are logged with their sip_status_code, because "the call didn't connect" is not a debuggable statement.
An agent that phones you unprompted has a different consent posture than one you open in a browser tab. That is why the greeting says who it is and how to end it before anything else happens.
Day 7: handing a person to a person
Some conversations should not end with an AI. If someone describes a red-flag symptom, or asks Saathi to diagnose them, Saathi gives the emergency guidance first, then offers to pass a short summary to a human support person — and asks permission.
Only after an explicit yes does it write a ticket with a reference ID like ESC-9F3A2B7C — eight hex characters from secrets.token_hex(4) — which it reads back to the caller. What it never does is promise a callback or a response time, because I have no support team on the other end and inventing one would be the most harmful thing in this entire project. The ticket stores a two-to-three sentence summary, never the transcript.
Day 8: a dashboard with no comforting numbers on it
Call analytics were tempting to fake. A dashboard with impressive figures screenshots well. I decided the numbers had to be real or absent, and that meant deciding what "success" is before the call ends.
No LLM grades it. CallAnalyticsTracker listens to actual session events — function_tools_executed, conversation_item_added, close — and resolves an outcome from five explicit conditions in a fixed priority: escalation created, then substantive guidance given, then error, then no response, then no success condition met. Every database write is exception-guarded, because telemetry must never be able to interrupt a live voice stream.
The dashboard reads real rows from PostgreSQL and polls every eight seconds. On the day I built it, it displayed a very small number of calls, all mine. That is what it should have displayed.
Day 9: one specialist, and the handoff
Day 9 was multi-agent. I added exactly one specialist — the Clinic & Appointment Specialist — because a second agent is only worth it if it has a boundary that the first one cannot hold.
The main agent hands over by returning a new agent and a line of speech from a tool:
@function_tool
async def transfer_to_clinic_specialist(self, context: RunContext) -> tuple[Agent, str] | str:
language = self._last_detected_language
try:
specialist = ClinicAppointmentSpecialist(
language=language,
chat_ctx=self.chat_ctx.copy(exclude_instructions=True),
)
except Exception as e:
logger.error(f"[HANDOFF] failed to create clinic specialist: {e}")
return "The clinic and appointment specialist is temporarily unavailable. ..."
announcement = HANDOFF_ANNOUNCEMENTS.get(language, HANDOFF_ANNOUNCEMENTS["English"])
return specialist, announcement
Three details carry the experience. chat_ctx.copy(exclude_instructions=True) gives the specialist the conversation but not the main agent's instructions, so the caller never repeats their location or their problem. The announcement is pre-written in all three languages, so the transfer is spoken in the language the caller was already using. And if construction fails, the tool returns a plain string instead of an agent — the handoff simply does not happen and Saathi keeps talking, which is a test I wrote on purpose.
The specialist's scope is narrower than its name suggests, and deliberately so. It cannot analyse symptoms — it does not even have that tool. It will not tell you which hospital is best; it has a scripted answer for that question that redirects to what it can compare, like distance and type. It treats "near me" as not-a-location and asks for a city or district, because a facility list built on a guessed location is worse than no list.
And it does not book appointments. It explains what to bring, what to ask, and how booking generally works, and it is instructed to say plainly that it cannot see real availability and that you should contact the facility. Writing that limitation into the prompt was more work than a fake booking confirmation would have been, and it is the difference between a demo and something that does not hurt anyone.
The day I realised I had forgotten to submit
Somewhere around Day 4 and 5, I forgot to submit the daily form. I was heads-down in code, the deadline for that day passed, and I found out afterwards.
That hit harder than any bug. I had been working long days, I was mentally exhausted, and for a while I was convinced the whole thing was over for me — that the work still on my screen no longer counted. I was frustrated with myself in a way that is hard to describe to someone who has not been nine days into something they cared about.
What got me moving again was the Murf AI Discord. I asked, half-expecting silence, and people answered. Vortexedits, a helpful member of the Murf AI Discord community, took the time to explain how things worked when I was confused and anxious about where I stood. I do not know what the final call on my missed submissions is, and I am not going to pretend I do. What I know is that a stranger spending ten minutes on my confusion is the reason there was a Day 6.
If you are doing one of these challenges: put the submission itself in your checklist, at the same priority as the code. And when you are lost, ask in public. The community is the underrated part of the stack.
Day 10: my test suite lied to me, and finding out was the best part
My suite collects 145 tests. On my last full local run: 135 passed, 3 skipped, 7 failed in 288 seconds. The 3 skips are opt-in live-network tests. Here is what those seven failures actually were, because "7 failed" and "7 bugs" are very different sentences.
Four of them died on this:
429 RESOURCE_EXHAUSTED — quotaId: GenerateRequestsPerMinutePerProjectPerModel-FreeTier
quotaValue: 15, model: gemini-3.5-flash-lite
Fifteen requests per minute on the free tier. Ten multilingual eval tests, two or three turns each, plus tool calls and retries — I was rate-limiting myself. The symptom pytest showed me was AssertionError: Expected another event, but none left, which sounds like an agent that went silent and is actually an agent that never got to answer. One of the other three failures passes on its own; it is an LLM-as-a-judge disagreeing with itself about wording between runs.
Then I dug into the last two, and found something I would not have found any other way. AgentSession.run(user_input=...) calls generate_reply() directly. Agent.on_user_turn_completed — where my language instruction, my consent rules, my escalation rules and my caller memory are all injected — only fires on the audio path.
Which means my multilingual test suite was never testing my multilingual logic. Those tests were measuring how well Gemini mirrors a language from the system prompt alone, with the entire deterministic layer switched off. One of the two failures is a Gujarati-to-English switch that fails precisely because the test never delivers the instruction the live agent would inject. The other one fails because I asked the judge to demand code-mixed English words while my own system prompt mandates replying in the caller's script — my eval contradicted my spec, and the agent obeyed the spec.
Nothing in my agent was broken. My confidence was. A green test suite that exercises the wrong code path is worse than a red one, because it feels like evidence.
What the voice itself taught me
I underestimated how much of "does this feel trustworthy" lives in the audio and not in the words. The same sentence about chest pain lands completely differently depending on pacing, and a half-second of dead air after a scared person stops talking reads as hesitation.
That is why the Falcon settings in the Day 1 snippet are not cosmetic. Sentence-level streaming with text_pacing means the reply begins while the model is still finishing it, so the gap between "I stopped talking" and "it started answering" collapses. Falcon being the fastest TTS API is not a benchmark I care about in the abstract — it is the reason a caller does not think the line went dead. I have not measured end-to-end latency with instrumentation, so I am not going to quote you a number. What I can say is that the streaming configuration is the single change that most improved how the conversation felt.
Choosing one voice, "Anisha", in the "Conversation" style, was also a deliberate narrowing. A health tool that sounds like a call centre gets treated like one.
What I did not build
This is the section I would skip if I were trying to impress you, so here it is instead.
Saathi does not book appointments. There is no scheduling integration, no availability data, nothing. The specialist explains the process and tells you to contact the facility yourself.
It does not rank facilities by quality. It sorts by distance and type, and says so.
The dashboard has no charts — four KPI cards, a success-rate bar, and the eight most recent calls, read from real rows. It also has no authentication, and neither does the analytics API route behind it. That is fine for a laptop and a challenge submission and it is not fine for anything else; it is written down in the README as a known limitation rather than quietly left for someone to find.
The language column in my analytics table is never populated, because the tracker is constructed without it. Small bug, real bug, listed rather than hidden.
I have no users. No production deployment, no uptime, no adoption numbers. Everything in this post is from my own machine, my own calls, my own test runs.
What I would do next
Three things, in order. First, close the eval gap with a test helper that actually invokes on_user_turn_completed and forwards the mutated message, plus request pacing of roughly one call every four seconds so the free tier stops masking real results. Second, put authentication in front of /dashboard and the analytics route before this ever touches a network anyone else is on. Third, more languages — the continuity layer is written to extend, and Marathi or Bengali would test whether the design generalises or whether I just tuned it for three.
Further out, the thing I actually want is offline-tolerant behaviour. The people this is for often have the worst connectivity, and a voice agent that needs a perfect uplink is a voice agent for people who already have options.
The stack, in one place
Backend is Python on the LiveKit Agents SDK 1.4.5, with two workers — the browser agent and saathi-outbound for SIP calls. Deepgram Nova-3 does speech-to-text, running two concurrent streams for the code-mixing problem. Gemini 3.5 Flash Lite reasons. Murf Falcon, the fastest TTS API, speaks. Silero VAD plus LiveKit's multilingual turn detector decide when you have finished a sentence. PostgreSQL through asyncpg holds caller memory, escalation tickets and call analytics. Facility data comes live from OpenStreetMap via Nominatim and Overpass. Frontend is Next.js 15 with React 19 and Tailwind 4. Tests are pytest with httpx.MockTransport for the offline HTTP paths and LLM-as-a-judge for the conversational ones, with Ruff and GitHub Actions in CI.
Everything is in the repo, including the parts I have criticised in this post: https://github.com/manthansingh26/murf-livekit-starter
Ten days, honestly
The engineering lesson I will keep is that in a health-adjacent tool, every decision you can move out of the model and into code, you should. Language selection, consent, red-flag detection, whether data was actually fetched — those are if statements, not vibes. The model is there to be warm and to ask good follow-up questions in your language. It is not there to decide whether you consented.
The other lesson is that constraints are the feature. The hardest work in this project went into things Saathi refuses to do, and the reason is simple: if it guesses wrong about a fever, someone drives four hours to the wrong place, or does not drive at all.
Thank you to Murf AI for running 10 Days of Voice Agents — VoiceForBharat Edition, and for Falcon, which is the piece a caller actually experiences. Thank you to the Murf AI Discord community, and specifically to Vortexedits, a helpful member of that community who answered a confused person's questions on a day when that mattered more than any commit.
If you are building for Bharat: build for the way people already behave. My grandmother is not going to learn to type. The software can learn to listen.
#10DaysOfVoiceAgents #VoiceForBharat #MurfAI #MurfFalcon #Murf #VoiceAI #VoiceAgents #ConversationalAI #SpeechAI #TextToSpeech #LiveKit #AIIndia
Top comments (1)
cool as i know u dont know thatsnot my fault see your self