The problem and the users
For a huge number of farmers across India, the biggest barrier to useful information isn't availability, it's interface. Government schemes, weather forecasts, and mandi prices all exist somewhere online, but they're locked behind text-heavy apps and websites that assume comfort with English, typing, and navigating menus. A farmer standing in their field, phone in hand, doesn't want to fill out a form. They want to ask a question out loud and get a useful answer back, the same way they'd ask a neighbor or a local officer.
That's the gap Kisan Sahay (किसान सहाय) tries to close. It's a voice-first assistant for farmers, built over 10 days for the Farm & Field track of Murf's 10 Days of Voice Agents — VoiceForBharat Edition challenge. You call it, you talk to it in Hindi, English, or a natural mix of both, and it helps with crops, weather, government schemes, and knows when to bring in more help.
The landing page — one clear button to start, and a plain-language explanation of what Kisan Sahay can help with.
What the voice agent actually does
By the end of the challenge, Kisan Sahay could:
- Hold a natural spoken conversation in Hindi (in proper Devanagari script), English, or Hinglish, matching whatever register the farmer just used
- Remember returning farmers by name, with their consent, and pick up from what was discussed last time
- Look up a live weather forecast for any district
- Answer questions about major government farmer schemes from a reference dataset
- Place outbound calls to warn a farmer about weather relevant to their crop
- Recognize when a problem is beyond it and create a tracked request for a human, with the farmer's permission
- Track whether each call actually achieved something useful, visible on a live dashboard
- Hand off detailed crop-health troubleshooting to a dedicated specialist agent, without making the farmer repeat themselves
How the system works
The core pipeline is the same shape most voice agents use:
Farmer speaks
-> Deepgram STT (multilingual mode) turns speech into text
-> Gemini LLM decides what to say, or which tool to call
-> Murf Falcon TTS turns the response into speech
-> LiveKit streams the audio back and forth in real time
What makes it more than a simple loop is what sits around that pipeline: a SQLite database for memory, escalations, and call outcomes; a couple of real tools (weather, schemes); an outbound calling path over SIP; and a second, focused agent it can hand off to.
A normal conversation in progress — the transcript panel updates live as Kisan Sahay answers a weather question with a real, dated forecast instead of a guess.
The most important features, and why they mattered
An Indian voice, and code-mixed language handling. Farmers don't switch neatly between Hindi and English — they mix them mid-sentence. Getting the agent to reliably detect Hindi in Devanagari, English, and Hinglish (Hindi words in Roman script) and reply in the same register, every single turn, took real iteration. The fix that finally worked was making the language rule explicit and absolute in the system prompt: judge only the most recent message, never carry language over from earlier turns, and always write Hindi in Devanagari script even when the farmer typed it in Roman letters — because the TTS engine needs the correct script to pronounce it properly.
Memory with real consent. Day 4 added a lookup_caller and save_caller_info tool pair. The important design choice wasn't the database schema, it was that consent is enforced in code, not just described in the prompt. The save_caller_info function takes a consent boolean, and if it's false, nothing gets written — the tool itself refuses, regardless of what the model "intends."
Real tools with honest failure. A weather tool that calls a live API is only actually useful if it also handles the API being slow, wrong, or down. The agent is instructed to say so honestly and suggest another source rather than inventing a forecast, and a genuinely failed lookup is recorded as a failure on the analytics dashboard, not silently swallowed.
Knowing when not to answer. This was, in hindsight, the most important feature. A voice agent giving farming advice should not pretend to have information it doesn't. When a lookup fails and the farmer needs a real answer, or when a crop problem sounds serious, Kisan Sahay asks permission and creates a tracked escalation for a human, instead of guessing.
A specialist handoff. By Day 9, Kisan Sahay had grown into a generalist. Rather than stretch its main prompt to cover deep crop-disease troubleshooting too, it hands that conversation off to a separate CropSpecialistAgent with its own narrower prompt and guardrails. The full conversation history carries across the handoff automatically, so the farmer never repeats themselves.
The handoff in action — Kisan Sahay announces the transfer in one line, and the crop specialist picks up immediately, already aware of what the farmer described, no repeating required.
Outbound calling, tested for real. Day 6 took the agent off the browser entirely — it dials out over a SIP trunk (a free Linphone account, since Twilio's trial ran out) to deliver a weather warning the farmer didn't ask for. Since the call is unsolicited, the opening line has to do real work: say who's calling, why, and how to opt out, all before anything else.
The call log on the receiving end — proof the outbound call actually connects over SIP, not just something running locally in a browser tab.
Challenges, and how I got through them
The forced handoff wouldn't actually hand off. When I first built the specialist handoff, the main agent would say "let me connect you to the crop specialist" and then just... stop, waiting for the farmer to say "okay" before actually calling the tool. It was treating the handoff like a consent-based action, the same pattern used for memory and escalations. The fix was making the tool description explicit that this was not a consent flow: the agent should call the tool in the same turn, immediately after announcing the handoff, with no waiting.
Gemini's turn-ordering rule broke the outbound greeting. For outbound calls (Day 6), I initially passed per-call context, like the farmer's name and district, straight into the generate_reply() instructions when the call connected. This intermittently broke with a 400 error from Gemini: "function call turn comes immediately after a user turn or after a function response turn." Because the very first assistant turn on an outbound call has no preceding user turn, any instruction that nudged the model toward an immediate tool call (like "check the weather right after your opening") violated that ordering rule. The fix was to bake all per-call context into the agent's instructions at construction time, and keep the generate_reply() trigger itself completely generic and static.
Call success tracking was quietly wrong. Early on, every call that didn't hit a specific tool (weather, schemes, escalation) was marked a failure by default, even a call where the agent answered a general farming question perfectly well from its own knowledge. The lesson: default to success for a call that completes without error, and mark failure only on genuine, specific failure paths — not on "no tool happened to fire."
How you can build your own
If you want to build something like this, here's the shape of it:
The four components you need:
-
Speech-to-text (STT) — turns the caller's voice into text. I used Deepgram, with
language="multi"for reliable code-switched language detection. - An LLM — decides what to say and which tools to call. I used Gemini.
- Text-to-speech (TTS) — turns the response back into speech. I used Murf Falcon, the fastest TTS API available, for low-latency, natural-sounding Indian voices.
- Real-time transport — moves audio back and forth with low latency. I used LiveKit, which also handles SIP telephony for outbound calling.
Setting up and running it:
Fork github.com/MEHWISH310/murf-livekit-starter, then:
git clone <your-repo-url>
cd murf-livekit-starter
# Terminal 1 — LiveKit server (skip this if you're using LiveKit Cloud instead of self-hosting)
livekit-server --dev
# Terminal 2 — Backend
cd backend
uv sync
uv run python src/agent.py download-files
uv run python src/agent.py dev
# Terminal 3 — Frontend
cd frontend
pnpm install
pnpm dev
Where API keys go: every key lives in a .env.local file (one in backend/, one in frontend/), which is git-ignored and never committed. Never hardcode a key directly into source files, and never publish .env.local contents in a blog post, screenshot, or public repo.
Connecting and testing: once both the backend agent and frontend are running, open http://localhost:3000, click the start button, allow microphone access, and speak. The agent should greet you, and you can talk it through a real conversation from there.
The full code for Kisan Sahay is public: github.com/MEHWISH310/murf-livekit-starter
What I'd improve next
- A live mandi price lookup, instead of the agent always deferring to "check locally"
- Phone-number-based caller identification for a real telephony deployment, instead of name-based lookup
- A broader government scheme dataset, ideally backed by a live source instead of a static local one
- More specialist agents for other common deep topics (irrigation planning, government scheme eligibility walkthroughs)
Closing thought
Ten days ago this was a generic customer-support template. Now it's an assistant that remembers who it's talking to, knows the difference between something it can answer and something it shouldn't guess at, and calls in the right help, human or specialist, when it needs to. That last part, knowing its own limits, ended up mattering more than any single feature.
Built for 10 Days of Voice Agents — VoiceForBharat Edition, powered by Murf Falcon.
Top comments (0)