My father has a master's degree in agriculture. Growing up around someone who had studied the subject deeply made me notice that a farming question is rarely as small as it sounds.
“Which crop should I grow?” can depend on soil, season, water, location, cost, and risk. A mark on a leaf might be harmless, or it might be the first sign of a problem that needs an expert. The useful answer is not the longest answer. It is the answer that understands the situation and helps someone decide what to do next.
A friend of mine made this gap feel real in a different way. He had land, money to invest, and an idea for an agriculture-related business. What he did not have was fluency in the subject. He could search online, but he did not always know which question to ask, which source to trust, or when general advice was no longer enough.
My father had the knowledge. My friend had the intent and the resources. Between them was a problem of access.
That problem is larger than one conversation. The Government of India's Digital Agriculture Mission describes the need for timely, reliable information and customised advisories for crop planning, crop health, pest management, and irrigation. Data may exist in many places, but a person standing in a field still needs a simple way to reach the right information.
I kept returning to one thought: what if asking for farming help could feel less like operating software and more like speaking to someone who listens?
That thought became Kisan Sahayak.
The assistant I wanted to build
Kisan Sahayak is a browser and phone-based voice AI assistant for Indian farmers and people beginning an agricultural venture. A user can speak in English, Hindi, or Hinglish and ask about crops, soil, irrigation, local weather, or an emerging crop problem.
The assistant can fetch live data, remember useful details with permission, place a controlled advisory call, create a human-help request, measure whether a conversation achieved its purpose, and transfer crop symptoms to a separate specialist agent.
It is not meant to replace an agronomist. Its job is to shorten the distance between a spoken question and a safe next step. When it does not have reliable data, it should say so. When a case needs a specialist or a person, it should stop pretending to be enough.
I built it during 10 Days of Voice Agents — VoiceForBharat Edition. Its speech is powered by Murf Falcon. Murf describes Falcon 2 as a real-time speech model with roughly 100 ms time-to-first-audio and the ability to switch languages within a sentence, both of which matter in a conversation that may move naturally between Hindi and English. You can read the technical details in the Falcon 2 documentation.
The source code is available here: github.com/Reet24-del/murf-livekit-starter
What happens when a farmer speaks
A voice assistant sounds like one system, but it is a chain of systems working quickly enough to feel like one conversation.
- Deepgram Nova-3 converts speech into text. The project uses multilingual recognition so the caller can speak English, Hindi, or a mixture of both. Deepgram documents its supported models and languages in its model overview.
- Google Gemini reads the latest request, conversation context, safety instructions, and available tools. It decides whether to answer, fetch data, ask for consent, or transfer control.
- Murf Falcon converts the reply into speech. The main assistant uses the Anisha voice. The crop specialist uses Samar, so the caller can hear that a real handoff has happened.
- LiveKit carries the audio and manages the real-time agent session between the browser or phone and the Python backend. Its documentation describes the same sequential STT–LLM–TTS pipeline used in this project.
Farmer speaks
|
v
LiveKit room
|
v
Deepgram STT ---> latest-turn language detection
| |
v v
Google Gemini <--- memory, tools, consent, safety rules
|
+---- routine question ----> Kisan Sahayak
|
+---- crop symptoms -------> Crop Problem Specialist
|
v
Murf Falcon TTS
|
v
The farmer hears the answer
Supporting services:
Open-Meteo -> live district weather
SQLite -> consented memory, help requests, and call outcomes
This architecture gave me a useful separation of responsibilities. The language model handles conversation. Tools handle facts that can change. SQLite handles information that must survive after a call. The specialist handles a narrower class of problems. The frontend tells the user what the system is doing.
Ten days, one layer at a time
The project did not begin with all of these abilities. Each daily task exposed a weakness in the version from the day before.
Day 1: Giving the idea a voice
The first version had one goal: complete a live voice loop.
I connected Deepgram speech-to-text, Gemini, Murf Falcon text-to-speech, and LiveKit transport. For the first time, I could speak into the browser and hear the assistant answer. During my local tests, the observed time-to-first-audio ranged from 137 ms to 263 ms. Those numbers are local observations, not a universal benchmark, because network, region, model, and device all affect perceived latency.
The early success was exciting, but it also revealed a truth about voice products. A correct answer can still feel wrong if the pause before it is too long. Latency is part of the conversation design.
Day 2: Deciding who the agent should be
A working voice loop is not yet an assistant. On Day 2, I gave Kisan Sahayak a clear role, tone, and boundary.
It would answer only farming and local-weather questions. It would keep spoken replies short. It would never invent current market prices, subsidy details, pesticide doses, or long-range weather. Hindi replies would use Devanagari. English replies would remain English. Hinglish could stay in Roman script.
I also tested hostile and out-of-scope prompts. This was where the project moved from “Can it answer?” to “Can it answer responsibly?” The biggest lesson was that refusal behavior must be designed with as much care as the happy path.
Day 3: Making the invisible conversation visible
Voice interfaces can be confusing when the screen gives no feedback. Is the microphone active? Is the agent thinking? Has the session ended?
I built a browser interface that shows clear states: ready, connecting, listening, thinking, speaking, transferring, and ended. I added a live transcript, a weather panel, saved memory, and controls that avoid ending a call through an accidental tap.
The interface uses a dark field-inspired visual language, but its main purpose is practical. A user should never have to guess whether the assistant heard them.
Day 4: Building memory without taking ownership of the user's data
The assistant could remember a conversation while the session was open, then forget everything when the user returned. Day 4 added persistent memory through SQLite.
Kisan Sahayak can save a caller's name, district, crops, land size, irrigation type, preferred language, and a short summary. It saves new information only after permission. A direct instruction such as “Remember that my district is Lucknow” also counts as clear permission.
This distinction became important:
- Conversation context helps during the current call.
- Stored memory helps during a future call.
The database stores selected facts, not the full conversation. Python's sqlite3 documentation describes SQLite as a lightweight disk-based database that does not require a separate server, which made it a good fit for a local challenge build. A deployed version would need encryption, authentication, retention rules, and stronger access controls.
Day 5: Teaching the assistant to use real data
The model once gave the wrong current time. That failure made the next design decision obvious: anything described as “current” must come from a current source.
I added a function tool that fetches district weather from Open-Meteo and another that reads the system clock in the Asia/Kolkata timezone. The weather result includes its source time, current conditions, wind, temperature range, and rain probability. Open-Meteo documents the available current and forecast variables in its Weather Forecast API.
The failure path mattered as much as the successful response. The weather request has a timeout. If the service is down or returns incomplete data, the assistant says that live weather is unavailable and refuses to guess.
The tool description also mattered. The model decides whether to call a function from that description, so “use this for current weather, rain chance, temperature, wind, spraying, irrigation, sowing, or harvesting” produced better routing than a vague label such as “weather tool.”
Day 6: Moving from browser conversations to outbound calls
Some useful information should reach a farmer before they open a website. On Day 6, I added controlled outbound rain-advisory calls through LiveKit SIP, with Linphone or Twilio as the call provider.
This feature includes rules that are easy to overlook in a demo:
- the recipient must have consented;
- the destination must be valid;
- calls are limited to 08:00–20:00 IST;
- saved opt-outs are honoured;
- the weather tool must run before a current advisory is spoken;
- failed, unanswered, or declined calls receive a controlled outcome.
The hard part was not writing a dial command. It was connecting the SIP trunk, worker registration, dispatch rule, verified destination, and agent name without mixing credentials between services. Telephony made every configuration mismatch visible immediately.
Day 7: Knowing when to ask a human
An AI assistant should not stretch every problem until it sounds solvable. I selected two situations for human help: serious or rapidly spreading crop damage, and market information that is missing or stale.
The assistant first says what it plans to share: who needs help, what happened, what it checked, urgency, language, and the requested follow-up method. It then asks for a clear yes or no. If the caller declines, no request is created.
After consent, the system stores a short redacted summary and returns a reference ID. It does not store passwords, OTPs, PINs, account numbers, or a full transcript.
This day changed my understanding of escalation. “Contact an expert” is advice. A consented request with a reference ID and visible status is a workflow.
Day 8: Measuring whether a call helped
Before Day 8, I could see calls happen but could not answer a basic product question: did the conversation achieve anything?
I defined success for Kisan Sahayak as one of four outcomes: farming guidance delivered, live weather delivered, an expert request created, or an outbound rain advisory delivered. A call that ends before one of those conditions is recorded as failed even when the software did not crash.
The analytics dashboard reads real browser and SIP session records from SQLite. It shows total, successful, and failed calls, along with filters, trends, channels, languages, durations, and controlled failure categories. It stores no caller identity and no transcript.
This feature forced me to separate technical reliability from user success. A perfectly stable call in which the user leaves before receiving guidance is still a failed product outcome.
Day 9: Handing crop problems to a specialist agent
The main assistant had accumulated weather, memory, time, safety, escalation, and general farming responsibilities. Crop symptom triage needed a narrower role.
I created a separate Crop Problem Specialist. Reports of leaf spots, curling, wilting, pests, disease, nutrient deficiency, or unexplained crop damage trigger the handoff. The main assistant announces the transfer. The website displays “Connecting you to the specialist.” The specialist then introduces itself in a different Murf voice and continues with a copy of the existing conversation, so the farmer does not have to repeat the problem.
The specialist asks one focused question at a time, offers only low-risk first steps, and never claims a certain diagnosis from a voice description. LiveKit's agents and handoffs guide supports this pattern by allowing a tool call to return another agent while preserving conversation context.
The first version did not route correctly. Crop symptoms competed with the older human-escalation tool, so the assistant sometimes offered a human request instead of transferring the call. I fixed that by giving specialist-eligible turns deterministic priority and temporarily removing the escalation tool from that turn.
Day 10: Turning the build into something another person can use
The final day was not another agent feature. It was the work of explaining the system honestly.
I organised the repository, documented setup, checked that secrets and local caller data were excluded, added screenshots, and wrote down the failures that shaped the design. A public project is more useful when a reader can understand its limits, run it, and reproduce the important tests.
The ten-day sequence also made the product progression clear:
Conversation
-> character and guardrails
-> visible interface
-> consented memory
-> live tools
-> outbound reach
-> human help
-> measurable outcomes
-> specialist handoff
-> documented, reproducible project
The bugs that changed the design
The useful lessons came from moments when the assistant behaved confidently but incorrectly.
English questions received Hindi answers
The original prompt described a bilingual assistant, but the bilingual greeting and earlier context sometimes outweighed the caller's newest sentence. An English question could receive a Hindi answer.
I moved language detection to every completed user turn. The latest turn now decides the response mode: English, Devanagari Hindi, or Roman-script Hinglish. Tests cover all three instead of checking only the opening greeting.
The fix taught me that language is not a profile setting. In a multilingual conversation, it is live session state.
The assistant sounded current without having current access
Knowing that a user is in India does not give a language model access to India's current clock. General weather knowledge is not today's weather either.
Time and weather now sit behind tools. Each returned value carries a time or timezone, and each failure has a spoken fallback. This removed the most dangerous kind of voice-agent error: a made-up answer delivered with a natural voice.
The call ended while the user was still trying to speak
Voice activity detection, turn detection, browser audio, silence timeouts, and session lifecycle all interact. In early runs, silence handling could mark the user away too quickly or the browser could disconnect before the final analytics record was written.
I increased the away timeout, avoided automatic disconnect on silence, added controlled re-prompts, and finalised analytics through the session shutdown path. The lesson was practical: real-time state bugs rarely belong to one layer.
The specialist and human-help paths collided
Both features dealt with difficult crop problems, but they served different purposes. The specialist continues the AI conversation. Human escalation creates a request after consent. Without an ownership rule, the model could choose either.
The final routing order is explicit. A crop symptom goes to the specialist first. Serious cases can later move to a human-help request, but only after the specialist path and only with permission.
How to run Kisan Sahayak
You will need Python 3.10–3.14, Node.js 20 or newer, uv, pnpm 9 or newer, a LiveKit Cloud project, and API keys for Murf, Deepgram, and Google AI.
1. Clone and install
git clone https://github.com/Reet24-del/murf-livekit-starter.git
cd murf-livekit-starter
cd backend
uv sync
uv run python src/agent.py download-files
cd ../frontend
pnpm install
cd ..
2. Create private environment files
cp backend/.env.example backend/.env.local
cp frontend/.env.example frontend/.env.local
Add the following values to backend/.env.local:
LIVEKIT_URL=
LIVEKIT_API_KEY=
LIVEKIT_API_SECRET=
MURF_API_KEY=
DEEPGRAM_API_KEY=
GOOGLE_API_KEY=
The frontend needs the same LiveKit project and the explicit agent name:
LIVEKIT_URL=
LIVEKIT_API_KEY=
LIVEKIT_API_SECRET=
AGENT_NAME=kisan-sahayak-primary
Both .env.local files are ignored by Git. Never publish API keys, SIP trunk IDs, phone numbers, caller records, or backend/memory.db.
3. Start the application
chmod +x start_app.sh
./start_app.sh
Open http://localhost:3001, select Start Call, and allow microphone access. The frontend and backend must use the same LiveKit project and agent name.
4. Test the two agent routes
Ask a routine question:
What is today's rain chance in Lucknow?
The main assistant should call the live weather tool and answer in English.
Then begin another call and report a crop symptom:
My tomato leaves have black spots and are curling.
The main assistant should announce a transfer. The interface should display the specialist connection state, and the crop specialist should continue without asking you to repeat the symptom.
End the calls and open http://localhost:3001/call-analytics. The total should increase, and each call should be classified from its real outcome.
What Kisan Sahayak could change, and what comes next
My friend's original difficulty was not a lack of ideas. It was the gap between an idea and informed action. Kisan Sahayak reduces that gap in a few specific ways.
It lets someone begin with the words they already know. It remembers location and crop details after consent, so a returning user does not start from zero. It brings live weather into the same conversation as crop planning. It recognises when a problem belongs to a specialist. It also leaves a visible trail when a human request is created or a call fails to reach its goal.
The current version remains a prototype. Its crop guidance should be strengthened with an agronomist-reviewed knowledge base. District-level mandi prices need a dependable live source and clear timestamps. Photo-assisted crop triage could add evidence that voice alone cannot provide. The local SQLite database should move to encrypted hosted storage with authentication and role-based access before public deployment. Finally, field testing across different accents, devices, and network conditions should shape the next round of changes.
My father’s education reminds me how much context sits behind a reliable agricultural answer. My friend’s plan reminds me how difficult that context can be to reach when someone is starting out. Kisan Sahayak is my attempt to make the first step easier while preserving the point at which software should call a specialist or ask a person for help.
Top comments (0)