What if a farmer could simply pick up a phone, speak naturally in Malayalam or English, and ask:
“What is today's market price?”
or
“It's going to rain tomorrow. Should I spray my crops today?”
or even:
“My crop is getting damaged. Can someone help me?”
No typing. No complicated menus. Just a conversation.
That was the idea behind Farm & Field, the voice agent I built during the 10 Days of Voice Agents — VoiceForBharat Edition challenge by Murf AI.
Over these 10 days, I went from building a basic voice conversation to creating an agricultural voice assistant with memory, real-time tools, outbound calling, human escalation, call analytics, multilingual conversations, and a specialist agent.
I built Farm & Field, a voice assistant designed to help farmers with everyday agricultural questions.
What Farm & Field Does
Farm & Field can:
- Answer general farming questions
- Provide weather information
- Look up agricultural market prices
- Remember returning users with consent
- Handle Malayalam, English, and Manglish conversations
- Make outbound calls for alerts
- Escalate serious problems to a human
- Hand crop-related problems to a specialist agent
- Track call outcomes through a dashboard The main idea was simple:
The AI should not try to do everything itself.
If it needs current information, it uses a tool.
If the problem needs specialized knowledge, it hands the conversation to a specialist.
If the situation is beyond what the AI can safely handle, it asks a human for help.
How the system works
A voice agent is basically four main components connected together:
Speech-to-Text (STT) — converts the farmer's speech into text. I used Deepgram.
LLM — understands the conversation and decides what to do. I used Google Gemini.
Text-to-Speech (TTS) — converts the response back into voice. I used Murf Falcon.
Real-time transport — carries the audio between the user and agent. I used LiveKit.
On top of this, Farm & Field has:
- SQLite for user memory and call data
- Weather and market-price tools
- Human escalation
- A Crop Problem Specialist agent
- Outbound calling through LiveKit SIP
- A call analytics dashboard
- A customized Next.js frontend
The important features
🇮🇳 Malayalam, English and Manglish support
Farmers don't always speak in one language.
For example:
"Ente rubber crop-il leaves yellow aakunnundu."
The agent can recognize Malayalam/Manglish conversations and adapt the TTS voice accordingly.
This made the conversation feel more natural for Indian users.
Murf Falcon for fast voice responses
I used Murf Falcon for TTS because latency is very important in voice conversations.
A slow response makes a voice agent feel like a chatbot that is reading messages aloud.
With a faster TTS response, the conversation feels much more natural.
Memory with consent
Farm & Field can remember useful information about returning farmers, such as:
- Name
- District
- Crops
- Land size
- Irrigation type
- Language preference
But the agent asks for permission before saving information.
This was important because memory should be useful without automatically storing everything a user says.
Tools instead of guessing
The agent doesn't rely on the LLM to guess live information.
For example, when a farmer asks:
"What is today's rubber price?"
the agent uses the market-price tool.
The same approach is used for weather information.
This makes the system more reliable.
Human escalation
The agent can recognize situations where AI should not be the final answer.
I implemented escalation for cases such as:
- Important market data being unavailable
- Serious or spreading crop problems
- Unknown pests or major crop damage
The farmer gives consent before a human-help request is created.
Specialist agent
I created a separate Crop Problem Specialist for crop disease and pest-related conversations.
The main agent can hand the conversation to the specialist without making the farmer repeat everything.
The specialist also has a different voice, making the handoff clear to the user.
Outbound calls
Farm & Field can initiate calls for specific alerts, such as:
- Weather warnings
- Market price alerts
I used LiveKit SIP for the outbound calling workflow.
Call analytics
The system also tracks call information such as:
- Call duration
- Call outcome
- Start and end time
- Notes
This helped me understand what happened during conversations instead of treating every call as an isolated interaction.
How to build your own
1.The four core pieces you need:
- An STT provider (Deepgram, etc.)
- An LLM (Gemini, OpenAI, etc.)
- A TTS provider — I'd recommend Murf Falcon for latency
- A real-time transport layer (LiveKit is the easiest path)
2.Setting up and running the project:
- Fork the Murf LiveKit Starter repository
- Follow its README for uv sync (backend) and pnpm install (frontend)
- Run the backend with uv run python src/agent.py dev and the frontend with pnpm dev
3.Where API keys go — and where they must NOT go:
- Copy .env.example to .env.local in both backend/ and frontend/ — never commit .env.local, it's already gitignored
- You'll need: a LiveKit Cloud project (URL, API key, secret), a Murf API key, a Deepgram API key, and an LLM provider key
- Never paste real keys into chat, screenshots, or your repo — treat them like passwords
4.Connecting and testing:
- With both the backend and frontend running, open localhost:3000, click the start button, allow microphone access, and talk
- Watch the backend terminal for a "registered worker" log line as confirmation the agent is live
5.The code:
My repository: https://github.com/safdarsidhik/murf-livekit-starter/tree/main
What I'd improve next
- Add support for more Indian regional languages and improve Malayalam/Manglish detection
- Connect the market-price tool to more reliable and up-to-date agricultural data sources
- Add image-based crop disease detection to help identify visible crop problems
- Improve the human escalation workflow so farmers can connect with agricultural experts more easily
- Add more advanced outbound calling for weather, crop, and market alerts
- Improve the call analytics dashboard with more detailed conversation insights
- Test Farm & Field with real farmers and use their feedback to improve the experience
Links
Repository: https://github.com/safdarsidhik/murf-livekit-starter/tree/main
Built with Murf Falcon — the fastest TTS API I used across this build
Part of 10 Days of Voice Agents — VoiceForBharat Edition by Murf AI
If you're building your own voice agent for an underserved use case, happy to answer questions — drop a comment.
Top comments (0)