Introduction
Over the last 10 days, I built English Learning Coach, a voice AI that helps learners practice English through natural conversations. Instead of being just another chatbot, it can remember users, use tools, escalate to a teacher when needed, track call outcomes, and hand off math questions to a specialist.
This project was built for the Learning & Literacy track in 10 Days of Voice Agents — VoiceForBharat Edition, powered by Murf Falcon.
The Problem
Many learners hesitate to practice spoken English because they don't always have someone available to practice with. A voice-based assistant makes practice feel more natural than typing and gives immediate feedback during conversations.
What My Voice Agent Can Do
By the end of the challenge, my agent includes:
🎙️ Natural voice conversations using Murf Falcon
🛡️ Clear behavior and safety guardrails
🧠 Memory for returning users
🧰 Tool-based actions instead of only generating text
📞 Outbound calling support
👩🏫 Human escalation with consent and reference IDs
📊 Real-time call analytics dashboard
🔀 Specialist agent handoff for math practice
How the System Works
The project combines four main components:
Speech-to-Text to understand spoken input
LLM to decide how the agent should respond
Murf Falcon Text-to-Speech for natural voice replies
LiveKit for real-time voice communication
SQLite stores memory, escalation requests, and call analytics.
My Favorite Features
Human Escalation
If a learner becomes frustrated or asks for a teacher, the agent asks permission before sharing a short summary, creates a support request, and provides a reference ID.
Call Analytics
The dashboard tracks:
Total Calls
Successful Calls
Failed Calls
These values come from real conversations instead of hardcoded numbers.
Specialist Handoff
The main English coach hands math questions to a dedicated Math Practice Specialist, keeping responsibilities focused.
The Hardest Part
The biggest challenge was integrating LiveKit's telephony and shutdown callbacks.
I encountered issues such as:
SIP configuration problems
callback errors during shutdown
analytics not saving correctly
API differences between LiveKit versions
Instead of restarting the project, I fixed each issue step by step by testing, reading logs, and making small changes until the workflow became stable.
That debugging process taught me much more than a perfect first attempt would have.
How to Run the Project
Clone the repository.
Install dependencies.
Add API keys inside .env.local.
Start the backend.
Start the frontend.
Open the browser and begin a voice conversation.
Never commit API keys or private user data.
GitHub Repository
👉 https://github.com/kusumaranikusumarani3232-hub/Murf-ai
What's Next
If I continue improving this project, I'd like to add:
Better multilingual conversations
More specialist agents
Smarter progress tracking
Richer analytics
Final Thoughts
This challenge helped me understand that building a voice agent isn't just about generating speech. It's about designing conversations, protecting users, knowing when to ask for human help, and creating experiences that solve real problems.
Thanks to Murf AI for organizing this learning journey.
Top comments (0)