Building SpeakEasy AI: A Multilingual Voice Agent for Bharat

Priyal Mangesh Mandloi — Sat, 15 Aug 2026 16:39:43 +0000

Voice AI is becoming more accessible, but language and accessibility still remain important challenges for many users in India.

As part of the Murf AI Voice for Bharat Challenge, I built SpeakEasy AI — a voice-based AI assistant designed to help users practice English through natural conversations.

🚀 What is SpeakEasy AI?

SpeakEasy AI is a conversational voice agent that allows users to speak naturally and receive AI-generated voice responses.

The idea is simple:

Speak naturally. Practice confidently. Improve one conversation at a time.

The agent is designed around everyday English practice and supports English, Hindi, and Hinglish interactions.

🛠️ Tech Stack

The project uses:

Python
LiveKit Agents
Murf AI for voice generation
LLM-based conversation
Speech-to-text and text-to-speech
FastAPI
Next.js / frontend UI
Git & GitHub

🎙️ Voice Interaction

The main goal was to make the interaction feel like a real conversation rather than a traditional chatbot.

Users can start a voice conversation, speak naturally, and receive spoken responses from the AI agent.

I also integrated a friendly female voice to make the experience more natural and approachable.

🧠 Multilingual Conversation

One of the important parts of the project is handling different ways users communicate.

For example, a user can switch between:

English
Hindi
Hinglish

This makes the experience more comfortable for users who may not be completely confident speaking English.

🔄 Specialist Handoff

I also experimented with conversational routing.

For example, when a user asked for help with a mathematics problem, the main assistant could route the conversation toward a dedicated maths practice specialist.

This makes the architecture more flexible because different specialists can handle different tasks.

📊 Call Analytics

Another feature I implemented was a simple call analytics dashboard.

The dashboard tracks:

Total calls
Successful calls
Failed calls

This provides a basic view of how the voice agent is performing.

💡 What I Learned

Building SpeakEasy AI helped me understand that a voice agent is much more than simply connecting speech-to-text with an LLM.

Important parts include:

Natural conversation flow
Voice quality and latency
Multilingual interaction
Agent routing
Error handling
Call analytics
User experience

The biggest learning for me was understanding how different AI components work together to create a complete voice experience.

🇮🇳 Why Voice AI for Bharat?

India has a huge diversity of languages and communication styles.

Voice interfaces can make AI more accessible to people who may find traditional text-based interfaces difficult or less natural.

With better multilingual support and localized voice experiences, conversational AI can become much more useful for everyday learning and communication.

🚀 What's Next?

I plan to continue improving SpeakEasy AI by working on:

Better multilingual conversations
More specialist agents
Improved conversation memory
Better analytics
More real-world use cases

This project has been a great hands-on experience in building production-oriented AI voice applications.

Thanks to the Murf AI Voice for Bharat Challenge for providing the opportunity to explore voice AI more deeply.

DEV Community: Priyal Mangesh Mandloi