๐ Building Revora AI: My 10-Day Voice Agent Journey
From a raw audio pipeline to an intelligent, multilingual, multi-agent AI learning companion for Bharat.
๐ Project Highlights & Hackathon Track
- Challenge: 10 Days of Voice Agents โ #VoiceForBharat Edition by Murf AI
- Track: ๐ Learning & Literacy
- Voice Engine: โก Murf Falcon (Ultra-low latency multilingual TTS)
- Real-Time Transport: ๐๏ธ LiveKit WebRTC
- Database & Memory: ๐ Supabase (PostgreSQL + Knowledge State)
- GitHub: https://github.com/AadarshJain07/murf-livekit-starter
๐ The Vision: Why Voice-First Learning for Bharat?
In India, learning has always been conversational. When students study, they don't think in rigid text prompts โ they discuss, debate, ask doubts, and switch effortlessly between English, Hindi, and Hinglish.
Yet, most modern AI tutors are text-locked:
- Friction in Expression: Typing complex mathematical equations, chemical formulas, or nuanced conceptual doubts on a mobile keyboard kills curiosity.
- Language Barrier: Rigid single-language interfaces fail Indian students who naturally code-switch ("Bhaiya, Doppler effect mein apparent frequency increase kab hoti hai?").
- Passive Consumption: Text chatbots output overwhelming paragraphs that students skim rather than truly understand.
Revora AI was built to bridge this gap. Created over 10 intense days during the Murf AI #VoiceForBharat challenge, Revora turns learning into an active, real-time, voice-first dialogue where students simply talk, ask, practice, debate, and master concepts naturally.
๐บ๏ธ The 10-Day Build Roadmap
Day 1โ2 โโโโโโโบ Day 3โ4 โโโโโโโบ Day 5โ6 โโโโโโโบ Day 7โ8 โโโโโโโบ Day 9โ10
Voice Core Personality Long-Term Actionable Multi-Agent
& LiveKit & Pedagogical Memory & Tools & Live Specialists &
Streaming Guardrails Gamification Telemetry Debate Mode
| Phase | Milestone & Focus | Key Tech / Breakthrough |
|---|---|---|
| Days 1โ2 | Voice Pipeline Foundations | Real-time audio streaming over LiveKit WebRTC + Murf Falcon TTS integration |
| Days 3โ4 | Pedagogical Persona & Guardrails | Socratic teaching style, safe educational guardrails, interruptibility & latency tuning |
| Days 5โ6 | Long-Term Memory & Gamification | Supabase session tracking, XP rewards, Quests & diagnostic weakness detection |
| Days 7โ8 | Actionable Tools & Telemetry | Tool calling, automated parent/student check-in calls, real-time analytics dashboard |
| Days 9โ10 | Specialist Agent Swarm & Debate Mode | Domain expert handoffs (Maths, Physics, Chemistry) + Adversarial Debate Mode |
๐๏ธ System Architecture & Workflow
Revora combines real-time WebRTC audio streaming, large language model reasoning, persistent knowledge graphs, dynamic tool execution, and specialized domain agents into a cohesive learning experience.
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ ๐๏ธ Student Voice โ
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ LiveKit WebRTC โ
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Speech-to-Text (STT) โ
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Revora Agent Orchestrator & Router โ
โโโโโโโโฌโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโฌโโโโโโโโโโฌโโโโโโโ
โ โ โ โ
โโโโโโโโโโโโโโโโดโโ โโโโโโโโดโโโโโโ โโโโโโโโดโโโโ โโโโดโโโโโโโโโโโโโโโโ
โ ๐ง Memory & DB โ โ ๐ ๏ธ Tools โ โ ๐ Stats โ โ ๐ฅ Specialists โ
โ (Supabase) โ โ (Actions) โ โ(Analyticsโ โ (Domain Swarm) โ
โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโ โโโโโโโโโโโโ โโโโฌโโโโโโโโโโโโโโโโ
โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Samar (Maths) โ Pooja (Physics) โ
โ Abhinav (Chemistry) โ Debate Specialist (Socratic) โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Murf Falcon Streaming TTS โ
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ ๐ Voice Response โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ก Deep Dive: Core Innovations & Features
1. ๐ Multilingual Voice & Code-Switching (English โข Hindi โข Hinglish)
Students in Bharat rarely speak pure Queen's English or formal textbook Hindi. Revora is engineered to recognize Indian phonetic nuances and respond with natural bilingual fluidity.
Student: "Sir, mujhe photoelectric effect ka threshold frequency samajh nahi aaya, please simple Hindi mein samjhao na?"
Revora (via Murf Falcon): "Bilkul! Dekho, threshold frequency woh minimum energy barrier hai jo electron ko metal surface se bahar nikalne ke liye chahiye. Agar light ki frequency isse kam hui, toh chahe kitni bhi bright light daal do, koi electron bahar nahi aayega!"
2. ๐ฅ Specialist Agent Swarm with Zero-Context Loss Handoff
Rather than relying on a single generic prompt that tries to do everything, Revora implements an explicit routing architecture to domain-specific specialist agents:
- ๐ Samar (Mathematics Specialist): Step-by-step algebraic breakdowns, mental math shortcuts, and geometric visualization.
- โ๏ธ Pooja (Physics Specialist): Intuitive thought experiments, kinematics, thermodynamics, and real-world mechanics.
- ๐งช Abhinav (Chemistry Specialist): Reaction mechanisms, periodic trends, stoichiometry, and molecular structures.
- ๐๏ธ Debate Specialist: Socratic sparring partner for structured argumentation, evidence assessment, and critical thinking.
๐ How Contextual Handoff Works:
When a student says "Connect me to the Physics Specialist," Revora doesn't simply switch models โ it packages the entire conversation state, current topic, and student learning profile:
# Conceptual Agent Handoff Handler
async def route_to_specialist(student_id: str, specialist_name: str, conversation_ctx: dict):
# 1. Fetch persistent learner profile from Supabase
learner_profile = await supabase.table("student_profiles").select("*").eq("id", student_id).single()
# 2. Package handoff state payload
handoff_payload = {
"student_name": learner_profile["name"],
"mastery_level": learner_profile["mastery_level"],
"weak_areas": learner_profile["weak_areas"],
"current_topic": conversation_ctx.get("current_topic"),
"recent_dialogue": conversation_ctx.get("history")[-5:]
}
# 3. Voice announcement before handoff
await agent.speak(f"Switching you over to {specialist_name}, who has your context ready.")
# 4. Activate specialist with preserved state
return await activate_specialist_agent(specialist_name, handoff_payload)
Key Benefit: The student never has to repeat their question or explain where they got stuck.
3. ๐ง Memory & Gamified Closed-Loop Learning Engine
Revora goes far beyond a passive conversational bot by embedding a continuous learning feedback loop:
๐ Learn โ Practice โ Measure โ Diagnose Weakness โ Targeted Quests
- ๐ฏ Interactive Quests: Generates short 2-minute voice quizzes tailored to the student's current topic.
- โญ XP & Mastery Badges: Provides instant positive reinforcement upon completing concepts.
- ๐ Diagnostic Weakness Engine: Tracks recurring mistakes in Supabase PostgreSQL and automatically schedules spaced-repetition reinforcement during subsequent sessions.
4. โ๏ธ Socratic Debate Mode: Building Critical Thinkers
Most educational AI simply gives direct answers, encouraging passive memorization. Revora's Debate Mode trains students to defend their ideas:
- The student chooses a debate topic (e.g., "Should AI replace traditional examinations?" or "Is nuclear power the best clean energy solution for India?").
- The AI takes the counter-position, challenging premises and demanding supporting evidence.
- At the end of the round, the AI provides a constructive scorecard evaluating Reasoning, Evidence, Counterargument Handling, and Clarity.
5. ๐ ๏ธ Actionable Tools & Real-Time Telemetry
Revora is an agent with hands. Using function calling, it can:
- Query student learning history and update mastery graphs in Supabase.
- Trigger automated outbound progress calls to students or parents.
- Escalate to human mentors when a student shows severe confusion or distress.
- Stream live call performance metrics into an analytics dashboard.
๐ Live Benchmark Metrics:
| Metric | Benchmark Result | Status |
|---|---|---|
| Total Test Calls | 14 | โ |
| Successful Completions | 12 | ๐ |
| Graceful Fallbacks / Handled Errors | 2 | ๐ก๏ธ |
| Call Success Rate | 85.7% | ๐ |
| TTS Synthesis Latency (Murf Falcon) | ~250ms | โก |
๐ ๏ธ Build Your Own Voice Agent: Technical Blueprint
If you are building your own production-grade voice agent, here are the four foundational pillars:
- Speech-to-Text (STT): Captures microphone input and converts it into a real-time text stream with low word-error rates.
- LLM Orchestration: Understands intent, accesses memory, executes tool calls, and generates concise, conversational responses.
- Text-to-Speech (TTS): Converts text into ultra-realistic, low-latency audio chunks using Murf Falcon.
- Real-Time WebRTC Transport: Connects bidirectional audio streams using LiveKit for sub-second, interruptible voice interactions.
๐ Secure Environment Configuration
Store all credentials in server-side environment variables:
# Murf AI Ultra-Low Latency Voice Engine
MURF_API_KEY="your_murf_api_key_here"
# LiveKit WebRTC Infrastructure
LIVEKIT_API_KEY="your_livekit_key"
LIVEKIT_API_SECRET="your_livekit_secret"
LIVEKIT_URL="wss://your-project.livekit.cloud"
# Supabase Vector & State Database
SUPABASE_URL="https://your-project.supabase.co"
SUPABASE_SERVICE_ROLE_KEY="your_supabase_service_role_key"
[!CAUTION]
Security Best Practice: Never expose private API keys or database service role keys in frontend client bundles or public GitHub repositories. All agent routing and database mutations must execute server-side.
๐ฎ What's Next for Revora AI?
This 10-day challenge established the core voice engine. Here is what is on the roadmap:
- [ ] ๐ผ๏ธ Multimodal Whiteboard Sync: Enable students to sketch math proofs or circuit diagrams on screen while speaking with Revora.
- [ ] ๐ฃ๏ธ Regional Language Expansion: Expand beyond Hindi/Hinglish to native Tamil, Telugu, Marathi, and Bengali voice models.
- [ ] ๐ NCERT / State Board Curriculum Alignment: Auto-generate syllabus-aligned quests and revision packs directly from standard textbooks.
- [ ] ๐ฑ Low-Bandwidth Telephony Gateway: Bring Revora to rural students via standard phone calls (SIP / PSTN integration).
๐ญ Key Takeaways & Final Thoughts
Building Revora during this 10-day sprint reinforced a vital engineering truth:
A great voice agent is not just an LLM wired to a speaker.
It is a carefully crafted symphony of:
๐๏ธ Voice + ๐ง AI Reasoning + ๐พ Memory + ๐ ๏ธ Tools + ๐ Analytics + ๐ฅ Domain Specialists + ๐ Pedagogy
What started as a simple audio loop on Day 1 became an intelligent, empathetic, multi-agent learning mentor for Indian students.
A huge shoutout to Murf AI for spearheading the #VoiceForBharat initiative and making ultra-fast voice AI accessible with Murf Falcon!
๐๏ธ Revora AI โ Learn by Talking. ๐ฎ๐ณ
Tags: #VoiceForBharat #10DaysOfVoiceAgents #MurfAI #VoiceAI #EdTech #AIForEducation #LiveKit #Supabase #MultiAgent
Top comments (0)