DEV Community: Ayontika-pal

Building an AI Interview Coach with VoiceForBharat: A 10-Day AI voice agent journey

Ayontika-pal — Sat, 15 Aug 2026 09:45:14 +0000

How I built a multi-agent, multilingual AI Interview Coach powered by Murf Falcon, LiveKit, Deepgram, and Gemini — complete with caller memory, real-time guardrails, outbound calling, human escalation, and call analytics.

1. Introduction & Problem Statement

Job interview preparation in India is often high-stakes, intimidating, and inequitable. While mock interviews with senior engineers or professional coaches cost thousands of rupees per session (typically ₹1,000–₹3,000 per hour), millions of job seekers in India—especially engineering students and candidates from Tier-2/Tier-3 cities—struggle with technical confidence, articulation, and language barriers when balancing English and Hinglish.

To solve this, I participated in 10 Days of Voice Agents — VoiceForBharat Edition organized by Murf AI. Over nine intensive days, I built Anisha (and her technical companion Dev), an AI Interview Coach under the Learning & Literacy track.

Target Audience & Core Pain Points

Target Audience: Engineering graduates, job seekers, Tier-2/Tier-3 college students, and career upskillers in India preparing for technical, behavioral, and DSA (Data Structures & Algorithms) interview rounds.
The Problem It Solves:
1. High Cost & Lack of Mentorship: Professional mock interview platforms are financially out of reach for students needing daily practice.
2. The Articulation & Anxiety Gap: Candidates frequently possess strong theoretical coding knowledge but freeze up or struggle when explaining their problem-solving steps out loud under real-time pressure.
3. Rigid Language Expectations (English vs. Hinglish): Candidates often feel most comfortable explaining complex logic in Hinglish (code-mixed Hindi + English), whereas conventional platforms enforce strict English-only environments.

Why Voice over Text Chatbots?

Interviews are fundamentally oral, dynamic, high-latency human interactions! Text-based LLM chatbots (like ChatGPT) cannot simulate real-world vocal pressure, tone, speech pacing, or natural spoken interruptions. A real-time voice agent provides immediate conversational feedback, allowing candidates to practice spoken flow, eliminate filler words, and build true interview-day confidence.

2. How the System Works: High-Level Architecture & Pipeline

Building an interactive voice agent requires orchestrating four key building blocks in near-zero latency:

┌─────────────────┐       ┌─────────────────┐       ┌──────────────────┐       ┌──────────────────┐
│  🎙️ User Speech  │ ────> │  Deepgram STT   │ ────> │ Gemini 2.5 Flash │ ────> │ Murf Falcon TTS  │ ────> 🔊 User Hears
│  (Microphone)   │ Audio │ (Nova-3 / Auto) │ Text  │ (LLM & Logic)    │ Text  │ (55ms Latency)   │ Audio
└─────────────────┘       └─────────────────┘       └──────────────────┘       └──────────────────┘
                                                             │
                                                  LiveKit Real-Time WebRTC
                                            ┌────────────────────────────────┐
                                            │ SQLite Memory / DB & Dashboard │
                                            └────────────────────────────────┘

The Real-Time Audio Loop

Speech-to-Text (STT): Deepgram Nova-3 transcribes live audio in real-time with multi-lingual auto-detection support (English & Hinglish), capturing candidate speech with high accuracy.
Brain & Logic (LLM): Gemini 2.5 Flash acts as the agent's brain, processing conversation context, evaluating candidate answers, enforcing safety guardrails, and triggering function tools.
Text-to-Speech (TTS): Murf Falcon streams ultra-fast (55ms latency), natural Indian English audio (Anisha voice) back to the user.
Real-Time Transport & Orchestration: LiveKit manages WebRTC audio streaming, frame synchronization, VAD (Voice Activity Detection), and turn management between the browser/phone and Python backend.
Persistence Layer: SQLite (caller_memory.db) persists user profiles, past interview history, human escalation tickets, and call logs.

3. Deep Dive into Key Features Built Across the 10 Days

Here is a comprehensive breakdown of the core capabilities powering the AI Interview Coach:

1. Indian Voice & Cultural Persona (`Anisha`)

Powered by Murf Falcon's Anisha voice, the agent delivers warm, professional, and encouraging Indian English pronunciation. The system defaults to clear English but dynamically adapts to code-mixed Hinglish when candidates express themselves in Hindi/Hinglish, rendering Hinglish text in clean Roman script for pristine Murf TTS playback.

2. Safety Guardrails & Consent Protocols

Data Protection: Strict system guardrails prohibit collecting sensitive personal data like Passwords, Aadhaar, PAN numbers, or banking credentials.
Do-Not-Call / Opt-Out Support: Candidates can state "Stop calling me" or "Opt out". The agent immediately calls opt_out_caller(user_id), updates the database, and terminates phone outreach.

3. Long-Term Caller Memory & Database Persistence (`db.py`)

Returning candidates are greeted by name ("Hello Ramesh, welcome back! Last time we practiced your Python interview..."). The agent uses get_caller() and upsert_caller() to track:

Candidate Name & Language Preference (Hinglish/English)
Target Role (e.g., Frontend Engineer, Backend Python Developer)
Skill level (Fresher, Mid-Level) & weak areas noted during prior practice sessions

4. Live API Tool Integration (`lookup_interview_question`)

When candidates request a coding or behavioral prompt, the agent invokes lookup_interview_question(). This tool queries the public Tech Interview Handbook repository on GitHub API to retrieve live, up-to-date exercises.

Freshness Transparency: Explains data freshness out loud (e.g., "From Tech Interview Handbook updated as of 29 July 2026").
Graceful Failover: If the network request times out, it switches instantly to a verified local cache (status: partial_offline_fallback) without awkward silences.

5. Outbound Phone Calling & SIP Telephony (`outbound_call.py`)

Candidates can schedule daily practice calls. Using LiveKit SIP and Twilio/Linphone integration, the script dispatches outbound calls directly to the candidate's phone:

Mandatory Opening Protocol: The agent announces who it is, why it's calling, and explicitly explains how to opt out in the very first two sentences.

6. Human Escalation Ticket Workflow (`create_escalation`)

When candidates express severe interview distress or explicitly request senior human mentor assistance (e.g., placement guarantees or human resume review):

The agent asks for explicit consent before logging details.
It invokes create_escalation() to write a ticket to caller_memory.db.
It speaks a tracking Reference ID (e.g., ESC-8A3F21) to the caller and promises a human follow-up within 24 hours.

7. Multi-Agent Specialist Handoff (`DSASpecialist` / `Dev`)

When an interview transitions from general behavioral prep to Data Structures & Algorithms:

Anisha asks: "I can connect you to our DSA practice expert, Dev, who specializes in technical coding problems. Would you like me to connect you to Dev now?"
Upon candidate agreement, the agent triggers handoff_to_dsa_specialist().
State transfers seamlessly to Dev (DSASpecialist), who greets the candidate ("Hello! I am Dev, your DSA practice expert...") and continues with full candidate context without asking the user to repeat themselves.

8. Privacy-Compliant Call Analytics Dashboard (`/dashboard`)

Built into the Next.js frontend, the dashboard tracks real-time session performance for the Learning & Literacy track:

Metrics Tracked: Total Calls, Successful Completed Interviews, Failed/Dropped Calls, and Overall Success Rate %.
Privacy Shield: Strictly aggregates counts while automatically filtering out caller transcripts, passwords, phone numbers, and private data.

4. Difficult Challenges Faced & How They Were Solved

Challenge 1: Audio Overlap & Interruption Latency

The Problem: In early builds, when a candidate interrupted the agent mid-sentence, the agent's TTS stream continued playing for 1–2 seconds, causing overlapping audio and chaotic conversations.
The Root Cause: High latency in silence detection and turn-taking signals created a mismatch between streaming audio frames and user voice activity detection.
The Solution: Combined LiveKit's MultilingualModel turn detector with Murf Falcon's ultra-low 55ms streaming TTS latency. Because Murf Falcon delivers time-to-first-audio (TTFA) in milliseconds, the pipeline can halt audio playback instantly the moment user speech is detected.

Challenge 2: Network Delays During Real-Time Tool Calling

The Problem: Fetching live interview questions from external APIs mid-session caused 3+ second pauses, leading users to ask "Are you still there?".
The Solution: Implemented a strict 4-second timeout on network socket calls in Python and created a fallback mechanism (status: partial_offline_fallback) that returns a pre-cached offline question set, allowing the agent to answer immediately without breaking conversation flow.

5. Step-by-Step Guide: How to Build and Run Your Own Agent

Follow these practical steps to spin up your own Murf Falcon + LiveKit voice agent!

Step 1: System Prerequisites

Python 3.10+ with uv package manager installed
Node.js 18+ with pnpm
LiveKit Server account (LiveKit Cloud or local CLI)

Step 2: Environment Configuration

Create a .env.local file in both backend/ and frontend/ directories:

# LiveKit WebRTC Config
LIVEKIT_URL=wss://your-livekit-project.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

# AI Service Keys
MURF_API_KEY=your_murf_falcon_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
GOOGLE_API_KEY=your_google_gemini_api_key

Security Notice: Never commit your .env.local files or API keys to public repositories. Ensure .env.local is listed in your .gitignore.

Step 3: Backend Agent Setup & Execution

cd backend
# Install dependencies with uv
uv sync

# Pre-download required LiveKit model files
uv run python src/agent.py download-files

# Start agent in development mode
uv run python src/agent.py dev

Step 4: Frontend UI Setup & Execution

cd frontend
# Install Node dependencies
pnpm install

# Start Next.js dev server
pnpm dev

Step 5: Test the Voice Agent

Open http://localhost:3000 in your browser.
Click "Start talking" and grant microphone permissions.
Greet Anisha ("Hi Anisha, I want to practice for a Python developer interview").
Ask to practice DSA questions to test the handoff to Dev!

🔮 6. Future Enhancements

Multimodal Code Canvas: Integrating a side-by-side Monaco code editor into the Next.js UI, allowing Dev to evaluate written code syntax while giving verbal feedback.
Regional Language Expansion: Adding native voice support for regional Indian languages (such as Hindi, Tamil, Telugu, and Kannada) available in Murf Falcon's voice library.

🔗 7. Code Repository & Links

Public GitHub Repository: https://github.com/Ayontikapal/murf-livekit-starter/tree/day9
Murf Falcon TTS API Docs: https://murf.ai/api/docs/text-to-speech/streaming
Murf Voice Library: https://murf.ai/api/docs/voices-styles/voice-library

Gemma 4: models and setup

Ayontika-pal — Mon, 18 May 2026 15:22:21 +0000

Gemma 4 Model

Google’s Gemma family has quickly become one of the most practical and developer-focused open-weight AI ecosystems available today. With the release of Gemma 4, Google has introduced major improvements over Gemma 3, making it the company’s most advanced open model family so far.

But Gemma 4 is more than just another language model update. It reflects a broader move toward accessible AI that developers, researchers, students, and independent creators can run, customize, and experiment with directly on their own machines.

That changes everything.

What Is Gemma 4?

Gemma 4 is the newest lightweight open-weight AI model family developed by Google DeepMind.

The main goal behind the release is to improve reasoning abilities while maintaining efficient performance, faster response generation, and better support for complex multi-step tasks.

The Gemma 4 Model Lineup

The Gemma 4 family currently includes:

Gemma 4 E2B
Gemma 4 E4B
Gemma 4 26B-A4B
Gemma 4 31B
Gemma 4 E2B

Gemma 4 E2B is the smallest model in the lineup. It is built for:

low memory usage
fast inference speeds
and edge deployment environments.

The model works well on:

laptops,
Raspberry Pi devices,
embedded systems,
and lightweight offline applications.

Why It’s Important

Smaller AI models traditionally struggled with reasoning quality and consistency. Gemma 4 E2B demonstrates how much compact architectures have improved.

Even with minimal hardware, the model can:

summarize notes
answer questions,
assist with coding tasks,
and operate entirely offline.

Recommended Hardware

4–6GB RAM
Low-VRAM GPUs
Apple Silicon devices
Small edge AI hardware

Best Use Cases

Offline AI assistants
Educational applications
AI-powered note summarizers
Smart home automation
Lightweight chatbot systems

Gemma 4 E4B

Why E4B Stands Out

The E4B model is widely considered the sweet spot between:

speed,
reasoning quality,
overall performance,
and hardware efficiency.

For many developers, this is likely the model they’ll use most often.

Key Strengths
E4B performs especially well in:

coding tasks,
reasoning,
long-form conversations,
summarization,
and RAG-based systems.

Recommended Hardware

RTX 3060 / 4060 or better
Apple Silicon Macs
8–12GB VRAM
16GB+ system RAM
Best Use Cases
AI coding assistants
Research applications
Personal AI tools
Local productivity systems
Chat-based applications

Gemma 4 26B-A4B

What Makes It Different?

This model uses a Mixture-of-Experts (MoE) architecture.

Instead of activating the entire neural network for every token, it selectively activates specialized expert layers only when needed.

Why MoE Matters

MoE architectures improve:

efficiency,
scalability,
and inference performance.

Main Advantages

Faster inference
Reduced compute costs
Strong reasoning performance
Better scaling efficiency

Recommended Hardware

RTX 4090
Multi-GPU systems
24–48GB VRAM
High-performance workstations

Best Use Cases

AI agents
Research environments
Advanced coding systems
Long-context workflows
Autonomous AI pipelines

Gemma 4 31B

*The Flagship Model
*
Gemma 4 31B is the most powerful dense model in the family.

It is designed for:

advanced reasoning,
complex instruction handling,
multimodal workflows,
and enterprise-scale AI applications.

Why Dense Models Still Matter

Dense models are often preferred because they provide:

more stable outputs,
strong reasoning capabilities,
and more consistent responses.

The 31B model focuses heavily on maximizing output quality rather than only optimizing efficiency.

Features

256K context window
Multimodal support
Advanced reasoning
Long-form text generation
Strong coding performance

Recommended Hardware

RTX 4090 / A100 / H100
32GB+ VRAM
Quantized inference support
High-end workstation setups

Multimodal Capabilities

Gemma 4 models also support multimodal workflows.

That means they can process:

text,
images,
and audio. Why Multimodal AI Is Important

This opens the door for applications such as:

visual tutoring systems,
image analysis,
accessibility tools,
UI understanding,
and document interpretation.

Running Gemma 4 Locally

One of the biggest reasons Gemma 4 is gaining popularity is how easy it is to run locally. Unlike many large AI systems that require expensive cloud infrastructure, Gemma 4 can operate directly on personal hardware using tools like Ollama.

This allows developers to:

experiment more freely,
avoid API costs,
work offline,
and improve privacy because data stays on the local machine.

Local AI development is becoming increasingly important for:

students learning AI,
independent developers,
researchers,
and startups building prototypes.

Installing Gemma 4 with Ollama

Ollama offers one of the easiest ways to download and run local AI models.

After installing Ollama, you can pull Gemma 4 directly from the terminal.

Install Gemma 4

ollama pull gemma:4b

This command downloads the model weights and prepares the model for local inference.

Depending on your hardware and internet connection, the process may take several minutes.

Running the Model

Once installation is complete, you can start using the model immediately.

ollama run gemma:4b

Ollama will launch an interactive terminal session where you can type prompts directly.

Example:

>>> Explain neural networks in simple words

The model then generates responses locally on your device.

_Using Gemma 4 in Python Applications
_

Gemma 4 can also be integrated into Python applications very easily.

This is useful for:

chat applications,
AI assistants,
research tools,
automation software,
and web applications.

Python Example

from ollama import chat

response = chat(
   model='gemma:4b',
   messages=[
       {
           'role': 'user',
           'content': 'Explain transformers simply'
       }
   ]
)

print(response['message']['content'])

Understanding the Code

Importing the Chat Function

from ollama import chat

This imports Ollama’s chat interface into Python and allows your application to communicate with the local Gemma model.

Sending a Prompt

response = chat(
   model='gemma:4b',
   messages=[
       {
           'role': 'user',
           'content': 'Explain transformers simply'
       }
   ]
)

Here:

model='gemma:4b' selects the model,

role='user' identifies the speaker,

and content contains the prompt being sent.

The structure is very similar to modern chat-based AI APIs.

Printing the Response

print(response['message']['content'])

This extracts the generated text from the response and prints it to the console.

Why Local AI Development Matters

Running Gemma 4 locally changes the development experience in several important ways.

Privacy
Your prompts and data remain on your own machine.

Lower Costs
There are no token-based API fees.

Faster Experimentation
Developers can test ideas immediately without worrying about cloud usage limits.

Offline Access
Once installed, the model can operate without an internet connection.

Final Thoughts

One of Gemma 4’s biggest strengths is its accessibility. Only a few years ago, running advanced AI models required enterprise-grade infrastructure, complex CUDA configurations, and expensive GPUs.

Today, developers can:

download a model,
run it locally,
and build AI-powered applications within minutes.

That level of accessibility is one of the main reasons local AI development is growing so rapidly.

DEV Community: Ayontika-pal

Building an AI Interview Coach with VoiceForBharat: A 10-Day AI voice agent journey

1. Introduction & Problem Statement

Target Audience & Core Pain Points

Why Voice over Text Chatbots?

2. How the System Works: High-Level Architecture & Pipeline

The Real-Time Audio Loop

3. Deep Dive into Key Features Built Across the 10 Days

1. Indian Voice & Cultural Persona (Anisha)

2. Safety Guardrails & Consent Protocols

3. Long-Term Caller Memory & Database Persistence (db.py)

4. Live API Tool Integration (lookup_interview_question)

5. Outbound Phone Calling & SIP Telephony (outbound_call.py)

6. Human Escalation Ticket Workflow (create_escalation)

7. Multi-Agent Specialist Handoff (DSASpecialist / Dev)

8. Privacy-Compliant Call Analytics Dashboard (/dashboard)

4. Difficult Challenges Faced & How They Were Solved

Challenge 1: Audio Overlap & Interruption Latency

Challenge 2: Network Delays During Real-Time Tool Calling

5. Step-by-Step Guide: How to Build and Run Your Own Agent

Step 1: System Prerequisites

Step 2: Environment Configuration

Step 3: Backend Agent Setup & Execution

Step 4: Frontend UI Setup & Execution

Step 5: Test the Voice Agent

🔮 6. Future Enhancements

🔗 7. Code Repository & Links

Gemma 4: models and setup

Gemma 4 Model

What Is Gemma 4?

The Gemma 4 Model Lineup

Running Gemma 4 Locally

1. Indian Voice & Cultural Persona (`Anisha`)

3. Long-Term Caller Memory & Database Persistence (`db.py`)

4. Live API Tool Integration (`lookup_interview_question`)

5. Outbound Phone Calling & SIP Telephony (`outbound_call.py`)

6. Human Escalation Ticket Workflow (`create_escalation`)

7. Multi-Agent Specialist Handoff (`DSASpecialist` / `Dev`)

8. Privacy-Compliant Call Analytics Dashboard (`/dashboard`)