This guide provides instructions for building and integrating a custom AI voice agent using the MirrorFly AI-RAG dashboard and SDK into your platform.
The solution supports real-time audio streaming, a workflow builder, and an external knowledge base training (Retrieval-Augmented Generation). As a result, you can ensure it aligns with the brand and offers context-aware user experiences.
Overview
We have divided the implementation steps into two sections:
- Building Custom AI Voice Agent: Configure the agent's personality, train it with datasets, and design conversational flows in the MirrorFly dashboard.
- Voice Agent Integration: Embed the agent into your web app using the MirrorFly AI SDK.
Section I: 07 Steps to Building A Custom AI Voice Agent
1. Initial Setup
- **Get Access: **You have to get the developer credentials from the MirrorFly team and log in to the MirrorFly AI Dashboard.
Agent Creation: Click 'Create Agents' and then select 'Voice Agent'.
Configuration: Now, enter an agent name, add a description, and define the initial System Prompt to set the voice agent's core behavior.
2. Personality & Model Settings
- Personality: Set the welcome message, fallback responses, and adjust the formality and tone.
- Model Selection: Choose from multiple LLM models to enhance your agent. There is no single vendor lock-in.
3. RAG Training
- Datasets: Upload knowledge bases in PDF or CSV format (Maximum file size is 5MB/10 files) for a domain-specific agent.
- Website Sync: Alternatively, you can also sync information directly from a URL to keep the agent updated with your website.
4. Workflow Builder
Use the visual drag-and-drop canvas to define the overall conversation logic. What you can define are:
- Conversational paths and decision logic.
- API calls and form collection.
- Email triggers and message nodes.
5. Speech & Functions
- STT/TTS: Proceed with configuring Speech-to-Text and Text-to-Speech providers with your API keys. Deepgram, ElevenLabs, and custom modals are available.
- Call Handling: Enable 'end call' and 'transfer to human' functions (via SIP or Conference). Adjust the "interruption sensitivity" of how AI can interrupt human speech.
- Outbound Calling: Additionally, you can configure outbound call capabilities. You can select a single call for individual call initiation or batch calls by uploading a CSV file with contact records. You have the option to enable Post-Call Analysis.
6. Custom Tools
- Webhook: Configure webhooks for sending real-time event notifications to the app's backend for automation.
-
MCP Server: Manage external tools and APIs using MCP servers (connect Gmail, Google Calendar, Google Drive, and more). This enables actions such as email triggers, meeting scheduling, file access, and other event flows within the conversation.
7. Customize Widget
Brand Personalization & Live Preview: Here, upload chatbot avatar image, set the agent name, define primary theme colors, and message bubble colors. Test the widget’s appearance and interaction flow before deploying it to production.
Section II: Agent Integration
Prerequisites
- Valid Agent ID from the dashboard.
- Make sure the website runs on HTTPS for microphone access.
- Browsers: Chrome, Edge, and Safari. (latest version)
1. Install the SDK
Add the SDK to your HTML file using a single script tag:
<script src="https://d1nzh49hhug3.cloudfront.net/aiVoiceScript/uat/mirrofly/mirror-fly-ai.v1.1.1.js"></script>
2. Initialize the Agent
Define a container element and initialize the SDK:
`// HTML Container
// Initialization
MirrorFlyAi.init({
container: "#widget",
agentId: "",
title: "Voice Assistant",
theme: "dark",
triggerStartCall: true,
transcriptionEnable: true,
transcriptionInUi: true,
chatEnable: true,
agentConnectionTimeout: 500
});`
3. Handle Callbacks
Monitor the agent's status and capture transcriptions:
const callbacks = {
onTranscription: (data) => console.log("Transcription:", data),
onAgentConnectionState: (state) => console.log("Connection:", state),
onError: (error) => console.error("SDK Error:", error)
};
4. Dynamic Agent Switching
If your platform uses multiple agents (e.g., Sales vs. Support), use the following to switch contexts:
function switchAgent(newAgentId) {
MirrorFlyAi.destroy();
document.querySelector("#widget").innerHTML = "";
MirrorFlyAi.init({
container: "#widget",
agentId: newAgentId,
triggerStartCall: true
});
}
5. Security & Permissions
Now the browser will prompt the user for microphone permission once initialized. Ensure you handle potential errors via the onError callback (permission denials or network issues).
Conclusion
In this guide, we have demonstrated how to build, train, and deploy a custom AI Voice Agent using MirrorFly’s AI-RAG dashboard and SDK.
You get full control over behavior, integrations, and deployment through RAG-based knowledge integration, real-time voice streaming, SDK embedding, and workflow orchestration.
You can extend the voice agent’s capabilities using webhooks, MCP tools, and external APIs without modifying the core voice infrastructure.
The solution allows you to adapt multiple business workflows, such as support, sales, or internal automation. It supports dynamic agent switching, transcription handling, and call control features.
You can ensure predictable performance across environments by properly configuring datasets, speech providers, and SDK callbacks.
Why wait? Start building your own AI voice agents on MirrorFly now. Use our GitHub custom AI voice agent repository and have your first agent running in hours.
Top comments (0)