Adding AI Voice to Discord Bots

Why Voice Matters for Discord Bots

If you’ve built a Discord bot before, you already know how powerful text commands and reactions can be. But imagine a bot that can talk back to your server—answering questions, announcing events, or even just saying “Hey, you’re doing great!” in a natural‑sounding voice. That’s the magic of AI‑powered text‑to‑speech (TTS) and voice cloning, and it’s becoming surprisingly accessible thanks to modern cloud APIs.

In this post I’ll walk you through:

• The core concepts behind TTS and voice cloning
• Why ElevenLabs is a great fit for Discord bots
• How to hook up an ElevenLabs TTS endpoint to a Discord bot in Python
• Tips for handling latency, voice channels, and user permissions

Let’s dive in.

1. Text‑to‑Speech vs Voice Cloning

Feature	TTS (Text‑to‑Speech)	Voice Cloning
Input	Plain text	Text + a few voice samples
Output	Synthetic voice (pre‑configured)	Synthetic voice that mimics a real person
Use‑case	Quick, generic announcements	Personalized bots (e.g., “Your bot says like your favorite streamer”)
Complexity	Low	High (requires training data)
Latency	Usually < 1s	Can be higher due to model size

Most Discord bots don’t need full voice cloning unless you’re building a niche experience. For most projects, a high‑quality TTS engine is sufficient and far easier to implement.

2. Why ElevenLabs?

ElevenLabs offers an API that delivers near‑real‑time, expressive voices. Their models support dozens of languages, and the voice quality is on par with many premium TTS providers. What’s more, the API is straightforward to call from any language that can make HTTP requests.

Key benefits for Discord bots:

• Low latency – the API streams audio in real time, so you can play it instantly in a voice channel.
• Customizable voice – tweak pitch, speaking rate, and emotion to match your bot’s personality.
• No heavy local dependencies – everything runs in the cloud; you just send a request and get back an MP3 or raw PCM stream.

If you’re ready to give your bot a voice, give ElevenLabs a try at https://try.elevenlabs.io/kr07zfuqn1bp. The link grants you a free trial that’s perfect for experimentation.

3. Prerequisites

• A Discord bot token (with “Bot” intent and “Voice” permissions)
• Python 3.8+ (or Node.js if you prefer)
• discord.py (for Python) or discord.js (for JavaScript)
• aiohttp (Python async HTTP client)
• An ElevenLabs API key (obtainable via the link above)

Tip: Store your tokens in environment variables or a .env file to keep them secret.

4. Building the Bot (Python Example)

Below is a minimal example that:

1. Connects to Discord.
2. Listens for the !speak command.
3. Calls ElevenLabs to synthesize the text.
4. Joins the user’s voice channel and plays the audio.

import os
import discord
import aiohttp
import asyncio
from dotenv import load_dotenv

load_dotenv()

TOKEN = os.getenv("DISCORD_BOT_TOKEN")
ELEVENLABS_API_KEY = os.getenv("ELEVENLABS_API_KEY")
ELEVENLABS_ENDPOINT = "https://api.elevenlabs.io/v1/text-to-speech/eleven_monolingual_v1"

intents = discord.Intents.default()
intents.message_content = True

client = discord.Client(intents=intents)

async def synthesize(text: str) -> bytes:
    """
    Call ElevenLabs TTS API and return raw MP3 bytes.
    """
    headers = {
        "xi-api-key": ELEVENLABS_API_KEY,
        "Content-Type": "application/json",
    }
    payload = {
        "text": text,
        "voice_settings": {
            "stability": 0.5,
            "similarity_boost": 0.75,
            "style": 0.5,
            "use_speaker_boost": False
        }
    }
    async with aiohttp.ClientSession() as session:
        async with session.post(ELEVENLABS_ENDPOINT, json=payload, headers=headers) as resp:
            if resp.status != 200:
                raise Exception(f"ElevenLabs error: {resp.status}")
            return await resp.read()

@client.event
async def on_message(message):
    if message.author.bot:
        return

    if message.content.startswith("!speak "):
        text = message.content[len("!speak "):]

        # Find the voice channel the user is in
        if not message.author.voice:
            await message.channel.send("You’re not connected to a voice channel!")
            return
        vc = message.author.voice.channel

        # Connect to voice channel if not already
        if not vc.guild.me.voice:
            await vc.connect()

        # Synthesize audio
        try:
            audio_bytes = await synthesize(text)
        except Exception as e:
            await message.channel.send(f"Error: {e}")
            return

        # Create an audio source
        source = discord.FFmpegPCMAudio(io.BytesIO(audio_bytes))
        vc.play(source)

        await message.channel.send(f"Playing: {text}")

client.run(TOKEN)

What’s Happening?

1. synthesize – Sends a JSON payload to ElevenLabs and receives an MP3. The voice_settings can be tweaked to adjust the bot’s tone.
2. Voice Channel Handling – The bot checks if the user is in a channel and joins if necessary.
3. Playing Audio – FFmpegPCMAudio streams the MP3 directly to Discord. No disk writes!

5. Handling Latency and Streaming

When you call the ElevenLabs API, the response can take ~500 ms to 1 s depending on load. In a busy server, this can add up. Two common strategies:

1. Pre‑fetch – Keep a small queue of “ready‑to‑play” audio chunks. For example, cache the last 10 messages.
2. Streaming – ElevenLabs supports streaming audio in chunks. You can pipe the HTTP response straight into FFmpeg without buffering the entire file.

Below is a quick snippet using asyncio.StreamReader to stream directly:

async def stream_to_ffmpeg(audio_url: str, vc: discord.VoiceClient):
    async with aiohttp.ClientSession() as session:
        async with session.get(audio_url) as resp:
            stream = resp.content
            ffmpeg = await asyncio.create_subprocess_exec(
                "ffmpeg",
                "-i", "pipe:0",
                "-f", "s16le",
                "-ar", "48000",
                "-ac", "2",
                "-",
                stdin=asyncio.subprocess.PIPE,
                stdout=asyncio.subprocess.PIPE
            )
            while True:
                chunk = await stream.read(1024)
                if not chunk:
                    break
                ffmpeg.stdin.write(chunk)
                await ffmpeg.stdin.drain()
            await ffmpeg.communicate()
            vc.play(discord.FFmpegPCMAudio("pipe:0"))

6. Voice Cloning (Optional)

If you’re adventurous, ElevenLabs also offers voice cloning. The workflow is:

1. Upload 10–20 seconds of a target voice (audio file or URL).
2. Receive a voice_id.
3. Use that voice_id in your TTS request.

{
  "voice_id": "your-voice-id",
  "text": "Hello, world!"
}

Because cloning requires more compute, it’s best for bots that need a distinctive persona (e.g., a bot that impersonates a popular streamer). Keep in mind the ethical considerations: always have consent from the person whose voice you’re cloning.

7. Security & Rate‑Limiting

• Rate limits – ElevenLabs limits requests to 200 per minute (free tier). For heavy usage, consider caching or batching.
• Discord permissions – Only let the bot join channels you trust. Use the guild_permissions flag to restrict its voice channel access.
• Token safety – Never commit your API keys to version control. Use environment variables or secret managers.

8. Testing and Tweaking

• Voice Settings – Play with stability, similarity_boost, and style. Lower stability can make the voice more natural but also more unpredictable.
• Language – ElevenLabs supports multiple languages. Make sure to set the correct language code in the payload.
• Volume – Discord’s volume slider can adjust the bot’s output. If you find the audio too loud, set volume=0.5 when creating the FFmpegPCMAudio source.

9. Common Pitfalls

Symptom	Cause	Fix
Bot says “I’m sorry, I couldn’t understand that.”	ElevenLabs returned an error	Check API key, payload JSON, and network connectivity
Audio cuts off mid‑sentence	Network hiccup or too slow	Add retries or use streaming approach
Bot joins but doesn’t speak	No voice channel permission	Verify bot’s role has “Connect” and “Speak” permissions

10. Going Beyond

Once you have a working TTS bot, consider adding:

• Voice commands – Use speech‑recognition APIs (e.g., Whisper) to let users talk to the bot.
• Dynamic responses – Combine the bot’s voice with AI-generated content (ChatGPT + TTS).
• Multi‑channel playback – Let the bot broadcast to multiple voice channels simultaneously.

Call to Action

Ready to make your Discord bot sound like a human? Start experimenting with ElevenLabs today and unlock high‑quality TTS for free. Grab your trial at https://try.elevenlabs.io/kr07zfuqn1bp, integrate the API, and watch your bot’s conversations come to life. Happy coding!