VoiceDeveloper

Posted on Oct 20

Build a Multi-Voice Dialogue Generator

#webdev #tutorial #javascript #ai

Why Multi‑Voice Dialogue Matters

When you’re building conversational apps, interactive stories, or even training data for AI, the human touch is everything. A single, generic voice can feel robotic; a handful of distinct voices turns a script into a living conversation. Whether you’re scripting a chatbot, creating an audiobook, or animating a game, the ability to switch between voices on the fly unlocks richer user experiences.

Today we’ll walk through a practical, end‑to‑end pipeline that lets you:

Define a simple dialogue script (who says what).
Assign each character a unique voice using ElevenLabs’ high‑quality TTS engine.
Generate the audio files and stitch them together into a single, seamless track.

All of this in under an hour, with no deep audio‑engineering background required.

1. Get Your ElevenLabs API Key

ElevenLabs offers a straightforward REST API that lets you synthesize text into speech, retrieve voice models, and even clone new voices. Sign up for a free tier (or paid, if you need higher quality) and grab your API key.

Tip: Store the key in an environment variable so you don’t hard‑code it.

export ELEVENLABS_API_KEY="YOUR_API_KEY_HERE"

(You’ll need this key in every request.)

2. Pick Your Voices

ElevenLabs ships with dozens of pre‑trained voices. If you need something truly unique, you can also clone a voice from a short audio clip. For this tutorial, we’ll use a couple of ready‑made voices:

Character	Voice ID (ElevenLabs)
Alice	`EXAMPLE_ALICE`
Bob	`EXAMPLE_BOB`

You can find the exact IDs in the ElevenLabs dashboard or via the API:

curl -H "xi-api-key: $ELEVENLABS_API_KEY" \
     https://api.elevenlabs.io/v1/voices

Pro tip: Save the JSON response; it’s handy for quick lookups.

3. Build the Dialogue Script

Let’s store our dialogue in a JSON array. Each object contains the speaker’s name and the line of text.

[
  { "speaker": "Alice", "text": "Hey Bob, have you seen my notebook?" },
  { "speaker": "Bob",   "text": "I left it on the kitchen counter." },
  { "speaker": "Alice", "text": "Thanks! I’ll grab it right away." }
]

You can keep this in a file (dialogue.json) or embed it directly in your code.

4. Synthesize Audio in Python

Below is a minimal Python script that:

Loads the dialogue JSON.
Maps each speaker to a voice ID.
Calls ElevenLabs’ TTS endpoint.
Saves each line as a separate WAV file.
Concatenates the WAVs into one final track.

import os
import json
import requests
from pathlib import Path

ELEVENLABS_API_KEY = os.getenv("ELEVENLABS_API_KEY")
BASE_URL = "https://api.elevenlabs.io/v1"

# Map speakers to voice IDs
VOICE_MAP = {
    "Alice": "EXAMPLE_ALICE",
    "Bob":   "EXAMPLE_BOB",
}

# Load dialogue
with open("dialogue.json") as f:
    script = json.load(f)

OUTPUT_DIR = Path("output")
OUTPUT_DIR.mkdir(exist_ok=True)

def synthesize(text, voice_id, filename):
    """Call ElevenLabs to synthesize a line."""
    url = f"{BASE_URL}/text-to-speech/{voice_id}"
    headers = {
        "xi-api-key": ELEVENLABS_API_KEY,
        "Content-Type": "application/json"
    }
    payload = {
        "text": text,
        "voice_settings": {
            "stability": 0.5,
            "similarity_boost": 0.75
        }
    }
    r = requests.post(url, headers=headers, json=payload)
    r.raise_for_status()
    with open(filename, "wb") as f:
        f.write(r.content)

# Synthesize each line
wav_files = []
for i, line in enumerate(script, 1):
    speaker = line["speaker"]
    voice_id = VOICE_MAP[speaker]
    out_path = OUTPUT_DIR / f"{i:02d}_{speaker}.wav"
    synthesize(line["text"], voice_id, out_path)
    wav_files.append(out_path)

# Concatenate WAVs using ffmpeg
ffmpeg_cmd = f"ffmpeg -y -i \"concat:{'|'.join(str(f) for f in wav_files)}\" -acodec copy {OUTPUT_DIR}/final_dialogue.wav"
os.system(ffmpeg_cmd)

print(f"✅ Done! Final dialogue at {OUTPUT_DIR / 'final_dialogue.wav'}")

Note:

ElevenLabs returns raw audio bytes.

The ffmpeg concat protocol requires a text file listing files in the format file 'path'.

If you don’t have ffmpeg, install it (brew install ffmpeg on macOS, apt-get install ffmpeg on Ubuntu).

5. Do the Same in JavaScript (Node.js)

If you prefer JavaScript, here’s the equivalent logic using node-fetch:

const fs = require('fs');
const fetch = require('node-fetch');
const path = require('path');
require('dotenv').config();

const API_KEY = process.env.ELEVENLABS_API_KEY;
const BASE_URL = 'https://api.elevenlabs.io/v1';
const VOICE_MAP = { Alice: 'EXAMPLE_ALICE', Bob: 'EXAMPLE_BOB' };

async function synthesize(text, voiceId, outFile) {
  const url = `${BASE_URL}/text-to-speech/${voiceId}`;
  const res = await fetch(url, {
    method: 'POST',
    headers: {
      'xi-api-key': API_KEY,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      text,
      voice_settings: { stability: 0.5, similarity_boost: 0.75 }
    })
  });
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  const buffer = await res.buffer();
  fs.writeFileSync(outFile, buffer);
}

(async () => {
  const script = JSON.parse(fs.readFileSync('dialogue.json'));
  const outDir = path.join(__dirname, 'output');
  if (!fs.existsSync(outDir)) fs.mkdirSync(outDir);
  const wavFiles = [];

  for (let i = 0; i < script.length; i++) {
    const line = script[i];
    const voiceId = VOICE_MAP[line.speaker];
    const outFile = path.join(outDir, `${String(i+1).padStart(2,'0')}_${line.speaker}.wav`);
    await synthesize(line.text, voiceId, outFile);
    wavFiles.push(outFile);
  }

  // Use ffmpeg to concatenate
  const ffmpeg = require('fluent-ffmpeg');
  ffmpeg()
    .input('concat:' + wavFiles.join('|'))
    .audioCodec('copy')
    .save(path.join(outDir, 'final_dialogue.wav'))
    .on('end', () => console.log('✅ Done!'));
})();

Tip: Install fluent-ffmpeg (npm i fluent-ffmpeg) and make sure ffmpeg is in your PATH.

6. Adding Voice Cloning (Optional)

If you want a truly unique character voice, ElevenLabs lets you clone a voice from a short sample. Upload a 30‑second clip, and the platform will generate a new voice ID that you can use just like any other:

curl -X POST \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -F "file=@my_clip.wav" \
  https://api.elevenlabs.io/v1/voice-cloning

The response will include a voice_id. Plug it into the VOICE_MAP and you’re good to go.

7. Common Pitfalls & Debugging

Problem	Fix
API request fails	Check that `ELEVENLABS_API_KEY` is correct and that your account has enough quota.
WAV files are empty	Verify the `synthesize()` function is writing the response correctly.
ffmpeg concat error	Make sure you’re using the correct concat syntax. For a list of files, create a `files.txt` with `file 'path'` on each line and use `-f concat -safe 0 -i files.txt`.

8. Wrap‑Up & Next Steps

You now have a fully functional multi‑voice dialogue generator that:

Pulls dialogue from a simple JSON file.
Maps each speaker to a distinct voice from ElevenLabs.
Generates and concatenates the audio into one polished track.

What to do next?

Add pauses: Insert small silence chunks between lines for natural breathing.
Dynamic tone: Use ElevenLabs’ voice_settings to adjust pitch or speaking rate per character.
Integrate into a web app: Serve the generated WAV files over HTTP and play them in the browser with <audio>.

Call to Action

Ready to make your scripts sound like real conversations? Sign up for ElevenLabs today and bring your dialogues to life with high‑fidelity, customizable voices.

Try ElevenLabs now and hear the difference.

DEV Community