How to Build a Voice-Over Tool for YouTube Videos

Why Build a Voice‑Over Tool?

If you’re a content creator, podcaster, or just a developer who loves automating the mundane, a voice‑over tool can save you hours of work. Instead of recording every narration manually, you can:

1. Generate spoken content from text – perfect for scripts, subtitles, or quick demos.
2. Clone your own voice – give your brand a consistent tone without re‑recording.
3. Add dynamic audio – change the narration on the fly for different regions, languages, or moods.

The good news is that the technology has become mature enough to be production‑ready. In this post, we’ll walk through building a minimal yet powerful voice‑over pipeline using the ElevenLabs API. By the end, you’ll have a script that pulls a YouTube transcript, turns it into speech, and stitches the audio back into a video.

Pro tip: ElevenLabs offers a free tier with 50,000 characters per month. Sign up at https://try.elevenlabs.io/kr07zfuqn1bp to get started and unlock higher‑quality models.

---

1. Grab the Video Transcript

The first step is to get the raw text. YouTube exposes captions through its API, but the easiest route for a quick prototype is the yt-dlp command‑line tool. It can download the transcript as a simple JSON file.

yt-dlp --skip-download --write-auto-sub --sub-lang en --sub-format json --output "%(title)s.%(ext)s" https://www.youtube.com/watch?v=YOUR_VIDEO_ID

This command will produce a file like MyVideo.json that contains an array of subtitle entries, each with a text field.

import json

with open("MyVideo.json", "r", encoding="utf-8") as f:
    subs = json.load(f)

# Flatten the transcript
transcript = " ".join([entry["text"] for entry in subs])

Now you have a plain string of the entire video’s spoken content.

---

2. Choose a Voice Model

ElevenLabs provides a catalog of high‑fidelity voice models. If you want to clone your own voice, you’ll need to upload a short (30‑second) sample and let the service train a custom model. For most projects, the pre‑built “English – Male / Female” voices are sufficient.

VOICE_ID = "21m00Tcm4TlvDq8ikWAM"  # Example: “Rachel”

Tip: Use the official ElevenLabs documentation to list available voices: curl https://api.elevenlabs.io/v1/voices.

---

3. Generate Speech with ElevenLabs

The core of our pipeline is a simple POST request to ElevenLabs’ /v1/text-to-speech/{voice_id} endpoint. Below is a Python wrapper that handles authentication, chunking, and streaming back the MP3 data.

import requests

API_KEY = "YOUR_ELEVENLABS_API_KEY"
BASE_URL = "https://api.elevenlabs.io/v1"

def text_to_speech(text: str, voice_id: str = VOICE_ID) -> bytes:
    headers = {
        "xi-api-key": API_KEY,
        "Content-Type": "application/json",
    }
    payload = {
        "text": text,
        "voice_settings": {
            "stability": 0.5,
            "similarity_boost": 0.75
        },
    }
    response = requests.post(
        f"{BASE_URL}/text-to-speech/{voice_id}",
        headers=headers,
        json=payload,
        stream=True
    )
    response.raise_for_status()
    # The API streams MP3; we’ll just read all bytes
    return response.content

audio_bytes = text_to_speech(transcript)
with open("output.mp3", "wb") as f:
    f.write(audio_bytes)

Chunking the text

ElevenLabs limits request size to 5,000 characters. If your transcript is longer, split it:

def chunk_text(text, size=5000):
    return [text[i:i+size] for i in range(0, len(text), size)]

chunks = chunk_text(transcript)
audio_parts = [text_to_speech(chunk) for chunk in chunks]

# Concatenate MP3 parts
with open("full_output.mp3", "wb") as f:
    for part in audio_parts:
        f.write(part)

---

4. Sync Audio Back to Video

Once you have the MP3, you can overlay it onto the original video. ffmpeg is the de‑facto tool for this.

ffmpeg -i original.mp4 -i full_output.mp3 -c:v copy -c:a aac -strict experimental -map 0:v:0 -map 1:a:0 output_with_voice.mp4

• -c:v copy preserves the original video stream.
• -c:a aac re‑encodes the audio to AAC, the format most players expect.
• -map ensures we keep the video from the first input and the audio from the second.

Run this command from within a Python script if you want full automation:

import subprocess

subprocess.run([
    "ffmpeg",
    "-i", "original.mp4",
    "-i", "full_output.mp3",
    "-c:v", "copy",
    "-c:a", "aac",
    "-strict", "experimental",
    "-map", "0:v:0",
    "-map", "1:a:0",
    "final_video.mp4"
])

---

5. Automate the Full Pipeline

Putting it all together, here’s a minimal command‑line tool:

python voice_over.py --video https://www.youtube.com/watch?v=YOUR_VIDEO_ID --voice 21m00Tcm4TlvDq8ikWAM

# voice_over.py
import argparse
import json
import os
import subprocess
import requests

API_KEY = "YOUR_ELEVENLABS_API_KEY"
BASE_URL = "https://api.elevenlabs.io/v1"

def download_transcript(url: str) -> str:
    # Use yt-dlp to fetch auto‑captions
    cmd = [
        "yt-dlp",
        "--skip-download",
        "--write-auto-sub",
        "--sub-lang", "en",
        "--sub-format", "json",
        "--output", "%(title)s.%(ext)s",
        url,
    ]
    subprocess.run(cmd, check=True)
    json_file = os.path.basename(url).split('/')[-1].replace('watch?v=', '') + ".json"
    with open(json_file, "r", encoding="utf-8") as f:
        subs = json.load(f)
    return " ".join([entry["text"] for entry in subs])

def chunk_text(text, size=5000):
    return [text[i:i+size] for i in range(0, len(text), size)]

def text_to_speech(text: str, voice_id: str) -> bytes:
    headers = {
        "xi-api-key": API_KEY,
        "Content-Type": "application/json",
    }
    payload = {
        "text": text,
        "voice_settings": {
            "stability": 0.5,
            "similarity_boost": 0.75
        },
    }
    response = requests.post(
        f"{BASE_URL}/text-to-speech/{voice_id}",
        headers=headers,
        json=payload,
        stream=True
    )
    response.raise_for_status()
    return response.content

def main(video_url, voice_id):
    transcript = download_transcript(video_url)
    chunks = chunk_text(transcript)
    audio_parts = [text_to_speech(chunk, voice_id) for chunk in chunks]
    with open("full_output.mp3", "wb") as f:
        for part in audio_parts:
            f.write(part)
    # Get original video filename
    original_video = "original.mp4"  # rename or pass as param
    subprocess.run([
        "ffmpeg",
        "-i", original_video,
        "-i", "full_output.mp3",
        "-c:v", "copy",
        "-c:a", "aac",
        "-strict", "experimental",
        "-map", "0:v:0",
        "-map", "1:a:0",
        "final_video.mp4"
    ])

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--video", required=True, help="YouTube video URL")
    parser.add_argument("--voice", default="21m00Tcm4TlvDq8ikWAM", help="ElevenLabs voice ID")
    args = parser.parse_args()
    main(args.video, args.voice)

Feel free to tweak the voice settings or add error handling. The core idea is: fetch text → synthesize audio → overlay audio.

---

6. Common Pitfalls & How to Avoid Them

Issue	Fix
API rate limits	Cache generated MP3s locally; reuse them for the same transcript.
Chunk overlap	Add a short pause (" " * 5) between chunks to prevent abrupt cuts.
Video‑audio sync drift	Use -shortest flag in ffmpeg to cut the video when audio ends.
Audio quality	Experiment with voice_settings values (stability, similarity_boost).

---

7. Going Beyond

• Multi‑language support: ElevenLabs offers voice models for many languages; just change the voice_id.
• Dynamic content: Build a web UI where users paste text and instantly download the MP3.
• Voice cloning: Upload a short voice sample to ElevenLabs’ custom model endpoint and replace voice_id with your cloned model ID.

---

Call to Action

Ready to add a professional‑sounding voice to your YouTube videos without the studio? Sign up for ElevenLabs today and start building your own voice‑over tool. Get a free tier, try the API, and experiment with different voices—all at https://try.elevenlabs.io/kr07zfuqn1bp.

Happy coding, and may your videos sound as good as they look!