DEV Community

VoiceDeveloper
VoiceDeveloper

Posted on

Create Multilingual Voice Content with ElevenLabs

#javascript #ai #webdev #tutorial

Why Multilingual Voice Matters in 2024

If you’ve ever built an app that reaches users beyond your native tongue, you know the pain of juggling separate audio pipelines for each language. Subtitles are a quick fix, but they don’t capture the warmth of a human voice, and they’re inaccessible for people with visual impairments.

Enter multilingual text‑to‑speech (TTS). Modern TTS can generate natural‑sounding speech in dozens of languages, letting you:

  • Deliver onboarding tutorials in the user’s preferred language.
  • Create localized podcasts or audiobooks without hiring a separate voice actor for each market.
  • Keep brand consistency by using the same cloned voice across languages.

Among the tools that make this possible, ElevenLabs stands out for its high‑fidelity models, easy‑to‑use API, and support for voice cloning. You can start experimenting right away by signing up here: https://try.elevenlabs.io/kr07zfuqn1bp.

Getting Your Hands on the ElevenLabs API

1. Grab an API key

  1. Sign up (or log in) at the link above.
  2. Navigate to Dashboard → API Keys and click Create new key.
  3. Copy the key – you’ll need it in every request.

Security tip: Store the key in an environment variable (ELEVENLABS_API_KEY) rather than hard‑coding it.

2. Install the Python client (optional)

ElevenLabs provides a thin wrapper around the REST endpoints. If you prefer raw HTTP, skip to the curl section.

pip install elevenlabs
Enter fullscreen mode Exit fullscreen mode

Generating Multilingual Speech with Python

Below is a minimal script that takes a piece of text, selects a language, and streams the resulting audio to a file. The same voice (cloned from a short reference recording) is used for every language, preserving brand identity.

import os
import elevenlabs
from elevenlabs import Voice, generate, save

# Load API key from env
elevenlabs.set_api_key(os.getenv("ELEVENLABS_API_KEY"))

# -------------------------------------------------
# 1️⃣  Create or fetch a cloned voice
# -------------------------------------------------
# If you already have a voice ID, skip this step.
# Replace 'my_reference.wav' with a 5‑10 second sample of your brand voice.
voice = Voice.from_file("my_reference.wav", name="BrandVoice")
voice_id = voice.id
print(f"Cloned voice ID: {voice_id}")

# -------------------------------------------------
# 2️⃣  Define multilingual payload
# -------------------------------------------------
texts = {
    "en": "Welcome to our app! Let’s get started.",
    "es": "¡Bienvenido a nuestra aplicación! Vamos a empezar.",
    "de": "Willkommen in unserer App! Lassen Sie uns loslegen."
}

# -------------------------------------------------
# 3️⃣  Generate and save each language version
# -------------------------------------------------
for lang, txt in texts.items():
    audio = generate(
        text=txt,
        voice=voice_id,
        model="eleven_multilingual_v2",  # multilingual model
        language=lang  # ISO‑639‑1 code
    )
    filename = f"welcome_{lang}.mp3"
    save(audio, filename)
    print(f"Saved {filename}")
Enter fullscreen mode Exit fullscreen mode

What’s happening?

  • Voice.from_file uploads your reference audio and returns a voice ID you can reuse forever.
  • The model="eleven_multilingual_v2" flag tells the API to use the multilingual model, which supports over 30 languages.
  • The language parameter accepts standard ISO‑639‑1 codes (en, es, de, etc.).

You can now serve welcome_en.mp3, welcome_es.mp3, and welcome_de.mp3 from the same endpoint, confident that the speaker sounds identical across locales.

The Same Thing with a Simple curl Call

If you’re building a CI pipeline or just want to test quickly, curl works just as well.

# Export your key once
export ELEVENLABS_API_KEY="sk_..."

# 1️⃣  Upload a reference voice (run once)
curl -X POST "https://api.elevenlabs.io/v1/voices/add" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -F "name=BrandVoice" \
  -F "files=@my_reference.wav" \
  -F "description=Brand voice for multilingual content"

# The response contains "voice_id". Save it for later.
Enter fullscreen mode Exit fullscreen mode 
# 2️⃣  Generate Spanish audio
curl -X POST "https://api.elevenlabs.io/v1/text-to-speech/{voice_id}" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "text": "¡Bienvenido a nuestra aplicación! Vamos a empezar.",
        "model_id": "eleven_multilingual_v2",
        "language": "es",
        "voice_settings": {
          "stability": 0.75,
          "similarity_boost": 0.85
        }
      }' \
  --output welcome_es.mp3
Enter fullscreen mode Exit fullscreen mode

Replace {voice_id} with the ID you received from the upload step. Adjust stability and similarity_boost to fine‑tune how “steady” the voice sounds versus how closely it matches your reference.

Voice Cloning Across Languages – Why It Works

You might wonder: How can a voice recorded in English sound natural when speaking Japanese? The secret lies in the model’s phoneme‑level embedding. ElevenLabs extracts a speaker’s timbre independent of language, then re‑applies that timbre to the phonetic sequence of the target language. The result is a voice that retains its characteristic warmth, breathiness, and cadence, no matter whether it’s saying “Hello” or “こんにちは”.

A practical tip: keep your reference recording neutral—avoid heavy accents or region‑specific slang. A clean, studio‑like sample gives the model the purest timbre to work with.

Real‑World Use Cases

Use case How multilingual TTS helps Sample code snippet
In‑app tutorials One voice guides users in every market, reinforcing brand trust. Python loop that pulls translations from a JSON file and generates MP3s.
Podcast localization Produce a single episode in 10+ languages without re‑recording. curl batch script that reads a CSV of language‑text pairs.
Accessibility for e‑learning Students with visual impairments get the same teacher‑like voice in their native language. Node.js server that calls the ElevenLabs API on demand.

Best Practices & Gotchas

✅ Do ❌ Don’t
Use short, high‑quality reference audio (5–10 s, 44.1 kHz, no background noise). Upload a noisy phone call as a reference – the model will amplify the noise.
Cache generated audio files when possible to avoid unnecessary API calls and costs. Generate the same phrase on every request – you’ll hit rate limits fast.
Respect language‑specific punctuation (e.g., commas vs. full‑width commas) to guide prosody. Feed raw markdown or HTML directly – the model will read the tags aloud.
Test the output with native speakers before shipping. Assume the model’s pronunciation is perfect for every dialect.

Pricing Snapshot (as of 2024)

ElevenLabs offers a free tier with a few hundred minutes per month – enough for experimentation. Paid plans unlock higher concurrency, longer audio limits, and priority support. Check the dashboard for the latest rates; the pricing page is linked from the sign‑up portal.

Wrap‑Up

Multilingual voice content used to be a costly, time‑consuming endeavor. With a modern TTS service like ElevenLabs, you can:

  • Clone a single brand voice once and reuse it in any supported language.
  • Generate high‑quality audio on the fly or batch‑process large catalogs.
  • Keep your codebase clean with straightforward REST or Python calls.

Give it a spin today—sign up, upload a quick reference clip, and let the API do the heavy lifting. Your global audience (and your dev timeline) will thank you.

Ready to build multilingual voice experiences? Try ElevenLabs now: https://try.elevenlabs.io/kr07zfuqn1bp. Happy coding!

Top comments (0)