VoiceDeveloper

Posted on Nov 12

Build a Voice Cloning Demo App

#webdev #javascript #tutorial #ai

Why Voice Cloning Is the Next Big Thing

If you’ve been playing with text‑to‑speech (TTS) APIs for the past few years, you’ve probably noticed a shift from robotic “read‑aloud” voices to something that actually sounds like a real person. Voice cloning takes this a step further: you feed a handful of recordings, and the model learns to generate speech that mimics the speaker’s tone, cadence, and even their favorite filler words.

Why does this matter for developers?

Personalized assistants – give each user a voice that feels uniquely theirs.
Content creation – turn blog posts or podcasts into audio without hiring a voice actor.
Accessibility – let people with speech impairments hear their own voice read text back to them.

In this article we’ll build a minimal demo app that lets you upload a short audio sample, generate a voice fingerprint, and then synthesize arbitrary text with that cloned voice. The heavy lifting is done by ElevenLabs, a cloud service that provides state‑of‑the‑art voice cloning with a simple REST API. (You can sign up via this affiliate link: https://try.elevenlabs.io/kr07zfuqn1bp – it’s free to start.)

Overview of the Architecture

┌───────────────┐      ┌─────────────────────┐
│   Front‑end   │ ◀──▶ │   Backend (Flask)   │
│  (HTML/JS)    │      │  - Upload audio     │
│               │      │  - Call ElevenLabs │
└──���────────────┘      └─────────────────────┘

Front‑end – A tiny HTML page with a file input and a text box. When you click Synthesize, it sends a multipart/form-data POST to the backend.
Backend – A Flask server that:
1. Saves the uploaded sample.
2. Calls ElevenLabs to create a voice clone (if it doesn’t already exist).
3. Sends the desired text to the TTS endpoint and streams back the MP3.

All the heavy AI work stays in the cloud, so the demo runs locally on any laptop.

Getting Started – Prerequisites

Python 3.9+
pip install flask requests python-dotenv
An ElevenLabs API key (grab one from the dashboard after signing up via the link above).
(Optional) ffmpeg if you want to convert the MP3 to WAV for playback in the browser.

Create a .env file in the project root:

ELEVENLABS_API_KEY=your_api_key_here

Never commit this file to Git – it contains secrets.

Step 1 – The Flask Backend

Create app.py:

import os
import uuid
from flask import Flask, request, send_file, jsonify
import requests
from dotenv import load_dotenv

load_dotenv()
API_KEY = os.getenv("ELEVENLABS_API_KEY")
BASE_URL = "https://api.elevenlabs.io/v1"

app = Flask(__name__)
UPLOAD_FOLDER = "uploads"
os.makedirs(UPLOAD_FOLDER, exist_ok=True)

def create_voice_clone(sample_path, voice_name="my_clone"):
    """Upload a short sample and ask ElevenLabs to create a clone."""
    url = f"{BASE_URL}/voices/add"
    headers = {"xi-api-key": API_KEY}
    files = {
        "sample_file": open(sample_path, "rb"),
        "voice_name": (None, voice_name),
        "description": (None, "Demo clone created via API"),
    }
    resp = requests.post(url, headers=headers, files=files)
    resp.raise_for_status()
    return resp.json()["voice_id"]

def synthesize(text, voice_id):
    """Generate speech from text using the cloned voice."""
    url = f"{BASE_URL}/text-to-speech/{voice_id}"
    headers = {
        "xi-api-key": API_KEY,
        "Content-Type": "application/json",
        "Accept": "audio/mpeg",
    }
    payload = {
        "text": text,
        "model_id": "eleven_multilingual_v2",
        "voice_settings": {"stability": 0.75, "similarity_boost": 0.85},
    }
    resp = requests.post(url, json=payload, headers=headers, stream=True)
    resp.raise_for_status()
    return resp.raw

@app.route("/clone-and-speak", methods=["POST"])
def clone_and_speak():
    # 1️⃣ Grab uploaded files
    sample = request.files.get("sample")
    text = request.form.get("text")
    if not sample or not text:
        return jsonify({"error": "Missing sample or text"}), 400

    # 2️⃣ Save sample locally
    sample_path = os.path.join(UPLOAD_FOLDER, f"{uuid.uuid4()}.wav")
    sample.save(sample_path)

    # 3️⃣ Create (or reuse) a voice clone
    voice_id = create_voice_clone(sample_path, voice_name="demo_clone")

    # 4️⃣ Synthesize the requested text
    audio_stream = synthesize(text, voice_id)

    # 5️⃣ Return the MP3 to the browser
    return send_file(
        audio_stream,
        mimetype="audio/mpeg",
        as_attachment=False,
        download_name="output.mp3",
    )

if __name__ == "__main__":
    app.run(debug=True)

What’s happening?

create_voice_clone calls ElevenLabs’ /voices/add endpoint. The service expects a short (10‑30 s) audio sample.
synthesize sends the text to /text-to-speech/{voice_id} and streams back an MP3.
The route /clone-and-speak ties everything together, handling the multipart request from the front‑end.

Tip: ElevenLabs caches voice IDs for the same voice_name. If you run the demo repeatedly with the same name, the service will reuse the existing clone, saving you API quota.

Step 2 – A Minimal Front‑End

Create index.html in the same folder:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Voice Cloning Demo</title>
  <style>
    body {font-family: Arial, sans-serif; margin: 2rem;}
    input, textarea {display: block; margin: 1rem 0; width: 100%;}
    button {padding: .5rem 1rem;}
  </style>
</head>
<body>
  <h1>Try Voice Cloning</h1>
  <form id="cloneForm">
    <label>Upload a short voice sample (WAV/MP3, ≤30 s):</label>
    <input type="file" name="sample" accept="audio/*" required />
    <label>Text you want spoken:</label>
    <textarea name="text" rows="4" required>Hello, world! This is a demo.</textarea>
    <button type="submit">Synthesize</button>
  </form>

  <h2>Result</h2>
  <audio id="player" controls></audio>

  <script>
    const form = document.getElementById('cloneForm');
    const player = document.getElementById('player');

    form.addEventListener('submit', async e => {
      e.preventDefault();
      const data = new FormData(form);
      const resp = await fetch('/clone-and-speak', {
        method: 'POST',
        body: data,
      });

      if (!resp.ok) {
        const err = await resp.json();
        alert('Error: ' + err.error);
        return;
      }

      const blob = await resp.blob();
      const url = URL.createObjectURL(blob);
      player.src = url;
    });
  </script>
</body>
</html>

Run the app:

export FLASK_APP=app.py
flask run

Visit http://127.0.0.1:5000 in your browser, upload a 10‑second clip of your own voice, type any sentence, and hit Synthesize. Within a couple of seconds you’ll hear the cloned voice reading your text.

Step 3 – Testing with cURL (Optional)

If you prefer a quick sanity check from the terminal, you can bypass the UI:

curl -X POST http://127.0.0.1:5000/clone-and-speak \
  -F "sample=@my_voice_sample.wav" \
  -F "text=Hello from the command line!" \
  --output result.mp3

Play result.mp3 with your favorite player to confirm the output.

Tweaking Voice Settings

ElevenLabs exposes a handful of parameters that let you control how “stable” the voice sounds and how closely it matches the original speaker:

Parameter	Range	What It Does
`stability`	0‑1	Smoother, more predictable output (higher = less variance).
`similarity_boost`	0‑1	Pushes the synthesis toward the cloned voice (higher = more likeness).

You can experiment by editing the voice_settings object in the synthesize function. For a more expressive narration you might lower stability and raise similarity_boost.

Deploying the Demo

Because the backend is just Flask, you can push it to any platform that supports Python—Heroku, Render, Fly.io, or even a cheap VPS. Just remember to set the ELEVENLABS_API_KEY environment variable in your hosting dashboard.

If you go serverless, you could replace the Flask app with an AWS Lambda + API Gateway pair. The request/response shape stays the same; you only need to adjust the file‑handling logic (e.g., store the sample in S3 temporarily).

Where to Go From Here

Batch synthesis – loop over a CSV of sentences and generate an entire audiobook in minutes.
Realtime streaming – use WebSockets to pipe the MP3 chunks directly to the browser as they arrive.
Multi‑voice UI – let users pick from a gallery of pre‑cloned personalities (celebrity, cartoon, etc.).

All of these extensions build on the same core API calls you just saw.

Give It a Spin!

The magic of voice cloning is only a few lines of code away. By leveraging ElevenLabs’ powerful API (sign up through this link: https://try.elevenlabs.io/kr07zfuqn1bp), you can turn any short recording into a fully fledged TTS voice in seconds. Clone your own voice, generate podcasts, or add a personal touch to your chatbot—whatever you build, the possibilities are huge.

Ready to experiment? Grab your free ElevenLabs API key, run the demo locally, and start building the next generation of voice‑first experiences today. Happy coding!

DEV Community