DEV Community

VoiceDeveloper
VoiceDeveloper

Posted on

Create an Interactive Audio Story Generator

#webdev #javascript #tutorial #ai

Overview

Building an interactive audio story generator is a fun way to combine natural‑language processing, text‑to‑speech (TTS), and voice cloning into a single, immersive experience. Whether you’re creating a choose‑your‑own‑adventure podcast, a dynamic audiobook, or a voice‑controlled game, the core idea is the same: generate spoken narration on the fly from text, optionally using a cloned voice that sounds like a character or a narrator you’ve created.

In this article we’ll walk through a practical, end‑to‑end implementation in Python. We’ll use ElevenLabs for TTS and voice cloning because it offers a straightforward REST API, high‑quality neural voices, and a generous free tier (plus the affiliate link you’ll see below). By the end, you’ll have a reusable script that can take any story prompt, branch based on user input, and play back the result—all in a few lines of code.

Tip: If you prefer JavaScript or want to run the generator in the browser, the same concepts apply. The snippets below can be adapted with fetch or axios.

Prerequisites

Item Why you need it How to get it
Python 3.10+ Modern async support and type hints sudo apt install python3 or download from python.org
Requests library Easy HTTP requests pip install requests
pydub Simple audio playback and manipulation pip install pydub
ffmpeg Required by pydub for MP3 handling sudo apt install ffmpeg
ElevenLabs API key Access to TTS and voice cloning Sign up at https://try.elevenlabs.io/kr07zfuqn1bp

Quick note on the affiliate link:

Every time you use https://try.elevenlabs.io/kr07zfuqn1bp, you’ll receive a small commission that supports this content. Feel free to use it when you sign up—no extra cost to you!

Step 1: Set Up Your ElevenLabs Account

  1. Visit the affiliate link: https://try.elevenlabs.io/kr07zfuqn1bp
  2. Sign up for a free account.
  3. In the dashboard, locate the API Key section and copy the key.
  4. (Optional) Explore the Voice Library and create a cloned voice from a short audio sample. This will let your story narrator sound unique.

Step 2: Create a Basic TTS Wrapper

Below is a simple Python wrapper that sends text to ElevenLabs and returns an MP3 URL. It also handles voice selection, including a cloned voice if you provide its ID.

import requests
import json

ELEVENLABS_API = "https://api.elevenlabs.io/v1"
API_KEY = "YOUR_API_KEY_HERE"  # Replace with your actual key

def synthesize(text: str, voice_id: str = None, output_path: str = "output.mp3") -> str:
    """
    Send a text prompt to ElevenLabs and save the resulting audio.
    """
    headers = {
        "xi-api-key": API_KEY,
        "Content-Type": "application/json",
    }

    payload = {
        "text": text,
        "voice_settings": {
            "stability": 0.75,
            "similarity_boost": 0.75
        }
    }

    if voice_id:
        payload["voice_id"] = voice_id

    response = requests.post(f"{ELEVENLABS_API}/text-to-speech/{voice_id or 'default'}",
                             headers=headers, json=payload)

    if response.status_code != 200:
        raise RuntimeError(f"ElevenLabs error: {response.text}")

    with open(output_path, "wb") as f:
        f.write(response.content)

    return output_path
Enter fullscreen mode Exit fullscreen mode

Why the voice_settings?

Stability controls how “steady” the voice sounds; similarity_boost nudges the model toward the reference voice if you’re using a cloned voice.

Step 3: Build an Interactive Prompt Loop

Let’s create a simple CLI that asks the user for choices and stitches together story segments. We’ll use a minimal branching structure:

def main():
    story_segments = {
        "start": "You stand at the crossroads of a dark forest. Do you go left into the mist or right toward the light?",
        "left": "The mist thickens, and you hear whispers. Do you follow the whispers or retreat?",
        "right": "The light reveals an abandoned cabin. Do you enter or keep walking?",
        "left_follow": "A friendly owl offers you a key. Do you take it?",
        "left_retreat": "You find a hidden path leading home. The adventure ends here.",
        "right_enter": "Inside, you find a treasure chest. Do you open it?",
        "right_keep": "You discover a hidden garden. The adventure ends peacefully."
    }

    current = "start"
    while True:
        print("\n" + story_segments[current] + "\n")
        choice = input("Choose (1/2): ").strip()
        if current == "start":
            current = "left" if choice == "1" else "right"
        elif current == "left":
            current = "left_follow" if choice == "1" else "left_retreat"
        elif current == "right":
            current = "right_enter" if choice == "1" else "right_keep"
        else:
            break

    # Synthesize the final segment
    audio_file = synthesize(story_segments[current], voice_id="YOUR_CLONED_VOICE_ID")
    print(f"Audio saved to {audio_file}")
Enter fullscreen mode Exit fullscreen mode

Tip: Replace "YOUR_CLONED_VOICE_ID" with the ID you got from ElevenLabs. If you don’t have a cloned voice, simply omit the argument and the default voice will be used.

Step 4: Play Back the Audio

We can play the MP3 using pydub and simpleaudio. Install if you haven’t:

pip install pydub simpleaudio
Enter fullscreen mode Exit fullscreen mode

Add the following to the end of main():

from pydub import AudioSegment
import simpleaudio as sa

def play_audio(file_path):
    song = AudioSegment.from_mp3(file_path)
    play_obj = sa.play_buffer(
        song.raw_data,
        num_channels=song.channels,
        bytes_per_sample=song.sample_width,
        sample_rate=song.frame_rate
    )
    play_obj.wait_done()

# After synthesize
play_audio(audio_file)
Enter fullscreen mode Exit fullscreen mode

Now the script will synthesize the chosen story segment and immediately play it back.

Step 5: Expand the Story – Adding Voice Cloning

If you want each character to have a distinct voice, you can create multiple cloned voices in ElevenLabs and map them to story segments. Example:

voice_map = {
    "start": "default",          # Narrator
    "left": "cloned_owl",
    "right": "cloned_cabin",
    "left_follow": "cloned_owl",
    "left_retreat": "default",
    "right_enter": "cloned_cabin",
    "right_keep": "default"
}

# In synthesize call:
audio_file = synthesize(story_segments[current], voice_id=voice_map[current])
Enter fullscreen mode Exit fullscreen mode

Remember to replace "cloned_owl" and "cloned_cabin" with the actual voice IDs you generated.

Step 6: Deploy as a Web Service (Optional)

If you want to expose this generator as an API or a web page, you can wrap the logic in FastAPI:

pip install fastapi uvicorn
Enter fullscreen mode Exit fullscreen mode 
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

class StoryRequest(BaseModel):
    choices: list[int]  # e.g., [1, 2, 1]

@app.post("/generate")
async def generate_story(req: StoryRequest):
    # Use the same logic as above to traverse the story
    # Return the audio URL or base64 string
    ...
Enter fullscreen mode Exit fullscreen mode

Run it with:

uvicorn script:app --reload
Enter fullscreen mode Exit fullscreen mode

Now you can build a front‑end that sends the user’s choices and streams the generated audio.

Best Practices & Tips

Practice Why it matters
Cache generated audio Avoid re‑synthesizing the same segment on every request. Store MP3s locally or in cloud storage.
Use asynchronous requests ElevenLabs can handle multiple synth requests in parallel. Use aiohttp or async requests.
Fine‑tune voice settings Experiment with stability and similarity boost to match the tone of your story.
Add pauses Insert \n\n or \n in the text to create natural breathing pauses.
Localize ElevenLabs supports multiple languages—use the language parameter if you want multilingual stories.

Wrap‑Up

You now have a fully functional, interactive audio story generator that leverages ElevenLabs’ high‑quality TTS and voice cloning. From a simple CLI to a web service, the core building blocks are the same:

  1. Collect user choices → 2. Determine the story segment → 3. Send text to ElevenLabs → 4. Play or stream the audio.

Feel free to extend the branching logic, add more characters, or even integrate a natural language model to generate new story content on the fly. The possibilities are endless!

Call‑to‑Action

Ready to bring your stories to life with crystal‑clear, personalized voices? Sign up at https://try.elevenlabs.io/kr07zfuqn1bp and start building your next audio adventure today!

Top comments (0)