DEV Community

VoiceDeveloper
VoiceDeveloper

Posted on

Create an AI Voice Assistant with ElevenLabs and Node.js

Overview

If you’ve ever wanted to give your apps a personality that can actually talk back, you’re in the right place. In this tutorial we’ll build a simple AI voice assistant using Node.js and the powerful text‑to‑speech (TTS) engine from ElevenLabs. The service not only produces natural‑sounding speech, it also supports voice cloning, so you can create a custom “brand voice” in minutes.

By the end of the guide you’ll have:

  1. A Node.js server that accepts a text prompt via HTTP.
  2. A call to ElevenLabs’ TTS API that returns a high‑quality audio file.
  3. Playback of the generated audio directly from the server (or saved to disk).

Let’s get coding!


Prerequisites

  • Node.js (v18+ recommended) and npm installed.
  • A free ElevenLabs account – you’ll need an API key. Grab yours at the affiliate link: https://try.elevenlabs.io/kr07zfuqn1bp.
  • Basic familiarity with Express (or any HTTP framework) and async/await.

Setting Up the Project

# Create a new folder and initialise npm
mkdir ai-voice-assistant && cd $_
npm init -y

# Install dependencies
npm install express node-fetch@2 @ffmpeg-installer/ffmpeg fluent-ffmpeg
Enter fullscreen mode Exit fullscreen mode
  • express – lightweight HTTP server.
  • node-fetch – makes it easy to call the ElevenLabs REST API.
  • fluent-ffmpeg – converts the raw MP3 stream into a format we can play or save.

Create an index.js file and add a minimal Express server:

// index.js
import express from 'express';
import fetch from 'node-fetch';
import ffmpeg from 'fluent-ffmpeg';
import ffmpegPath from '@ffmpeg-installer/ffmpeg';
import { pipeline } from 'stream';
import { promisify } from 'util';
import fs from 'fs';

ffmpeg.setFfmpegPath(ffmpegPath.path);
const pipe = promisify(pipeline);
const app = express();

app.use(express.json());

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`🚀 Server listening on ${PORT}`));
Enter fullscreen mode Exit fullscreen mode

Now we have a server ready to receive JSON payloads.


Getting an ElevenLabs API Key

  1. Sign up (or log in) at the affiliate link: https://try.elevenlabs.io/kr07zfuqn1bp.
  2. Navigate to API → Keys and click Create new key.
  3. Copy the key – you’ll store it in an environment variable called ELEVENLABS_API_KEY.

Add a .env file (remember to add it to .gitignore):

ELEVENLABS_API_KEY=your_secret_key_here
Enter fullscreen mode Exit fullscreen mode

Load it in your code:

import dotenv from 'dotenv';
dotenv.config();

const ELEVEN_API_KEY = process.env.ELEVENLABS_API_KEY;
if (!ELEVEN_API_KEY) {
  throw new Error('Missing ELEVENLABS_API_KEY environment variable');
}
Enter fullscreen mode Exit fullscreen mode

Making a TTS Request

ElevenLabs exposes a straightforward POST endpoint:

POST https://api.elevenlabs.io/v1/text-to-speech/{voice_id}
Enter fullscreen mode Exit fullscreen mode

For a quick start we’ll use the default “Rachel” voice (voice_id = 'EXAVITQu4vr4xnSDxMaL'). Feel free to explore other voices in the dashboard.

Add the route handler:

app.post('/speak', async (req, res) => {
  const { text } = req.body;
  if (!text) return res.status(400).json({ error: 'Missing "text" in request body' });

  const voiceId = 'EXAVITQu4vr4xnSDxMaL'; // default voice
  const url = `https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`;

  try {
    const response = await fetch(url, {
      method: 'POST',
      headers: {
        'xi-api-key': ELEVEN_API_KEY,
        'Content-Type': 'application/json',
        // Optional: tweak stability, similarity, etc.
      },
      body: JSON.stringify({
        text,
        // You can add voice settings here, see ElevenLabs docs
      }),
    });

    if (!response.ok) {
      const err = await response.text();
      throw new Error(`ElevenLabs error: ${err}`);
    }

    // The API streams raw audio (MP3). Pipe it to a file or buffer.
    const outputPath = `output-${Date.now()}.mp3`;
    await pipe(response.body, fs.createWriteStream(outputPath));

    // Send back the file path (or stream directly)
    res.json({ audioFile: outputPath });
  } catch (err) {
    console.error(err);
    res.status(500).json({ error: err.message });
  }
});
Enter fullscreen mode Exit fullscreen mode

Why we pipe the response

ElevenLabs streams the audio, which means we can start writing to disk immediately without loading the entire file into memory. This keeps the server lightweight and responsive.


Playing Audio in Node (Optional)

If you want to play the generated speech on the server (useful for Raspberry Pi projects), you can use fluent-ffmpeg together with a system player like ffplay or aplay. Here’s a quick helper:

function playAudio(filePath) {
  return new Promise((resolve, reject) => {
    ffmpeg(filePath)
      .format('wav')
      .audioCodec('pcm_s16le')
      .on('end', resolve)
      .on('error', reject)
      .pipe();
  });
}

// Usage inside the route after saving:
await playAudio(outputPath);
Enter fullscreen mode Exit fullscreen mode

On a headless machine you might prefer to send the MP3 back to the client and let the browser handle playback.


Optional: Voice Cloning

One of ElevenLabs’ standout features is voice cloning. After you upload a few minutes of clean audio, the platform creates a custom voice you can reference via a new voice_id. The workflow is:

  1. Upload samples through the ElevenLabs UI.
  2. Wait for the model to finish training (usually a few minutes).
  3. Use the returned voice_id in the /speak endpoint just like we did above.

The API call is identical; only the voice_id changes. This opens doors for personalized assistants, brand‑consistent narrations, or even recreating historical figures (respecting legal constraints, of course).


Wrapping It Up

You now have a functional AI voice assistant that:

  • Accepts arbitrary text via an HTTP POST.
  • Sends the request to ElevenLabs’ state‑of‑the‑art TTS engine.
  • Streams the resulting audio to a file (or directly to a client).

From here you can:

  • Hook the endpoint into a chatbot (e.g., OpenAI’s GPT) to create a full‑duplex conversational agent.
  • Add caching so repeated prompts don’t re‑synthesize the same audio.
  • Experiment with voice cloning to give your app a unique auditory brand.

The combination of Node.js for rapid prototyping and ElevenLabs for high‑quality speech makes it easy to turn any text‑based service into a voice‑first experience.


Call to Action

Ready to give your app a voice that sounds truly human? Grab your ElevenLabs API key today at https://try.elevenlabs.io/kr07zfuqn1bp and start building. Happy coding!

Top comments (0)