Create an AI Voice Assistant with ElevenLabs and Node.js

Why ElevenLabs for a Voice‑First Bot?

If you’ve ever wanted to give your app a “real” voice—something that sounds natural, expressive, and even mimics a specific speaker—ElevenLabs is the go‑to service today. Their API delivers high‑quality text‑to‑speech (TTS) and voice‑cloning capabilities with just a few HTTP calls, and the pricing tier for developers is generous enough for hobby projects and prototypes.

In this tutorial we’ll wire up a simple AI voice assistant using Node.js and the ElevenLabs API. By the end you’ll have a command‑line tool that can:

1. Take a user’s typed query,
2. Send it to an LLM (e.g., OpenAI’s ChatGPT) for a response,
3. Convert the response into spoken audio with ElevenLabs,
4. Play the audio back to the user.

If you’re ready to start building, grab your free API key from the ElevenLabs dashboard via this link: https://try.elevenlabs.io/kr07zfuqn1bp.

---

Prerequisites

What you need	Why
Node.js ≥ 18	Built‑in fetch support makes HTTP calls a breeze.
npm or yarn	Package management.
ElevenLabs API key	Authenticates your requests (see link above).
OpenAI API key (optional)	Powers the conversational logic.
ffmpeg (optional)	Enables audio playback on most OSes.

You can verify Node with node -v and install ffmpeg via your system’s package manager (brew install ffmpeg, apt-get install ffmpeg, etc.).

---

Project Layout

voice‑assistant/
├─ .env                # API keys
├─ index.js            # Main entry point
├─ tts.js              # Tiny wrapper around ElevenLabs
└─ package.json

Create the folder and initialize npm:

mkdir voice-assistant && cd voice-assistant
npm init -y
npm install dotenv openai

We’ll keep the code lightweight and avoid extra dependencies for HTTP – the native fetch works perfectly.

---

Storing Secrets Securely

Create a .env file (add it to .gitignore later):

ELEVENLABS_API_KEY=your_elevenlabs_key_here
OPENAI_API_KEY=your_openai_key_here   # optional

Load these values at runtime:

// index.js
import * as dotenv from 'dotenv';
dotenv.config();

const ELEVEN_API_KEY = process.env.ELEVENLABS_API_KEY;
const OPENAI_API_KEY = process.env.OPENAI_API_KEY;

---

The ElevenLabs Wrapper (tts.js)

// tts.js
import fs from 'fs';
import path from 'path';
import { pipeline } from 'stream';
import { promisify } from 'util';
import fetch from 'node-fetch';

const streamPipeline = promisify(pipeline);

/**
 * Convert text to speech using ElevenLabs.
 *
 * @param {string} text   The text you want spoken.
 * @param {string} voiceId The ID of the voice to use (default: “Rachel”).
 * @param {string} outPath Where to save the resulting .mp3 file.
 */
export async function synthesize(text, voiceId = 'EXAVITQu4vr4xnSDxMaL', outPath = 'output.mp3') {
  const url = `https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`;
  const response = await fetch(url, {
    method: 'POST',
    headers: {
      'xi-api-key': process.env.ELEVENLABS_API_KEY,
      'Content-Type': 'application/json',
      'Accept': 'audio/mpeg',
    },
    body: JSON.stringify({
      text,
      voice_settings: {
        stability: 0.75,
        similarity_boost: 0.85,
      },
    }),
  });

  if (!response.ok) {
    const err = await response.text();
    throw new Error(`ElevenLabs error: ${response.status} – ${err}`);
  }

  await streamPipeline(response.body, fs.createWriteStream(outPath));
  console.log(`🔊 Saved audio to ${outPath}`);
}

Note: The default voice ID above points to the “Rachel” voice. You can list your available voices via the ElevenLabs API or the dashboard. For custom voice cloning, upload a short audio sample and retrieve the new voice ID—still the same endpoint, just a different voiceId.

---

Hooking Up an LLM (Optional)

If you want the assistant to answer questions instead of just echoing what you type, add a tiny OpenAI helper:

// openai.js
import fetch from 'node-fetch';

export async function askChatGPT(prompt) {
  const response = await fetch('https://api.openai.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'gpt-3.5-turbo',
      messages: [{ role: 'user', content: prompt }],
      max_tokens: 150,
    }),
  });

  const data = await response.json();
  return data.choices?.[0]?.message?.content?.trim() ?? '';
}

---

Putting It All Together (index.js)

// index.js
import readline from 'readline';
import { synthesize } from './tts.js';
import { askChatGPT } from './openai.js';
import { exec } from 'child_process';
import * as dotenv from 'dotenv';
dotenv.config();

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

function playAudio(file) {
  // Cross‑platform playback: macOS (afplay), Linux (ffplay), Windows (powershell)
  const cmd = process.platform === 'win32'
    ? `powershell -c (New-Object Media.SoundPlayer "${file}").PlaySync();`
    : process.platform === 'darwin'
    ? `afplay ${file}`
    : `ffplay -autoexit -nodisp ${file}`;

  exec(cmd, (err) => {
    if (err) console.error('Playback error:', err);
    prompt();
  });
}

async function handleInput(input) {
  const trimmed = input.trim();
  if (!trimmed) return prompt();

  // 1️⃣ Get a response from the LLM (or just echo)
  const reply = await askChatGPT(trimmed);
  console.log(`🤖 ${reply}`);

  // 2️⃣ Convert reply to speech
  const audioFile = 'reply.mp3';
  await synthesize(reply, undefined, audioFile);

  // 3️⃣ Play the audio
  playAudio(audioFile);
}

function prompt() {
  rl.question('\nYou: ', handleInput);
}

// Kick off
console.log('🚀 Voice Assistant ready! Type something and hear it back.');
prompt();

Run the app with:

node index.js

You should see the prompt, type a question, watch the LLM answer in the console, and hear the spoken response through your speakers.

---

Adding Voice Cloning (Bonus)

ElevenLabs lets you upload a short 10‑second sample of a target speaker and generate a custom voice ID. Here’s a quick curl example:

curl -X POST "https://api.elevenlabs.io/v1/voices/add" \
  -H "xi-api-key: YOUR_ELEVENLABS_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "name=MyClone" \
  -F "files=@/path/to/sample.wav"

The response includes a voice_id. Replace the voiceId argument in synthesize() with that value to hear your clone speak the LLM’s answer. For a deeper dive, check the ElevenLabs docs (again, the same affiliate link: https://try.elevenlabs.io/kr07zfuqn1bp).

---

Debugging Tips

Symptom	Likely cause	Fix
401 Unauthorized from ElevenLabs	Wrong or missing API key	Verify ELEVENLABS_API_KEY in .env.
No audio file created	fetch returned non‑200	Log response.status and await response.text() to see the error payload.
Playback fails on Linux	Missing ffplay	Install ffmpeg (sudo apt-get install ffmpeg).
LLM returns empty string	OpenAI quota exhausted	Check usage in the OpenAI dashboard.

---

Next Steps

• Add hot‑word detection with a microphone stream and a speech‑to‑text service (e.g., Whisper).
• Persist conversation context so the assistant can remember prior turns.
• Deploy to the cloud (Render, Railway, or Fly.io) and expose a simple HTTP endpoint that any front‑end can call.
• Experiment with different voice settings (stability, similarity_boost) to get the exact tonal quality you need.

The possibilities are endless once you have the core TTS loop working.

---

Give It a Spin!

ElevenLabs makes high‑fidelity voice synthesis and cloning accessible with just a few lines of code. Grab your free API key now, clone this repo, and start building the next generation of voice‑first applications: https://try.elevenlabs.io/kr07zfuqn1bp. Happy coding!