Getting Started with ElevenLabs Text-to-Speech API

ElevenLabs: Advanced AI Speech Tool

ElevenLabs is an advanced AI speech tool that allows you to generate high-quality spoken audio in any voice and style. Their text-to-speech API is a service that developers can use to convert written text into speech in a variety of voices, including both pre-existing and custom voices.

With ElevenLabs, you can train your own voice model using your own recorded speech to create a custom voice for your application. The API is capable of rendering human-like intonation and inflections with unprecedented fidelity, resulting in speech that sounds natural and lifelike.

Getting Started with ElevenLabs API

To get started with the ElevenLabs API, you'll need to sign up for an API key on their website https://beta.elevenlabs.io/. Once you've created an account click on your profile icon and go to profile settings there you can obtained an API key, you can start using their text-to-speech service.

Making a Basic Request

To make a basic request to the API, you'll need to send a POST request to their endpoint with your API key and the text you want to convert to speech. Here's an example of how to do this in javascript using the axios:

import axios from 'axios';

// Define a function called textToSpeech that takes in a string called inputText as its argument.
const textToSpeech = async (inputText) => {
  // Set the API key for ElevenLabs API. 
  // Do not use directly. Use environment variables.
  const API_KEY = ELEVEN_LABS_API_KEY;
  // Set the ID of the voice to be used.
  const VOICE_ID = '21m00Tcm4TlvDq8ikWAM';

  // Set options for the API request.
  const options = {
    method: 'POST',
    url: `https://api.elevenlabs.io/v1/text-to-speech/${VOICE_ID}`,
    headers: {
      accept: 'audio/mpeg', // Set the expected response type to audio/mpeg.
      'content-type': 'application/json', // Set the content type to application/json.
      'xi-api-key': `${API_KEY}`, // Set the API key in the headers.
    },
    data: {
      text: inputText, // Pass in the inputText as the text to be converted to speech.
    },
    responseType: 'arraybuffer', // Set the responseType to arraybuffer to receive binary data as response.
  };

  // Send the API request using Axios and wait for the response.
  const speechDetails = await axios.request(options);

  // Return the binary audio data received from the API response.
  return speechDetails.data;
};

// Export the textToSpeech function as the default export of this module.
export default textToSpeech;

Convert to audio file

Once you have received the audio data in the form of an ArrayBuffer from the ElevenLabs API, you can convert it into an MP3 blob file that can be played or saved. Here's how to do it in React js:

import { useState, useEffect } from 'react';

const AudioPlayer = () => {
  // Define a state variable to hold the audio URL
  const [audioURL, setAudioURL] = useState(null);

  // Define a function to fetch the audio data and set the URL state variable
  const handleAudioFetch = async () => {
    // Call the textToSpeech function to generate the audio data for the text "Hello welcome"
    const data = await textToSpeech("Hello welcome")
    // Create a new Blob object from the audio data with MIME type 'audio/mpeg'
    const blob = new Blob([data], { type: 'audio/mpeg' });
    // Create a URL for the blob object
    const url = URL.createObjectURL(blob);
    // Set the audio URL state variable to the newly created URL
    setAudioURL(url);
  };

  // Use the useEffect hook to call the handleAudioFetch function once when the component mounts
  useEffect(() => {
    handleAudioFetch()
  }, []);

  // Render an audio element with the URL if it is not null
  return (
    <div>
      {audioURL && (
        <audio autoPlay controls>
          <source src={audioURL} type="audio/mpeg" />
        </audio>
      )}
    </div>
  );
};

export default AudioPlayer;

For more detailed documentation on how to use ElevenLabs' API, you can visit their API documentation page at https://api.elevenlabs.io/docs#/ where you can try it out the API.

Comments

[Greg]

Would this approach expose your api key?

[Pranav]

Yes, this will include the API key in your frontend code. You should instead create an API endpoint or use Server Actions (if you're using nextjs) for this.

[Sanjay 🥷]

Hello @greg, we should always use API key as environment variables in .env files.

[msopacua]

That doesn't help. The variables will be exposed, since the build process will include them in the generated bundle. Dotenv only prevents it to be exposed to the git repository.

[Dennis]

Hello Can you please help me. I am trying to upload an audio file instead of text with the "input_audio" param which essentially takes a audio public url. But no mateer how many times I give a downlodable url, it shows "invalid audio url"
Can you please help!!!!

[RobAdkerson]

Any way to get the audio in a different file format (wav) or frame rate?