DEV Community

Cynxio
Cynxio

Posted on

ElevenLabs vs four free voice clones: a blind test on my own voice

I pay ElevenLabs $6 a month for a clone of my voice. Four free voice models say they are as good. So I made a clone of my own voice with each of them, hid which clip was which, and ranked them by ear. One clip in every round is the real me.

The test

One recording of me, made for this test, is the source of every clone. ElevenLabs' Instant clone takes 1 to 2 minutes of it; each free model took the length its maker's card asks for. Every model ran on its default settings and the first take is the clip. The free models are Chatterbox (Resemble AI), Qwen3-TTS (Alibaba), IndexTTS-2.5 (bilibili) and Fish S2 Pro (Fish Audio), run on one rented Runpod A40 each.

Four rounds, six clips in each (five clones and the real me), shuffled by a script. I ranked all six blind and the key was opened after each round:

  1. A plain sentence: "I make videos about AI, and this is the first one where the AI is talking as me."
  2. A name and a number: "Hey, Derwin here, we just shipped version 2.3 today."
  3. A sad line: "I really thought this one would work out, but I guess it's time to let it go."
  4. Indonesian: "Aku besar di Indonesia, jadi bahasa ini yang paling dekat sama aku."

Only ElevenLabs and Fish S2 Pro list Indonesian among their languages; Qwen3-TTS and IndexTTS-2.5 do not, and Chatterbox Multilingual lists Malay, not Indonesian.

How the clones ranked

Clone rank in each round, 1 is the best clone (the real me was my number one every time and is left out):

Model R1 R2 R3 R4 Average
Fish S2 Pro 2 1 1 1 1.25
Qwen3-TTS 1 2 3 3 2.25
ElevenLabs 3 5 2 2 3.00
Chatterbox 4 4 4 4 4.00
IndexTTS-2.5 5 3 5 5 4.50

Fish beat ElevenLabs in all four rounds. Qwen beat it in rounds 1 and 2, Chatterbox only in round 2, where ElevenLabs came last. My two guesses on camera, ElevenLabs in round 1 and Qwen in round 2, were both wrong.

One person made one ranking, and the fan noise in my own recording gave the real clip away, which the film admits. So I also ran a speaker-similarity score (ECAPA, SpeechBrain), each clip against that round's real line. Average: Qwen .61, Fish .57, Chatterbox .46, ElevenLabs .43, IndexTTS .36. My real voice against its other three lines scores .69. The score and my ears disagree about who is closest, which is the point of doing both.

Cost and speed

Measured on 31 October 2026, four lines per model, one rented A40 at $0.00034 a second, execution time only:

Model Seconds of processing per second of audio Dollars per minute of audio Cold start
Chatterbox 0.44 $0.009 291 s
Qwen3-TTS 1.72 $0.035 207 s
Fish S2 Pro 2.00 $0.041 347 s and 227 s
IndexTTS-2.5 not measured not measured 476 s and 357 s
ElevenLabs Starter 0.49 $0.054 none

IndexTTS-2.5 never stayed up long enough to time. A free model is not free: a rented GPU boots for 3 to 8 minutes, at 7 to 16 cents, before it makes a sound. ElevenLabs' $0.054 a minute holds only if the whole 40,000-credit plan, about 112 minutes of audio, is used.

The catch

My verdict on camera: considering the ranking, I'd cancel ElevenLabs and go with Fish for future videos. Fish S2 Pro's licence is non-commercial, so earning from its voice needs a commercial licence, and Fish lists no price for one. Qwen, the next best clone, already allows earning.

The film is a code-rendered animation built from generated artwork. Nyx, the narrator, speaks in an authorized synthetic ElevenLabs voice; the music bed and the sound effects are generated with ElevenLabs too. Product logos belong to their owners.

Top comments (0)