DEV Community

Great Sage
Great Sage

Posted on

Self-hosting an OpenAI-compatible TTS API with Kokoro-82M (no GPU needed)

Most self-hosted TTS is a tradeoff: espeak/piper are fast and free but sound robotic, and anything that sounds actually good usually means a GPU box or an ElevenLabs bill. Kokoro-82M is the first open model I've used that dodges both problems — 82M parameters, runs fine on CPU, and the output is close enough to ElevenLabs-tier for narration and voiceover work that I stopped reaching for paid APIs.

The part that made it actually usable for me is Kokoro-FastAPI — a wrapper that exposes Kokoro behind an OpenAI-compatible /v1/audio/speech endpoint. If you already have code that talks to OpenAI's TTS API, you don't rewrite anything, you just swap the base URL:

from openai import OpenAI

client = OpenAI(
    base_url="https://your-app.up.railway.app/v1",
    api_key="your-api-key",
)

response = client.audio.speech.create(
    model="kokoro",
    voice="af_bella",
    input="Hello world!",
)
response.stream_to_file("output.mp3")
Enter fullscreen mode Exit fullscreen mode

What you get out of the box:

  • OpenAI-compatible POST /v1/audio/speech
  • multi-language: English, Japanese, Chinese
  • output formats: MP3, WAV, Opus, FLAC, M4A, PCM
  • streaming with configurable chunk sizes
  • weighted voice combinations, e.g. af_bella(2)+af_sky(1) for a 67/33 blend
  • word-level timestamped captions via POST /dev/captioned_speech
  • a built-in web UI (/web) and API docs (/docs) so you can poke at it without writing a client

Tradeoff worth knowing up front: the image I run is the CPU build, so it's not going to touch a GPU pipeline on raw throughput or handle heavy concurrent load. For narration, scripts, voiceover, IVR prompts — anything not real-time-at-scale — it's been solid.

Deploying it

I maintain a one-click Railway template for this — it's just the stock ghcr.io/remsky/kokoro-fastapi-cpu image, nothing patched. Full disclosure: I get a kickback if you deploy through it.

Deploy: https://railway.com/deploy/kokoro-tts-api?referralCode=Z1xivh&utm_medium=integration&utm_source=template&utm_campaign=generic

If you'd rather not go through a referral link, the same image runs anywhere Docker does:

docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest
Enter fullscreen mode Exit fullscreen mode

Upstream repo (all credit to the actual authors): https://github.com/remsky/Kokoro-FastAPI

Happy to answer questions on memory sizing or voice quality in the comments — it's been running steady for me without needing much tuning.

Top comments (0)