I built a fully local speech translator that runs offline, stays private, and fits in your terminal

Lucas Hornung — Wed, 24 Jun 2026 19:55:34 +0000

Last October, someone posted "Ask HN: Real-time speech-to-speech translation?" on Hacker News. 158 points, 70 comments. They wanted a Babelfish, something that listens, translates, and speaks, all offline. Every alternative people suggested (RTranslator, 3PO, Samsung Interpreter) failed on at least one axis: too awkward, requires cloud, not open source.

I decided to build one for English and Mandarin.

What it does

You speak English, it says the same thing in Mandarin. You speak Mandarin, it says it in English. Everything runs on your machine.

npm install -g live-translate
live-translate start
live-translate
# Hold SPACE, speak, release, hear translation

No API keys. No cloud. Works in airplane mode. Works behind the Great Firewall.

Why I built this

The existing options all have the same problem.

Google Translate sends your audio to Google's servers, is blocked in China, and renders "我请你吃饭" as "I invite you to eat rice" instead of "I'll treat you to dinner."

Microsoft Translator is not much better. A paying Azure customer publicly filed a ticket saying Mandarin output is "literal or word-for-word" while Spanish and Portuguese work fine.

DeepL has no voice mode at all for Chinese, and academic testing found it mishandles basic Chinese grammar.

ChatGPT Voice has excellent quality when it works, but it forgets it's supposed to be translating and starts answering your questions instead.

I wanted something private (my conversations with my family are not Google's business), that worked offline (I travel to places with no signal), and that was open source so I could inspect and fix it.

The architecture

Three ML models chained together:

Microphone
    |
Whisper (speech to text + language detection)
    |
Ollama + Qwen 2.5 (text to translated text)
    |
Piper TTS (translated text to speech)
    |
Speaker

Whisper handles speech recognition and language detection in one step. It tells me "this is English" or "this is Mandarin" so the routing is free.

Qwen 2.5 via Ollama handles translation. I started with Opus-MT (Helsinki-NLP's lightweight model) but the quality was bad. It would repeat "你好你好你好" for a simple "Hello." Qwen 2.5 at 7B parameters produces natural, idiomatic output.

Piper TTS handles speech synthesis. It's fast, MIT licensed, and has decent voices for both English and Chinese.

All three run locally. No data leaves your machine. The only network call is pulling the Docker containers on first install.

One thing I learned the hard way

Translation quality is dramatically worse in one direction. English to Chinese is decent across most tools. Chinese to English, the direction where a Mandarin speaker needs to be understood, is significantly worse. A clinical study tested live spoken translation and found that English to Chinese scored 62-76% acceptable while Chinese to English scored only 36-41%.

This matters because a live conversation needs both directions to work. Most translation demos only show the easy direction. I benchmark both.

The Whisper hallucination problem

Whisper has a known issue. During silence or background noise, it hallucinates phrases from its training data. The most common ones are "Translated by Amara.org Community" and "Transcribed by Otter.ai", because Whisper was trained on YouTube subtitle files that contained these credit lines.

Common hallucinations: "Translated by Amara.org Community" and "Transcribed by Otter.ai"

I added filtering for known hallucination strings and surface Whisper's confidence score to the user. If confidence is low, you see a warning instead of garbage output.

It's also an MCP

If you use Claude Code or Claude Desktop, you can use the same engine as an MCP server:

npx @anthropic-ai/claude-code mcp add live-translate-mcp

Give Claude an audio file and it'll transcribe, translate, and save the translated audio, all locally.

waxberry-dev / live-translate-mcp

MCP server for local speech translation (EN ↔ 中文) via Whisper + Claude + Piper

live-translate-mcp

Real-time English ↔ Mandarin speech translation for Claude — powered by Whisper, Claude AI, and Piper TTS.

Give Claude the ability to listen, translate, and speak. live-translate-mcp is a Model Context Protocol (MCP) server that adds speech translation as a native tool inside Claude Desktop and Claude Code. Hand it an audio file, and it transcribes, translates, synthesises, and plays the result — entirely on your machine, with Claude handling the translation.

What it does

Step	Technology	Where it runs
Speech → Text	OpenAI Whisper (via `@huggingface/transformers`)	Local
Text → Translation	Claude (Opus 4.8)	Anthropic API
Translation → Speech	Piper TTS (ONNX)	Local

Audio never leaves your machine except for the translated text sent to the Claude API. ASR and TTS run fully on-device.

Tools

`translate_file`

Translate a WAV audio file. Pass an absolute path — the server transcribes it, translates the text via Claude, synthesises speech, saves…

View on GitHub

Try it

npm install -g live-translate
live-translate doctor    # check prerequisites
live-translate start     # start the translation backend
live-translate           # hold SPACE to speak

Click here for prerequisites

Node.js: v18 or higher
Docker: Required for the local ML backends
Sox: Used for audio processing

Run live-translate doctor to verify your setup.

The whole thing is MIT licensed. PRs welcome, especially if you speak Cantonese. That's the most-requested language to add and Whisper already supports it.

waxberry-dev / live-translate

Fully local English ↔ Mandarin speech translator. Open source, offline, privacy-first. CLI + MCP server.

live-translate — English ↔ 中文

Fully local speech-to-speech translator between English and Mandarin Chinese No cloud APIs required. All audio processing runs on your machine. MIT licensed.

Install

npm install -g live-translate

Quick Start

live-translate   # press SPACE to record, SPACE again to translate
                 # downloads models and starts services automatically on first run
live-translate stop   # stop services when done

Commands

Command	Description
`live-translate`	Start translating (SPACE to start/stop recording, Q or Ctrl-C to quit)
`live-translate config`	Configure your translation provider
`live-translate start`	Download models and start backend services (runs automatically when needed)
`live-translate stop`	Stop all backend services
`live-translate status`	Show service health and active provider
`live-translate doctor`	Check prerequisites

Translation Providers

Run live-translate config to choose a backend. The default is Opus-MT (fully local, no API key needed).

Provider	Type	Quality	Cost
Opus-MT (default)	Local model	Good	Free
Ollama (Qwen 2.5)	Local LLM	High	Free — needs

…

View on GitHub

Website: waxberry.dev

If you found this useful, a star on GitHub means a lot. And if you try it, tell me how the translation quality is for you, especially Mandarin to English. That's the direction I'm working hardest to improve.

DEV Community: Lucas Hornung

I built a fully local speech translator that runs offline, stays private, and fits in your terminal

What it does

Why I built this

The architecture

One thing I learned the hard way

The Whisper hallucination problem

It's also an MCP

waxberry-dev / live-translate-mcp

MCP server for local speech translation (EN ↔ 中文) via Whisper + Claude + Piper

live-translate-mcp

What it does

Tools

translate_file

Try it

waxberry-dev / live-translate

Fully local English ↔ Mandarin speech translator. Open source, offline, privacy-first. CLI + MCP server.

live-translate — English ↔ 中文

Install

Quick Start

Commands

Translation Providers

`translate_file`