Apple quietly shipped everything you need to build a real-time translator — so I built one

I work at a German company. Meetings are in German, sometimes English — and even though I speak both, there are moments in fast meetings where I zone out for two seconds and think: wait, what did they just say?

I couldn't use any of the existing captioning tools, because they all pipe your meeting audio to a cloud server. Sending confidential work calls to a third party was a non-starter.

Then I realized something: macOS 26 quietly shipped every building block you need for a real-time translator. On-device speech recognition (SpeechAnalyzer), on-device machine translation (Translation framework), and an on-device LLM (FoundationModels / Apple Intelligence). No servers, no API keys, no per-minute fees.

So I built Wakaru — a menu bar app that turns any audio playing on your Mac into movie-style translated subtitles, in real time.

TL;DR

• 🎬 Live translated subtitles for anything your Mac plays — Teams, Zoom, YouTube, webinars, podcasts
• 🔒 100% on-device. Your audio never leaves your Mac. No account, no API keys, no cloud
• 🌍 Recognizes 10 spoken languages, subtitles in 22
• 🧠 Optional high-accuracy mode using the on-device Apple Intelligence LLM
• ⚡ Built entirely with Apple's native frameworks — zero external dependencies, ~3,000 lines of Swift

This post covers how it works and the four gotchas that cost me the most time. If you're planning to build anything on macOS 26's new speech or translation APIs, this might save you a few days.

First attempt: Electron (and why I threw it away)

My first prototype was Electron. It worked — but transcription + translation took about 2 seconds per sentence. For subtitles, 2 seconds might not sound like a lot, but it is: by the time the caption showed up, the conversation had moved on.

When I saw that macOS 26 had the entire pipeline available natively and on-device, I rebuilt it in Swift. The difference was dramatic — captions now appear while the sentence is still being spoken.

(The Electron version wasn't wasted, though. After a lot of tuning it got reasonably fast, and I'm planning to release it for Windows soon.)

Architecture: four stages, zero dependencies

ScreenCaptureKit ──▶ SpeechAnalyzer ──▶ Translation framework ──▶ Subtitle overlay
 (system audio)      (speech-to-text)    or FoundationModels        (NSPanel)
                                          (translation)

The key decision: capture system audio (what the Mac is playing) instead of the microphone. That's what makes Wakaru app-agnostic — it doesn't integrate with Zoom or Teams; it doesn't need to know they exist.

The subtitles are drawn on a borderless, transparent, click-through NSPanel, so you can click straight through the captions to whatever is underneath.

Making it feel like movie subtitles (not a log viewer)

The naive version of this app is a window where original text and translations pile up like a chat log. That's easy to build — and unusable as subtitles.

What I ended up with:

• Don't translate while a sentence is still forming. I initially translated the live transcription in real time. For a verb-final language like German, the translation reshuffled itself on every update — completely unreadable. Now Wakaru waits until a sentence (or a long clause) is complete, then translates it exactly once.
• Caption lifetime scales with reading speed. Each caption stays on screen for text.count / 7 + 2 seconds (clamped to 4–12s). Short interjections vanish quickly; long sentences stay until you can actually finish them.
• Every caption gets a minimum of 2.5 seconds, even when someone is talking fast — new captions queue up instead of instantly evicting the old ones.

let lifetime = min(max(4, Double(text.count) / 7 + 2), 12)

Gotcha #1: Set the sample rate at capture time

SpeechAnalyzer prefers 16 kHz audio. My first version captured at 48 kHz and resampled with AVAudioConverter — and the first caption took seconds to appear. The converter buffers audio internally before it emits anything.

The fix: ScreenCaptureKit lets you pick the sample rate at capture time.

let cfg = SCStreamConfiguration()
cfg.capturesAudio = true
cfg.sampleRate = 16_000   // match SpeechAnalyzer's preferred format
cfg.channelCount = 1

Capture at 16 kHz mono from the start and the only conversion left is Float32 → Int16, sample by sample. The latency disappeared.

Gotcha #2: SpeechAnalyzer is slow by default — in three different ways

Even after fixing the sample rate, captions were still sluggish. It turned out to be three separate problems:

1. Reporting cadence. By default, SpeechAnalyzer batches up partial results and delivers them in bursts, seconds late. Pass .fastResults in reportingOptions to get them as they happen. For live captions this is non-negotiable.
2. Lazy model loading. The recognition model is big, and by default it loads when the first audio arrives — so your first caption is seconds late. Call prepareToAnalyze when the user hits start, and recognition is instant from the first word.
3. Slow shutdown. Stopping with finalizeAndFinishThroughEndOfInput() drains the entire audio backlog before returning — a stop/restart (e.g. switching languages) took seconds. For subtitles you don't care about queued audio, so use cancelAndFinishNow().

Gotcha #3: Recognition results rewrite the past

This was the biggest trap of all.

SpeechAnalyzer's partial results don't just grow at the end. Text you already displayed gets rewritten retroactively — filler words ("uh, uh") get collapsed, words get swapped, punctuation appears late.

Wakaru cuts completed sentences out of the growing transcript and translates each one. That means it has to remember where the already-translated part ends. If you store that boundary as a character offset, it silently drifts every time the recognizer rewrites history. The symptoms: the same sentence gets translated twice, or fragments go missing.

The fix: stop trusting positions, and use content as a bookmark. Wakaru remembers the last few words of the most recently committed sentence, and on every update, searches for that anchor in the rewritten text to re-derive the boundary.

One wrinkle remains: if the speaker literally says the same thing twice ("Thank you. Thank you."), the anchor appears in two places. So the rule is "pick the occurrence closest to the previous boundary estimate" — the character offset survives, demoted from source-of-truth to tie-breaker.

Gotcha #4: The Translation framework only works inside SwiftUI

This one surprised me the most. TranslationSession can only be obtained inside SwiftUI's .translationTask view modifier. There is no "just give me a session" API. Wakaru is a menu bar app — there was nowhere natural to put it.

The workaround: the subtitle overlay is a SwiftUI view anyway, so I attached an invisible .translationTask to it. It receives the session and hands it to a hub object that the rest of the app calls into.

.translationTask(hub.configuration) { session in
    await hub.serve(session)   // publish the session to the pipeline
}

Bonus trap: if you stop and restart with the same language pair, the new configuration compares equal to the old one and the task never restarts — you have to call invalidate() explicitly.

Bonus: taming the on-device LLM

On Apple Intelligence Macs, Wakaru has a high-accuracy mode that translates with the on-device LLM instead of the NMT model, passing the previous 3 sentences as context — pronouns, idioms, and short replies come out much more natural.

But a ~3B on-device model in a real-time loop needs guardrails:

• 2.5s deadline — if the LLM is slower, that sentence silently falls back to the standard engine
• Rebuild the session every 10 sentences — as the session transcript grows, the model starts echoing source-language fragments into translations (measured: around sentence 27)
• Quarantine degenerate output — greedy decoding occasionally locks into repetition loops (a normal German sentence once became the same Japanese word ×25). There's no repetition penalty setting, so I detect and reject it at the output

The rule that matters: the LLM is never allowed to stall the captions. Every failure mode falls back per-sentence to the standard engine.

Try it / tell me what you think

Wakaru runs on macOS 26+ (Apple silicon). It's free for 14 days (and stays free for 1 hour/day after that), with a one-time purchase for unlimited time — no subscription:

📱 Wakaru on the Mac App Store

Happy to answer any questions about the implementation in the comments! 👇