I kept pausing Japanese streams — so I built tab-audio captions on Cloudflare Workers

XTSoft — Wed, 09 Sep 2026 01:49:53 +0000

I'm a Chinese speaker living in the US. Weekdays are mostly English: standup on Zoom, design docs, Slack threads that somehow always explode at 5pm. Nights and weekends are Japanese — VTuber streams, cooking livestreams, the occasional anime episode I refuse to watch with English-only subs.

For months my "language practice" looked like this:

Twitch / YouTube open in Chrome
Host says something fast
I miss it
Pause
Scrub back 8–12 seconds
Still not sure → open a dictionary tab
Stream moves on without me

That loop is survivable for a recorded course. It is miserable for a live stream where chat is already three jokes ahead.

I didn't want another upload-a-video translator. I wanted: stay on the same tab, see the original line + a Chinese (or English) translation almost immediately, keep watching.

So I built a Chrome/Edge extension that captions whatever audio is playing in the tab, and put the realtime path on Cloudflare Workers + Durable Objects.

The product constraint (before the stack)

Hard requirements from day one:

Works on any tab with audio — YouTube, Twitch, Netflix-in-browser, Zoom/Meet in the browser, random course sites
No file upload
Bilingual overlay (source + target)
Latency low enough that a live stream still feels watchable (~0.5s ballpark for me)
Prefer keeping raw audio in the tab; only ship text / small chunks for processing

That ruled out "export MP4 → wait → download SRT." The extension had to own capture; the edge had to own the session.

How the pieces fit (CF-shaped)

High level, nothing fancy:

Extension (Chrome MV3)

Uses tabCapture / offscreen document patterns to grab the tab's audio stream, run VAD-ish chunking so I'm not uploading silence, and render a floating bilingual overlay. Settings and history live locally first.

Worker (stateless edge)

Auth, plan checks, routing. A request comes in: "here's a short audio/text segment for session X." The Worker doesn't hold long-lived state — it just verifies the user and hands the work to the right place.

Durable Object (one per caption session)

This is the part that made the architecture click for me.

Each Start click maps to a Durable Object keyed by session id. That object:

Owns the WebSocket to the extension (hibernation-friendly)
Queues / serializes ASR + translation steps so chunks don't arrive out of order on screen
Tracks minute usage for the free tier without racing a shared Redis
Keeps a short rolling buffer of recent caption lines (enough to recover a reconnect without rebuilding history from scratch)
Tears down cleanly when the user hits Stop or the tab dies

I tried the "just shove everything through a single Worker + external WebSocket server" version first. It worked until two tabs and a flaky Wi‑Fi night. Per-session Durable Objects made the failure domain obvious: one stuck session doesn't take down everyone else's captions.

Why Cloudflare specifically

I didn't want to babysit a caption server in us-west-2 while I'm watching a stream at 1am PT. Workers put the handshake close to me; Durable Objects give me single-threaded session semantics without standing up Redis + a sticky-session LB for a side project that started as "please stop pausing."

ASR / translation providers sit behind the Worker as normal HTTPS calls. The interesting bit for me wasn't swapping models — it was keeping the session glue boring and correct at the edge.

What "good enough" latency felt like

I didn't optimize for leaderboard numbers. I optimized for: can I leave this on for a 90-minute cooking stream without rage-pausing?

For me, captions landing around half a second after speech is the line between "tool" and "distraction." Faster is nicer; much slower and my thumb goes back to the spacebar.

So the goal stayed: watchable live captions, not court-ready transcripts.

A normal evening with it

Typical flow now:

Install the extension
Open the Japanese stream (or the English Zoom tab for work)
Pick target language — Chinese when I'm tired, English when I'm taking notes
Hit Start
Read along: original on top, translation under it

Meetings: browser Meet/Zoom tab → Start → stop pretending I caught every acronym.

Language nights: dual subtitles on, dictionary tab closed unless I really want a deep dive later.

I still export / copy history when I need searchable notes after a lecture. Live comprehension and archival transcripts turned out to be different jobs — which is why the Durable Object keeps only a short buffer, not a forever archive in SQLite by default.

Privacy, said plainly

"Listens to tab audio" sounds creepy if you care about privacy (I do — this runs on my work laptop too).

Design intent:

Capture stays in the extension / tab path
The session DO sees segments for that session, not a permanent dump of every stream I've ever watched
UI makes it obvious when captioning is running

Not a formal security audit. Just: I wouldn't ship something I refuse to enable myself.

The thing I actually shipped

All of that is packaged as SonicCaption — Chrome/Edge extension, freemium minutes, bilingual overlay.

Site: soniccaption.com

Short demo: YouTube

I'm not doing a pricing pitch here. The interesting part was the constraint set and putting session state in Durable Objects instead of "yet another Node process."

What I'd tell myself six months ago

1. Fix the pause loop, not "AI translation" as a vibe.

The job was staying present on a live Japanese stream while my brain is half in English from work Slack.

2. One Durable Object per session beats a clever global queue.

Ordering, reconnects, and usage counters got simpler when the unit of isolation matched the unit of user intent (Start → Stop).

3. Universal tab audio beats site-specific hacks for v1.

Special-casing YouTube forever is how you never ship Twitch.

4. Dual subtitles change behavior.

Source + target together cut how often I alt-tabbed to a dictionary mid-stream.

Open questions

Where do you split live captions vs archival transcripts in product and in storage?
How do you explain tabCapture permissions without sounding like malware?
For bilingual UI on a 13" laptop: stacked lines, side-by-side, or toggle — what feels least noisy?

If you've built realtime stuff on Workers / Durable Objects (or fought the same pause-rewind ritual on JP streams), I'd love to hear what you landed on in the comments.

Built this because I needed it. Feedback: support@soniccaption.com

DEV Community: XTSoft