DEV Community: Nichola Chaus

The host's question kept landing under the guest's name: fixing speaker labels in a live tab transcriber

Nichola Chaus — Sun, 25 Oct 2026 02:46:47 +0000

I build Transcribe Video to Text, a Chrome extension that turns any video playing in a tab into text while it plays: YouTube, course platforms, webinars, a local file opened in Chrome. For interviews and calls it also labels who is speaking.

Last week I recorded a short demo interview for a YouTube video about the extension. Two synthetic voices: Maya, the host, asks questions, and Daniel, the guest, answers. I exported the transcript to Word and the host's questions sat under the guest's label. Not once, again and again.

It turned out to be two separate bugs, one on the server and one in the side panel, plus a limit of live diarization that I could not fix. Here is the whole path, from capturing tab audio in Manifest V3 to the labels.

How tab audio gets to the speech model in MV3

There is no background page in MV3, and the service worker can't hold a media stream. So the capture is split across three places.

1. The service worker gets a stream ID. When the user clicks Transcribe in the side panel, the worker calls chrome.tabCapture.getMediaStreamId({ targetTabId }). This only works for a tab where the user actually invoked the extension. If Chrome refuses, the panel doesn't show the raw error. It says: "To transcribe this tab, click the Transcribe icon in the toolbar above. This grants audio access for the current tab."

2. An offscreen document turns the ID into audio. The worker creates it with chrome.offscreen.createDocument({ reasons: ['USER_MEDIA'], ... }) and passes the stream ID. The offscreen page calls getUserMedia with chromeMediaSource: 'tab' and that ID.

One thing you notice in the first minute of testing: capturing a tab mutes it for the user. The fix is to route the captured stream back to the speakers:

audioCtx = new AudioContext();
audioCtx.createMediaStreamSource(mediaStream).connect(audioCtx.destination);

3. MediaRecorder sends small chunks over a WebSocket. Opus in WebM at 32 kbps, a chunk every 250 ms. The recorder starts only after my server answers ready, which means the connection to the speech model is already open, so the first words have somewhere to go.

A small trap here: the first chunk is the WebM header, about 220 bytes, and it arrives even when the tab is completely silent. I used "first chunk arrived" to switch a "sound is coming in" indicator on, and it lit up on a video with no sound at all. In that session the server got 233 bytes in 37 seconds. Now the first chunk is ignored, and the indicator waits for a later chunk that actually carries audio.

4. The server relays audio to Deepgram. A live connection with nova-3, interim_results, smart_format and diarize: true. Deepgram sends back messages with the text, an is_final flag, a speech_final flag at the end of an utterance, and a list of words. With diarization on, every word carries a speaker number.

Bug 1: one label for the whole message

Note where the speaker lives: on each word, not on the message. My server took the speaker of the first word that had one and put that label on the whole message:

const speaker = alt.words?.find(w => w.speaker != null)?.speaker ?? null;

Most of the time a final message is one person, so this looked fine. But when the host asks a short question and the guest answers right away, one final message can hold both voices. The whole thing went out under the label of whoever spoke first.

The fix splits every final message into runs of consecutive words by the same speaker. A word without a speaker number joins the current run:

function splitBySpeaker(words) {
  const runs = [];
  for (const w of words || []) {
    const cur = runs[runs.length - 1];
    const curSpk = cur && cur.find(x => x.speaker != null)?.speaker;
    if (cur && (w.speaker == null || curSpk == null || w.speaker === curSpk)) cur.push(w);
    else runs.push([w]);
  }
  return runs;
}

If there is more than one run, each run goes to the client as its own final segment with its own start time, duration and speaker. Interim messages are left alone: they get replaced a moment later anyway.

The test has five cases: question and answer in one message, a single speaker, an interim message, a word without a speaker number, and a response with no diarization at all. The two-voice case fails on the old code, all five pass on the new one. The server fix went live on October 18, so transcripts saved in the web account got correct labels right away.

Bug 2: the side panel closed the paragraph one chunk too late

The panel doesn't print every final chunk as a separate line. It collects chunks in a buffer and closes a paragraph when the text ends a sentence and is longer than 80 characters, or grows past 250 characters, or has 4 chunks, or the speaker changes. The paragraph gets the label of its first chunk.

The bug was in the order of operations. A new chunk was pushed into the buffer first, and only then did the code check whether the speaker had changed. So on a speaker change, the guest's first sentence was already inside the host's buffer and went out as part of the host's paragraph, under the host's label. The Word export copies the panel, so the document had the same mistake.

The fix is boring and that's the point: if the speaker changed, close the previous speaker's paragraph first, then push the new chunk.

speakerChanged && deepgramChunkBuffer.length && flushBuffer();
deepgramChunkBuffer.push({ text, start: start || 0, speaker: speaker ?? null });
// ...then the usual length and sentence checks

I reproduced the demo interview in a browser test: load the side panel page, push the same sequence of transcript messages into its listener, read the paragraphs from the DOM. On the old code two checks failed, exactly the way my Word file looked. On the new code they pass. This part shipped in version 1.3.5 on October 21.

A bonus bug found while checking the fix

While I was checking labels on a TED interview, I found that the server only stored messages that had speech_final. A long reply can produce several is_final messages before the utterance ends, and everything except the last one was lost in the saved transcript. In one recording, the stretch from 0:29 to 0:54 was saved as a single word. The live panel showed it fine, so it went unnoticed since July.

Now the server keeps is_final text in a buffer and writes it out on speech_final, on a speaker change, or when the connection closes.

The limit I could not fix: live diarization is not stable

The same TED interview had another complaint: the whole first minute was labeled as one speaker, and the second label showed up at 1:01 in the middle of a sentence. This time my code was not the problem. Deepgram itself sent speaker 0 for everything up to 60.6 seconds.

I ran a small experiment with the same SDK, nova-3, diarization on, English, on an 80 second clip from that talk:

Live stream, run 1: speaker changes at 12.8, 27.3, 37.0, 57.9 and 78.0 seconds. Correct.
Live stream, run 2, same file, same parameters: speaker 0 from 12.8 all the way to 78.0. Wrong, exactly what the user saw.
The whole file sent as a prerecorded request: correct, and it even caught a short reply at 17.3 seconds that both live runs missed.

So in live mode, with two voices that sound alike, the labels can be perfect on one run and collapse into one speaker on the next. The fix I see is to keep the session audio in server memory and, after the user presses Stop, send it once more as a whole file with diarization and relabel the saved segments. That roughly doubles the speech model minutes per session, so it is not shipped yet.

What I took away

With diarization, the speaker belongs to the word, not to the message. Anything that labels a whole message by its first word will mislabel quick back-and-forth.
Buffering for readability has an order of operations. Close on the boundary before you add what is past it.
A synthetic two-voice interview is a cheap and repeatable test for speaker labels. It caught both bugs in one recording.
Before you blame your code for wrong labels, log what the model actually sent. One of my two "label bug" reports was the live model, not my code.

If you want to try it on your own interviews or lectures, the extension is here: Transcribe Video to Text. YouTube videos with subtitles are free and unlimited, AI transcription of other videos has a free monthly allowance.

Chrome kills an MV3 service worker at 5 minutes. Here is how my AI image extension stopped losing results

Nichola Chaus — Sun, 18 Oct 2026 04:13:35 +0000

I build a small Chrome extension that upscales and unblurs photos with an AI model running right on your computer. No server, the image never leaves the browser. On a laptop with a decent GPU an upscale takes seconds. On an old office PC with no GPU, the same model runs on the CPU through WebAssembly, and a 4x upscale of a big photo can take six or seven minutes.
For two weeks those slow machines kept losing their results at the very end. Here is what was going on and the fix that took the error count to zero.
The symptom
Users on slow PCs pressed Enhance, watched the progress bar crawl for several minutes, and then got an error instead of a picture. Under the hood it was Chrome's "The message port closed before a response was received" (sometimes worded as "message channel closed"). In our anonymous error stats it showed up 6 times from 5 users between September 27 and October 6, including Japan, Mexico and Canada. Every case had the same shape: no GPU, and a run longer than about five minutes. One user saw the raw English error text right before uninstalling.
The architecture that caused it
In Manifest V3 there is no background page. The model can't live in the service worker, which Chrome stops whenever it likes, so it runs in an offscreen document. The original message flow looked like this:
The editor page sends INFER to the service worker.
The service worker forwards it to the offscreen document and keeps sendResponse open with return true.
The offscreen document runs the model and answers.
The service worker passes the answer back to the page.
It worked perfectly in testing, because on my machine step 3 took seconds.
The 5-minute rule
Chrome's own docs on the extension service worker lifecycle say it plainly: the worker is terminated when a single request, like an event or an API call, takes longer than 5 minutes to process. A message the worker is still waiting to answer counts as that request. So on a slow PC Chrome killed the service worker in the middle of step 2, the port to the page died with it, and the result the offscreen document produced a minute later had nowhere to go.
The offscreen document itself was fine. It kept computing. The middleman was the problem.
The fix: take the service worker out of the long path
In version 1.1.1 the service worker does only the fast part: it creates the offscreen document if it isn't there yet and replies right away. The long INFER message goes from the page straight to the offscreen document.
On the page side it is one small helper. First chrome.runtime.sendMessage({ type: 'ENSURE_OFFSCREEN' }), which the worker answers in milliseconds. Then chrome.runtime.sendMessage({ type: 'INFER', target: 'offscreen', ...payload }), which the offscreen document answers when the model is done. The batch mode uses the same helper, so a folder of 30 photos goes through the same safe path.
One gotcha: runtime.sendMessage is delivered to every listener in the extension, the service worker included. If the worker's listener answers anything it doesn't recognise with an "unknown message" error, that answer wins the race and the page gets an error in a millisecond. So the first line of the worker's listener is now if (request.target === 'offscreen') return false; and the offscreen listener does the mirror check.
How I tested it without waiting 5 minutes
You don't need a slow PC to reproduce this. With the Chrome DevTools Protocol you can stop all service workers on demand: ServiceWorker.stopAllWorkers. The end-to-end test starts an upscale, kills the worker in the middle of the run, and waits for the result. On the old code the test fails with exactly the error users saw. On 1.1.1 the picture arrives.
The result
Version 1.1.1 went live on October 7. Since then the port error count is zero, on 1.1.1 and on 1.1.2 after it, while dozens of users have run enhancements. It is not the end of long-run problems (a busy model still makes a second tab wait), but no one loses a finished result anymore.
Takeaways for MV3 extensions
Never keep a service worker waiting on work that can take minutes. Let it start things, not carry them.
Talk to the offscreen document directly with a target field, and make every other listener ignore messages that aren't addressed to it.
Kill the worker in your tests with ServiceWorker.stopAllWorkers. If a run survives that, it survives a slow PC.
Map Chrome's channel errors to a human message in the user's language. A raw "message port closed" is an uninstall.
If you want to see the extension this happened in, it is Image Enhancer, a free AI photo upscaler that runs in your browser (https://www.devexthub.com/image-enhancer/?utm_source=devto). Happy to answer questions about running ONNX models in an offscreen document.

I built a bank statement to Excel converter that never uploads your file

Nichola Chaus — Sat, 10 Oct 2026 07:06:33 +0000

Most "bank statement to Excel" sites ask you to upload the PDF to their server. For a document with your account number, balance and every purchase you made this month, that always felt like the wrong trade.

So the converter I built runs entirely in the browser. Here is how it works and where it breaks.

Reading the PDF without a server

A statement downloaded from online banking is a text PDF: every date, description and amount is a positioned text item. pdf.js (Mozilla's PDF engine, the one inside Firefox) gives you those items with x/y coordinates via page.getTextContent(). No upload, no OCR, it all happens on the page.

Turning text items into a table

Cluster items into rows by their y coordinate, with a tolerance of about half the median text height.
Find column anchors by clustering x positions across rows.
Amounts are right-aligned, so their left edges drift (-4.75 vs -1,450.00). Before clustering I snap numbers that share a right edge to one column. Without this, debit and credit columns fall apart.
Merge sparse columns, cap the width, glue wrapped lines back to their row.

That last step bit me this week: a receipt with amounts like RUB 5,476.82 was not recognised as numbers, so the bigger bold total drifted left and got glued to the line above. Currency codes are now treated like currency symbols.

Writing a real .xlsx

An .xlsx is a zip of XML files, so JSZip plus a small writer is enough. The first version wrote every cell as text, and SUM over the amount column returned 0. Now amounts like 1,488.00, (25.40) or 1.488,00 are written as numbers, while dates and account numbers with leading zeros stay text.

Where it breaks

[x] Scanned statements are pictures, there is no text layer. The page detects this and says so instead of producing an empty sheet.
[x] The whole page lands in the sheet: bank name, address, footers. You delete those rows.
[x] Very wide statements (more than 6 columns) get neighbouring columns merged.

You can try it here, the file never leaves your machine: https://www.devexthub.com/pdf-to-excel/bank-statement/

If you have a statement layout that comes out wrong, I'd like to hear about it.