Building Browser-Based Transcription: Workers, Cancellation, and Honest Progress

A transcription demo can be surprisingly short: load a model, pass it some audio, display the text.

A transcription tool needs to answer more awkward questions.

Is it downloading the model or processing the recording? What happens when someone cancels halfway through? Does “local” mean it works without an internet connection? And what happens to a sentence that crosses the boundary between two audio windows?

I’m the author of FlowPick, a browser extension and collection of audio and video tools. This article covers the local transcription implementation in its web tools: Transformers.js for recognition, Mediabunny for reading media, and a dedicated Web Worker for each task.

It is an implementation walkthrough, not an accuracy or performance benchmark.

The file stays local. The model still has to arrive.

FlowPick’s local transcription path reads a user-selected file and runs recognition in the browser. It does not upload that media file for transcription.

The application still downloads model and runtime files when needed. These are different kinds of network traffic, and the interface should explain the difference.

The current implementation offers the multilingual Whisper Tiny and Base models through Transformers.js. Model and runtime assets are served from FlowPick’s static asset host.

The processing path looks like this:

This diagram describes the transcription path. Website analytics and optional cloud services are separate concerns; running inference locally does not establish that an entire website makes no network requests.

Give each task its own Worker

FlowPick creates a new module Worker when transcription starts:

const worker = new Worker(
  new URL('../workers/audio-to-text.worker.ts', import.meta.url),
  { type: 'module' }
)

worker.postMessage({ file, settings })

The Worker owns input inspection, model initialization, audio decoding, and inference. The main thread owns the interface.

This keeps the processing loop away from UI work. It also gives the task a clear lifetime: one transcription task, one Worker.

There is a cost. A new task initializes a recognizer again, even when its model files are already cached. A persistent Worker could reuse the model instance, but would need more lifecycle management around cancellation, stale messages, and concurrent requests.

For this implementation, task isolation makes that ownership easier to follow.

Progress is several different measurements

A single percentage cannot explain the whole job.

Downloading a model has a byte count. Transcribing a recording has a position on the media timeline. Checking whether an audio track can be decoded has neither.

FlowPick sends separate events for these:

type TranscriptionEvent =
  | {
      type: 'stage'
      stage:
        | 'checking'
        | 'loading'
        | 'decoding'
        | 'detecting'
        | 'transcribing'
    }
  | {
      type: 'download'
      file: string
      progress: number | null
      loaded: number
      total: number
    }
  | {
      type: 'segments'
      segments: TranscriptSegment[]
      progress: number
      processed: number
      duration: number
    }
  | { type: 'language'; language: string; confidence: number }
  | { type: 'complete' }
  | { type: 'error'; code: string }

During model loading, the UI can show the file being downloaded. When its total size is unavailable, progress remains indeterminate.

During recognition, progress comes from the end of the completed audio interval:

progress = coreEnd / duration * 100

That means “this much of the recording has been processed.” It is not an estimate of elapsed computing time.

The distinction matters. Reaching 50% of the audio does not promise that the remaining work will take exactly as long as the first half.

Decode manageable windows

Before loading the recognizer, the Worker checks the input:

• Can the container be read?
• For the video tool, is there a video track?
• Is there an audio track?
• Can that audio track be decoded?
• Is its duration finite and greater than zero?

A file extension alone cannot answer these questions.

The Worker then uses Mediabunny’s AudioSampleSink to read audio over bounded intervals. Samples are downmixed and resampled into a 16 kHz mono Float32Array.

FlowPick uses 25-second core intervals with up to 2.5 seconds of context on either side:

for (let coreStart = 0; coreStart < duration; coreStart += 25) {
  const coreEnd = Math.min(duration, coreStart + 25)

  const start = Math.max(0, coreStart - 2.5)
  const end = Math.min(duration, coreEnd + 2.5)

  const audio = new Float32Array(
    Math.ceil((end - start) * 16_000)
  )

  // Decode this interval, then run recognition.
}

The maximum interval is 30 seconds, matching Whisper’s audio-window design. OpenAI describes that design in the Whisper repository.

The mono input array for a full window occupies about 1.92 MB:

30 seconds × 16,000 samples/second × 4 bytes/sample

That is only the input array. It excludes model weights, inference tensors, decoder buffers, the selected file, and accumulated transcript data.

Windowed decoding bounds this part of memory use. It does not make the whole application constant-memory.

Overlap needs an ownership rule

Context helps when a sentence crosses a window boundary. It can also produce duplicate text.

FlowPick assigns each recognized segment to a core interval according to the midpoint of its timestamp range. Simplified from the implementation:

const midpoint = (segment.start + segment.end) / 2

if (midpoint < coreStart || midpoint >= coreEnd) {
  continue
}

The timestamps are first converted from window-relative positions to positions in the original recording.

There is also a reconciliation step for a specific boundary case: a truncated cue at the end of one window followed by a fuller, overlapping recognition in the next. It compares normalized text prefixes before replacing the earlier cue.

These are application heuristics. They do not guarantee perfect sentence boundaries or remove the need to review subtitles.

Auto-detection should be allowed to say “uncertain”

Automatic language selection needs more than leaving a setting blank.

The implementation includes an explicit language-token prediction step. It passes the audio features and decoder start token to the model, then compares scores for the supported language tokens.

FlowPick’s acceptance rule requires:

• A normalized top score of at least 0.5.
• A margin of at least 0.15 over the next candidate.

These are application thresholds, not a published accuracy guarantee. The resulting “confidence” is a normalized score among language-token candidates, not a calibrated probability that the language is correct.

If the result is too ambiguous, the Worker returns language-uncertain so the user can choose a language.

The detected language is reused for subsequent windows in the task. That also means this approach should not be presented as reliable language switching for multilingual conversations.

WebGPU support is only the first check

A browser exposing navigator.gpu does not prove that GPU inference is the best default for that device.

FlowPick uses a conservative recommendation function. It considers whether an adapter is available, reported memory, buffer limits, and adapter information. Unknown or integrated adapters generally receive the WASM recommendation.

This is a product heuristic, not a hardware benchmark.

The current recognizer configuration also differs by backend:

const dtype = device === 'webgpu'
  ? {
      encoder_model: 'fp32',
      decoder_model_merged: 'q4'
    }
  : 'q8'

The WASM runtime is configured for one thread in this path.

In automatic mode, a WebGPU model-loading failure can trigger a retry using WASM. That retry is deliberately narrower than “retry every GPU error”: it applies to model initialization failures, rather than silently restarting every failed transcription.

For the relevant API and quantization options, see the Transformers.js documentation.

Cancel must end the task

Changing a button label to “Cancelled” is not sufficient if inference continues running.

FlowPick’s task wrapper terminates the Worker and rejects the task promise:

cancel() {
  if (settled) return

  finish() // marks settled and calls worker.terminate()

  rejectTask(
    new DOMException('Cancelled', 'AbortError')
  )
}

The UI treats AbortError as cancellation rather than a processing failure. Segments already delivered to the main thread can remain available as a partial result.

There is an important cleanup detail here. Worker.terminate() stops the Worker immediately; it does not wait for the Worker’s code to finish.

The Worker’s normal path has a finally block that disposes the media input and recognizer. Forced termination must not depend on that block running.

This design uses the Worker lifetime as the cancellation boundary. It does not promise that every underlying resource is reclaimed synchronously at the moment the user clicks Cancel.

Export is part of the implementation

FlowPick returns editable segments with a small internal shape:

interface TranscriptSegment {
  start: number
  end: number
  text: string
}

The exporter supports TXT, SRT, and WebVTT. Before formatting timed subtitles, it removes invalid or empty cues, sorts them, and clips an end time against the next cue’s start.

It also prevents edited text containing blank lines or --> from accidentally creating another cue structure.

For WebVTT, the output includes the WEBVTT header and dot-separated milliseconds. The format is defined in the W3C WebVTT specification.

Formatting valid subtitle text and recognizing speech accurately are separate jobs. Both need attention.

What I would verify before trusting a result

This article does not establish a word error rate, cross-browser compatibility matrix, or real-time processing speed.

For a particular recording and device, I would check:

• Cold-cache and warm-cache model loading separately.
• Cancellation during loading, decoding, and recognition.
• Silent input and files with no decodable audio.
• Speech crossing the 25-second core boundaries.
• Ambiguous language detection and manually selected language.
• Exported subtitles in the player where they will be used.

Names, technical vocabulary, overlapping speakers, and noisy recordings still deserve manual review.

The model call is one part of this tool. Most of the surrounding implementation exists to make its behavior understandable: which stage is running, what has been processed, what failed, and whether the user can stop it.

That is the part I would build early in another browser transcription project, before spending much time polishing the transcript screen.

---

Disclosure: I’m the author of FlowPick. This article was drafted with AI assistance based on the project’s implementation. It does not report new performance or accuracy measurements.