How Myna Player Generates Context-Aware Subtitles Ahead of Playback

Subtitles have a timing problem that becomes more visible when AI translation enters the pipeline.

Translate every tiny fragment immediately and the text may arrive on time, but important context disappears. Wait for a longer passage and the translation becomes more coherent, but the subtitle may appear after the dialogue has already passed.

That creates a practical tradeoff:

speed or context.

I wanted to see whether a media player could avoid making that choice at playback time.

That became Myna Player, an open-source, local-first AI video player that processes audio ahead of the current playback position, transcribes it locally, translates finalized subtitle cues with surrounding context, and renders them against the native player clock.

Myna Player is currently a public alpha. The core macOS application works, Windows code and packaging are under active verification, and the first stable public release has not been published yet.

This article explains the architecture, the subtitle timing contract, and why processing ahead of playback changes the problem.

The product boundary

Myna Player is not a general-purpose transcription editor with a video preview attached.

It is a media player with a just-in-time subtitle pipeline.

That distinction affects the architecture.

The system must keep several things true at the same time:

• playback remains responsive;
• transcription stays ahead of the viewer;
• seeking does not corrupt old work;
• completed subtitle work can resume later;
• translation receives enough context;
• translated cues keep the original timing;
• local media and audio remain on the device;
• the renderer never waits for an AI request.

The simplified flow is:

native player clock
        |
        v
persistent priority scheduler
        |
        v
FFmpeg audio windows
        |
        v
local Whisper worker + VAD
        |
        v
timed source subtitle cues
        |
        +--------------------+
        |                    |
        v                    v
local source cache     optional translation workers
        |                    |
        +----------+---------+
                   |
                   v
       synchronized subtitle renderer

The key idea is that the expensive work happens outside the playback loop.

Why listening to system audio would be the wrong abstraction

A live-caption application often listens to whatever audio is currently being played.

That is useful when the source is unknown or truly live.

Myna Player works with local media files. It can read the source directly.

That means it does not need to wait for sound to leave the speakers before analyzing it. It can extract audio from later sections of the file and prepare subtitles before playback reaches them.

Direct file access provides several advantages:

• deterministic time ranges;
• repeatable extraction;
• precise stream selection;
• the ability to jump ahead;
• resumable cached work;
• no dependency on system-audio capture permissions;
• no feedback from notification sounds or other applications.

The media file itself becomes the source of truth.

The processing unit is a canonical time window

Myna Player schedules deterministic, non-overlapping audio windows.

The current design uses canonical 30-second ranges.

Conceptually:

00:00–00:30
00:30–01:00
01:00–01:30
01:30–02:00
...

Canonical windows are important because they give the cache stable identities.

If the same video is reopened, the application can determine exactly which regions were completed. If the user seeks, already finished windows do not have to be transcribed again.

The scheduler assigns priority according to playback position:

• the first required window is urgent;
• roughly the next 90 seconds are promoted;
• later windows remain background work.

The result is not simply “transcribe the entire file as fast as possible.”

It is:

Keep the region the viewer is about to reach prepared, while using spare capacity to extend the cache farther into the video.

Media fingerprints and pipeline keys

A file path alone is not a safe cache key.

The media may change. The selected audio stream may change. The speech model, language mode, VAD model, chunk size, or cue-segmentation rules may change.

Myna Player creates a media fingerprint and combines it with a pipeline key that includes inputs such as:

• selected audio stream;
• Whisper runtime and model hash;
• transcription language mode;
• VAD model;
• chunk and extraction parameters;
• cue-segmentation version.

Only checkpoints with an exactly matching key are restored.

This prevents a subtle class of bugs where old subtitle data appears valid even though it was produced by a different pipeline configuration.

A cache should not merely remember that work happened. It should remember which work happened under which assumptions.

Seeking requires generations, not only cancellation

A viewer may seek from minute 3 to minute 48 while transcription is processing minute 4.

The current worker needs to stop, but cancellation alone is not sufficient.

A slow process may still return after the seek. If its result is accepted blindly, stale cues from the previous playback region can overwrite or pollute the new session.

Myna Player increments a processing generation when the active region changes.

The seek flow is conceptually:

1. cancel the active ASR and translation connection;
2. increment the generation;
3. preserve completed cache entries;
4. promote the newly requested region;
5. reject late results from older generations.

Generations turn cancellation into a verifiable contract.

The system does not have to assume that every subprocess stopped instantly. It can recognize that an old result belongs to an obsolete plan.

Extracting audio with context

Each canonical window is expanded slightly during extraction.

Myna Player reads approximately two seconds of neighboring audio context around the canonical range and converts it to:

mono
16 kHz
signed 16-bit PCM

The context helps speech recognition around boundaries, but the persisted output still belongs to the canonical time range.

This is a common pattern in stream processing:

• read overlapping input for better local decisions;
• write non-overlapping canonical output for deterministic storage.

Without boundary context, words split across windows can be lost or duplicated. Without canonical output boundaries, overlapping windows can produce conflicting cues.

A long-lived local Whisper worker

Starting a speech-recognition process for every 30-second segment would add significant overhead.

Myna Player uses a long-lived local whisper.cpp worker.

The worker:

• binds only to a random 127.0.0.1 port;
• loads the selected model once;
• accepts extracted audio windows;
• can apply Silero voice activity detection;
• returns word-level timestamps;
• is terminated on cancellation or application shutdown.

Word timestamps are essential because the language model should not invent the subtitle timing.

The transcription layer produces timed words. A deterministic segmentation stage then converts those words into readable, non-overlapping subtitle cues.

That separation lets the system improve visual subtitle grouping without asking the translation provider to decide when a cue should appear.

Voice activity detection reduces wasted work

A 30-second media window may contain music, silence, or non-speech sound.

Optional Silero VAD identifies speech regions before transcription.

That can reduce unnecessary inference and avoid producing unstable text from silence or background audio.

VAD is part of the pipeline fingerprint because changing the speech-region detector can change the resulting transcript and cue boundaries.

Again, the cache key must represent the actual computation, not only the source file.

Persisting source cues transactionally

After a source window is transcribed and segmented, Myna Player stores the source cues and marks the checkpoint complete in the same SQLite transaction.

This prevents a failure state such as:

• checkpoint says complete;
• subtitle rows were only partially written.

Or the reverse:

• subtitle rows exist;
• checkpoint still says incomplete;
• the application repeats the work and creates overlap.

The storage layer uses SQLite with WAL mode and versioned migrations.

The database contains local information such as:

• media fingerprints;
• playback position;
• processing checkpoints;
• source transcript segments;
• provider-specific translations;
• model metadata;
• user corrections.

Persistent storage is what turns the look-ahead pipeline from a temporary live effect into resumable product behavior.

Transcription and translation are separate workers

Translation should not block source transcription.

Myna Player runs ASR and cloud translation as independent scheduling lanes.

The source pipeline can continue preparing timed cues while the translation worker processes bounded groups of completed cues.

This matters because cloud providers have variable latency. A slow translation response should not stop the application from building the source-language cache.

It also means a user can switch translation providers or target languages without retranscribing the media.

Source text and provider-specific translations are stored separately.

The structure is conceptually:

source cue
  ├── DeepL / French
  ├── Gemini / Turkish
  ├── OpenAI / English
  └── MiniMax / German

The source timing remains the shared foundation.

The translation contract: context may change words, not time

This is the central rule of the translation layer.

A translation provider receives:

• a bounded batch of finalized cues;
• stable cue IDs;
• limited previous-dialogue context;
• the requested target language;
• instructions to return every requested cue exactly once.

The provider may improve wording using context.

It may not:

• merge cue IDs;
• split one cue into new IDs;
• reorder cues;
• omit cues;
• duplicate cues;
• invent timestamps;
• change the source timing.

Responses are validated before persistence.

Missing, duplicate, unknown, or reordered IDs are rejected.

This creates a useful division of responsibility:

Layer	Responsibility
ASR	Recognize words and timestamps
Cue segmentation	Create readable timed source units
Translation	Produce context-aware text for those units
Player clock	Decide which cue is visible now

The language model handles language. Deterministic code handles time.

Why surrounding context still matters

A subtitle cue is often too small to translate correctly in isolation.

Pronouns, formality, verb tense, gender, idioms, and ambiguous words may depend on previous dialogue.

Sending the entire film for every request would be expensive, slow, and unnecessary.

Myna Player sends bounded prior dialogue context with the active batch.

The contextual text helps the provider interpret the current lines, but only the requested cue IDs are accepted as output.

This separates context for understanding from scope for mutation.

The provider can read more than it is allowed to rewrite.

That is a useful pattern beyond subtitles. AI systems often benefit from broad context while still needing narrow, validated output boundaries.

Source edits invalidate translations

Users can correct subtitle text and timing.

When the source text of a cue changes, every provider translation derived from the old text becomes stale.

Myna Player invalidates those translations rather than silently continuing to display them.

Source and translated data therefore have an explicit dependency relationship.

A correction is not just a visual edit. It changes the input to downstream language processing.

Incremental updates over Tauri Channels

The UI does not reload the entire subtitle set every time a worker finishes a window.

The backend sends:

1. one full initial snapshot;
2. ordered incremental cue patches.

These updates travel through Tauri Channels.

Ordering matters because source cues, translations, corrections, and invalidations may arrive from different workers.

The frontend applies a known sequence of patches instead of trying to reconstruct state from unordered events.

This keeps the WebView focused on presentation while the native backend owns persistence and pipeline state.

Rendering against a native player clock

The subtitle renderer does not call Whisper or a translation API.

It reads already cached cues and compares them to a player clock updated at roughly 100 ms intervals.

Cue lookup uses binary search rather than scanning every subtitle on every tick.

This provides a strict runtime rule:

Playback rendering must remain deterministic even when background AI work is slow, offline, or failing.

If the next translated cue is not ready, the system can display source text or wait according to the selected mode. It does not freeze playback while waiting for a model.

Native video below a transparent WebView

Myna Player uses Tauri 2 and a Leptos interface compiled to WebAssembly, but video rendering is native.

On macOS, libVLC renders into an application-owned NSView positioned below the transparent Tauri WebView.

The web layer owns controls and generated subtitles. AppKit owns the video surface geometry and updates it during resize and fullscreen transitions.

On Windows, the application creates a child HWND below WebView2 and binds libVLC to that surface.

This composition avoids forcing decoded video frames through the web interface.

It also creates a clear boundary:

• native layer: playback, clock, surfaces, media tracks;
• web UI: controls, settings, subtitles, progress state;
• Rust backend: scheduling, storage, processing, providers.

Unsafe libVLC interaction remains isolated inside a limited player crate.

The Rust workspace follows product boundaries

The repository is split into focused crates:

myna-player-core       domain models and scheduling
myna-player-media      FFprobe and FFmpeg integration
myna-player-player     native libVLC engine
myna-player-jobs       persistent priorities, retry, resume
myna-player-storage    SQLite/WAL and media fingerprints
myna-player-pipeline   ASR, VAD, translation, cue processing
myna-player-providers  provider registry and credentials
src-tauri              desktop composition and IPC
myna-player-ui         Leptos/WASM interface

The separation is not only organizational.

It helps keep several risky concerns from leaking into each other:

• shell process execution stays in media and pipeline boundaries;
• unsafe player calls stay in the player layer;
• credentials stay behind the provider boundary;
• persistent scheduling stays independent from UI state;
• serializable domain types remain usable in tests.

Local-first privacy

The following operations run on the user's computer:

• FFprobe media inspection;
• FFmpeg audio extraction;
• voice activity detection;
• whisper.cpp speech recognition;
• subtitle timing and segmentation;
• playback;
• editing and export;
• caching and checkpoint management.

Cloud translation is opt-in.

When enabled, the provider receives finalized transcript text and limited neighboring context. It does not receive the video file or extracted audio.

Provider credentials are stored through the operating system credential store and are never returned to the web UI.

The local Whisper worker binds only to localhost, and diagnostic logs are bounded and redact home-directory paths.

Local-first does not mean every optional feature is offline. It means the user can understand and control which data crosses the device boundary.

Per-video deletion

Generated data is grouped under the media fingerprint.

SQLite foreign keys use cascading deletion so a video can be reset atomically from the application's point of view.

Reset this video:

1. cancels the active ASR and translation generation;
2. pauses playback and seeks to the beginning;
3. deletes checkpoints, transcripts, translations, cache, and remembered position;
4. recreates only the open-media identity;
5. leaves global settings, installed models, and credentials intact.

This is another place where product wording matters.

“Clear cache” is too vague. A user should know whether a reset deletes subtitle history, corrections, translations, or provider credentials.

Packaging AI runtimes is part of the application

A local-first desktop AI product is not complete when it builds on the developer's machine.

Myna Player needs native runtimes and models such as:

• VLC and plugins;
• FFmpeg and FFprobe;
• whisper.cpp;
• optional Silero VAD data;
• Whisper model files.

The packaging scripts download or build pinned components, verify SHA-256 checksums, stage required licenses, and create native bundles.

Models are activated only after expected size and checksum verification.

The release workflow is designed to:

• build on clean macOS and Windows runners;
• verify packaged runtimes;
• exercise native playback smoke tests;
• sign nested binaries;
• notarize macOS artifacts;
• verify Windows signatures;
• publish checksums.

The project is not marked release-verified until those workflows pass with the required signing configuration.

That conservative status is intentional. A successful development build is not the same as a trustworthy public installer.

Export and interoperability

Myna Player can export:

• source subtitles;
• translated subtitles;
• dual-language subtitles;
• SRT;
• VTT.

The generated subtitles are not trapped inside the application.

Export is useful for users, but it is also a debugging tool. A persisted subtitle file makes it easier to inspect cue boundaries, timing drift, translation consistency, and corrections outside the player.

What I learned

1. Processing ahead changes the latency budget

The system does not need every AI operation to be faster than human speech. It needs the prepared region to remain ahead of the playback clock.

That turns the problem from instantaneous inference into scheduling.

2. Context and timing should belong to different layers

Translation needs surrounding dialogue. Timing should remain deterministic. Stable cue IDs create the boundary between them.

3. Cancellation needs versioned state

Generation numbers make late worker results rejectable. Without them, a seek can create subtle stale-data races.

4. Cache validity depends on the complete pipeline

A transcript produced with another model, language mode, VAD configuration, or segmentation version is not necessarily reusable.

5. AI must not enter the render loop

The player should remain responsive even when models are slow, credentials are missing, or the network is unavailable.

6. Packaging is an engineering domain of its own

Bundling VLC, FFmpeg, Whisper, models, licenses, signatures, and platform-specific surfaces required as much architectural attention as the subtitle pipeline.

7. Local-first is a data-flow promise

It is not enough to say “privacy focused.” The product must document which operations are local, what optional text leaves the device, where credentials live, and how generated data can be deleted.

Current status

Myna Player is open source under GPL-3.0-or-later and remains in public alpha.

The core macOS experience is integrated. Windows playback and packaging code exists and is being verified through clean-runner workflows. Signed public alpha artifacts and broader clean-machine smoke tests remain release milestones.

The repository includes architecture, privacy, security, contribution, packaging, release, and third-party licensing documentation.

ademisler / Myna-Player

Open-source, local-first AI video player for ahead-of-playback transcription, contextual translation, and precisely timed subtitles.

Myna Player

Watch in any language.

A local-first, open-source AI video player that transcribes ahead, translates with context, and displays each subtitle at the right moment

Website · Architecture · Roadmap · Contributing · Security

Project status

Myna Player is a public alpha. The core macOS application works, Windows code and packaging are under active verification, and the first stable public release has not been published yet. Expect interface, storage, and packaging changes before 1.0.

The repository is open for review and contribution. Please read CONTRIBUTING.md before proposing substantial changes.

Why Myna Player exists

Live subtitle systems often choose between speed and coherence. Translating tiny fragments quickly loses context; translating long blocks makes subtitles arrive late. Myna Player separates those concerns:

1. it processes audio ahead of playback;
2. Whisper produces word-level timing locally;
3. natural subtitle cues are derived from those timestamps;
4. translation sees surrounding context but may not rewrite cue…

View on GitHub

I would especially value feedback on these areas:

• How far ahead of playback should a subtitle pipeline normally process?
• Which subtitle segmentation rules produce the best balance between readability and faithful timing?
• Should contextual translation include only previous dialogue, or a limited amount of future dialogue as well?
• What should a player display when source subtitles are ready but translation is still pending?
• Which clean-machine tests would make you trust an alpha desktop AI application?

This is the fifth and final article in this first series documenting the products I have built—from a browser toolbox to local AI desktop systems.