NVIDIA Nemotron 3.5 Transcribe: One Offline Checkpoint for Prompted Transcription, Speakers, and Translation

On 28 October 2026 NVIDIA released Nemotron 3.5 Transcribe under the OpenMDW-1.1 license. The checkpoint is a speech-augmented language model (SALM) that takes recorded audio plus a natural-language instruction and returns one of several output styles: verbatim transcript, cleaned written form, speaker-attributed text, rich transcription with events, translation, or English audio question answering. It covers 40 languages for transcription, 31 for translation to or from English, and speaker attribution in 13 languages with up to eight speakers. Maximum recording length is one hour (65 536-token context). It is explicitly offline—not a streaming recognizer.

The numbers that matter for local use: roughly 30 B total parameters with about 4.3 B active (including the audio encoder). Checkpoints ship in BF16, FP8, and NVFP4. On a single RTX PRO 6000 Blackwell with the NVFP4 checkpoint plus DFlash2 speculative decoding the published throughput reaches 2 238× real-time. Mean WER on the English Open ASR Leaderboard is reported at 4.71 % across nine public sets; multilingual Open ASR is 4.62 %.

Architecture that collapses the pipeline

Classic ASR pipelines split the work: acoustic encoder → CTC or RNN-T decoder → optional language-model rescoring → separate diarizer → post-processor for punctuation and capitalization. Nemotron 3.5 Transcribe folds most of that into one forward pass.

A parallel speech encoder (Transformer) and a frozen Nemotron 3 Sortformer diarizer read the same 16 kHz mono spectrogram. Their embeddings are projected into the Nemotron 3.5 Lightning hybrid Mamba–Transformer MoE backbone. Word-level timestamps come from a CTC alignment head on the encoder rather than from prompted generation. The instruction token sequence decides the output style at inference time; no per-style fine-tune is required.

Internal encoder chunks are 30 seconds—an implementation detail, not a streaming contract. The model still expects the full recording (or a long segment) before it can emit a coherent multi-speaker or translated result.

What the prompt actually controls

A short instruction selects the mode. Examples from the technical report and model card:

• verbatim: keep fillers, false starts, and exact wording
• written form: clean punctuation, capitalization, and normalized numbers/dates
• contextual: bias toward supplied vocabulary (proper names, product terms)
• speaker-attributed: prefix turns with speaker labels and optional timestamps
• translation: target language specified in the instruction

Because the backbone is an LLM, the same weights also answer questions about the audio content when the prompt asks for it (English-only evaluation). That is useful for post-call summaries but dangerous if you treat the model as a pure transcription engine—hallucinated details can appear when the instruction drifts.

Windows capture path and the offline constraint

For a desktop tool that must stay under a few hundred milliseconds of added latency on a live call or interview, the offline nature is the dominant constraint. You cannot feed the model 30-second chunks and expect progressive text the way a cache-aware FastConformer-RNN-T streaming model does.

Practical pattern on Windows:

1. Capture with WASAPI loopback (or exclusive-mode WASAPI) at 16 kHz mono. cpal or NAudio both work; the critical part is exclusive access so the audio engine does not resample behind your back.
2. Buffer until voice activity ends or a hard timeout (e.g. 20–30 s).
3. Optionally run a small local VAD (Silero) to trim silence before the heavy model.
4. Hand the trimmed buffer plus the instruction to the Transcribe checkpoint (vLLM, Transformers, or a NeMo-Speech.cpp / GGUF path when available).
5. Overlay the result.

// sketch: WASAPI loopback capture into a ring buffer
let device = find_output_device(name)?;
let default_cfg = device.default_output_config()?;
let config = StreamConfig {
    channels: 1,
    sample_rate: SampleRate(16_000),
    buffer_size: cpal::BufferSize::Fixed(480),
};
// build_input_stream on the output device yields loopback
// push frames into a lock-free ring; VAD or timer decides when to flush

The flush latency is dominated by the model, not the capture. On an RTX 5090-class card with NVFP4 the 2 238× figure means a 30-second utterance finishes in well under a second of wall time once the weights are resident. The cold-start and the decision to wait for end-of-utterance are the real costs.

Trade-offs versus streaming small models

A 600 M cache-aware FastConformer-RNN-T (the separate Nemotron 3.5 ASR streaming checkpoint) gives you partial hypotheses every 80–160 ms. Accuracy is lower, diarization is absent or requires a second model, and style control is limited to language ID or a short prompt kernel. You pay for progressive text with higher word error rate and the engineering of speculative decoding or partial-result stabilization.

Transcribe buys you one-checkpoint style control, speaker labels, and translation at the price of batching. For post-meeting notes or for an overlay that only updates when the speaker pauses, the trade-off is favorable. For a live subtitle that must track the speaker with <300 ms lag, you still want the small streaming model (or a cascade: small model for the overlay, Transcribe for the final clean version).

Practical notes for local deployment

• BF16 needs an 80 GB H100-class card for the reference profile. FP8 and NVFP4 drop that to a single high-end Blackwell workstation GPU.
• GGUF files exist for NeMo-Speech.cpp; expect the usual quantization quality drop on proper names and code-switched speech.
• Fine-tuning recipes live in NeMo Speech. Domain vocabulary can be injected via the instruction or via light adaptation; full speaker-adaptive training is still a multi-GPU job.
• The model is not a drop-in replacement for a real-time STT engine inside a Windows audio pipeline that must also drive an overlay at 60 Hz. Treat it as the high-quality offline or end-of-turn stage.

If you are already capturing loopback audio for a call or interview tool, the cheapest experiment is to keep the existing small streaming model for the live overlay and add a background Transcribe pass that writes the cleaned, speaker-labeled version once the segment ends. The weights are open; the latency budget is not.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.