DEV Community

Cover image for Building a Portuguese (PT-BR) Audio & Video Transcription Pipeline with Whisper Large-v3, Pyannote Speaker Diarization, and MCP
dongdong
dongdong

Posted on

Building a Portuguese (PT-BR) Audio & Video Transcription Pipeline with Whisper Large-v3, Pyannote Speaker Diarization, and MCP

#ai #opensource #typescript #tutorial

Transcribing multi-speaker audio and video in Brazilian Portuguese (PT-BR) comes with unique acoustic challenges: regional accents, colloquial expressions, overlapping speech in interviews, and low-bitrate voice notes (such as .opus / .ogg files from WhatsApp).

In this guide, we walk through how we architected Transcreva — a fast, browser-accessible AI transcription and subtitle generation platform with automatic speaker diarization — and how we exposed its workflow to AI coding agents using the Model Context Protocol (MCP).

1. Acoustic Transcription + Speaker Diarization Architecture

When you run standard speech-to-text on a 45-minute podcast or academic interview, a wall of unsegmented text is hard to read. To produce dialogue-ready transcripts with Speaker 1, Speaker 2 labels and word-level timestamps, we combine two stages:

  1. ASR (Automatic Speech Recognition) with Whisper Large-v3: Generates high-accuracy Portuguese and multilingual text segments with word-level alignment.
  2. Speaker Diarization with Pyannote: Computes speaker embeddings across sliding windows, clusters speaker turns, and aligns each Whisper word timestamp with the active speaker cluster. 
interface TranscriptSegment {
  id: string;
  speaker: string; // e.g., "Speaker 1"
  startTime: number; // seconds
  endTime: number;
  text: string;
}

export function formatSrtTimestamp(seconds: number): string {
  const hrs = Math.floor(seconds / 3600);
  const mins = Math.floor((seconds % 3600) / 60);
  const secs = Math.floor(seconds % 60);
  const ms = Math.round((seconds % 1) * 1000);
  return `${String(hrs).padStart(2, "0")}:${String(mins).padStart(2, "0")}:${String(secs).padStart(2, "0")},${String(ms).padStart(3, "0")}`;
}
Enter fullscreen mode Exit fullscreen mode

2. Handling WhatsApp Voice Notes (.opus / .ogg) and Video Subtitles (.srt / .vtt)

Two of the most common real-world workflows for Brazilian journalists, researchers, and creators are:

  • WhatsApp Audio Transcription: Voice notes exported from WhatsApp use the Opus codec inside an .ogg or .opus container. Our dedicated WhatsApp Audio Transcription workflow normalizes sample rates on the fly and outputs clean paragraphs or speaker-labeled dialogue.
  • Automatic Subtitle Generation: Video editors working on YouTube, Reels, or TikTok need frame-accurate captions. Using word-level timestamps from Whisper Large-v3, our Automatic Subtitle Generator segments lines by character-per-second (CPS) reading speed and exports ready-to-use .srt and .vtt files.

3. Integrating with Cursor & Claude Desktop via MCP (transcreva-mcp)

To bring transcription utilities directly into AI coding assistants (Cursor, Claude Desktop, Windsurf), we also built an open-source Model Context Protocol server: transcreva-mcp.

You can add it to your MCP configuration in seconds:

{
  "mcpServers": {
    "transcreva": {
      "command": "npx",
      "args": ["-y", "transcreva-mcp"]
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Try It Out

Top comments (0)