DEV Community: navidseyedain

How I Built ALAD Mobile: Real-Time AI Audio Dubbing for Any Android App

navidseyedain — Tue, 11 Aug 2026 01:07:52 +0000

Have you ever wanted to watch a foreign YouTube video, a live stream on Twitch, or a show on Netflix without constantly staring at subtitles?

To solve this, I built ALAD Mobile (Live Audio Dubbing) — an open-source native Android application that captures internal system audio in real-time, streams it through Google's Gemini Multimodal Live API, and instantly plays back the translated voice over any active media.

💡 What It Does

ALAD Mobile creates a system-wide floating overlay widget that sits comfortably over your active applications. Once activated:

🎧 Intercepts Audio: It captures the internal playback audio of any app playing media.
⚡ Live Translation: Streams audio chunks to Gemini's bidirectional Multimodal Live API via WebSockets.
🎙️ Real-Time Playback: Plays back the dubbed voice in real time in your selected language (supporting 78 languages).
📱 Seamless Overlay: Lets you toggle dubbing or switch target languages without ever leaving YouTube, Netflix, or Spotify.

🛠️ Tech Stack & Architecture

Building low-latency real-time voice translation on mobile required a carefully structured stack:

Language: 100% Kotlin
UI Framework: Jetpack Compose (Material 3)
Async Programming: Kotlin Coroutines, StateFlow, and SharedFlow
Networking & Streaming: WebSockets for low-latency bidirectional audio streaming
Android Audio APIs: MediaProjection / AudioRecord for internal audio capture + AudioTrack for low-latency playback
System Service: Floating Overlay Window using TYPE_APPLICATION_OVERLAY
AI Engine: Google Gemini Multimodal Live API

🧠 How It Works Under the Hood

1. Internal Audio Capture

Using Android's MediaProjection API and AudioRecord, the app captures raw PCM audio streams from the device's internal sound engine without relying on external microphone input.

2. Bidirectional Live WebSockets

Traditional REST API requests introduce too much latency for live translation. ALAD Mobile opens a persistent WebSocket connection directly to Gemini's Live API, continuously streaming PCM audio chunks in real-time.

3. Audio Track Playback

As Gemini processes and translates the stream, incoming audio payloads are decoded and pushed directly into a low-latency AudioTrack buffer for smooth, near-instant voice playback.

4. Floating System Overlay

To ensure zero friction, a Floating Control Widget built with Android's System Alert Window lets users control dubbing and switch target languages seamlessly over any running app.

🚀 Technical Challenges & Lessons Learned

Minimizing Latency: Balancing buffer sizes between raw PCM capture and WebSocket transmission frames was crucial for keeping speech output synchronized with active playback.
State Management: Handling connection lifecycle events, audio buffer underruns, and background service states seamlessly using Kotlin StateFlow.

🌐 Open Source & How to Try It

ALAD Mobile is 100% Free and Open Source. It runs using a free API key directly from Google AI Studio — no subscriptions or user accounts required.

🔗 GitHub Repository: navidseyedain/ALAD-Mobile

If you find this project interesting, feel free to drop a ⭐ star on GitHub, submit a PR, or leave your architectural feedback below!

Building ALAD: Real-Time AI Video Audio Dubbing in Chrome with Gemini API & WebSockets

navidseyedain — Wed, 05 Aug 2026 22:05:14 +0000

Building ALAD: Real-Time AI Video Audio Dubbing in Chrome with Gemini API & WebSockets
Imagine watching a video, live stream, or online course in a language you don't speak, and hearing it dubbed in real-time into your native language—right inside your browser, with almost zero latency.

That is exactly why I built ALAD (Live Audio Dubbing / LAD), an open-source Chrome Extension that leverages Google’s Gemini API and WebSockets to capture live tab audio, translate it on the fly, and play back natural dubbed audio alongside the original video.

In this article, I’ll walk through why I created ALAD, how its architecture works under the hood, the technical hurdles of handling real-time audio in Chrome Manifest V3, and how you can try or contribute to the project!

💡 The Problem: Why Real-Time Dubbing?
Subtitles are great, but they require your constant visual attention. If you are watching a technical tutorial, a lecture, or a fast-paced presentation in a foreign language, scanning subtitles while trying to focus on code or visual slides can be exhausting.

Existing dubbing solutions usually fall into two categories:

Offline post-processing tools: High quality, but slow and impossible for live or streaming video.

Heavy cloud SaaS platforms: Expensive, subscription-based, and locked behind proprietary players.

I wanted a solution that was lightweight, open-source, privacy-friendly, and integrated directly into the browser.

🚀 Introducing ALAD (Live Audio Dubbing)
ALAD connects your active browser tab directly to Gemini's real-time multimodal capabilities using WebSockets.

Key Features:
🎙️ Real-Time Tab Audio Capture: Captures clear raw audio directly from YouTube, Udemy, Twitch, or any HTML5 video player without capturing room ambient noise.

⚡ Low-Latency Streaming: Streams audio chunks over WebSockets for rapid response and minimal delay between visual speaker cues and audio output.

🤖 Powered by Gemini API: Uses Gemini's advanced multimodal audio understanding and natural speech generation.

🎚️ Smart Audio Ducking & Volume Balancing: Lowers the original video audio slightly while playing the dubbed translation so you can comfortably hear both or focus solely on the translated audio.

⚙️ Manifest V3 Compliant: Built strictly for modern Chrome extension security and performance standards.

🔑 BYO Key (Bring Your Own API Key): Keeps your requests private and cost-effective—your API key is stored locally in your browser.

🛠️ Tech Stack & Architecture
Here is how data flows through ALAD in real time:

[ Active Tab (HTML5 Video) ]
│
▼ (tabCapture API / Web Audio API)
[ Audio Processing & PCM Chunking ]
│
▼ (WebSocket Stream)
[ ALAD Background Engine ]
│
▼ (Gemini API Multimodal Endpoint)
[ AI Translation & Audio Synthesis ]
│
▼ (AudioBuffer Queue / Playback)
[ Dubbed Audio Streamed to User ]
Main Technologies Used:
Frontend / Popup: HTML5, CSS3, Modern JavaScript (ESNext)

Extension Platform: Chrome Extension API Manifest V3 (Service Workers, Offscreen Documents, tabCapture)

Audio Engineering: Web Audio API (AudioContext, PCM 16-bit encoding)

Real-Time Transport: WebSockets for low-latency full-duplex binary audio streaming

AI Engine: Google Gemini API (Multimodal Live API)

🔬 Under the Hood: Key Technical Challenges
Developing a real-time audio extension on Manifest V3 came with a few interesting technical challenges:

Tab Audio Capture in Manifest V3 In Manifest V3, background scripts run inside transient Service Workers, which do not have access to the DOM or AudioContext.

To solve this, ALAD utilizes Chrome’s Offscreen API (chrome.offscreen). When audio capturing starts:

The background service worker spawns an offscreen document.

The offscreen document calls chrome.tabCapture.getMediaStreamId() to capture the active tab's media stream.

Raw audio is piped into a MediaStreamAudioSourceNode inside an AudioContext.

...JavaScript
// Offscreen script snippet for tab audio capturing
const stream = await navigator.mediaDevices.getUserMedia({
audio: {
mandatory: {
chromeMediaSource: 'tab',
chromeMediaSourceId: streamId
}
}
});

const audioContext = new AudioContext({ sampleRate: 16000 });
const source = audioContext.createMediaStreamSource(stream);
// Process and chunk PCM audio data...

...

PCM Audio Chunking & WebSocket Encoding For Gemini to process live audio seamlessly, audio must be downsampled (typically to 16kHz mono) and chunked into base64-encoded PCM frames before being pushed through the WebSocket connection.

We process incoming audio buffer arrays, convert floating-point samples to 16-bit signed integers (PCM16), and dispatch small binary frames continuously:

JavaScript
function convertFloat32ToPCM16(buffer) {
let l = buffer.length;
let buf = new Int16Array(l);
while (l--) {
buf[l] = Math.min(1, Math.max(-1, buffer[l])) * 0x7FFF;
}
return buf.buffer;
}

Jitter-Free Audio Queue & Playback Synchronization Receiving streamed audio chunks over a network means packet arrival times can fluctuate. If played back immediately upon receipt, the synthesized voice would sound choppy.

ALAD implements a custom Audio Queue Scheduler. Received PCM chunks are decoded into AudioBuffer objects and scheduled sequentially on the timeline (audioContext.currentTime), creating continuous, stutter-free real-time dubbing.

📦 How to Install and Run ALAD Locally
You can test ALAD right now on your machine in just a few minutes:

Prerequisites:
Google Chrome (or any Chromium-based browser like Brave or Edge)

A Gemini API key (obtainable for free from Google AI Studio)

Installation Steps:
Clone the repository:

Bash
git clone https://github.com/navidseyedain/ALAD.git
cd ALAD
Load into Chrome:

Open Chrome and navigate to chrome://extensions/

Enable Developer mode (toggle in the top right corner).

Click Load unpacked.

Select the ALAD project directory.

Configure & Start Dubbing:

Click the ALAD icon in your Chrome toolbar.

Paste your Gemini API key in the settings tab.

Select your target language.

Open any video on YouTube, click Start Live Dubbing, and enjoy!

🛣️ What's Next? (Roadmap)
ALAD is an active open-source project, and there are several exciting features planned:

[ ] Multi-speaker Detection: Identifying different voices in the video and assigning distinct synthetic voices.

[ ] Offline Cache & Subtitle Overlay: Generating dual synchronized subtitles alongside audio dubbing.

[ ] Chrome Web Store Publishing: Packaging and releasing ALAD to the official Chrome Web Store for one-click installation.

[ ] Custom Voice Parameters: Pitch, speed, and emotion adjustments for dubbed voices.

🤝 Open Source & Contributions
ALAD is completely open-source and released under the MIT License. Contributions are super welcome! Whether you want to fix a bug, improve audio synchronization, refine the UI, or add support for new languages, feel free to open an issue or pull request.

🌐 GitHub Repository: https://github.com/navidseyedain/ALAD

👤 Author: Navid Seyedain (@navidseyedain)

If you find this project useful or interesting, please consider dropping a ⭐ Star on the repository to support development!

Thank you for reading! Feel free to leave your thoughts, feedback, or questions in the comments below.

Gemini SEO - Parallel Multi-Agent SEO & GEO Engine

navidseyedain — Mon, 27 Jul 2026 17:39:41 +0000

Search Engine Optimization (SEO) is evolving rapidly into Generative Engine Optimization (GEO). To help developers and webmasters optimize their content for modern AI search engines, I built Gemini SEO.

🚀 What is Gemini SEO?

Gemini SEO is a high-performance, parallel multi-agent engine built with TypeScript and powered by Google's Gemini API. It automates deep technical SEO and GEO audits using specialized autonomous AI agents running concurrently.

✨ Key Features

⚡ Parallel Multi-Agent Execution: Runs multiple AI agents in parallel for fast, multi-dimensional analysis.
🤖 Generative Engine Optimization (GEO): Prepares your content not just for traditional search engines, but for modern AI search platforms.
🛡️ Built with TypeScript: Fully typed, scalable, and maintainable codebase.
🎯 Actionable Insights: Generates structured reports and actionable optimization suggestions.

🛠️ Getting Started

You can easily set up and run the project locally:


bash
# Clone the repository
git clone [https://github.com/navidseyedain/gemini-seo.git](https://github.com/navidseyedain/gemini-seo.git)

# Navigate to directory
cd gemini-seo

# Install dependencies
npm install

Make sure to set your Gemini API key in your environment variables before running the engine.
🔗 Open Source & Contribution
The project is completely open-source under the MIT License. I would love to get feedback from the community!
GitHub Repository: navidseyedain/gemini-seo
⭐ If you find this project useful, please consider giving it a star on GitHub! It helps the project reach more developers.
Feel free to leave your thoughts, questions, or feature requests in the comments below!