Hinglish reel captions, made offline on your own laptop

This is a submission for the Hacktoberfest Week 2 "Side Hustle" challenge.

What I Built

Scroll through Indian Instagram and you will keep running into captions like this one:

aaj hum baat karenge ki kaise aap apne reels mein captions laga sakte ho

Hindi and English in one sentence, typed in English letters. It is how a lot of people talk and post, and Roman script is what many viewers find easiest to read on a phone.

Someone has to caption those reels, and for plenty of creators that someone gets paid: a freelance editor, a two-person agency, a cousin who "does the editing". Their problem is not that speech recognition can't hear Hindi, because open models like Whisper hear it well. The problem is what comes out. Hindi arrives in Devanagari, English words arrive in whichever script the model fancied, and nothing turns that into the spelling people actually type (kya, nahi, bahut) with a timestamp on every word. Cloud caption tools tend to bill by the minute or by the month, and they want the client's unreleased footage on their servers.

Hinglish Reel Captioner is a small local web app for that job. You drop in a video or a voice note. It listens with open-weight Whisper, converts each Hindi word to Roman Hinglish while leaving English words English, and plays big word-by-word captions over a live preview. You fix any word in an editor that keeps the timing intact, pick a look (or save the creator's brand colours), and either burn the captions into the MP4 or export SRT, VTT, ASS or a plain transcript.

The person who would pay is the person who captions other people's reels. Their price per reel is set by their clients, and their cost per reel is their time. The point of the tool is to turn the slowest part of that job, typing Hinglish against a waveform, into skimming and fixing, without adding a per-minute line to their costs.

• Roman Hinglish, word by word. क्या becomes kya, वीडियो stays video, एपीआई becomes API.
• An editor that keeps the timing. Respell a word and nothing moves. Add or delete words and the neighbours keep their timing.
• Four looks, your brand colours. Pumpkin, Pop, Classic and Box, with one-click "my brand look".
• Burned in, or as files. An MP4 with the captions drawn in, or .srt, .vtt, .ass and a plain transcript.
• Voice notes too. Audio-only files get a branded background and become a captioned video.
• My words. One line fixes a spelling for every future video: doston=dosto, or insta gram=Instagram.

Demo

There is nothing to sign up for and nothing to deploy, because the point is that it runs on your own machine. Demo mode plays a scripted Hinglish transcript over any video you drop in, so you can try the whole studio without downloading a model. You need Python 3.10 or newer and FFmpeg:

{% https://github.com/SaurabhEffect/HacktoberFest-2026/tree/main/hinglish-reel-captioner %}
./run.sh --demo

On Windows PowerShell, from the cloned project folder, run:

.\run.bat --demo

It opens http://127.0.0.1:8000. Drop a video and you land in the studio. The preview plays with the same fonts, line breaks and colours as the final render. The panel on the right has Captions (edit, split, join, click a time to jump), Style (four looks and every control behind them) and Export.

The Options panel holds the language, the speech model and the My words rules:

It is a small FastAPI app (reelcaptioner/) and a front end of plain HTML, CSS and JavaScript modules: no build step, no CDN, fonts bundled. There are 395 tests. The README covers the settings, the HTTP API (so a batch of reels can be captioned from a shell script) and troubleshooting.

How I Built It

I directed an AI coding agent (Claude) to build this. I chose the idea and the shape of the product, and the agent wrote and tested the code, so everything below is about the product and about what the tests in the repo actually check.

The open pieces

• Whisper (OpenAI, MIT licence), run through faster-whisper and CTranslate2, does the listening: word timestamps, voice-activity detection, your own names as hotwords, and a Hinglish prompt. Any CTranslate2 Whisper model works, so you can pick a size from tiny to large-v3, or point the Custom model box at a Hindi fine-tune.
• FFmpeg and libass probe the file, pull out 16 kHz audio, make a small preview copy and burn the captions in.
• FastAPI and plain HTML, CSS and JavaScript make the app.
• Ollama is optional. It runs a local model, Gemma by preference, to tidy spellings card by card.

Three problems that took more thought than the model

1. Whisper writes Hindi in Devanagari, so the app converts. Rather than coax Roman text out of the model, it lets Whisper write what it writes and converts each word afterwards, in a fixed order: your own rules first, then English loanwords (kept as English), then a lexicon of common Hindi words with their usual spellings, then rule-based transliteration for everything else. The rules are where the craft is. Devanagari writes an inherent a after every consonant and then does not pronounce some of them: समझ is samajh, not samajha, and बनता is banta, not banata. Deleting those schwas correctly does a lot of work for words that are in no lexicon. The whole thing is deterministic, so a wrong spelling is fixed once and stays fixed.

2. The preview has to match the render. The studio draws captions in the browser, but the export burns them in with libass. If the two disagree, the editor approves one thing and ships another, and by default they do disagree. Browsers place text using a font's ascent and descent (for Poppins, 1.05 and 0.35 em), while libass uses the Windows metrics (1.135 and 0.627 em) and scales font size by their sum. So the exporter converts every size (×1.762) and places each baseline explicitly, the server measures the line breaks once with the real font and hands the same ones to both renderers, and the outline is drawn in the browser as rings of hard text-shadows, because -webkit-text-stroke draws mitred spikes that libass does not.

To check it, a script captions the same flat-colour clip for nine looks and positions, once in the live preview in headless Chromium and once burned in by FFmpeg, and compares where the caption lands. The worst difference was 0.19% of the frame, about one pixel on a 640 px tall video. The tests derive each constant from the font file, so changing the font cannot quietly pull the two apart.

3. An editor that never loses the timing. Fixing a spelling moves nothing: when the word count is unchanged, every timing is kept. When you add or delete words, the words before and after the edit keep their timing and the new ones share the gap. The optional LLM pass follows the same discipline. It may only respell: each card has to come back with the same number of words and mostly the same letters, or the original card is kept, and it is only ever shown the text, so it cannot move a timestamp.

How it was tested without a model

The build environment could not download a Whisper model, so the tests exercise the real transcription code against a stand-in model that returns faster-whisper's real output shapes. Every FFmpeg render is run for real and measured pixel by pixel, the command-line tests boot an actual server, and the Ollama pass is tested against a fake one. There are 395 tests, passing on Python 3.10, 3.11, 3.12 and 3.13, each in a freshly built environment.

What that cannot prove is accuracy on real Hinglish audio, which depends on your recording and your model. That is why the editor is a first-class feature, and measuring word error rates on real reels is the first thing I would do next.

Why Does Open Innovation Matter?

Four things in this project only work because the model is open.

It costs nothing per reel. A captioning side hustle runs on thin margins and many small jobs. When the model runs on the captioner's own laptop, the hundredth reel costs the same as the first: a few minutes of CPU. There is no meter to watch and no plan to outgrow, so what they charge a client stays their own decision.

The footage stays put. Reels for a brand deal or a launch are often unreleased, and sometimes under NDA, by the time they reach the editor. With local inference the audio is extracted and transcribed on the machine that already holds the file. The server listens on 127.0.0.1 by default, there are no analytics and no CDN, and a test fails if the front end ever references an outside address. The only network traffic is the one-time model download.

The spelling is mine to change. Hinglish has no standard spelling (hai, hain, h), so the spelling layer is where the value is. Hosted captioning APIs typically hand back finished text in their own conventions. With open weights I get word timestamps and hotwords instead, and everything after the model is code I control: the romanizer, the My words rules and the editor. When someone says "my audience writes dosto, not doston", that is one line in a text box and it holds for every video after. When a better Hindi model appears, it is a drop-down choice or a folder path, not a migration.

Nobody can change the terms. The weights are a file on disk. They cannot be repriced, deprecated or rate-limited out from under a business that depends on them.

Where would a closed API have been better? Probably out-of-the-box accuracy on noisy audio, since a large hosted model will beat small on a laptop CPU, and setup is friction because you need Python and FFmpeg. I would rather say that plainly than pretend open wins every column. It wins the ones this customer cares about: cost per reel, where the footage lives, and who controls the spelling.