DEV Community: Rahul Rao

Am I ready to sleep? I built a 75-second phone check to answer that

Rahul Rao — Sat, 14 Nov 2026 05:29:20 +0000

This is a submission for the Presage SmartSpectra SDK Challenge

What I Built

Every night I ask the same question: am I ready to sleep, or do I need a bit more time to get there? For me a good night starts with the state of mind I go to bed in. Sleep trackers tell me how I slept after the fact. I wanted a simple check-in before bed, while I could still try a wind-down routine. So I built Nidra to compare an evening pulse-and-HRV reading with my own recent evenings. It's an experiment, not a prediction of how I'll sleep.

Nidra is an Android app. You stand your phone up in room light, close your eyes, and for 75–90 seconds the camera reads your pulse and heart rate variability. You get one number, Sleep Readiness, 0 to 100%, and one thing to try tonight. Then you check again and see before and after.

In the morning you drag the same scale to say how rested you feel. Over the week, the history puts your evening number next to your mornings, so you can see whether it's worth listening to.

No wearable, no account, no server of Nidra's own. A phone you already own, and your eyes closed.

Demo

The voice-over uses an AI clone of my own voice. What the video shows, recorded hand-held on my phone with the waits sped up: a real check with eyes closed and the moon filling as readings arrive, the score and "See why", tonight's suggestion, the three themes and the privacy note in Settings, then a second check in the Diya theme, where Rotate turns the camera picture upright, and a different suggestion when Calm comes out lower.

Code

rahulrao85 / nidra

Sleep Readiness in 75 seconds: a camera check before bed using the Presage SmartSpectra SDK (Android, Kotlin)

Nidra

Most sleep trackers tell you how you slept once the night is over. Nidra asks a different question: how ready are you for sleep right now, while you can still do something about it?

Nidra is an Android app. Put your phone on a stand, close your eyes for 75–90 seconds, and the camera reads your pulse and heart rate variability. You get one number, Sleep Readiness from 0 to 100%, and one thing to try tonight.

Built for the Presage SmartSpectra SDK Challenge on DEV.

Nidra is a wellness estimate, not a medical device. It doesn't diagnose or treat anything, and it shouldn't guide health decisions.

The routine

Check, 75–90 seconds. It runs at least 75 seconds and up to 90 while it waits for an HRV reading. A tone tells you to close your eyes. At the end, one beep means it scored; three low beeps and…

View on GitHub

Kotlin and Jetpack Compose, com.presagetech:smartspectra 3.4.0 from Maven Central, MIT licence. The README has setup steps: add your own free Presage API key to local.properties and run on a physical Android phone. There is also a simulated mode that runs every screen without a camera or a key, and 14 unit tests for the scoring, all passing on 13-Nov-2026.

The whole SDK integration is one file: SdkVitalsSource.kt.

How It Responds to Human Signals

SmartSpectra signal	What Nidra does with it
Pulse rate (`cardio.pulseRateList`)	The Body slider: tonight's pulse against your own usual evening
HRV: RMSSD and Baevsky stress index (`cardio.hrvList`)	The Calm slider, 60% of the score
Confidence values and `ValidationStatus`	Decide whether to show a number at all
SDK `hint` text	Shown on screen so you can fix your framing
Blinking and face landmarks	Reduced on arrival to "eyes closed %" and a blink count, recorded but not scored. The points are dropped.

While you sit: the main slider and the moon move as readings arrive. That live number is only the pulse part, measured against your pulse earlier in the same check; the final score adds HRV. Tones mean you never have to open your eyes: a beep to start, one beep when it has scored, three low beeps and a buzz when it couldn't.
The score: Sleep Readiness is an experimental personal comparison, not a probability, a medical measure or a forecast of your sleep. Body (40%) and Calm (60%) combine into one number, rounded to the nearest 5%; the 40/60 weights are provisional. Both compare you with your own past evenings (the median of up to 14), not a population norm, so 50% means "your usual evening". Your first check has nothing to compare with, so it starts at 50%.
The suggestion: whichever slider is lower picks it. Lower Body: three minutes of guided slow breathing, paced towards 6 breaths a minute. Lower Calm: five minutes writing tomorrow's to-do list.
When it won't answer: Nidra withholds the score when pulse or HRV confidence is below threshold (40 for pulse, 50 for HRV), or when fewer than 70% of the sampled seconds had an accepted view. Brief invalid frames, like a second with no face, are tolerated.

What testing on a real phone taught me

All of this came from checks on my own phone (a Samsung Galaxy M35, which isn't on Presage's tested list).

One minute isn't enough. Pulse was confident from about 16 seconds, but HRV first appeared at 69 seconds in my runs. So the check runs at least 75 seconds, and up to 90 while it waits for HRV. Other phones may differ.
Breathing didn't make it. I planned a Breath slider. On a stand, in room light, breathing rate swung from 4.6 to 75 a minute with confidence never above 43. I'd rather drop a signal than show a number I don't trust, so breathing is a guided exercise that Nidra paces but doesn't measure.
Closing your eyes looks like looking away. My first properly eyes-closed checks gave no score. The log showed pulse confidence 100 and HRV confidence 95, but the SDK marked the face "not forward" for 61 of 75 seconds. Closed eyes read as not facing the camera. Nidra now accepts that one status, while pulse and HRV confidence still have to pass.
Is it repeatable? Calm is 60% of the score, and my early HRV readings on this one phone were 21, 21, 46 and 74 ms across four separate runs. So I ran three checks in a row on 14-Nov-2026: two with the phone on a stand, then one hand-held, the way most people would actually do it:

Check	Phone	Pulse (bpm)	Pulse confidence	RMSSD (ms)	Stress index	HRV confidence	Eyes closed	Score
1	Stand	83	97%	76	1.5	71%	53%	80%
2	Stand	79	99%	19	2.0	88%	5%	45%
3	Hand-held	80	44%	43	0.9	63%	13%	75%

Pulse held steady: 79 to 83 bpm across all three, even hand-held. HRV did not: RMSSD swung from 19 to 76 ms within two minutes, and because Calm is 60% of the score, the score swung from 80% to 45% with it. So one reading isn't something to trust on its own. That's why Nidra compares you with your own median of past evenings and asks for a re-check, and why I'd read it as "roughly where am I tonight", not a precise number.

The hand-held check still gave a score, but pulse confidence fell from about 98% to 44% and the camera flagged movement. Holding the phone also means watching the screen, so your eyes stay open (13% closed here). The phone-on-a-stand design with start and end tones exists for exactly that reason.

Prize Categories

Best Thoughtful Real-World Use. A lot of people go to bed wired and only find out once they're lying awake. Nidra is built for that moment: one number, a dark screen by default, tones so your eyes stay closed, a check short enough to actually do every night, and a refusal to score when the camera can't be trusted. Consent comes first, Nidra keeps its data on the phone, and the consent screen says plainly what the Presage SDK may send (details below).

Most Creative or Delightful Use. The same number drives three bedtime themes: Moonlight, where the moon fills from crescent to full; Diya, where a lamp flame grows and steadies as you settle; and Plain, for people who only want the number. The check asks you to close your eyes, so tones, not the screen, tell you when to start and stop.

Consent and Data Handling

Consent: the first screen explains what the camera does and what is kept, and nothing runs until you agree. Android then asks for camera permission. The camera runs only during a check, shows a "Camera on" label, and stops when the check ends, when you tap Cancel, or when you leave the app. It's for checking yourself, never someone else.
Processing: the SDK measures on the phone. Nidra doesn't store camera frames or images, and SmartSpectra image output is switched off.
Storage: no video, photos, face images or face landmarks are kept. For each check Nidra saves a few numbers in the app's private storage: pulse, RMSSD, stress index, the confidence values, eyes-closed %, blink count and the score. It also keeps your morning ratings and to-do list.
Sharing: Nidra itself has no account, no analytics and no backend, and Android backup is off. The SDK contacts Presage to validate the API key and record usage.
AI services: Nidra uses none; the score and suggestions come from fixed rules. Presage says some vital metrics may be sent to its AI insights service. Nidra doesn't request, use or show those insights. I haven't verified whether SDK 3.4.0 sends vitals to that service with Nidra's settings. During my checks the app did make regular small network requests, so treat pulse and HRV as potentially shared with Presage until that's confirmed.
SDK telemetry: the optional telemetry is off, with enableTelemetry = false.
Retention and deletion: data stays on the phone until you delete it or uninstall. Settings has one button that deletes everything.

Nidra is a wellness estimate, not a medical device. It makes no diagnosis or treatment claims and doesn't guide health decisions. One evening reading can't predict a night's sleep, which is why it's called readiness, why it's treated as an experiment, and why the history puts your mornings next to it.

I built Nidra with help from Claude Code. The testing, the decisions about what to drop, and the evenings in front of the camera were mine.

Hisaab: Gemma 4 reads my bank SMS on my phone, and the app has no internet permission

Rahul Rao — Tue, 20 Oct 2026 16:00:34 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

In India, money talks by SMS. Every card swipe, every UPI payment, every autopay sends a text. I have accounts at more banks than I'd like to admit, so my inbox gets dozens of these a week. I read almost none of them.

That's a problem, because the SMS is often the only place a mistake shows up. A bill that says ₹1,199 but debits ₹1,499. An Uber ride charged twice, three minutes apart. A dollar subscription billed at 2 AM that I forgot I'd signed up for. Each one is right there in my inbox, unread.

The usual fix is an expense app that "reads your SMS". The catch is how it reads them: most upload your messages to their servers and parse them there. Your bank SMS are some of the most private data on your phone: what you pay, who you pay, when you're awake, and the last digits of every account and card.

So I built Hisaab (हिसाब, Hindi for "the accounts"). It's an Android app that reads my bank SMS with Gemma 4 E2B running on the phone itself, and it can't send anything anywhere: the app has no internet permission at all.

What it does, so I don't have to:

"Was this you?" alerts, within a second of the SMS. It flags:
- a debit that doesn't match its bill
- a double charge
- a payment far above my usual at that merchant
- a large first payment to someone new
- a foreign-currency or 2 AM charge
- a bill that jumped from last time
- a new auto-debit I never set up

It was me and Not me buttons sit right on the notification.

A morning note at 08:00: one calm line about yesterday, then the exact figures: what I spent, the month so far, and bills due soon.
Bills, matched: every bill SMS is paired with the debit that paid it, so "did I pay Tata Power?" takes a glance, not a search.

Demo

Today	Bills, matched	No internet, by design

The video and screenshots use the app's demo mode: 30 made-up SMS in a separate database. No real account appears anywhere in this post. The narration is an AI clone of my own voice.

Code

rahulrao85 / hisaab

Bank SMS read on your phone by Gemma 4 — alerts for wrong bills, double charges and new merchants. No internet permission. Hacktoberfest 2026 Week 3.

Hisaab

Your bank SMS, read on your phone by Gemma. Nothing leaves the device.

Hisaab (हिसाब) is Hindi for "the accounts". In India every card swipe, UPI payment and autopay sends an SMS. Hisaab reads them for you, so you never have to open six banking apps to know what you spent:

Was this you? An alert within a second of the SMS, with It was me / Not me buttons right on the notification, when
- a debit doesn't match the bill it pays (Airtel billed ₹1,199, ₹1,499 left your account)
- the same charge lands twice within 10 minutes
- you pay a merchant far more than usual (3× your median there)
- you pay a new merchant more than ₹1,000
- a charge is in a foreign currency, or comes in at 2 AM
- a bill jumps more than 25% over the last one
- a new auto-debit or subscription gets set up
…

View on GitHub

How I Built It

The rule I started with: the model never touches money. A phone-sized model is small, and small models guess. So I split the work by how wrong each part is allowed to be:

Job	Who does it	Why
Amount, account, reference number, due date	Plain regex	Must be exact. An unparsed SMS means "no alert", never a wrong ₹ figure
When to alert	Plain rules	A phone-sized model shouldn't be the one calling something fraud
Who the merchant really is, and its category	Gemma 4 E2B, on the phone	This is where a language model earns its place
One friendly line in the morning note	Gemma 4 E2B, with no numbers allowed	The figures underneath come from code

Why Gemma is needed at all. Bank SMS don't say "Uber" or "Blinkit". They say UBER INDIA SYSTEMS, blinkit@hdfcbank, HPCL MAHALAXMI SERVICE STN or TATA POWER DDL. A lookup table can't keep up with payment gateways, UPI handles and company names. A language model that has read the internet mostly can. Gemma gets a short few-shot prompt and answers in one line, Name | Category. Each merchant is asked about once, and the answer is cached forever. So the model runs a handful of times a week, not on every SMS. That's the slow-and-steady part: it does very little, and only when there's something new to learn.

I started with the wrong Gemma. My first build used Gemma 3 1B (a 4-bit, 550 MB .task file on MediaPipe). On its very first run it didn't answer at all. It repeated my instructions back, because I hadn't wrapped the prompt in Gemma's chat turns. With that fixed, I wrote a 24-merchant check that runs on the phone (Setup → Accuracy check). It uses real-looking Indian payee strings that never appear in the prompt, each with the answer a person would give. Then I swapped in Gemma 4 E2B (2.6 GB, Apache-2.0, on Google's open-source LiteRT-LM runtime) and ran the same check:

On a Galaxy M35 (Exynos 1380), CPU	Gemma 3 1B	Gemma 4 E2B
Merchant names right	15 / 24	16 / 24
Categories right	9 / 24	18 / 24
Time per merchant	4.5 s	2.6 s
Model load, warm	0.4 s	0.5 s
Peak app memory	—	about 1 GB

The bigger model is also the faster one here. That surprised me, and it's why Hisaab now ships with Gemma 4 E2B and keeps the 1B as a fallback for smaller phones. The 1B filed Tata Power under "Investments"; Gemma 4 files it under Bills.

What Gemma 4 still gets wrong is the interesting part. Every name it missed is a legal company name that hides the brand: KIRANAKART TECHNOLOGIES is Zepto, RAZ*DREAMPLUG TECHNOLOGIES is CRED, ONE97 COMMUNICATIONS is Paytm. That's a lookup problem, not a language problem, so I didn't fake it with a bigger prompt. You tap the transaction, type "Zepto" once, and every past and future payment to that payee follows. Lazy, but only once.

The morning note, and why Gemma isn't allowed numbers. My first version asked Gemma to rewrite the day's facts, with a guard that rejected any number that wasn't in the facts. Gemma 4 passed the guard and was still wrong: it called the month's total "today's spending". Every number was correct, and one label was wrong. A number guard can't catch that. So now Gemma writes one sentence with no figures at all ("It sounds like a busy day with a mix of spending and a bill to watch."), and the code prints the exact numbers underneath. If Gemma slips a digit or a ₹ into its line anyway, the line is dropped.

Running it on the phone. Both models load on the CPU with greedy decoding (temperature 0), so the same SMS always gets the same answer. Naming runs in a background job, so I never wait on it. The alert itself doesn't use the model at all, so it fires the moment the SMS lands, and Gemma's name for the payee appears a couple of seconds later.

Parsing Indian bank SMS. Every bank has its own format, and a few of them nearly broke the parser:

ICICI writes "Acct XX107 debited for Rs 1,199.00; AIRTEL credited". Both "debited" and "credited" appear in a single debit. Whichever verb comes first decides.
SBI writes "debited by 4999.0", with no ₹ sign at all.
Airtel's bill SMS says "Post*paid*", which my first parser read as "paid", turning a bill into a payment. Word boundaries fixed it.
Card statements carry two amounts, total due and minimum due. Hisaab takes the total.
Billers send "Payment of ₹1,499 has been received, thank you". These are skipped, because the bank's own debit SMS already counts that money.

All of this sits in plain Kotlin with no Android code, so it's unit-tested on a laptop. 17 tests run against 30 synthetic SMS covering HDFC, ICICI, SBI, Axis, Kotak and HSBC.

The first scan. When you install it, Hisaab reads your last 90 days of SMS. The first version alerted on every "new merchant" from three months ago, and nobody needs that. Now anything older than 7 days is quietly marked as yours, and only recent surprises ask for an answer.

Why Does Open Innovation Matter?

For this app, the open model is the whole idea.

A closed API would need the internet, and this app has none. To use a hosted model, Hisaab would have to send my bank SMS to someone else's server, which is exactly the problem with existing expense apps. With Gemma's open weights running on the phone, I could remove the INTERNET permission entirely. That's a claim you can check: the Setup screen reads the app's own manifest and shows it, and aapt dump permissions on the APK shows the same. "Private by policy" asks you to trust a company. "Private because it physically can't connect" doesn't ask you to trust anyone.

It costs nothing to run, forever. There's no API key, no per-call price and no subscription. That matters because the app is meant to run every day for years. Even a cheap API charges on every call, and that adds up over years of daily use.

It works offline. On a flight, in a basement, or in airplane mode, the alerts and merchant labels still work.

I can change how it behaves. The prompt, the categories and the guard are all mine to change. I swapped Gemma 3 1B for Gemma 4 E2B in an afternoon and measured the difference, and the next Gemma will drop in the same way. LiteRT-LM and MediaPipe are open source too, so when something misbehaved I could read the code instead of guessing.

Prize Categories

Best Use of Gemma: Gemma 4 E2B runs on the phone through LiteRT-LM, with a measured comparison against Gemma 3 1B, and no server at any step.

Zubaan: I dubbed my English Shorts into Hinglish (same face, same voice) with open models

Rahul Rao — Fri, 16 Oct 2026 09:58:13 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

I run a small AI channel, Grounded AI Lab, for Indian viewers. On 11 October I switched it to Hinglish only, because that's how my audience actually talks. That left four finished English Shorts stuck on hold. Re-recording them meant another evening in front of the camera for videos that were already done.

So I built Zubaan. In Hindi, zubaan means "tongue", and it also means "language". This tool changes both. You give it an English Short and get back the same Short in Hinglish, with my face, my voice, my lips moving to the new words, and Hinglish captions. The cards, cuts, music and sound effects all stay where they were.

Who pays for this: Indian creators, teachers and coaches who record in English and want the much bigger Hindi-speaking audience, without recording everything twice. Paid dubbing tools charge per minute of video. Writing natural Hinglish (Roman-script Hindi with the English tech words left alone) is the hard part, and that's exactly the job I gave an open model.

What I'd charge: ₹99 per Short, or ₹1,499 a month for 20. My cost per 70-second Short is under ₹1 of DigitalOcean inference plus about 1,000 characters of the creator's own voice-clone plan. The GPU work runs on a gaming card I already owned.

Demo

Side by side: English original on the left, Hinglish dub on the right (with the Hinglish audio):

Same frame, same moment. Top: the English original. Bottom: Zubaan's Hinglish lip sync.

The finished Hinglish Short (unlisted demo):

Code

rahulrao85 / zubaan

Dub an English YouTube Short into Hinglish with the creator's own face and voice. Whisper + Gemma 4 on DigitalOcean + LatentSync.

Zubaan

Zubaan (zu-baan) is Hindi for "tongue", and also for "language". This tool changes both.

Turn an English YouTube Short into a Hinglish one, with the creator's own face and voice Open models handle the transcript, translation and lip sync, and the only closed piece is the creator's voice clone.

Built for the Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle on Shorts from my channel Grounded AI Lab.

Pipeline

Step	Tool	Open?	Runs on
English speech → timed transcript	faster-whisper `small.en`	Open (MIT)	Laptop CPU, 70 s of audio in ~31 s
Transcript → spoken Hinglish, line by line	Gemma 4 31B via DigitalOcean Serverless Inference	Open weights (Apache 2.0)	DigitalOcean
Hinglish → creator's voice + word timings	ElevenLabs voice clone (`eleven_v4`)	Closed	ElevenLabs API
Face video → lips that match the Hinglish	LatentSync 1.5	Open (Apache 2.0 code, OpenRAIL++ weights)	RTX 3060 12 GB

…

View on GitHub

How I Built It

Five steps. Every one except the voice uses open software.

Step	Tool	Where it runs
English speech → timed transcript	Whisper (`faster-whisper`, small.en)	My laptop's CPU: 70 s of audio in 31 s
Transcript → spoken Hinglish	Gemma 4 31B on DigitalOcean Serverless Inference	DigitalOcean, 6 s per Short
Hinglish → my voice + word timings	ElevenLabs voice clone	ElevenLabs
Face → lips that match the Hinglish	LatentSync 1.5 (ByteDance)	RTX 3060 12 GB, under WSL
Re-render with Hinglish captions	HyperFrames + ffmpeg	My laptop

Picking the translator. I gave the same transcript to four open models on DigitalOcean and compared the output line by line:

Model	Time	Verdict
Gemma 4 31B	6.4 s	Most natural, consistent "aap", fixed both mishearings. Picked.
DeepSeek V4 Pro	25 s	The punchiest lines ("Catch? Hai na ek catch."), but it switched between "aap" and "tum"
Llama 4 Maverick	24 s	Translated "catch" as pakad, which nobody says
Qwen 3.5 397B	—	Came back empty; I didn't chase it

The bug that taught me something. Whisper heard "GPT-6 Astra" as "Astra 6" and the model name "Jev" as "Gev". Gemma then translated the mistakes faithfully. Hand-written hints worked for that one video only, so I looked for a source of truth that every video already has: its own on-screen text. The composition has an overlay that says "Try GPT-6 Astra without a subscription". dub.py now pulls every overlay card out of the composition and gives it to Gemma as the spelling authority. Both names came out right, and Gemma even added "har mahine" (every month) from a card, where the transcript had only said "the list keeps changing".

Timing (version 2). Hinglish runs 10–15% longer than English. My first render placed each line at its English start time, and it looked wrong: the face kept talking in the gaps where the dub had already finished. Now Whisper's word timestamps give the exact span where each English line was spoken, and each Hinglish line is stretched or squeezed (0.85–1.2×) to fill that span. Most lines now start and end within about 0.1 s of the original. Two Hinglish lines are so much shorter than the English that they finish early even at 0.85×, and there the lip-sync model keeps the mouth at rest. In the 6 pauses where both the dub and the original are silent, the video switches back to the original frames, so the face is quiet when the voice is quiet.

The mistake that cost me 48 minutes. My machine already had LatentSync installed from an earlier experiment, and I assumed it was version 1.6 (512 px). I ran it with the 1.6 config, and it took 48 minutes for 6 seconds of video while my 12 GB card spilled into system RAM. Then it produced a mouth that looked like crumpled foil. While I was downloading the "real" 1.5 weights, I compared fingerprints: the installed file was 1.5, byte for byte. With the correct 256 px config, the same 6 seconds took 3.3 minutes and looked right.

Then the full 70-second run was killed for running out of memory. LatentSync keeps every frame in RAM, about 12 MB per 1440×1920 frame, so 70 seconds needed around 20 GB on a 24 GB machine. lipsync.sh now cuts the video into chunks of 16 seconds or less at the pauses between dubbed lines (dub.py's timing plan already knows where they are), syncs each chunk and joins them, so no word gets split. The final 70-second Short took about 23 minutes on the RTX 3060, in 5 chunks, with no visible seams. Both lessons are comments in lipsync.sh, so nobody else loses that hour.

What I couldn't use. I wanted to run the lip sync on a DigitalOcean GPU Droplet (RTX 4000 Ada, $0.76/h, enough for the sharper 1.6 model). My new account didn't have GPU access yet, and the limit request is still pending. So the GPU step runs locally for now, and the Droplet is the obvious next step for anyone without a gaming card.

Why Does Open Innovation Matter?

Three things here wouldn't have worked with a closed API:

The fix lives in my pipeline, not in someone's roadmap. The on-screen-glossary trick works because I control exactly what goes into the model and can swap models freely. When Gemma's first run regressed on a name, I changed the prompt and re-ran it for well under a rupee. With a closed dubbing product, I'd have filed a ticket.
Model choice is a 10-minute experiment. DigitalOcean serves a dozen open-weight models behind one endpoint, so comparing four translators cost less than a cup of tea, and I picked on quality, not on whichever vendor I happened to be locked into.
The expensive part runs on hardware I already own. Lip sync is the GPU-heavy step, and an open model meant it ran free on my gaming card. A per-minute service would make this side hustle's margins depend on someone else's pricing page.

Being honest about the one closed piece: the voice. I kept the ElevenLabs clone my channel already uses, because my audience knows that voice. Testing an open voice-cloning model on Hinglish is the next experiment, and the voice step is a single function, so swapping it out is a small change.

On consent: the tool is built to dub your own face and voice. The voice comes from your own clone, and lip sync only runs on footage you supply. The output gets YouTube's "altered or synthetic content" label, as it should.

Known Limitations

I watched the first full render and didn't like it, so here's what's still wrong, from worst to least bad:

The body still speaks English. Lip sync only redraws the mouth and jaw. Head nods, eyebrows and hand gestures keep the rhythm of the original English sentences. Version 2's timing fix (above) removed the worst of it, the mouth moving with no voice, but a dubbed face is never as natural as a re-recorded one.
The mouth is a little soft. LatentSync 1.5 works at 256 px. The 512 px model (1.6) needs an 18 GB GPU, which is exactly what the DigitalOcean GPU Droplet was for.
English names in a Hindi voice. A Hinglish voice may read "Astra" as the Hindi word astra (weapon). Zubaan takes voice-only respellings (--say "Astra=Ass-truh") while the captions keep the real spelling. I listened to three versions and kept the plain spelling: in a Hinglish sentence, that is how my audience would say it anyway.
Where it works best: Shorts where cards and screen recordings cover most of the face. For face-to-camera Shorts, my honest advice to creators is still to re-record, and to use Zubaan for the back catalogue they never would have.

My First User

Me, as a creator, and I'm being strict about it. After watching version 1, I decided not to publish the dubbed GAL-012 on the channel. For a face-to-camera Short, re-recording still looks better (see Known Limitations). Where Zubaan earns its place is my held English Shorts that are mostly cards and screen recordings, and that's where it goes next. Nobody has paid yet. The next test is whether a creator friend who records in English will pay ₹99 for one of theirs.

Prize Categories

Best Use of DigitalOcean: Gemma 4 runs on DigitalOcean Serverless Inference as the translation core, and four open models were compared on the same endpoint.
Best Use of Gemma: Gemma 4 31B writes all the Hinglish, using the on-screen-glossary technique.
ElevenLabs: my ElevenLabs voice clone speaks the Hinglish, and its word-level timestamps drive the captions.

Code and post drafted with AI assistance (Claude Code); all results, numbers and design decisions reviewed by me.