DEV Community: Tamajit-005

56 messages, 4 voice notes, 48 seconds: the briefing that reads my family's WhatsApp while I sleep

Tamajit-005 — Mon, 26 Oct 2026 06:04:05 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

You slept. It listened. Here's what you missed.

Last night, my father sent four voice notes to our family WhatsApp group. He was planning a trip to Shantiniketan, setting a budget, and — most importantly — updating us on my grandfather's health after a scare that initially sounded like a heart problem.

I was asleep.

By morning, I would have had to play all four notes, remember who said what, and mentally assemble a to-do list. Instead, I played a 48-second audio briefing that caught me up on all of it — in a warm voice, with a receipt for every claim.

This is the story of Amar Khobor (আমার খবর) — a briefing tool born the night a speech-to-text model mistook "heart problem" for "trouble at the market."

What I Built

Amar Khobor turns a WhatsApp group export into a warm, sub-minute audio briefing plus structured, verifiable show notes.

The digest arrives as two things at once:

A spoken script — warm and conversational, like a family member catching you up. No timestamps, no citations read aloud.
Show notes — structured cards below the player, mapping every claim to its sender and timestamp: Baba → Dadu's gastric issue, not heart → health → urgent.

Here's what it produced from my family's actual messages that night:

"Shobai kemon acho? Amar Khobor is here with a little update from Baba. Baba is already dreaming about our next trip! He's suggesting Shantiniketan—staying in a nice lodge and visiting the Sonajhuri Haat. He also mentioned a budget of around 2000, though he says we might spend a bit more if we enjoy some shopping there. So do think about it, okay? On a more serious note, there was a little scare with Dadu. Baba thought it might be a heart issue, but Dadu mentioned it's just gas. Edkat went to check on him, and Baba is also heading there. Thankfully, Dadu is doing much better now and has already had some khichdi. Baba will be taking him to see Dr. Sen tomorrow morning. So please, don't tension too much and everyone get some good sleep. We'll catch up in the morning. Shuvo ratri!"

Demo

▶ Listen to the sample briefing — generated from my family's real overnight voice notes, narrated with ElevenLabs.
▶ Try the live app

Code

View the source code on GitHub →

Tamajit-005 / Amar-Khobor

Turn overnight WhatsApp voice notes (Bengali/Hindi/English/Benglish) into a warm 2-minute audio briefing. Dual-STT bake-off: Sarvam Saaras vs Whisper-large-v3 · Gemma 4 digest · ElevenLabs narration.

Amar Khobor (আমার খবর)

"You slept. It listened. Here's what you missed."

Amar Khobor turns a chaotic overnight WhatsApp group — dozens of voice notes in Bengali, Hindi, English, Benglish, and Hinglish — into one warm, two-minute audio briefing you can play while making morning tea. No more scrolling through 40 voice notes to find out who needs what by when.

▶ Listen to a sample briefing · Read the build story

The Problem

Family WhatsApp groups don't sleep. While you were asleep, someone asked for a medicine refill, someone's flight got delayed, a school form needs signing by noon, and three people argued about dinner plans — all in voice notes, half of them in Benglish that autocorrect can't parse.

Catching up means playing every note, remembering who said what, and mentally assembling a to-do list. Every morning. For every family member who doesn't have time for that.

Amar…

View on GitHub

How I Built It

Here is what Whisper-large-v3 produced from my father's voice note about Dadu:

আমি হেবে ছিলাম হাটে সমসা কিন্তু দাদু ভললো গ্যাসে সমসা

And here is what Sarvam's Saaras produced from the exact same audio:

আমি ভেবেছিলাম হার্টের সমস্যা, কিন্তু দাদু বলল গ্যাসের সমস্যা।

Whisper wrote "trouble at the market" (হাটে সমসা). My father said "heart problem" (হার্টের সমস্যা). One is wordplay. The other is a family medical emergency.

That single mistranscription is why Amar Khobor exists. Family WhatsApp groups don't sleep. Overnight, voice notes pile up in Bengali, Hindi, English, Benglish, and Hinglish — and catching up means playing every note, remembering who said what, and mentally assembling a to-do list. Every morning. Every family member. Nobody has time for that.

The bake-off: what survived a human check

I didn't want to pick a transcription provider based on someone else's benchmark table. So I built a bake-off harness that scores Saaras and Whisper-large-v3 on my own family's audio.

One honest caveat first: the WER ground truth in test_data/manifest.yaml is the Saaras transcript. So Saaras's 0.00% WER is an floor, not an independent accuracy score — Saaras agrees with itself by construction. The evidence that survived a human check is the diff below.

The clip that matters — bn_003, verified by ear:

	Transcript
What my father said	আমি ভেবেছিলাম হার্টের সমস্যা... ("I thought it was a heart problem...")
What Saaras wrote	আমি ভেবেছিলাম হার্টের সমস্যা... ✓ correct
What Whisper wrote	আমি হেবে ছিলাম হাটে সমসা... ✗ "trouble at the market"

Full per-clip results, including the noise-guard test: test_data/eval_results.json · demo/stt_comparison.md

Metric	Sarvam Saaras (`saaras:v4`)	Groq Whisper (`whisper-large-v3`)
Monolingual WER ↓	0.00% (floor — see caveat)	67.13%
Noise hallucinations	0	1
Average latency ↓	499 ms	814 ms

That Whisper hallucination is real: on a one-second clip of near-silence, it produced the word "you" from nothing. Saaras produced nothing — correctly.

Why this matters for the product: Saaras is purpose-built for Indic languages and code-mixed speech, and it's what runs in production. Whisper is the open-weights baseline I benchmark against — inspectable, reproducible, and swappable.

The bug I shipped and fixed

Gemma 4 did something I didn't expect: it occasionally emitted its entire chain-of-thought — planning, drafting, self-correcting — before the JSON digest. Sometimes it wrapped the JSON in an array instead of returning a bare object.

My first extractor took the first { to the last }. That grabbed a { from mid-reasoning and produced invalid JSON. The pipeline crashed on a response that contained a perfectly good digest — just buried in chatter.

The fix was a tolerant backward extractor: find the last occurrence of "spoken_script", backtrack to the { that opens that object, and slice to the closing }. Now the parse succeeds even when Gemma rambles — and the prompt is hardened to discourage the rambling in the first place.

idx = content.rfind('"spoken_script"')
start = content.rfind('{', 0, idx)
end = content.rfind('}')
cleaned = content[start:end+1]

This turned a flaky LLM response into a deterministic parse — committed, with tests, in pipeline/summarize.py.

The engineering decisions worth explaining

Why filter audio before STT? Voice notes are the expensive part. An energy-based VAD pass tags silence and background noise before any API call, so a forwarded chain of muted clips costs nothing. Forwarded messages are also excluded from summarization — nobody needs a briefing about a chain-forwarded good-morning graphic.

Why one LLM call for both outputs? The spoken script and the show notes describe the same events. Generating them in a single structured Gemma call keeps them consistent — if the script says Dadu needs a doctor, the show-notes card for it exists too.

Why a 202-and-poll architecture? A real overnight export takes a minute to process. POST /digests/run returns 202 Accepted immediately, FastAPI BackgroundTasks does the work, and the frontend polls every 2 seconds. No Celery, no Redis, no message broker — this is a family tool, not a distributed system.

Why ephemeral memory by default? Family voice notes are some of the most private audio that exists. Audio and briefings live strictly in-memory during the session and vanish on server restart. MongoDB Atlas is an opt-in adapter, never a requirement.

Honest limitations

Gemma output formatting — the model sometimes emitted reasoning-like text before the JSON or wrapped the digest in an array. Prompt hardening and a tolerant extractor address the observed cases, but do not guarantee valid output for every response.
Saaras-derived references — the evaluation references were taken from Saaras's transcripts. Its reported 0.00% WER therefore measures agreement with those references, not independently established transcription accuracy. Independent human transcription is the next evaluation step.
Energy-based VAD — the filter detects audio energy, not speech itself. Loud background noise can pass through, and quiet speech can be missed.
No API authentication — the current demo does not provide authenticated, per-user access. Authentication and rate limiting are needed before broader public use.

Why Does Open Innovation Matter?

This is a tool for family voice notes — some of the most private audio that exists. That constraint shaped every choice:

Gemma 4 summarizes the digest because it's an open-weight model I can swap, self-host, or point at any provider without rewriting the pipeline. The provider is switched with one environment variable.
Whisper-large-v3 via Groq is the open-weights baseline, so the STT comparison is between two models whose behavior I can inspect — not a black box I have to trust.
Edge-TTS keeps development free and local-first; ElevenLabs is the production voice for the live deployment, opt-in and never a hard dependency.
Ephemeral in-memory storage means zero infrastructure setup and zero database footprint by default.

The honest asymmetry: Saaras — the model that wins my bake-off — is closed. I can't inspect or self-host it. I picked it for Bengali accuracy and kept the provider swap one environment variable away, which is the only honest position for a model you don't own.

The Lazy Payoff

Fifty-six messages. Four voice notes. Forty-five seconds of audio you cannot skim.

Morning-me would replay all four notes to find the budget and the health update, then scroll the chat to check who said what. The briefing is 48 seconds, and the receipts are already attached.

The saving isn't the seconds — it's that audio has no Ctrl-F.

Sloths don't catch up faster. They let the night do the work.

Prize Categories

Best Use of Gemma — Gemma 4 powers the single-pass dual-output digest (spoken script + show notes), with a hardened prompt and a tolerant backward JSON extractor that turns flaky LLM output into a deterministic parse.
Best Use of ElevenLabs — ElevenLabs narrates the production briefing in a warm Bengali-speaking voice; it's the voice in the committed sample MP3 and the live deployment.
Best Use of Render — the backend deploys as a containerized FastAPI service via a committed Dockerfile and render.yaml Blueprint, with a root health route and Vercel-aware CORS.

Sarvam's Saaras isn't a prize track, but it's the reason the comparison exists: it's the production STT that made the "trouble at the market" vs "heart problem" comparison possible in the first place.

Built for the Hacktoberfest Open-Source AI Challenge, Week 3: Slow and Steady. Speech-to-text by Sarvam AI and Groq; briefing voice by ElevenLabs.

Amar Voice — Giving My Friend Their Voice Back (EN / हिंदी / বাংলা)

Tamajit-005 — Sun, 04 Oct 2026 16:19:55 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Amar Voice — A Voice That Stays Theirs.

Built for a friend who has lost their voice, but still has old voice recordings.

Amar Voice turns those recordings back into a voice they can use every day. Upload a few of their old voice notes → the app clones their voice → they tap a big button and the phone speaks as them again, in English, हिंदी, or বাংলা. Anything beyond the daily phrases can be typed into the custom-message box (up to 1000 chars).

Built for tired eyes and shaky hands: giant high-contrast cards, three language pills, one speaker menu, zero settings pages. It installs like an app (PWA), quick cards play instantly from cache, and family can add / rename / delete voice profiles right from the phone — no login, each device gets its own private space.

The 6 quick cards (every tap speaks in their voice)

Tap the feeling	English	हिंदी	বাংলা
💧 Water — pleading, thirsty	I need some water, please...	मुझे थोड़ा पानी चाहिए...	আমার একটু জল দরকার, দয়া করে...
💊 Medicine — routine, calm	Time for my medicine...	मेरी दवाई का समय हो गया है...	আমার ওষুধ খাওয়ার সময় হয়েছে...
📞 Call — insistent, needs someone	Please call my family member right now!	कृपया मेरे परिवार के किसी सदस्य को बुलाओ!	দয়া করে এখনই আমার পরিবারের কাউকে ডাকো!
☕ Tea — comfort, warmth	I would like some warm tea.	मुझे थोड़ी गर्म चाय चाहिए।	আমাকে একটু গরম চা দাও।
😊 I'm fine — happy, reassuring	I am doing fine, do not worry.	मैं बिल्कुल ठीक हूँ, चिंता मत करो।	আমি ভালো আছি, চিন্তা কোরো না।
🚨 Help — urgent, distressed	I need help, please come here!	मुझे मदद चाहिए, कृपया यहाँ आओ!	আমার সাহায্য দরকার, দয়া করে এখানে এসো!

The emojis aren't decoration — they are the interface. 😊 means happy, 🚨 means urgent, 💧 means pleading. No reading needed: tap the feeling and the phone says the matching phrase in their own voice, with the feeling carried in the words themselves (... vs !) as rendered by the clone.

Demo

Live app: https://amar-voice.onrender.com

Video demo: https://youtu.be/DQC13FPReB4?si=tfCekrNp80R67fji

Tip for judges: open the link on your phone → Add to Homescreen → tap 💧 water → switch हिंदी / বাংলা pills → try custom text → open Upload to clone a voice from old recordings (max 10 clips, 800 KB each).

Code

https://github.com/Tamajit-005/Amar-voice

backend/main.py — routes, upload caps, rate limits
backend/engine.py — voice-ID map, clone / rename / delete
backend/eleven.py — ElevenLabs client
frontend/index.html — PWA UI
frontend/sw.js — offline cache
render.yaml + Dockerfile — deploy

How I Built It

FastAPI backend + single-file PWA frontend, deployed on Render free tier via Docker.

Voice: ElevenLabs v4 Instant Voice Cloning. Upload a few 10–25s clips from their old recordings → backend clones once, reuses that voice ID for all 3 languages. 18 presets pre-generated per speaker, cached on server + service worker for instant taps. Custom text synthesizes in a few seconds when warm (longer on a cold free-tier start), MP3 download capped at 350 KB.
Privacy without login: random amar_uid in localStorage, clips + voice-ID map + presets namespaced per device. default is shared, everything else is device-private.
Guardrails: 10 clips / 800 KB per clip / 2-min caps, per-IP rate limits on paid endpoints, atomic cache writes, honest 404s (never serves another voice on failure).
Open pieces: FastAPI, Pydantic, Uvicorn, pydub + ffmpeg, vanilla JS PWA with service worker, Docker — all open-source, and the repo is open for anyone to self-host with their own API key.
Problems faced: local models sounded distant from the real voice → moved to cloud, kept offline cache for presets; built for one friend but shipped per-device spaces so other families can reuse it; Render free idle wipes server voices (phone keeps ID, sees only default) — needs paid disk or external store.

Why Does Open Innovation Matter?

The parts that make this shippable and forkable are open: anyone can clone the repo, cp .env.example .env, deploy the same Dockerfile to Render, and have it running for their own family in minutes — no app-store review, no proprietary client, no accounts to manage.

That mattered practically: I swapped the entire voice backend (local → ElevenLabs) without touching the PWA, added per-device spaces with no auth service, and kept it runnable on a 512 MB free tier because ffmpeg/pydub/FastAPI are hackable. What I'd open next: a local-model fallback so custom speech works fully offline.

Prize Categories

Best Use of Render
Best Use of ElevenLabs