Yaadi (यादी): Speak a Task, Get a Calendar Event. No Cloud, Just Gemma 4

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Yaadi (यादी) turns a voice note into a task list and a calendar file, entirely on your own laptop. Yaadi means "list" in Marathi and Hindi.

You speak a note in Hinglish, Hindi, Marathi or English, something like "kal subah 10 baje Parth ko project ka link bhej dena, aur parso tak lab report submit karna hai". Yaadi transcribes it, pulls out the tasks (title, due date and time, people, the words each task came from), and lets you edit or untick them. One click exports a standard .ics file that imports into any calendar app.

Who gets to be a little lazier? Anyone who already sends themselves voice notes (WhatsApp "message yourself", Notes, Recorder) and then never turns them into calendar entries. That's me, and most of the students I know. There is no cloud and no account, and it costs nothing to run. Your audio never leaves your machine.

Demo

🎥 Video (about 4.5 minutes, narrated): https://drive.google.com/file/d/1VEqJwwaLsjQrgetuxr-6u5CO58h3w-B-/view?usp=sharing

The video is a screen recording of the real app: real uploads, the real model running on my laptop GPU, and the real .ics file it exports.

A real run (E4B, 4-bit):

heard : रिया के बर्थडे के लिए गिफ्ट खरीदना है अभी कोई डेट नहीं है और कल दोपहर 2 बजे कारण के साथ स्टैंडअप मीटिंग है
tasks : - [ ] Buy gift for Riya
        - [ ] Standup meeting with Karan - due 2026-10-21T14:00:00+05:30   -> tasks.json  tasks.ics  tasks.md

Code

Aashutosh-Mahajan / yaadi

Offline voice note to task list + .ics. Speak Hinglish, Hindi, Marathi or English; Gemma 4 does the listening and reasoning, on your own laptop.

Yaadi (यादी)

Speak a Hinglish, Hindi, Marathi or English voice note. Get a clean task list and a calendar file, on your own laptop, with no internet and no cloud. One open-weight multimodal model, Gemma 4, does both the listening and the reasoning. There is no separate speech-recognition model.

Yaadi means "list" in Marathi and Hindi. Built for the Hacktoberfest Open-Source AI Challenge, Week 3 ("Slow and Steady"), Best Use of Gemma.

Contents: Quick start · What it does · Measured results · Setup · Use · Privacy · How it got fast · Windows findings · Evaluate · Demo video · Layout

Quick start

git clone https://github.com/Aashutosh-Mahajan/yaadi.git; cd yaadi
python -m venv .venv; .\.venv\Scripts\activate
pip install torch --index-url https://download.pytorch.org/whl/cu126; pip install -e
python -c "import yaadi; from

…

View on GitHub

MIT licensed. The README has setup steps, every measured number, and the full list of limits.

How I Built It

Model: Gemma 4, E4B by default and E2B as the smaller option, both run 4-bit (NF4) through Hugging Face Transformers. Gemma 4 takes audio directly, so one model does both the listening and the reasoning. There is no separate speech-recognition model.

Hardware: an RTX 3050 Laptop GPU with 6 GB of VRAM and 16 GB of RAM, on Windows 11.

voice note -> ffmpeg (mono 16 kHz) -> chunker (<= 28 s, cut at the quietest pause)
  -> Gemma 4, pass 1: transcribe each chunk (original language and script)
  -> Gemma 4, pass 2: one line per task, with day numbers from a calendar table
  -> pydantic validation (1 retry) -> deterministic date resolver -> review UI
  -> tasks.json / tasks.ics / tasks.md

Gemma 4's audio clips top out at 30 seconds, so longer notes are split at pauses and transcribed chunk by chunk. A second pass reads the whole joined transcript and extracts the tasks.

What I measured (15 test notes: 5 Hinglish, 4 Marathi, 3 Hindi, 3 English):

Run	Task F1	Dates correct	Median time	Peak VRAM
E4B, two-pass (default)	0.97	100%	12.1 s	4.65 GB
E2B, two-pass	0.95	100%	9.2 s	2.96 GB
E4B, one-shot (audio to tasks in one call)	0.97	100%	6.9 s	4.74 GB

A 45-second note, split into two chunks, still produced all 9 of its tasks. I also ran the whole pipeline with every network connection blocked: it finished with zero outbound attempts.

Four things that went wrong, and what fixed them (the full write-up is in the repo):

1. It was too slow. My first pipeline took about 35 s for an 8 s note. Profiling showed the GPU doing 1.8 s of work while the CPU spent 3.6 s launching tiny kernels. On Windows, decoding costs a flat ~190 ms per output token. The lever was writing fewer tokens, so the model now emits one compact title | day | time | people | cue line per task, and my code expands it into the full JSON. I first tried compact nested JSON arrays, and the 4-bit model unbalanced the brackets about 1 note in 8. Lines can't do that. Median latency went from 34.6 s to about 10 to 12 s.
2. It got weekdays wrong. "Friday" came out as the wrong date. Language models are bad at weekday arithmetic, so the prompt now carries a 14-day calendar table, and a small deterministic resolver double-checks words like kal, parso, उद्या, शुक्रवार and Friday against each task's own cue. Date correctness went from 84% to 100%.
3. E4B crashed on load. Reading the stock 16 GB checkpoint through a memory map segfaults on this Windows stack. A small yaadi.prepare step re-shards it with plain file I/O.
4. It didn't fit in 6 GB. Gemma 4's per-layer embedding table is 5.6 GB, and it's just a lookup. Left alone it overflowed VRAM. Yaadi keeps that table in CPU RAM and runs only the lookup there, so E4B sits at about 4.4 GB.

Honest limits:

• My test notes are synthetic. They are text-to-speech clips (Microsoft neural voices), which are cleaner and more regular than real speech. Read the scores as an upper bound until it's repeated on real recordings.
• Marathi and names are the weak spots. In one note, both models heard "आजी" (grandmother) as "आजि" and wrote "Meet Ajila". E2B once heard "Professor Rao" as "Professor Ram". The app shows you the transcript so you can catch this.
• E4B misses my 30 s target slightly. A 26 s note takes about 31.5 s on E4B (about 25 s on E2B). Loading E4B takes about 75 s the first time, and the web server stays warm after that.
• One-shot matched two-pass on these short clips and was faster. Two passes still earn their place: you see the transcript, and notes over 28 s work.

Built with AI assistance: I used an AI coding assistant (Claude Code) while building this, and all the numbers above are from runs I did on my own machine.

Why Does Open Innovation Matter?

• Privacy that is a property, not a promise. The whole point is that voice notes stay on your laptop. With open weights I could prove it: a test that blocks every outbound connection and still passes. With a closed API the audio has to leave.
• Zero running cost. After a one-time download, a note costs nothing. That matters for a student tool.
• I could fix the runtime, not just call it. When E4B segfaulted and overflowed 6 GB, I could re-shard the weights, pull out the embedding table, and patch how it loads. None of that is possible behind an API, and it's the difference between "needs a 24 GB GPU" and "runs on my laptop".
• One flag to trade speed for accuracy. --model e2b or --model e4b. Same code, same weights format, honest numbers for both.
• Everyone can reproduce it. The weights, the code, the test notes and the results are all public.

One caveat: I did not run a closed-API comparison, so I can't claim it beats one on accuracy. What open weights made possible here is the privacy guarantee and running at all on a 6 GB laptop.

The Lazy Payoff

From a voice note to a ready-to-import calendar file takes about 12 seconds of processing for a typical short note on a 6 GB laptop GPU. For a 26 s note it's about 31 s.

I did not time myself typing the same tasks into a calendar by hand, so I won't claim a "3× faster" number. I only tested on my synthetic notes this week, and the next step is to use it on my own real voice notes.

Prize Categories

• Best Use of Gemma: Gemma 4 E4B and E2B do both the speech understanding and the task extraction, on-device.

devchallenge #hf26challenge