Atharv Soni

Posted on Oct 4

Dadi's Dawai Diary: Turning Doctor Voice Notes into Large-Print Hindi Medicine Schedules with Local Open-Source AI

#devchallenge #weekendchallenge #hf26challenge

Hacktoberfest Weekend Challenge: Build for a Friend Submission 🤝

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built "दादी की दवाई डायरी" (Dadi's Dawai Diary) for my grandmother ("दादी"), who lives in India and manages multiple daily medications for hypertension, diabetes, acid reflux, vitamins, and knee pain.

The Real-World Struggle

Every few weeks after a doctor visit, our family faces the exact same dilemma:

The Prescription Language Barrier: Doctor consultation slips are scribbled in medical English; medicine foil blister packs have tiny, illegible font. Dadi only reads Hindi and cannot decipher whether a white pill is for blood pressure or acidity.
Confusing Meal Timing: Dadi constantly asks: "Beta, ye khali pet leni thi ya khane ke baad?" (Should I take this on an empty stomach or after food?). Confusing morning pills with bedtime pills can be medically hazardous.
Medical Privacy Dealbreaker: We refused to upload Dadi's private doctor voice notes or consultation recordings to closed commercial cloud APIs (like OpenAI or AWS Transcribe). Grandma's personal health diagnoses and medical history belong strictly inside our family.
Spotty Internet: In Indian clinics and homes, cellular Wi-Fi frequently disconnects. An app that depends on an active cloud connection is useless when connectivity drops.

What Dadi's Dawai Diary Does:

Audio Consultation Intake: A family caregiver records a voice note of the doctor speaking (or uploads the audio), speaking naturally in Hindi or Hinglish.
100% Local Speech-to-Text: Transcribes the consultation locally on device using the open-weights Whisper model via WebAssembly / ONNX.
Medical Entity Extractor: Intelligently parses medicine names, dosages (e.g. 1 गोली), meal conditions (खाली पेट vs खाने के बाद), time-of-day slots (🌅 सुबह, ☀️ दोपहर, 🌙 रात, ⚠️ ज़रूरत पर), and purpose in Hindi (गैस के लिए, शुगर के लिए).
Caregiver Safety Verification: Before Dadi sees it, a safety modal allows a family member to review, edit, or add medicines.
Dadi's High-Contrast Accessible Schedule:
- Giant 40px+ Devanagari typography with high contrast for aging eyes.
- Time-of-day tabs with intuitive visual icons (🌅 Morning 8:00 AM, ☀️ Afternoon 1:30 PM, 🌙 Night 8:30 PM, ⚠️ SOS).
- Visual pill representations (🟡 Yellow capsule, ⚪ White round tablet, 🔴 Red capsule).
- 🔊 "दादी सुनो" (Gentle Hindi Audio Readout): With one tap, the app gently speaks each instruction aloud in calm Hindi ("दादी जी, सुबह खाली पेट पैंटोसिड की एक गोली लेनी है...").
- 🖨️ Print for Fridge: Formats a single-sheet, high-contrast timetable to stick on the refrigerator or bedside table.
- 📲 WhatsApp Family Share: Generates a clean Hindi message ready to copy into the family WhatsApp group.

Demo

Here is a walkthrough of Dadi's Dawai Diary running completely locally:

1. Doctor Consultation Audio Intake

Caregivers can record live voice notes, upload audio, or choose preset clinic scenarios:

2. Caregiver Safety Verification Modal

Safety first: A family member confirms the extracted medicines and dosages before generating Dadi's schedule:

3. Dadi's Accessible Large-Print Schedule

Giant cards, color-coded pill indicators, time-of-day slots, and the 🔊 "दादी सुनो" audio button:

Code

ath1614 / Dadi-s-Dawai-Diary

👵 दादी की दवाई डायरी (Dadi's Dawai Diary)

100% Local Open-Source AI that turns spoken Hindi/Hinglish doctor consultations into large-print, high-contrast medicine schedules for grandparents.

🎯 What It Does & Who It Is For

The Friend & Loved One: My Grandmother ("दादी")

My grandmother, like millions of elderly grandparents in India, manages multiple daily medications (hypertension, blood sugar, acid reflux, vitamins, pain management).

The Real-World Struggle

Cryptic Prescriptions & Language Gap: Doctor slips are scribbled in medical English, while blister packs have tiny unreadable text. Dadi speaks Hindi and struggles to remember which pill is which.
Confusing Timing & Meals: Dadi frequently confuses "खाली पेट" (empty stomach / before food) with "खाने के बाद" (after food), or forgets whether a pill was meant for afternoon or night.
Privacy Dealbreaker with Closed AI APIs: Grandma's personal health diagnoses and doctor recordings shouldn't be sent across the internet to closed, commercial US…

View on GitHub

The code is 100% open-source under the MIT License: GitHub Repository: ath1614/Dadi-s-Dawai-Diary

How I Built It

The entire project is built from scratch around open-source AI and local runtime technologies:

Local Open-Weights Whisper STT:
- Powered by @xenova/transformers (Transformers.js) running Xenova/whisper-tiny via ONNX runtime and WebAssembly directly in the browser.
- Zero audio packets leave the device.
Local Hindi Medical Consultation NLP Engine:
- Built a deterministic and regex-assisted entity extraction pipeline specifically tuned for Indian medical brand names (Pantocid, Metformin, Telma, Dolo 650, Neurobion Forte, Shelcal, Becosules, Azithral) and Hindi speech patterns (खाली पेट, खाने के बाद, नाश्ते से पहले, ज़रूरत पड़ने पर).
- Generates natural, respectful Hindi prompt strings formatted for elderly comprehension.
Local Speech Synthesis:
- Utilizes client-side speech synthesis configured for Indian Hindi (hi-IN) with adjusted pitch and slower cadence (0.88x speed) tailored for elderly ears.
Accessible Indian Design System:
- Built using Vanilla CSS and Google Fonts (Rozha One and Yantramanav) with WCAG AAA contrast compliance, warm Indian palette (saffron, turmeric, marigold, indigo), and a dedicated @media print layout for fridge sheets.

Why Does Open Innovation Matter?

Open-source AI was not just a design choice for this project; it was an absolute necessity. Closed proprietary APIs (like OpenAI Whisper or Google Cloud Speech) would have failed this application:

Feature	Closed Cloud APIs	Local Open-Source AI (Dadi's Dawai Diary)
Medical Privacy	Consultations and private health data uploaded to US servers.	100% On-Device: Zero data ever leaves Dadi's room. Complete HIPAA/privacy compliance.
Offline Reliability	Fails completely during internet outages in clinics or homes.	Works Anywhere: Runs without internet or SIM connection.
Ongoing Cost	Metered token fees and monthly subscriptions.	$0 Cost Forever: Open weights are completely free for families.
Cultural & Dialect Freedom	Closed black-box with no ability to tune for regional slang.	Hackable & Extendable: Anyone can adapt or fine-tune models for Bhojpuri, Gujarati, Tamil, or Bengali.

My Agent Session

This project was built pair-programming with Antigravity AI, utilizing DevRelay to manage tools, debug upstream Playwright driver dependencies, and capture automated browser testing demonstrations.

Building Dadi's Dawai Diary: Local Open-Weights AI for Grandparents

You

I am participating in Hacktoberfest DEV Weekend Challenge 1 (Build for a Friend). I want to build 'Dadi\'s Dawai Diary' from scratch: an open-source tool that turns doctor Hindi/Hinglish voice notes into a large-print medicine schedule for my grandmother, running 100% locally. Also, devrelay login was failing with unknown command, and browser automated testing driver was throwing errors.

Agent

Diagnosed and resolved the devrelay PATH issue, connected to MLH/DEV profile as Atharv Soni (@atharv_soni_8a5d9cd294432), and fixed the Playwright driver 404 issue by providing the offline ms-playwright-go 1.57.0 driver. Integrated browser testing is verified.

You

Let's complete this project and submit this for Hacktoberfest.

Agent

Implemented complete Dadi's Dawai Diary: 100% Local Whisper STT pipeline, Hindi/Hinglish Medical Consultation NLP parser, Caregiver Safety Verification modal, Dadi's accessible large-print cards with time-of-day tabs, gentle Hindi speech synthesis, printable fridge schedule, and automated browser demo recording.