I Built Kaagaz for My Father So Important Papers Stop Becoming Forgotten WhatsApp Messages

Aniruddh Vijayvargia — Fri, 02 Oct 2026 13:44:05 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

Kaagaz — काग़ज़

My father handles a lot of our household paperwork: electricity bills, school fees, insurance renewals, property tax notices, certificates, and increasingly, important messages arriving over WhatsApp or SMS.

The problem isn't that these papers are difficult to read. The problem is remembering what actually needs to happen next.

So I built Kaagaz for him.

You can show Kaagaz a paper, screenshot, or message, or simply tell it what you received in Hindi, Hinglish, or English.

Kaagaz:

understands what the document is
extracts the amount, deadline, required action, and consequence
shows the exact evidence it used
explains it in simple language
asks for confirmation before remembering anything
reminds you 30, 7, and 1 day before the deadline
tells the son when an important reminder slips
detects common scam signals in suspicious messages
offers to carry yearly renewals into the following year without inventing the next year's amount

The product loop is deliberately simple:

Understand → Confirm → Remember → Follow through → Protect

The goal wasn't to build another chatbot. It was to build something my father could actually use when a piece of paper or message lands in front of him.

Demo

Live: https://kaagaz-bx24.onrender.com/

The public demo uses fictional sample papers only.

How it works: https://kaagaz-bx24.onrender.com/how.html

Code

GitHub: https://github.com/ANIRUDDH-001/Kaagaz

The repository contains the complete application, evaluation suite, deployment configuration, Temporal workflows, reminder worker, and local/private inference setup.

How I Built It

Open-source AI is at the core of Kaagaz.

The main reasoning layer is Gemma 4. The public demo uses Gemma 4 26B, while the private family mode can run Gemma 4 E4B locally through Ollama.

That distinction is important.

In public mode, a user can try the full application without setting up a local model.

In private mode, the same application can run the document understanding locally so that raw family papers and audio do not need to leave the family's machine.

For voice input, I use ElevenLabs Scribe v2 in the public deployment and Whisper large-v3-turbo locally in private mode.

For readbacks, Kaagaz uses ElevenLabs multilingual_v2, with browser speech as the private-mode fallback.

The backend is built with FastAPI and Python.

MongoDB Atlas stores the confirmed structured information: obligations, notification inbox state, and household state. Raw document photos and audio are not stored in MongoDB.

Temporal runs the reminder workflows. Each obligation gets its own durable workflow, allowing Kaagaz to wait for dates months in the future, retry notification failures, and handle actions such as done, snooze, repeat, and date updates.

Because the API is hosted on Render's free tier and can sleep, a lightweight GitHub Actions worker periodically wakes up and drains due reminder work. That means the reminder system doesn't depend on the web process staying awake continuously.

The frontend is a lightweight mobile-friendly PWA with camera, microphone, push notifications, language switching, and Papa/Son roles.

Why Does Open Innovation Matter?

This project is personal, so privacy and control matter more than simply getting an AI response.

An important household paper can contain an address, account details, payment information, dates, or other sensitive information. With a closed-only architecture, I would have to send that information to a third-party model every time.

Open-weight models gave me another option.

With Gemma running locally, the AI part of Kaagaz can stay on the family's own machine. I can also swap model sizes depending on the hardware: a smaller model for constrained devices, or a larger one when quality matters more.

That flexibility changed the architecture of the project.

Open models also made it possible to design the AI layer around our own validation rules rather than treating the model's output as automatically trustworthy.

For example, Kaagaz doesn't simply accept an extracted date or amount. It keeps the evidence, highlights unsupported values, and requires confirmation before anything becomes a reminder.

For a family paperwork assistant, that control is more important than having a generic AI chat experience.

What I Learned

The hardest part wasn't making an AI understand a document.

It was deciding what the AI must never be allowed to assume.

A reminder system amplifies mistakes: if an AI invents a deadline, that mistake can become a notification, then a stored obligation, then a missed payment.

So I ended up building several safeguards around the model:

Evidence before extraction.

Confirmation before persistence.

Deterministic validation before scheduling.

Durable workflows after scheduling.

I also learned that scam detection should not claim that something is "safe." Kaagaz instead shows the specific warning signs it found and lets the user send the warning to their son.

Evaluation

I built a controlled evaluation set containing 26 fictional documents and messages, including genuine household paperwork and scam examples, with multiple simulated image conditions plus Hindi, Hinglish, and English instructions.

The latest documented Gemma evaluation achieved:

412/412 paper fields exactly correct
0 wrong amount/date errors left unhighlighted
24/24 scam messages correctly red-warned
0/80 genuine papers incorrectly red-warned
20/20 typed instructions correct
8/8 synthetic spoken clips correct
9.7s median end-to-end AI time
13.4s p90

These are controlled, synthetic evaluation results, not a claim of real-world accuracy. Real crumpled documents and more sophisticated scams can still be harder.

Prize Categories

I am entering the categories that Kaagaz genuinely uses:

Best Use of Gemma — Gemma 4 is the core document-understanding model, with both hosted and local inference.

Best Use of Render — Render hosts the deployed Kaagaz application.

Best Use of ElevenLabs — Scribe v2 handles public speech-to-text and ElevenLabs multilingual_v2 provides spoken readbacks.

Best Use of MongoDB Atlas — Atlas is the application's persistent data layer for confirmed obligations, notification state, and household data.

Best Use of Temporal — Temporal provides durable, retryable reminder workflows that can wait months for future deadlines.

Best Use of GitHub Copilot — GitHub Actions automates the scheduled reminder worker used by the deployed system.

DEV Community: Aniruddh Vijayvargia