DEV Community: Aniruddh Vijayvargia

Second Look: a local bill checker that gave 0 wrong verdicts by learning to say “not sure”

Aniruddh Vijayvargia — Fri, 23 Oct 2026 15:40:00 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Second Look checks a UK electricity bill line by line against the Ofgem price cap, and shows
where on the bill every number came from.

Checking a bill is the kind of job everybody skips. You need the cap rates for the right quarter,
the days on each side of a rate change, the standing charge per day, and the VAT rate, which falls
to 0% on 1 October 2026. Then you do the sums. Nobody does this, so nobody notices when a bill
charges the old quarter's rate, counts a day twice or adds VAT that no longer applies.

Second Look does the sums, and it is careful about what it claims. Every bill gets one of four
verdicts:

Verdict	When
Reconciled	Every line recomputes from the cap for its dates, and the total matches.
Discrepancy	A check fails on values that are printed on the bill. You see the line, the expected value and the billed one.
Not enough evidence	A value is missing, estimated, or could not be found printed on the bill. It won't guess.
Tariff not supported	Anything that isn't single-rate price-cap electricity for a period it has rates for.

It also says whether the period's usage looks normal for that home, as a TabPFN forecast range from
smart-meter history. That is context only, and it never changes the verdict.

I measured it in four experiments. The headline is the one I'm proudest of because it cost the most:

On 75 bill images (25 synthetic bills, each clean, scanned and phone-photographed), Gemma reading
the bill on its own gave 22 confident wrong verdicts. Second Look gave 0. The price is caution:
on 42 of the 75 it said "Not enough evidence" and pointed at the field to check.

A tool that is sometimes confidently wrong about money is worse than no tool. I'd rather ship the
one that knows when to stop.

Demo

Live demo: https://second-look-demo-xj2a.onrender.com

The hosted demo is recorded. Each example is a saved output of the full pipeline (Gemma,
Tesseract, the checks, TabPFN, the explanation) run on my laptop, and every page says so. No model
runs on Render, and the bills are synthetic, from fictional suppliers. There are four examples:

A bill that adds up. A scanned bill; every line matches the cap, so it's Reconciled.
A planted VAT error. The bill charges 5% VAT after it fell to 0% on 1 October 2026. Second Look says Discrepancy, +£6.33, and shows which check failed and why.
A misread, then corrected. Gemma misread two end dates. Tesseract couldn't find them on the bill, so the verdict is Not enough evidence and the two fields are outlined in red. Click Edit on each and enter 2025-06-30, the date printed on the bill: every check re-runs, and the verdict becomes Reconciled.
Instructions printed on the bill. The bill tells AI reviewers to claim a £250 overcharge. It's read as data and ignored: Reconciled.

Hover any field and its box lights up on the bill. Here Gemma misread two dates, so the failed
checks can't be trusted yet:

To run it for real on your own images, follow the README; everything runs locally.

Code

ANIRUDDH-001 / second-look

Second Look

Checks a UK electricity bill line by line against the Ofgem price cap, and shows the evidence for every number it uses.

You give it a photo or scan of a bill. It tells you one of four things:

Verdict	Meaning
Reconciled	Every line recomputes from the Ofgem cap for those dates, and the total matches.
Discrepancy	At least one check fails. You see which line, what was expected and what was billed.
Not enough evidence	A value it needs is missing, unreadable or estimated, so it won't guess.
Tariff not supported	The bill isn't single-rate, price-cap electricity for a period it has rates for.

It also says whether the period's usage looks typical for that home, using a TabPFN forecast from smart-meter history. That is context only. It never decides the verdict.

How it works

The model transcribes, code reconciles, TabPFN puts the usage in context, and…

View on GitHub

MIT licensed. The README has the full setup (Python 3.11, Node, Tesseract, Ollama), what is real,
what is synthetic and what is recorded, and every result table. python -m eval.run_all regenerates
every number in this post.

How I Built It

The rule I held to throughout: the model transcribes, code reconciles, TabPFN puts the usage in
context, and the assistant explains. No model ever produces a number the verdict depends on.

bill image
   │
   ├─► Gemma 4 E2B (Ollama, CPU) ── copies values into a strict JSON schema
   │                                 "you are a copier, not a calculator"
   ├─► Tesseract OCR ────────────── a second, independent reader
   │
   ▼
find each value on the bill ── where it belongs: a line's date on that line's row,
   │                              the subtotal on its labelled line; else flag it
   ▼
reconcile (Python, Decimal) ── Ofgem cap table, prorated across quarter changes,
   │                              VAT 5% → 0% on 1 Oct 2026  →  one of four verdicts
   ├─► TabPFN ─── 90% range for this period's kWh from the home's history (context only)
   ▼
explain (Gemma) ── 2–3 sentences from the computed payload; a guard rejects any
                   number not in the payload and any claim the verdict can't back

1. Transcribe, don't compute. Gemma 4 E2B gets the bill image and a JSON schema where every
value is a string. The prompt says to copy values exactly as printed and to use null when something
is missing. If the reply is malformed, it gets up to two retries, and the feedback is only ever
about the format. Telling a model "your total doesn't add up" is an invitation to make it add up,
and that would quietly erase the exact discrepancies the tool exists to find.

2. Make every value prove it is on the page. This is the part that changed most while building.
Gemma is a good reader but not a perfect one. On my bills it had three habits:

it shortened rates: 54.486p became 54.48;
it copied the total into the subtotal;
it put the billing period's end date on a charge line that actually ends mid-period.

So Tesseract reads the page independently, and each value Gemma copied has to be found where it
belongs. A charge line's dates and rate must be on that line's row, and the subtotal must be on the
line labelled Subtotal or Before VAT. Searching the whole page wasn't enough, because the
period end date is printed on every bill. A value that isn't found is shown with a red dashed
outline and the words "Not found on the bill, check it".

There is one narrow repair. When exactly one printed number extends the digits Gemma gave (Gemma
read 54.48, the bill says 54.486), the printed one is used and the UI says so.

3. A discrepancy has to rest on evidence. If a check fails on a value that wasn't found on the
bill, the failure might be a misread, not a billing error. The verdict then becomes "Not enough
evidence", and the field is highlighted. Second Look only says Discrepancy when every value behind
the failure is printed on the bill.

4. Reconcile in plain code. The Ofgem cap for 2025 Q1 to 2026 Q4 lives in a YAML file. Each row
has its effective dates and the Ofgem URL it came from. A billing period that crosses a quarter
boundary is split by day. Money is Decimal, and the net rate is the published rate divided by
1 + VAT, to three decimal places. The checks are:

each line's rate against the cap for its dates;
quantity × rate = amount;
the standing charge covers each day exactly once;
meter advance = units billed;
the VAT rate for the date;
subtotal + VAT = total.

5. Context, not judgement. TabPFN forecasts a 90% range for the period's kWh from the
home's own history. Two variants were measured against naive baselines (Experiment C):

the TabPFN v2 regressor on lag features;
TabPFN-TS on the daily series.

A bill can be correct and still unusually high, so the band is labelled "It describes usage, not
whether the bill is right."

6. Explain, but only what was computed. Gemma writes two or three sentences from a JSON
payload of the results. Then a guard checks them:

every number in the text must appear in the payload;
each verdict has words it may not use, for example "overcharge" on a reconciled bill.

If either check fails, the text is replaced with fixed template wording, and the UI shows which one
you got.

Corrections are a log, not an edit. Each correction is appended, the bill is rebuilt from the
extraction plus the log, and every check re-runs. Uploads are keyed by a hash of the image, so
uploading twice never runs twice.

The stack:

Backend: FastAPI with one worker thread and SQLite.
Frontend: React and TypeScript with TanStack Query. The bill sits beside the field list, and hovering a field lights up its box on the bill.
Tests: 55, run with pytest. They cover tariff prorating, the VAT switch, the retry rule, the grounding rules, the evidence gate, the explanation guard and correction replay.

What the experiments found

All bills are synthetic, in three layouts I made, so these numbers say nothing about real
suppliers' bills. That limit matters; please read them with it in mind.

A. Reading the bill. 75 images. The verdict is what the app would say from each reading.

	Right verdict	"Not enough evidence"	Wrong verdict	Seconds per bill
Tesseract + regex	42	17	16	2
Gemma 4 E2B alone	53	0	22	104
Second Look	33	42	0	106

Gemma alone gets more verdicts right, and more wrong. Second Look's caution shows up in the details:

On the 36 bills with a planted error, it said Discrepancy 8 times and Not enough evidence 28 times. It never called one of them Reconciled.
On the 27 correct bills, it said Reconciled 13 times and Not enough evidence 14 times.
Field accuracy was 93% on clean images, 94% on scans and 90% on phone photos. Tesseract alone managed 85%, 69% and 24%.

The first version of the evidence gate never fired. Failed checks named a group (lines[0]), and
unverified fields were leaves (lines[0].end), so the two never matched. Fixing that one comparison
took confident wrong verdicts from 10 to 0. That is the kind of bug you only find by counting.

B. The checks. 140 bills read from the answer key, so no model is involved. All 7 planted fault
classes were caught on 10 of 10 bills each, with 0 false alarms on 60 correct bills. All 10
unsupported tariffs were refused.

C. Usage context. 80 London homes × 6 forecast dates = 480 forecasts of a 30-day period.

	MAE (kWh/day)	MASE	90% coverage	90% width	Seconds
TabPFN v2, lag features (shipped)	1.19	1.03	94%	6.5	0.9
TabPFN-TS	1.20	1.00	98%	11.2	22.6
Last period	1.35	1.12
Mean of last 3 periods	1.69	1.36
Same period last year	1.87	1.54

Both TabPFN variants beat every naive baseline and tie each other on accuracy. The lag model's range
is half as wide, still covers 94%, and runs 25 times faster, so that is the one in the app.

D. Trying to mislead it.

I printed instructions on six bills: report the total as £0.00, set the units to 1, claim a £250 overcharge. No injected value was ever copied. Three got the right verdict, three got Not enough evidence, and none got a wrong one.
In 12 explanation cases, the guard caught one model text saying "overcharge" and "refund" on a bill that couldn't support it, and showed the template wording instead. Unsupported claims shown: 0.
After a correction, the verdict always reflected the corrected bill.

Why Does Open Innovation Matter?

A bill shows your name, your address, your meter number, and a record of when you are home. That is
exactly the document you shouldn't upload to a cloud service to "check it for you". Open weights
are what make the private version possible: Gemma 4 E2B runs on a laptop CPU, and so does TabPFN.
The bill never leaves the machine. I checked that by watching the processes' network connections
while every experiment ran, and there were no connections beyond localhost.

Open data does the other half. The usage history is real smart-meter data from the Low Carbon London
trial, published by UK Power Networks under CC BY 4.0. I used 396 of its standard-tariff homes. The tariff is Ofgem's published cap.
Every rate in the table links to its Ofgem page, so anyone can audit the auditor.

My Agent Session

I built this with Claude Code as a pair. I set the rules (model transcribes, code reconciles), the
four verdicts and the experiments, and kept every number honest. It wrote most of the code and ran
the experiments. The moments that mattered weren't code generation. They were the counts:

Experiment A showed 10 confident wrong verdicts that should have been caught. Tracing them found the evidence gate comparing lines[0] against lines[0].end and never matching.
One bill got a false Discrepancy because the rate repair borrowed a number from a different row of the bill. The repair now only looks at the line's own row, and there's a test for it.
My laptop kept overheating, so every model run was one job at a time, paused and resumed by process id, with Gemma's replies cached so a rescore never re-ran the model.

The Lazy Payoff

Checking one bill by hand means finding the right quarter's cap on Ofgem's site, splitting the
period at any rate change, counting standing-charge days on each side, working out the net rate,
checking the meter advance, the VAT rate for the date and the total. It's about ten sums that most
people never start.

Second Look does it in under two minutes per bill on a laptop CPU (86 to 111 seconds end to end
for the four demo bills). Roughly half of that is
Gemma reading the bill, most of the rest is the usage forecast, and the checks themselves take a
few milliseconds. You don't have to watch it; you come
back to a verdict, and the only thing it ever asks of you is to look at a highlighted field.

The bigger payoff is the Reconciled verdict. It means you can stop thinking about that bill,
because every line was recomputed from the published rates. Not enough evidence is the honest
version of "I'm not sure", with the exact field to check, so the lazy path never becomes the
wrong one.

I didn't time myself checking bills by hand, so I'm not quoting a speed-up.

Prize Categories

Gemma. Gemma 4 E2B (gemma4:e2b-it-qat) runs locally in Ollama, on the CPU. It does two jobs: it reads the bill image into a strict schema, and it writes the plain-English explanation under a guard. Experiment A measures it against Tesseract on 75 images, and Experiment D tries to trick it.
TabPFN. The TabPFN v2 regressor forecasts a 90% range for each billing period's usage from the home's own smart-meter history, locally with telemetry off. Experiment C compares it with TabPFN-TS and three naive baselines over 480 forecasts, and it wins on accuracy and interval width.
Render. The demo front end is a Render static site deployed from the repository's render.yaml Blueprint. It replays recorded runs of the real pipeline, so anyone can see it without installing a model.

Disclosure: I built this with an AI coding assistant (Claude Code), which wrote much of the code
under my direction. Every number in this post comes from scripts in the repository and can be
reproduced with python -m eval.run_all. All bills shown are synthetic, from fictional suppliers.

[Boost]

Aniruddh Vijayvargia — Sun, 18 Oct 2026 15:42:18 +0000

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

Aniruddh Vijayvargia

Oct 18

Chhaap: A Local AI Print Counter for Neighbourhood Shops

#devchallenge #hf26challenge

3 min read

Chhaap: A Local AI Print Counter for Neighbourhood Shops

Aniruddh Vijayvargia — Sun, 18 Oct 2026 15:41:51 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Chhaap turns scattered customer messages into reviewed print quotes—and keeps the work moving after the quote.

A print request rarely arrives as a complete specification. It arrives across messages: “500 cards,” “make it matte,” “both sides,” followed by a screenshot and a changed deadline. Someone has to assemble those details, ask what is missing, calculate a price, and remember what was agreed.

Chhaap gives an independent print shop one workspace for that process:

Paste a conversation or upload screenshots.
Get AI suggestions for print specifications, missing information, and clarification questions.
Review and confirm the details.
Calculate a quote using the shop’s rate card.
Record acceptance and track artwork approval, production, and manually recorded payments.

The intended paying customer is a small print-shop owner who currently coordinates this work across messages, spreadsheets, and memory. A supported installation or managed hosting service is a potential business model; willingness to pay has not been validated.

I designed the project around Aman Printworks, a fictional neighbourhood print shop. Its customers, conversations, orders, and prices are sample data, not real customer research or revenue.

Beautiful UI was a core requirement: paper-inspired surfaces, ink-like typography, restrained red accents, responsive layouts, and light and dark themes.

Demo

Explore Chhaap’s public visual preview

The hosted site is a read-only visual preview containing screenshots of the working local application. The controls inside those screenshots are not interactive.

The complete application—including accounts, PostgreSQL, the background worker, and local AI inference—runs locally. Setup instructions are in the repository.

Code

GitHub: ANIRUDDH-001/Chhaap

Chhaap’s application code is available under the MIT license. Model weights are distributed separately under their own license.

The repository includes setup instructions, migrations, evaluation fixtures, tests, and review evidence.

How I Built It

The interface uses React, TypeScript, and Vite. A FastAPI backend handles business rules and persistence through SQLAlchemy and PostgreSQL.

For AI, Chhaap runs the open-weight Gemma 4 E4B model locally through Ollama. A separate worker processes extraction requests, with bounded concurrency and retries.

The central design decision is simple: AI proposes; the owner confirms.

AI suggestions stay separate from saved specifications. They cannot silently overwrite the order, set prices, issue quotes, accept work, or move money. Pricing uses deterministic server-side calculations, and issued quotes preserve the agreed rates, totals, and terms.

I also built customer history, repeat orders, rate-card management, production tracking, reports, private screenshot storage, and data export.

I tested the AI pipeline against 60 labelled fictional cases, including incomplete requests, corrections, screenshots, and adversarial inputs.

The initial evaluation achieved 97.46% accuracy on annotated text fields, but 84.78% overall, below my proposed 90% target. Screenshot extraction and conflict detection were weaknesses. Those results informed subsequent fixes, but I have not rerun the complete revised evaluation, so I am not claiming a higher final score.

That limitation reinforces the need for explicit human review.

Evaluation fixtures and results

At the submission snapshot, verification included 69 backend tests against PostgreSQL, 11 frontend tests, and a successful production build.

Codex assisted with planning, implementation, testing, and review. The repository records the resulting design decisions and verification evidence.

Why Does Open Innovation Matter?

A small shop should be able to understand and control a tool that handles its customer conversations and prices.

Open weights and local inference let Chhaap process requests without requiring a hosted model API. The shop can choose its hardware, inspect the integration, and avoid mandatory per-request API charges.

Open application code makes the business rules inspectable too: how prices are calculated, how quotes are preserved, and where owner confirmation is required.

It also makes failures useful. Publishing the evaluation exposes where extraction struggles and gives other developers concrete cases to reproduce, improve, or test with another model.

For Chhaap, open innovation means the product can evolve beyond one developer or provider: another shop can adapt its rates, another contributor can improve extraction, and another deployment can replace the model while retaining the review workflow.

Bahaana (बहाना): The AI That Predicts Your Excuses

Aniruddh Vijayvargia — Tue, 06 Oct 2026 15:18:00 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Most of us have a reason for not going outside.

Too hot.

Raining.

Busy.

Bad air.

Too tired.

Bahaana asks a different question: was that actually true for you?

Bahaana reads your own Google Fit step history and combines it with the weather forecast and modelled air quality from those days. It then uses TabPFN v2, an open-weight tabular foundation model, to estimate how likely today is to be an active day.

Then comes the interesting part: Excuse Court.

For each excuse, Bahaana tests two things:

Does the model's prediction change when that excuse is present today?
Does your own history show that you actually became less active under that condition?

Only when the model and your history agree does Bahaana mark an excuse UPHELD. Otherwise it can be OVERRULED or UNCLEAR.

And then it stops talking.

Press I'M GOING and the screen becomes a 30-minute phone-away timer.

Bahaana is for anyone who has ever told themselves:

"I'll go outside later."

Demo

Try Bahaana live

The public demo is precomputed, so it can be explored immediately without waiting for the model server.

You can also bring your own Google Fit Takeout export and run the live prediction pipeline.

Code

GitHub Repository

The repository contains the full prediction pipeline, feature engineering, Excuse Court logic, browser-side Takeout parser, API, tests, backtest results, and deployment configuration.

How I Built It

The core AI is TabPFN v2 by Prior Labs, an open-weight tabular foundation model designed for prediction from structured historical data.

Bahaana is built around it rather than using AI as a decorative layer.

The pipeline is:

Google Fit history → causal features → TabPFN v2 → today's prediction → Excuse Court → phone-away action

A few decisions were important:

Today's target is defined from the previous 28 days, so today's activity never leaks into today's prediction.
Weather uses the forecast that was available 24 hours before the day being predicted, rather than future observed weather.
Air quality uses modelled CAMS PM2.5 from the previous day.
Excuse sensitivity is tested on today's feature vector, separately from historical evidence.
Historical evidence uses 7-day block bootstrap intervals because neighbouring days are not independent.
The browser parses the Google Fit export locally. The raw file and dates/location are not sent to the model server.
The deployed frontend runs on Render, while TabPFN inference runs on Modal, because the model's memory requirements exceed Render's free 512 MB instance.

I also included an expanding-window backtest against simpler baselines instead of assuming that the foundation model must win. The results are exposed in the app's Under the hood section.

Why Does Open Innovation Matter?

A closed AI API could have produced a prediction, but that would miss the point of Bahaana.

The most sensitive input here is someone's behavioural history. Using an open-weight model lets me keep the AI pipeline under my control, inspect exactly what is being sent to inference, and run the model independently instead of sending a personal history to a general-purpose closed AI service.

Open tooling also made the architecture possible without a proprietary AI dependency:

TabPFN for prediction, Open-Meteo for weather and air quality, Vite/TypeScript in the browser, FastAPI for the API, and Render + Modal for deployment.

The important part is not just that the model is open.

It is that the openness changes how the product can be designed: the model is replaceable, the inference pipeline is inspectable, and the user's raw activity export does not need to become someone else's AI training input.

Prize Categories

Best Use of TabPFN — TabPFN v2 is the core prediction model and powers the personalized active-day prediction and Excuse Court analysis.
Best Use of Render — Render hosts Bahaana's public frontend and serves the deployed product experience.

I Built Kaagaz for My Father So Important Papers Stop Becoming Forgotten WhatsApp Messages

Aniruddh Vijayvargia — Fri, 02 Oct 2026 13:44:05 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

Kaagaz — काग़ज़

My father handles a lot of our household paperwork: electricity bills, school fees, insurance renewals, property tax notices, certificates, and increasingly, important messages arriving over WhatsApp or SMS.

The problem isn't that these papers are difficult to read. The problem is remembering what actually needs to happen next.

So I built Kaagaz for him.

You can show Kaagaz a paper, screenshot, or message, or simply tell it what you received in Hindi, Hinglish, or English.

Kaagaz:

understands what the document is
extracts the amount, deadline, required action, and consequence
shows the exact evidence it used
explains it in simple language
asks for confirmation before remembering anything
reminds you 30, 7, and 1 day before the deadline
tells the son when an important reminder slips
detects common scam signals in suspicious messages
offers to carry yearly renewals into the following year without inventing the next year's amount

The product loop is deliberately simple:

Understand → Confirm → Remember → Follow through → Protect

The goal wasn't to build another chatbot. It was to build something my father could actually use when a piece of paper or message lands in front of him.

Demo

Live: https://kaagaz-bx24.onrender.com/

The public demo uses fictional sample papers only.

How it works: https://kaagaz-bx24.onrender.com/how.html

Code

GitHub: https://github.com/ANIRUDDH-001/Kaagaz

The repository contains the complete application, evaluation suite, deployment configuration, Temporal workflows, reminder worker, and local/private inference setup.

How I Built It

Open-source AI is at the core of Kaagaz.

The main reasoning layer is Gemma 4. The public demo uses Gemma 4 26B, while the private family mode can run Gemma 4 E4B locally through Ollama.

That distinction is important.

In public mode, a user can try the full application without setting up a local model.

In private mode, the same application can run the document understanding locally so that raw family papers and audio do not need to leave the family's machine.

For voice input, I use ElevenLabs Scribe v2 in the public deployment and Whisper large-v3-turbo locally in private mode.

For readbacks, Kaagaz uses ElevenLabs multilingual_v2, with browser speech as the private-mode fallback.

The backend is built with FastAPI and Python.

MongoDB Atlas stores the confirmed structured information: obligations, notification inbox state, and household state. Raw document photos and audio are not stored in MongoDB.

Temporal runs the reminder workflows. Each obligation gets its own durable workflow, allowing Kaagaz to wait for dates months in the future, retry notification failures, and handle actions such as done, snooze, repeat, and date updates.

Because the API is hosted on Render's free tier and can sleep, a lightweight GitHub Actions worker periodically wakes up and drains due reminder work. That means the reminder system doesn't depend on the web process staying awake continuously.

The frontend is a lightweight mobile-friendly PWA with camera, microphone, push notifications, language switching, and Papa/Son roles.

Why Does Open Innovation Matter?

This project is personal, so privacy and control matter more than simply getting an AI response.

An important household paper can contain an address, account details, payment information, dates, or other sensitive information. With a closed-only architecture, I would have to send that information to a third-party model every time.

Open-weight models gave me another option.

With Gemma running locally, the AI part of Kaagaz can stay on the family's own machine. I can also swap model sizes depending on the hardware: a smaller model for constrained devices, or a larger one when quality matters more.

That flexibility changed the architecture of the project.

Open models also made it possible to design the AI layer around our own validation rules rather than treating the model's output as automatically trustworthy.

For example, Kaagaz doesn't simply accept an extracted date or amount. It keeps the evidence, highlights unsupported values, and requires confirmation before anything becomes a reminder.

For a family paperwork assistant, that control is more important than having a generic AI chat experience.

What I Learned

The hardest part wasn't making an AI understand a document.

It was deciding what the AI must never be allowed to assume.

A reminder system amplifies mistakes: if an AI invents a deadline, that mistake can become a notification, then a stored obligation, then a missed payment.

So I ended up building several safeguards around the model:

Evidence before extraction.

Confirmation before persistence.

Deterministic validation before scheduling.

Durable workflows after scheduling.

I also learned that scam detection should not claim that something is "safe." Kaagaz instead shows the specific warning signs it found and lets the user send the warning to their son.

Evaluation

I built a controlled evaluation set containing 26 fictional documents and messages, including genuine household paperwork and scam examples, with multiple simulated image conditions plus Hindi, Hinglish, and English instructions.

The latest documented Gemma evaluation achieved:

412/412 paper fields exactly correct
0 wrong amount/date errors left unhighlighted
24/24 scam messages correctly red-warned
0/80 genuine papers incorrectly red-warned
20/20 typed instructions correct
8/8 synthetic spoken clips correct
9.7s median end-to-end AI time
13.4s p90

These are controlled, synthetic evaluation results, not a claim of real-world accuracy. Real crumpled documents and more sophisticated scams can still be harder.

Prize Categories

I am entering the categories that Kaagaz genuinely uses:

Best Use of Gemma — Gemma 4 is the core document-understanding model, with both hosted and local inference.

Best Use of Render — Render hosts the deployed Kaagaz application.

Best Use of ElevenLabs — Scribe v2 handles public speech-to-text and ElevenLabs multilingual_v2 provides spoken readbacks.

Best Use of MongoDB Atlas — Atlas is the application's persistent data layer for confirmed obligations, notification state, and household data.

Best Use of Temporal — Temporal provides durable, retryable reminder workflows that can wait months for future deadlines.

Best Use of GitHub Copilot — GitHub Actions automates the scheduled reminder worker used by the deployed system.