Madras Field Notes — A Fully Offline, In-Browser Field Guide for Chennai's Birds and Plants

pkbhalla — Mon, 12 Oct 2026 06:58:00 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Madras Field Notes is an installable PWA that turns your phone into a pocket naturalist for Chennai — and it works entirely in airplane mode.

Identify: Point your camera at a bird or plant. A CLIP zero-shot image classifier — running inside your browser, against a curated list of 15 species common to IIT Madras campus and the Pallikaranai marsh — tells you what you're looking at, with an honest "not sure" state below its confidence threshold.
Learn: Every species gets a card from a hand-built knowledge pack: 3 facts, identifying features, and one safety note — in both English and Tamil. No screen time bloat; read the card, look back up.
Ask: A Qwen2.5-0.5B open-weight model running locally in the browser tab answers follow-up questions ("Is this bird migratory?") grounded strictly in the species card — it cannot hallucinate edibility or medical advice because it never sees the open internet.
Log: Identifications save to an on-device field journal, exportable as JSON for citizen-science projects.

Live demo: https://madrasfieldnotes.onrender.com (open it once online, then switch on airplane mode — everything still works)
Source: https://github.com/pkbhalla/MadrasFieldNotes

Demo flow: load once → airplane mode → identify a rose-ringed parakeet from a photo → read its card in Tamil → ask the local LLM a question → check the field log. Zero network requests after first load.

How I Built It

Stack: Vite + TypeScript, no backend, no API keys, no analytics. Hugging Face Transformers.js v3 runs both models in Web Workers so the UI never blocks.
Identify: Xenova/clip-vit-base-patch32 zero-shot image classification on WASM, against label phrases built from the knowledge pack.
Chat: onnx-community/Qwen2.5-0.5B-Instruct at q4 quantization (~300 MB), with the species JSON injected into its system prompt as the only allowed knowledge source.
Offline: Service Worker caches the app shell; the Cache API persists model weights; navigator.storage.persist() guards against browser eviction.

Where Open Beat Closed

This project only exists because the models are open — and my biggest failure proves why that matters.

1. I could see the real cost. Closed APIs hide their footprint behind a spinner. Open weights forced me to confront the physics: my first build loaded a 1B model's fp32 weights (~2 GB) and Chrome killed the tab the moment CLIP and the LLM were resident together. Android caps a browser tab around 1–1.5 GB no matter how much RAM the phone has. Because the weights were mine to inspect and swap, the fix was a real engineering tradeoff — q4 quantization, a smaller instruct model, identification-first design — not a support ticket. Chat now ships desktop-first, with mobile chat as the honest next step.

2. Zero per-query cost means zero reason to stay inside. No API bill means the app nags nobody, tracks nobody, and never degrades when a free tier ends. The user "closes" the screen because using it worked.

3. The knowledge pack is the forkable product. species.json holds Chennai's birds and plants with Tamil names. Any community — Coimbatore, Colombo, Kuala Lumpur — can fork the repo, swap 15 JSON entries, and ship their field guide, in their language, without asking anyone for permission or a bill. That's what "gets people off the screen and into the world" looks like at ecosystem scale.

4. Privacy is architectural, not a policy page. Photos of what you find on a trail never leave the device, because there is no server to send them to.

What I'd Do Next

Shard-free sub-500 MB total budget so chat survives mobile tab limits
Bird-call identification via the open-source BirdNET browser port
Community-contributed packs (crowdsource species.json per region)

Built in ~24 hours for the hackathon. The screen is the shortest part of this experience — by design, and by memory ceiling.

DEV Community: pkbhalla

Madras Field Notes — A Fully Offline, In-Browser Field Guide for Chennai's Birds and Plants

What I Built

How I Built It

Where Open Beat Closed

What I'd Do Next