Fieldnotes: a nature journal that uses local Gemma 4, then tells you to put the phone away

Prakul Chandra — Sat, 10 Oct 2026 12:03:09 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Most nature apps want you to keep looking at them. Fieldnotes is built to end its own session.

You photograph a bird, plant or insect, answer a couple of plain-language questions ("not sure" is always an allowed answer), and a Gemma 4 model running on your own computer reads the photo. It does not hand back a confident label. It tells you what it can actually see, offers up to three candidates with an honest likelihood, and lists what you should look for to be surer.

Then comes the part I care about most. Saving the observation doesn't drop you into a feed. It opens a full-screen page with no navigation, one short quest written for that observation, and a timer that keeps running while your screen is locked. A typical quest: sit quietly for five minutes and listen for the calls this bird makes. The page tells you to keep your phone in your pocket. When you come back, you add one line about what you noticed, and it goes into your offline journal next to the photo.

It's for curious beginners, families, and anyone who would rather learn to look than collect labels.

Demo

I recorded it on my laptop: a real photo, Gemma analyzing it, the hedged result, saving to the journal, the go-and-look screen, and the journal entry afterwards.

There is no live link, and that's deliberate. The AI runs on your own machine through Ollama, so a hosted copy of the page would have nothing to talk to. The repo below runs in four commands.

Code

ollama pull gemma4:e2b
npm install
npm run check:gemma path/to/photo.jpg   # confirms the model can read an image
npm run dev

The README covers model setup, limitations, and which model sizes to use on which hardware.

How I Built It

The stack: React + Vite, installable as a PWA, with the journal and quests stored on the device. The AI is Gemma 4, an open-weight model from Google, served locally by Ollama. Photos and answers go from the app to Ollama on the same machine and nothing is sent to a cloud service.

Why Gemma 4: I needed a model that understands images and runs on a laptop. Gemma 4 comes in several sizes that accept images, and the small "E" variants are meant for modest hardware. I develop on an i7 (12th gen) with 16 GB RAM and an RTX 3050 Ti, which has only a small amount of video memory, so I started with gemma4:e2b and tested gemma4:e4b for better vision. The model name is one environment variable, so switching is a one-line change.

Structured output: I ask Gemma for JSON that matches a schema (what it can see, what's missing, up to three candidates with a likelihood band, a short quest) rather than free text. Then I don't trust it.

Guardrails in code, not just in the prompt. A small model's own confidence score isn't a real probability, so the app enforces its own rules after the model replies:

There is no "certain" level, only unlikely, possible and probable. Sentences like "this is definitely..." are removed.
Probable needs a photo, at least three visible features, and no equally likely rival.
With no photo, nothing can rise above possible. If the photo is unusable, every candidate drops to unlikely and no name is shown.
Anything below probable is displayed as "Possibly X", so the headline carries the doubt, not just the paragraph underneath.
Plants and fungi always carry a "never eat or handle based on this result" notice.

I wrote unit tests for these rules using deliberately overconfident fake model output, so they're checked without needing a model.

Smaller engineering details: browsers can't call Ollama directly (CORS), so Vite proxies /ollama to the local server. The service worker deliberately never caches Ollama requests, otherwise a stale cached reply could make the app claim the model was available when it wasn't.

What went wrong

I tested with a photo of a Great Tit (black head, white cheek patch, yellow belly). With gemma4:e2b and no location information, Gemma suggested a Yellow-rumped Warbler, a North American species. It was wrong, and it had no way of knowing where I was.

The app handled it the way I'd hoped: it said it couldn't determine the exact species from the image alone, listed what to check, and told me to keep my phone in my pocket. But the first version still put a confident-looking species name in the headline above that hedge. That was a design bug, and it's why anything below probable now shows as "Possibly...".

To see what actually helps, I ran the same photo three ways:

Setup	What Gemma said	Time
`e2b`, no region	Yellow-rumped Warbler (or similar)	9:00PM

[FILL: two or three sentences on what changed and what surprised you. Report it honestly, even if the answer was still wrong.]

Recently I went on a hike , pictured a birds or two , helped me to narrow down my choices

Limits

Gemma is a general vision-language model, not a species classifier. Expect sensible answers at the level of "a tit-like bird" and weaker ones between close look-alikes, juveniles, or poor photos. On a laptop with a small GPU, answers take noticeably longer than a cloud API. And this version runs on the computer, with the phone talking to it over the same network, so it's local rather than fully in-your-pocket.

Why Does Open Innovation Matter?

Privacy. Photos of a garden, a balcony, or a child's hand holding a leaf never leave the machine. With a closed API, every photo would be sent to someone else's server.
No signal needed. Trails rarely have coverage. Analysis needs no internet, only a local connection to the computer running the model.
Free to run, no account. There's no API key and no per-photo cost, which matters for something people should want to use every day.
Swappable and tunable. I changed the model size, the sampling temperature and a location hint just by editing environment variables, and I could point the same code at a fine-tuned species model later.
Auditable. The weights are released under Apache 2.0, so anyone can reproduce my demo and check what it does.

I also want to be honest about where closed would have been better. A larger closed model, or a dedicated species-identification service, would probably name species more accurately than a small local model on my hardware. For this app, I'd rather have a slightly less accurate guess that stays private, works offline, and openly shows its doubt.

My Agent Session

I used Claude (Anthropic) as a coding assistant while building this. Gemma is the model inside the app and does all the identification.

Prize Categories

Best Use of Gemma: Gemma 4 runs locally through Ollama and is the core of the app, and I tested more than one model size.

DEV Community: Prakul Chandra