Trailside: An Offline Field Guide Powered by Gemma 4, Ollama and Open Weights

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Trailside is a field guide that lives on your own laptop. You point your phone at a leaf, a mushroom, a bird or a beetle, snap a photo, and an open-weight model tells you what it probably is, how sure it is, what it could be confused with, and whether you should keep your hands off it. Every sighting goes into a field journal you can export as markdown.

The part I care about most is the last line of every answer: a look-up challenge. Instead of ending with "here's more to read", Trailside ends with one thing to go and check in person, like "count the lobes on the next leaf of this tree" or "stand still for a minute and see if you can hear it again". The screen is meant to be the shortest part of the experience: snap, read for ten seconds, pocket the phone, go look.

It's for anyone who walks somewhere with a bit of nature in it and gets curious: hikers, birders-in-training, parents on a park walk, run clubs that stop to stretch next to something interesting.

Demo

A 2-minute screen recording of me using Trailside live on my phone, on mobile data, photographing five plants around my building (click the image to watch):

The recording shows version 1 of the app. The write-up below explains the problems I found and the fix I made afterwards.

Code

sans-byte / trailside-hactoberfest

🌲 Trailside Field Guide

Point your phone at a plant, tree, mushroom, bird, insect or track and get an ID, look-alikes, a safety note and a "look up" challenge to go and check in the real world. Every sighting goes into a field journal you can export.

Runs entirely on your own machine with the open-weight model gemma4 via Ollama No account, no cloud, no API bill, and your photos and location never leave your hardware.

Stack

• Model: gemma4 (open weights, vision) served locally by Ollama, using JSON-schema structured output
• API: Node + Express, built-in node:sqlite for the journal
• UI: React (Vite), mobile-first, camera capture

Run

ollama pull gemma4
npm run install:all
npm run build
npm start          # http://localhost:8000 (also reachable from your phone on the same Wi-Fi)

Swap models with TRAILSIDE_MODEL=qwen3.5:9b npm start (any Ollama vision model works).

Honest limitations

• ID is best-effort. Never eat anything based on…

View on GitHub

Run it with ollama pull gemma4, npm run install:all, npm run build, npm start.

How I Built It

The model is gemma4 (open weights, ~7.5B parameters, Q4_K_M quantisation, a 6.6 GB download) running through Ollama on a normal Linux laptop. Everything else is a small stack I'd use for any web app:

• React (Vite): a mobile-first UI. Tapping the button offers the camera or the gallery, and the photo is downscaled to ~1024px in the browser before upload, so a 12 MP phone photo doesn't slow local inference for no gain.
• Express: one endpoint takes the image, adds the optional place and note, and calls Ollama's chat API.
• node:sqlite: the journal is a single SQLite file using Node's built-in module. No database server, which matters for something that should work with no signal.

The thing that made a 7B local model usable is structured output. I pass a JSON schema to Ollama (common_name, category, confidence, why, look_alikes, safety_note, look_up_challenge), so every answer comes back in the same shape and the UI never has to parse prose. The system prompt is strict about honesty: give a calibrated confidence, say what features you used, list look-alikes, and never encourage eating anything. (It started short and grew after the field test below showed it needed to.)

Where it didn't work (and what I did about it)

gemma4 advertises audio support, so my first plan was a bird-call identifier. I tested it before building the UI around it:

• Three seconds of pure silence was described as "a lively discussion between several people".
• A plain sine tone was identified as an American Robin, high confidence.

Through Ollama the audio wasn't actually reaching the model; it was inventing an answer. A confidently wrong bird ID is worse than no feature, so I cut the sound path instead of shipping it, and kept the part that I could verify works. Photos pass the same sanity check: a synthetic test image came back as "Unknown, low confidence, this is a series of coloured bars", which is the behaviour I want from a field guide. A real bird-call feature should use a purpose-built open model like BirdNET, which I'd add next.

How accurate is it? A benchmark

Before taking it outside I wanted numbers, so I ran 10 open-licensed Wikipedia photos of known species through the real app (the same endpoint the phone uses), unedited. A hit means the name or scientific name matched the known species.

Photo	What Trailside said	Confidence	Hit?
Fly agaric	Amanita muscaria	high	✅
Monarch butterfly	Monarch Butterfly	high	✅
Indian peafowl	Indian Peafowl	high	✅
Hibiscus	Hibiscus	high	✅
Oak	Oak	high	✅
Sunflower	Sunflower	high	✅
Aloe vera	Aloe species	high	✅
Peepal (Ficus religiosa)	"Likely Ficus (fig tree)"	medium	❌ right genus, wrong species
Tulsi (holy basil)	"Likely Stachys or another Lamiaceae"	medium	❌ right family, wrong genus
American robin	Black Redstart	high	❌ confidently wrong

7 out of 10. What I take from it:

• Common, distinctive things are easy. The misses were a look-alike-heavy plant family and a bird.
• When it was unsure, it said so: both plant misses were "medium" and hedged to the genus or family. That is the behaviour a field guide needs.
• The robin was wrong and high confidence, which is the failure to design around. It's why every answer carries look-alikes and a safety note, and why I'd never trust it for anything edible.
• It's slow. On my laptop (12 CPU cores, no GPU, so Ollama runs at 100% CPU) each ID took 58 to 100 seconds. That's fine for a "look, then pocket the phone" workflow and bad for rapid-fire use. A GPU machine would change this a lot.

Field test (the humbling part)

Benchmarks on tidy Wikipedia photos are one thing. So I took my phone out around my building, mostly after dark, used the app live on mobile data (that's the video above), and photographed five real plants. To reach the laptop from outside my Wi-Fi I put a passcode-protected Cloudflare tunnel in front of the server. The model still runs only on my laptop, but those photos did travel through Cloudflare's network on the way, which is worth knowing. I judged the answers myself by eye, so treat my verdicts as an amateur's, not a botanist's.

What I photographed	v1 said	v1 confidence	My verdict
Bulb plant with long strap leaves	Dracaena	high	❌ probably a crinum/spider lily
White 5-petal flower, at night	Catharanthus roseus	high	⚠️ right family at best (looks more like crape jasmine)
Jasmine vine, at night	White clover	high	❌ not even close
Papaya tree	Fiddle-leaf fig	high	❌
Red ti plant	Cordyline	high	✅

1 out of 5, and every answer was "high confidence". That's much worse than the 7/10 daylight benchmark, though five photos is a tiny sample. Darkness, tropical ornamentals and a 4-bit 7B model are a hard combination. The bigger problem is that the confidence label was meaningless. A field guide that is wrong and certain is worse than no guide.

I also noticed several "look-up challenges" told me to open a weather app or search something on my phone, which is the opposite of the point of this project.

So I made a second pass. I rewrote the system prompt with strict confidence rules ("high" only when a diagnostic feature is clearly visible and no look-alike remains; otherwise name the genus or family, or say Unknown) and banned any challenge that involves a phone. Then I re-ran the same photos:

Photo	v1	v2
Bulb plant	Dracaena, high	Dracaena, medium (still wrong)
White flower	Catharanthus roseus, high	"White star-like flower", medium (vague)
Jasmine	White clover, high	Unknown, medium
Papaya	Fiddle-leaf fig, high	"Ficus species", medium (still wrong)
Red ti plant	Cordyline, high	Cordyline, high ✅
(Phlox, a stock photo from earlier)	Phlox, high	Phlox, high ✅

What improved: the clover claim is gone, the wrong answers dropped to medium or hedged, and the two correct answers stayed "high". What didn't: two IDs are still wrong, and the new challenges got repetitive ("feel the texture of the nearest leaf") because I constrained them hard. They're now physical and phone-free, but not yet inspired.

The lesson I'm taking away: with an open model I could see the failure, change the prompt in a minute, and re-test on my own machine with no quota and no cost. The fix was partial, and I'd rather show that than pretend it was perfect. A real next step would be fine-tuning on regional plants (which an open-weight model allows and a closed API doesn't) or pairing it with an iNaturalist-style classifier for plants.

Why Does Open Innovation Matter?

A field guide has requirements that a closed API quietly breaks:

• The AI needs no internet. The model, the server and the journal all run on one machine, so identification works with no signal at all if you use the app on the laptop itself or from a phone joined to the same local network or hotspot. (My field test used a tunnel for convenience, which does need internet. That was a choice, not a requirement.)
• Your data stays yours by default. A photo of what's in your garden plus GPS coordinates is a surprisingly personal record. Run locally, nothing is sent to any AI provider, and there's no account to create. The tunnel I used for the field test is the one exception, and it only carried traffic to my own machine.
• It costs nothing per identification. No API key, no metering, no "you've used your quota" halfway up a hill.
• I could find the model's limits and route around them. Because I was running the weights locally, I could poke at the audio path directly, see it was fabricating, and make an honest call. With a closed API I'd have had a black box and a feature that looked like it worked.
• The model is swappable. TRAILSIDE_MODEL=qwen3.5:9b npm start changes the brain without touching the app, and any Ollama vision model works. If a better open model for plants shows up tomorrow, it's a one-line change, and with open weights I can fine-tune it on regional species myself, which most hosted models don't let me do.

The trade-off is real: a 7B model is not a botanist, and the app says so. Confidence levels, look-alikes and a "never eat anything based on an app" note are in every result for that reason. Open innovation didn't make the model perfect; it made it something I can inspect, adapt and trust to the degree it deserves.

My Agent Session

I built this with Claude Code, from first prompt to the benchmark, the field test and the prompt fix described above.

Prize Categories

• Best Use of Gemma: Trailside runs gemma4 locally through Ollama, using its vision capability and structured JSON output as the core of the app.