Pocket Naturalist

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Pocket Naturalist is an offline field guide that helps you learn the plants and trees around you without staring at a screen.

You snap one photo of a leaf, bark, or flower, then put the phone back in your pocket. A local vision-language model identifies the likely species, and a short spoken note plays in your earbuds: the name, one interesting fact, and a "look for this next" prompt, like the shape of the bark or the berries nearby. The screen is only used for the single photo, and everything else is audio.

It's for casual hikers, parents on weekend walks, and anyone who has wondered "what is that tree?" but didn't want to stop and search. It works with no signal, which is where most good trails are.

Demo

Coming soon. I'll add a short video of a real trail walk after my first field test.

Code

Coming soon. The repo will be open-source and linked here.

How I Built It

• Model: a small open-weight vision-language model (such as a Gemma or Qwen-VL variant), quantized to run on a laptop or recent phone-class device.
• Inference: fully local through llama.cpp / Ollama. No API calls and no network needed after the initial model download.
• Text-to-speech: an open-source offline TTS engine (such as Piper) reads the result aloud.
• Agent logic: a lightweight loop that takes the photo, produces a species guess with a confidence level, adds a short fact, and generates a "look for this next" prompt so the walk keeps going.
• Safety rail: the app never says anything is safe to eat. If confidence is low, it says "I'm not sure, here are two possibilities" instead of guessing. Foraging is deliberately out of scope.

Why Does Open Innovation Matter?

• It works where the trail has no signal. A closed API can't identify a plant from a ridgeline with zero bars. A local open-weight model can.
• Your photos and location stay on your device. Nobody's walking habits get uploaded to a server they don't control.
• It costs nothing per scan. With no API fees, you can scan 200 leaves on one walk.
• It can be fine-tuned for a region. Because the weights are open, a local nature club could tune the model on regional flora and share it.
• Models are swappable. As better small open models appear, upgrading means changing one line instead of rewriting around a new vendor.

My Agent Session

Coming soon. I'll save my build session with DevRelay and link it here.

Prize Categories

Open-weight models, local inference.