Create account

DEV Community

Aakash Mehta

Posted on Oct 10

GrassQuest: a local Gemma scavenger hunt that sends you outside

#hf26challenge #ai #opensource #programming

Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

GrassQuest turns a short walk into a photo scavenger hunt. You type where you're going, the season and how long you have. Gemma writes five things to find: a leaf shape, bark, a flower, a puddle reflection, a small insect. You snap a photo of each, and Gemma checks it. At the end it writes a short field journal from only what you actually found.

The idea is simple: the screen should be the shortest part of the walk. You plan in about 20 seconds, then put the phone away.

It runs entirely on my laptop. No cloud, no account, and no photo ever leaves my machine.

Demo

There is no live URL on purpose. The model runs locally through Ollama, so a hosted copy would defeat the point.

Screenshots from the demo:

Code

aakashamy777 / Touch_Grass

GrassQuest

A local-first app for the Hacktoberfest "Touch Grass" challenge.

Prerequisites

Node.js installed
Ollama installed and running locally
The gemma3:4b model pulled in Ollama

Setup

Install dependencies:
```
npm install
```
Pull the required model if you haven't already:
```
ollama run gemma3:4b
```
Start the server:
```
npm start
```
Open your browser and go to http://localhost:3000

View on GitHub

Setup: install Ollama, run ollama pull gemma3:4b, then npm install and npm start, and open http://localhost:3000.

How I Built It

Model: Gemma 3 4B (vision) through Ollama. It writes the quest, judges each photo and writes the journal.
Server: Node.js and Express, with small endpoints for quests, photo checks and the journal.
Client: one plain HTML page. Photos are resized to 1024px in the browser before upload, and progress is saved in localStorage so a refresh doesn't lose the quest.
Structured output: every call asks Gemma for JSON, validates it, and retries once if the output is bad.
Built in stages: skeleton and health check first, then the quest generator, the photo check and the journal, testing each stage before starting the next.

Why Does Open Innovation Matter?

Privacy. Your photos and your area stay on your own machine. A closed API would receive every photo you take.
No signal needed. Once the model is downloaded, the whole app works without internet, which is what you want on a trail.
Cost. It is free to run, however many walks you take.
Swappable. Changing one model name lets me try another open vision model without rewriting the app.

What I Tested

I tested the photo check with real photos on my laptop. Five matching photos passed their tasks, and mismatched photos were rejected. A photo of roses was correctly rejected on a different flower task.

Task	Photo	Result
Leaf with five lobes	maple leaf	pass
Close-up of rough bark	tree bark	pass
Cluster of pink flowers	flowers	pass
Building reflection in a puddle	puddle	pass
Small insect on a leaf	stink bug	pass
Any task	unrelated photo	rejected

What It Does Not Do

Look-alikes can slip through. A chrysanthemum passed on a task that asked for a similar pink flower. A 4B model recognises "flower" better than an exact species. I added a "Gemma is not sure" note for low confidence, but it is not perfect.
Quests aren't always local. Gemma sometimes suggests things that don't grow near you, such as maple leaves in Bengaluru.
I have not done a full outdoor field test or a phone test yet. Testing so far was on my laptop, using saved photos.
The journal needed a fix. The first version invented dates and details that were not in the input, so I changed it to use only the photo notes.

AI Disclosure

I used Antigravity to help write the code, and a Claude assistant helped me plan the build and write this post. The app itself runs on Gemma locally.

Prize Categories

Best Use of Gemma: Gemma 3 4B does all three jobs (planning, vision and writing) locally.

Top comments (3)

Harun - solo dev • Oct 10

@aakash_mehta_48e67c642976 , this is a brilliant execution of the "touch grass" challenge! building a local-first, offline-capable ai app is exactly the kind of privacy-respecting innovation the space needs. no cloud api means zero data leakage and zero running costs, which is a massive win.

i especially love that you implemented structured json output with a validation and retry loop. that is a senior-level pattern that prevents the ui from breaking when a small model inevitably hallucinates the schema.

regarding your self-critique about quests not being locally accurate (like maple leaves in bengaluru): a lightweight fix without adding complex geolocation apis would be to simply add a "biome" or "region" dropdown in the initial prompt (e.g., "tropical urban", "temperate forest", "arid"). passing that single variable to gemma's quest generator would drastically improve the relevance of the suggestions while keeping the app 100% offline and private.

also, huge respect for catching the journal hallucination and fixing it by strictly grounding the output in the photo notes. that is the exact "fail closed" / strict-context philosophy that makes local models actually usable.

fantastic work. can't wait to see this tested on an actual phone in the wild! 🐯🌿