Color Loop: a walking game where code checks the color and open-source Gemma checks you went outside

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Color Loop is a walking game with one rule: each day has a color.

You open the page, tap Use my location, and get a 1.3 to 3.5 km walking loop that starts and ends at your door. Then you put the phone away and walk. When you spot today's color, say red, you take a photo with your normal camera. Five different spots, then home.

Back home, you upload the photos. Color Loop tells you which ones counted and why, turns the five counted photos into a palette of your walk, and writes a short field note about what you found.

The whole design goal was: the screen is the shortest part of the experience. About 30 seconds before the walk, about a minute after, and nothing in between. There's no live tracking, no map to stare at mid-walk, and no notifications. The app hands you a quest and gets out of the way. You can even download the route as a GPX and never open the page during the walk.

It's for anyone who needs a small excuse to go outside: a lunch-break walk, a run club that wants a shared daily quest (everyone gets the same color on the same date), or someone who wants to notice their street again. Hunting for one color does something surprising: you start seeing red post boxes, red doors and red leaves you've walked past a hundred times.

Demo

▶ Try it: https://colorloop.streamlit.app/

If the app shows a "wake up" screen, that's Streamlit's free tier napping. Click it and give it ~30 seconds.

How to test it without walking: tap Use my location (or type any address), then upload a few photos of today's color. Try one taken indoors: Gemma will reject it.

Code

PranavG1203 / Color_Loop

Color Loop

Each day has a color. Walk a 1.5 to 3 km loop from your door, photograph that color in five different spots, and close the loop. The phone plans the route and judges the photos. Between those two moments it stays in your pocket.

Open-source AI sits in the judge. Gemma, Google's open-weight vision model family, looks at every photo that passes the color test. It names what you found ("red postbox") and rejects shots taken indoors or of a screen, so the game can only be won outside. Plain, testable code measures the color and applies the distance rules. Gemma answers what pixels can't. It also writes the quest card and a walk story built only from what it actually saw.

How it works

Piece	File	What it does
Color of the day	colors.py	Date-seeded pick from six hue windows, so everyone gets the same color.

…

View on GitHub

How I Built It

The key design decision: code checks the color, Gemma checks reality

My first instinct was to let an AI judge everything. I decided against it, because a game needs a judge players can trust and argue with. So the work is split:

Question	Who answers	Why
Is there enough red in this photo?	Plain code (OpenCV, HSV)	Measurable, repeatable, explainable: "19% red, need 10%"
Was it taken on the loop, 150 m from your last spot, not a burst shot?	Plain code (EXIF + haversine)	Deterministic rules, easy to unit-test
What is the red thing? Was this taken outdoors? Is it a photo of a screen?	Gemma (open-weight vision model)	Pixels can't answer these. A model can.
Quest card and walk story	Gemma	Grounded only in the stats and what it actually saw

That last row of the judge is what makes it a Touch Grass game: you can't win from your couch. A red cushion passes the color test, then Gemma looks at it and says no:

postbox.jpg  COUNTED   red post box · 19% red · outdoors
sofa.jpg     REJECTED  red tufted ottoman looks indoors; this game is played outside
leaves.jpg   REJECTED  too little red: 8%, need 10%

Every rejection comes with a reason. People accept a "no" when they understand it.

The pipeline

1. Color of the day: picked from six hue windows using the date as a seed, so everyone in a run club gets the same color.
2. The loop: three waypoints on a circle that passes through your start point, routed on foot by OSRM over OpenStreetMap data. If the route comes back too long or short, the circle is rescaled and retried until it's within 20% of the target distance.
3. The color test: each photo is converted to HSV; a photo passes if at least 10% of its pixels fall in today's hue window with enough saturation and brightness. All thresholds live in one place so they're easy to tune.
4. The vision check: only photos that pass the color test go to Gemma, which gets one prompt and must answer in JSON:

   {"subject": "red post box", "outdoors": true, "screen": false}

1. The place rules: EXIF time and GPS give dedupe (shots within 20 s count once), spread (150 m between counted spots) and an on-route check (within 150 m of the loop).
2. The palette: the mean color of the matching pixels in each counted photo becomes a five-swatch PNG, your walk reduced to color.
3. The field note: Gemma gets a small JSON of stats plus the things it saw, with an explicit "do not invent places, objects, weather or people", and writes a four-sentence recap.

One model, two ways to run it

llm.py talks to the same open Gemma family over two transports:

• Local: Ollama with gemma3:4b. Fully offline. Your photos never leave your laptop.
• Hosted: Google AI Studio with gemma-4-26b-a4b-it, for the public demo on Streamlit Community Cloud, where I can't run Ollama.

The app picks whichever is available, and the top bar always shows which model is judging you.

Things that broke (and what I learned)

• Gemma 4 thought so hard it never answered. Gemma 4 reasons before answering. My story prompt came back empty with finishReason: MAX_TOKENS: it had spent 1,298 tokens thinking and zero answering, and took 28 seconds. Its reasoning also arrives as separate "thought": true parts that I was gluing into the answer. Filtering those out and setting thinkingLevel: "minimal" took the story from 28 s with no answer to 2.6 s, and the vision check to about a second per photo.
• The "inf m off the planned loop" bug. A real photo came back as infinitely far from the route. Phones write GPS as 0/0 when they don't have a fix, and newer Pillow reads that as nan instead of raising an error. One nan coordinate and the distance math turns to infinity. Now any coordinate that isn't a real place on Earth is discarded.
• Browsers strip GPS. Many phones remove location from photos chosen through a web file picker. The judge falls back to time + color + Gemma in that case, and says so on the result screen instead of pretending.
• A black map. My first map used a WebGL renderer that drew nothing on devices without GPU acceleration. I switched to Leaflet with grayscale OpenStreetMap tiles, which works everywhere.
• On a laptop CPU, local Gemma takes ~40 s per photo. Fine for a once-a-day game, and a fair price for photos that never leave your machine.

Stack

Gemma (open weights) · Ollama · Google AI Studio · OpenStreetMap · OSRM · OpenCV · Pillow · NumPy · Folium/Leaflet · Streamlit. The UI is a monochrome, grid-based "field instrument". The interface stays gray so that the only color on screen is the one you're hunting.

Field Test

Why Does Open Innovation Matter?

It keeps your location off a server you don't control. A walking game sees exactly where you live and where you walk every day. With an open-weight model running in Ollama, the whole judge (route, photos, GPS, the vision model) can run on your own laptop with the Wi-Fi off. A closed API can't offer that. Its whole model is that your photos go to them.

Open weights meant my project survived a model being retired. During the build, the hosted API stopped serving the Gemma 3 model I'd started with. On a closed platform, that's the end of the story. Here, the same Gemma 3 weights kept running locally in Ollama, untouched, while I moved the hosted demo over to Gemma 4. I can pin a model version forever, because the weights are on my disk.

It costs nothing per walk. A game you're meant to play every day shouldn't have a per-photo bill. Locally, the marginal cost is zero.

Open data made the routes possible. The loops come from OpenStreetMap, mapped by volunteers, and routed by OSRM. Both are open, so anyone can point the app at their own OSRM server and run the whole thing offline, or build their own city-specific version.

Small and open is enough when the design is right. Because code handles everything measurable, the model only has to answer narrow questions ("is this outdoors?"). That's why a 4B open model on a laptop CPU does the job. A giant closed model wouldn't make the game fairer; it would only make it costlier and less private.

My Agent Session

I built Color Loop with an AI coding agent, from my design doc to the deployed app, including the debugging above.

Prize Categories

• Best Use of Gemma: Gemma is part of the judge, not decoration. It's the only thing that can tell a red post box on a street from a red cushion on a sofa, and so the only thing that makes this game impossible to win without going outside. It also writes the quest card and a walk story grounded in what it saw, and the same Gemma family runs both fully offline (Ollama, gemma3:4b) and hosted (AI Studio, gemma-4-26b-a4b-it).