This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
Most apps want you to keep looking at them. I wanted to make one that sends you away. So I turned touching grass into a game: quests, XP, a plant that grows, and an AI that checks your photos without anything leaving your phone.
Touch Grass Quest gives you five outdoor quests every day ("find a pinecone", "spot a puddle") plus one shared quest that everyone gets. You open the in-app camera, take a photo, and an on-device model decides whether you really found it. Finds earn XP that grows a plant from a seedling to a blossoming tree. Streaks multiply your XP, and every find becomes a stamp in your Grass Dex.
The screen is meant to be the shortest part of the experience:
- Live camera only. There's no gallery upload, so you actually have to go somewhere.
- A cooldown between quests, so you can't farm XP from the couch.
- An away bonus: XP for the time the app spent in your pocket.
- Collecting, not gambling. Rewards are stamps and a growing plant, with no random pulls, on purpose.
It's for anyone who wants a small, playful reason to leave the house.
Demo
A quest written on-device by Qwen2.5 0.5B, with the XP it's worth.
Taking it outside
A pinecone passed at 90% confidence, a pile of pebbles passed as "a stone" at 93%, and a photo of my laptop screen was rejected. The away bonus showed up after I locked my phone for a few minutes. The app also reopened with airplane mode on after the first load, and that first load took only a few seconds on my Wi-Fi, by my rough count.
Pebbles pass as "a stone" at 93%, for +24 XP (the base reward with my x1.2 streak multiplier). The next quest is already dealt, and the camera is locked for a 3-minute cooldown.
The quest was "find a bush". I photographed leaves up close, and the model said "leaf" at 93%, so it didn't count.
Every find becomes a stamp in the Grass Dex. Unfound ones stay locked.
The check isn't perfect, though. I tried a real tree and it failed twice: once from far away, and once up close. In the failure screenshot above, a close-up of foliage was read as "leaf" instead of "bush". The model seems to want the subject at a medium distance, framed clearly. That was the most useful thing I learned from testing, and it's the first thing I'd fix.
Code
albertbenedict
/
touch-grass
Touch Grass Quest
Touch Grass Quest
A scavenger-hunt game that sends you outside. Every day you get outdoor quests ("find a pinecone", "spot a puddle"), take a photo with the in-app camera, and an AI running on your phone checks whether you found it. Finds earn XP that grows a plant, and every find becomes a stamp in your Grass Dex.
Built for the Hacktoberfest Open-Source AI Challenge, Week 1: Touch Grass.
Live app: https://albertbenedict.github.io/touch-grass/
How it works
- Daily quests: five a day (2 easy, 2 medium, 1 hard) plus one shared quest that everyone gets on the same day.
-
Live camera only: there's no gallery upload, so you have to go somewhere. (Append
?devto the URL to enable file upload for desktop testing.) - Photo check: CLIP compares your photo against the quest target and a set of "none of the above" labels. The target must be the top guess with…
How I Built It
The whole app is vanilla JavaScript running in the browser with transformers.js. There's no backend, no account, and no API key.
-
Photo check: CLIP (
Xenova/clip-vit-base-patch32) does zero-shot classification. It compares your photo against the quest target plus a set of "none of the above" labels, and the target has to be the top guess with enough confidence. - Quest text: a small open LLM writes each quest sentence. The app tries Gemma 3 270M first, then falls back to Qwen2.5 0.5B. On my setup, Qwen2.5 0.5B is the one that actually loaded.
- Offline: a service worker caches the app, and transformers.js caches the model files after the first download.
-
Progress: XP, streaks, the plant, and the Grass Dex all live in
localStorageon your device.
Two things went wrong
- CLIP accepted a photo of a tiger as "a leaf." In my test, the app called the tiger "a leaf" at 46% confidence and gave me XP. CLIP always picks the closest label from a short list, so any photo gets forced onto something. I added generic "none of the above" labels (an animal, a person, a screenshot, and more) and a confidence threshold.
- The small quest writer wrote quests about "ancient Egyptian trade routes." Now its output is validated, and if it drifts, the app uses a plain template instead. Small models need guardrails.
Limitations
- It runs on the honor system. A determined person could still photograph a screen, and real anti-cheat would need a server, which I chose not to add for now.
- The photo check is picky about framing. A tree photographed from too far or too close failed in my tests.
- A fixed shared quest can be impossible for some people. Not everyone lives near a lake.
- The first load downloads large model files.
Why Does Open Innovation Matter?
- Photos never leave your phone. The only network traffic is downloading the app and the model files. A closed vision API would mean uploading every picture you take outdoors to someone else's server.
- It works with no signal. The place you need this app is a trail, and a cloud API can't follow you there. After the first load, mine doesn't need the internet.
- It costs nothing to run. It's a static site, so there's no per-photo bill and no API key to leak.
- Models are swappable. When Gemma didn't load for me, falling back to Qwen was just the next line in a list, and I got a working app instead of a broken one.
The honest tradeoff: small open models are less capable than big closed ones, which is exactly why I needed the guardrails above. For a game where the whole point is that nothing leaves your phone, I'd make that trade again.
Try it on your phone: https://albertbenedict.github.io/touch-grass/. Allow the camera, go find a leaf, and tell me in the comments what your first stamp was.
Top comments (3)
This app saved my marriage. 10/10 give it a shot
Great work buddy 👍
thanks mate