Touch Grass 🌳: An AI Scavenger Hunt That Gets You Outside

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

What if AI gave you a reason to put down your phone and actually go outside?

That's the idea behind Touch Grass, an AI-powered outdoor scavenger hunt that turns your real surroundings into a game.

Instead of giving every player the same fixed list of objectives, Touch Grass uses the player's location and nearby geographic information to generate five objects to find. A player exploring a park might receive different targets from someone exploring a busy residential or commercial area.

Here's how a game works:

1. Create a game or join your friends using a game code.
2. Allow location access so the game can understand your surroundings.
3. Receive five scavenger-hunt targets generated with the help of Gemma and local geographic context.
4. Head outside, explore, and find the objects.
5. Take a photo using the in-game camera.
6. Let the YOLO object-detection model verify the photo.

The first player to verify all five targets wins!

There's also multiplayer gameplay, collaborative target replacement, location-context refreshes as players explore, and optional step-count and distance statistics.

One important design decision was to restrict targets to classes supported by the actual YOLO detector. There's no point asking players to find an object the verification model cannot recognize.

The goal is simple: make the screen the shortest part of the experience.

Demo

🚀 Try Touch Grass here:

https://touch-grass-api-uwhz.onrender.com/

The application is deployed on Render. Open it on your phone, grant location and camera permissions, and try creating a game.

The first visit may take a little time because the free hosting service can spin down when idle. For the best experience, use the deployed HTTPS link so the browser can access the camera and location APIs.

Code

💻 GitHub repository: ma10-yt/touch-grass

The source code includes the FastAPI backend, frontend, AI target generator, geographic context service, YOLO verification pipeline, MongoDB integration, and automated tests.

How I Built It

My main goal was to make AI an actual part of the gameplay rather than simply adding a chatbot to an application.

The project uses:

• Python and FastAPI for the backend and game APIs.
• HTML, CSS, and JavaScript for the frontend.
• MongoDB Atlas for persistent game and multiplayer state.
• OpenStreetMap, Nominatim, and Overpass for locality and nearby-feature context.
• Gemma for AI-generated targets and hints.
• Ollama for local language-model inference during development.
• Cloudflare Workers AI for production Gemma inference.
• YOLOv8n with ONNX Runtime for server-side photo verification.
• Render to host the deployed application.
• GitHub Copilot in VS Code to help me write code, debug issues, and iterate on the implementation.

The journey from local Gemma to Cloudflare

One of the most interesting parts of this project was figuring out how to run the language model reliably after deployment.

First, I used Ollama with Gemma 3 1B locally.

I downloaded the model and ran it through Ollama on my own computer. This let me experiment with prompts and build the initial target-generation flow without depending on a hosted inference provider.

However, my production application runs on Render. The Ollama model running on my laptop wasn't automatically available to that remote server. I needed a way to run inference remotely.

Next, I tried OpenRouter.

It gave me access to a hosted Gemma model, so I could connect the deployed backend to a remote inference API. But I ran into free-model rate limits and provider-capacity errors. A game should not become unusable just because a model provider is temporarily unavailable.

Finally, I integrated Cloudflare Workers AI.

I switched production inference to Google's Gemma 4 26B A4B Instruct model:

@cf/google/gemma-4-26b-a4b-it

I tested the REST API, handled the model's response format, disabled thinking mode to avoid wasting the output-token budget on reasoning, and integrated the model into the existing backend.

I also added handling for quota exhaustion, invalid credentials, temporary rate limits, malformed output, and other provider failures. When generation is unavailable, the backend can select eligible targets through its location-ranked fallback rather than blindly relying on a fixed demo list.

The final architecture keeps the AI provider separate from the game logic. Ollama remains useful for local development, while the deployed application can use Cloudflare for remote inference.

Making location actually matter

Getting a model to return five objects was only the beginning. The targets needed to make sense for the player's surroundings.

I integrated OpenStreetMap context into the generation pipeline. The backend retrieves nearby geographic features, classifies the surrounding area, ranks eligible object classes, and includes that context in the model prompt.

During testing, I discovered that map requests could time out and push the generator into a generic-context fallback. Caching could also reuse context for nearby locations. I improved the request timeout, retry behaviour, cache precision, and diagnostic logging to make it easier to distinguish actual location-aware generation from generic fallback behaviour.

I also addressed repetitive target generation and a target-replacement bug. The replacement logic now excludes the class being replaced, so the game doesn't simply offer the same object again.

The backend test suite most recently passed 157 tests, with 2 skipped.

These problems taught me that getting a successful LLM response isn't the same as building a reliable AI feature. Geographic data quality, candidate selection, validation, retry behaviour, and application state all matter.

Why Does Open Innovation Matter?

For Touch Grass, open innovation mattered because I wanted control over how the AI fits into the application.

Using an open-weight model such as Gemma meant I could start locally with Ollama, experiment with prompts, and then move to a hosted inference provider when I needed to deploy the application. I didn't need to redesign the entire game around one provider.

That flexibility was valuable when OpenRouter's free-model limits became a problem. I could test another way of serving Gemma while keeping the same core target-generation and validation logic.

Open innovation also matters for the computer-vision side. The game doesn't blindly trust whatever the language model suggests. The backend constrains target classes, validates generated responses, and uses an actual object detector to check submitted photographs.

There is an important trade-off: production Gemma inference is hosted through Cloudflare, so the deployed application is not an entirely local or offline AI system. The local Ollama setup remains available for development, while production benefits from managed inference. The target-generation prompt and coarse geographic context are sent to Cloudflare, but camera photographs are processed by the application's backend for YOLO verification rather than being sent to the Gemma API.

I also learned that using an open-weight model doesn't automatically make an application reliable or private. The surrounding architecture, the data sent to each service, and the way failures are handled matter just as much.

For me, open innovation meant being able to experiment, change deployment strategies, inspect and improve the application logic, and keep the project adaptable rather than being locked into one AI provider.

And most importantly, it helped me build something where AI encourages people to interact with the world instead of spending more time staring at a screen.

My Agent Session

I used GitHub Copilot in VS Code throughout development to help write code, investigate bugs, and refine the implementation.

I haven't included a DevRelay session link in this submission, but the project source and commit history are available in the GitHub repository.

Prize Categories

I'm entering the following categories based on the technologies used in Touch Grass:

• Best Use of Gemma ($200): Gemma generates contextual scavenger-hunt targets and hints. I experimented with local Gemma inference through Ollama before deploying a hosted Gemma model through Cloudflare Workers AI.
• Best Use of Render ($200): Render hosts the deployed FastAPI application and frontend, making the game accessible through a public live demo.
• Best Use of MongoDB Atlas ($100): MongoDB Atlas provides persistent storage for the game's state and multiplayer sessions behind an application built around an open-weight model.
• Best Use of GitHub Copilot ($100): I used GitHub Copilot in VS Code extensively while writing and debugging the project and iterating on implementation and tests.

Thanks to the DEV team and the Hacktoberfest partners for organizing this challenge. Building Touch Grass pushed me to work through real deployment and AI-integration problems, not just get a prototype running locally.

Now it's time to close the editor and touch some grass. 🌳🎮

devchallenge #hf26challenge