I built an outdoor game that uses AI to get you off your phone

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What if an app's most important feature was knowing when to get out of your way?

Most apps compete for your attention. They want you to open them more often, spend more time inside them, and come back tomorrow.

For this challenge, I wanted to build something with the opposite goal.

I built GrassQuest, an outdoor exploration game that uses AI to turn your actual surroundings into quests, then rewards you for putting your phone away and exploring the real world.

The interesting part isn't just that it encourages people to go outside. It's how it decides what to do outside, how difficult that activity should be, and how to verify what you discovered without uploading your photographs to a cloud server.

What I Built

GrassQuest is a location-aware outdoor quest game built around one simple loop: choose a quest, leave your screen behind, explore, and return with proof of your adventure.

Here's how it works.

Your surroundings determine your quests

GrassQuest checks local weather, sunrise and sunset times, and nearby outdoor features using Open-Meteo and OpenStreetMap data.

It uses this context to filter its quest library. A quest that requires a pond should only appear when a suitable water feature exists nearby. A sunset quest makes more sense around golden hour. When severe weather makes outdoor activities unsuitable, the game can offer gentler alternatives and protect your streak.

The application has 248 hand-written quest templates, expanded into 1,324 concrete variations. The aim is to make exploration feel relevant to your surroundings rather than repeatedly presenting the same generic checklist.

TabPFN decides how challenging your next adventure should be

Instead of showing three random quests, GrassQuest uses TabPFN to estimate completion probability and walking duration.

It presents three choices:

• Easy Win: The quest with the highest predicted chance of completion.
• Sweet Spot: The quest closest to a 65% predicted completion probability.
• Stretch: A more challenging quest that meets a minimum predicted probability threshold.

These predictions use structured information such as weather, time of day, quest effort, distance to nearby features, and available player history. The choices are selected from different categories.

The probabilities are estimates, not guarantees. When player history is limited, the system can fall back to a heuristic rather than pretending to know more than it does.

The Ranger sends you on your way

After accepting a quest, you receive an audio briefing from The Ranger, GrassQuest's narrator.

ElevenLabs generates the voice lines, which cover different weather conditions, times of day, and quest outcomes. I pre-generated 16 audio clips instead of repeatedly generating identical lines during gameplay.

Once the briefing is over, the expedition timer and compass bearing help you get moving.

Then you put your phone in your pocket.

Gemma verifies your adventure locally

When you return, you submit a photograph as proof of your quest.

GrassQuest uses Gemma 4 Vision through Ollama to evaluate the photograph on your own computer. It can return a successful verdict, a close attempt that earns partial rewards, or an unsuccessful attempt with feedback. It can also identify plants, trees, birds, insects, animals, fungi, and interesting sky features for your Field Guide.

The photograph never needs to leave your computer. The image is processed locally, and GrassQuest's backend receives the resulting verdict data rather than the original photograph.

This does require a compatible local Ollama and Gemma setup, but it gives the player control over where their images are processed.

There is a reason to come back tomorrow

Completed quests earn XP and Seeds. XP advances your level, while Seeds can be spent on Basecamp upgrades and streak freezes.

Your Basecamp evolves as you progress, and the Field Guide becomes a collection of things you've discovered outdoors. Badges and leaderboards add milestones and friendly competition.

The point isn't to create another digital chore. It's to give people a reason to keep exploring.

Demo

Live application:

https://grass-quest-woad.vercel.app

Live demo video:

Some Images

The deployed frontend is the entry point to GrassQuest. Local vision judging requires Ollama and a compatible Gemma model on the player's computer.

Code

GitHub repository:

NotNirav / GrassQuest

🌿 GrassQuest

Tiny real-world quests that get you off the screen. Get a quest, go outside, come back with a photo, and Gemma, an open-weight model running on your own computer through Ollama, judges whether you did it. Your photo never leaves your machine.

Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 1: "Touch Grass" (Oct 2026).

How it works

 Browser (React)                                   Your computer
 ┌───────────────────────────────┐   photo +      ┌──────────────────────────┐
 │ quest card · "go outside"     │ ── prompt ───▶ │ Ollama · Gemma 4 (vision)│
 │ proof upload · verdict · XP   │ ◀── verdict ── │  judge + narrator lines  │
 └──────────────┬────────────────┘   (JSON)       └──────────────────────────┘
                │ verdict only (no photo)
                ▼
 ┌───────────────────────────────┐        ┌──────────────────────────────────┐
 │ FastAPI on Render             │ ─────▶ │ MongoDB Atlas                    │
 │  quest picker + dedupe        │        │ users · attempts · quests        │
 │  TabPFN: P(you finish quest)  │        │ + bge-m3 embeddings (Vector      │
 │

…

View on GitHub

The repository contains the frontend, FastAPI backend, quest library, model integrations, game mechanics, and setup instructions.

The main technologies are React, Vite, Python, FastAPI, MapLibre GL, MongoDB Atlas, TabPFN, Gemma 4 through Ollama, and ElevenLabs.

How I Built It

I started with a smaller MVP containing 41 hand-written quests and basic XP mechanics. But the first version had a problem: the individual integrations existed without forming a convincing gameplay experience.

The quests didn't adequately account for the player's surroundings. XP didn't lead to much. TabPFN barely influenced the player's choices, and Gemma only returned basic pass-or-fail results.

I decided to rebuild the experience around a more coherent idea: the environment should determine what's possible, AI should help choose what's achievable, and local vision should help verify what happened.

1. Environmental context and quest selection

The FastAPI backend aggregates weather and nearby-place data from Open-Meteo and OpenStreetMap Overpass. It filters quests according to environmental requirements, weather, time of day, and distance.

I also had to deal with Overpass timeouts. I split large queries into smaller groups, added caching, and introduced background cache warming to make the context engine more resilient.

2. TabPFN-powered recommendations

TabPFN's classifier estimates the probability of completing a quest, while a regressor estimates its duration.

The backend evaluates eligible candidates and uses those predictions to produce the Easy Win, Sweet Spot, and Stretch choices. I added request limits, caching, and a heuristic fallback to avoid making the entire game dependent on model availability.

3. Local Gemma inference

My first attempt was to run the vision model directly in the browser using WebGPU and Transformers.js. Downloading a multi-gigabyte model into mobile browser tabs proved too brittle across devices.

I pivoted to Ollama on the player's computer.

The browser sends the resized photograph directly to the local Ollama service. Gemma returns structured JSON containing the verdict, confidence, feedback, and identification information. The backend receives the verdict data needed for game progression, not the photograph itself.

This added a setup requirement, but it established a much clearer privacy boundary.

4. MongoDB Atlas and semantic quest deduplication

MongoDB Atlas stores player data, quest attempts, progression, and the expanded quest library.

I also integrated Atlas Vector Search using 1,024-dimensional embeddings generated locally with bge-m3 through Ollama. The system compares new candidates against recent attempts and filters semantically similar quests, reducing repetitive recommendations.

This makes the quest library more than a collection of randomly shuffled tasks.

5. ElevenLabs and the audio-first experience

I used ElevenLabs Flash v2.5 to generate The Ranger's voice lines. Rather than spending API credits on repeated narration, I pre-rendered 16 situational clips and stored them as static audio assets.

The result is an audio layer that responds to the expedition's context without requiring a fresh voice-generation request for every interaction.

6. The game and its visual identity

The frontend uses React and Vite, with MapLibre GL for maps and a custom CSS design inspired by outdoor field guides and topographic maps.

I built the Basecamp as an evolving SVG scene, with upgrades that reflect player progression and a visual environment that changes according to the local solar phase.

The backend handles the game economy, XP, Seeds, streaks, streak freezes, badges, and leaderboard data.

The overall architecture combines local AI inference with a hosted application backend. This means the public application is accessible through a browser, while the vision model remains on the player's computer.

Why Does Open Innovation Matter?

The most important reason is privacy.

An outdoor photograph can reveal far more than the object a player is trying to identify. Sending every image to a remote AI service would introduce a data transfer that isn't necessary for the core experience.

Running Gemma locally lets GrassQuest evaluate photographs without requiring them to be uploaded to a cloud vision API. The player retains control of the original image, while the backend receives only the structured result needed to award rewards.

Open-weight models also give developers more control over their AI architecture. I could choose where inference happens, integrate the model into my own workflow, and design the application around a local privacy boundary rather than depending entirely on a hosted inference endpoint.

TabPFN demonstrates another kind of flexibility: structured predictions can directly influence product decisions rather than existing as a disconnected AI feature.

MongoDB Atlas provides persistence and semantic search, while ElevenLabs supplies the narration that makes the expedition experience more engaging. These are different technologies with different roles; I haven't tried to make every component solve the same problem.

GrassQuest does still rely on hosted services for parts of its architecture, and local Gemma inference requires compatible hardware and setup. Open innovation doesn't eliminate every trade-off.

What it makes possible is a design in which the most sensitive part of the experience — evaluating a personal photograph — can happen on the user's own machine.

For a project about spending less time on screens, that distinction matters.

I don't want AI to become another reason to stare at a screen. I want it to help people notice what's beyond it.

Prize Categories

Best Use of Gemma

Gemma 4 Vision runs locally through Ollama to evaluate outdoor quest photographs, return structured verdicts, and identify natural discoveries for the Field Guide. Its local execution also enables the application's photograph-privacy boundary.

Best Use of TabPFN

TabPFN predicts quest completion probability and estimated duration. These predictions directly influence the three quest choices, making the model part of the core gameplay rather than a cosmetic integration.

Best Use of MongoDB Atlas

MongoDB Atlas stores the quest library, player data, attempts, and progression. Atlas Vector Search also helps filter semantically repetitive quests using locally generated bge-m3 embeddings.

Best Use of ElevenLabs

ElevenLabs Flash v2.5 generates The Ranger's narration, including context-specific departure briefings and reactions to quest outcomes. Pre-generated audio clips make these interactions available without repeatedly generating the same voice lines at runtime.

---

Built Solo

Built for the Hacktoberfest Open-Source AI Challenge, Week 1: Touch Grass.

If you try GrassQuest, I'd love to hear what quest it gives you and what you discover along the way.