An Offline AI Scavenger Hunt to Touch Grass

tejas raj — Sun, 11 Oct 2026 14:22:38 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Outdoor Quest is an offline-first Progressive Web App (PWA) designed to get developers out of their desk chairs and into the sunlight. It is a real-world, AI-powered scavenger hunt that runs entirely in your browser.

Players are given a list of outdoor targets to find (like a "park bench," "bird," "bicycle," or "stop sign"). To prove they found it, they must point their phone's live camera at the object. The app uses an on-device object detection model to instantly draw a bounding box and verify the item, awarding points completely offline.

To make it truly "Touch Grass" ready, I implemented a Live-Camera Anti-Cheat system (preventing users from uploading downloaded photos) and designed a Rugged Tactile UI featuring high-contrast, oversized elements that remain highly readable under direct, harsh sunlight.

Demo

🌍 Live Application: Play Outdoor Quest Here
(Note: Please open this on a mobile device and install it to your home screen for the full offline PWA experience!)

🎥 Video Walkthrough: Watch the Demo on ScreenPal

Code

Tejas-Raj01 / hacktober-fest-week-2-

🌲 Outdoor Quest — Rust & TypeScript Terminal Game

A real-time multiplayer outdoor quest game built with Rust (Backend WebSocket Server) and TypeScript (Interactive Terminal Client).

🎯 Game Features & Rules

Simple Room Code Sharing:
- Host starts a game and gets a Room Code (e.g. QUEST-4921).
- Friends join by simply entering the Room Code from their terminal. No need to type or know any server URLs!
5 Fixed Outdoor Tasks:
- 🌲 Task 1: Leaf & Bark Challenge — Step outside! Touch a natural green leaf or tree bark and submit photo proof.
- 🪨 Task 2: Unique Stone Quest — Look down! Touch a smooth or unique outdoor rock/stone and submit photo proof.
- ☁️ Task 3: Sky & Cloud Spotting — Look up! Snap a photo of a cloud, bird, or flying plane in the open sky.
- 🚶 Task 4: 30-Step Color Hunt — Walk 30 steps outdoors…

View on GitHub

How I Built It

This project relies on a hybrid architecture bridging edge AI and a robust backend:

Edge AI Vision (Transformers.js): The core offline vision experience uses the Xenova/detr-resnet-50 open-weight model directly in the browser to map live video frames from getUserMedia against the COCO dataset entirely on-device.
Edge AI Logic (Gemma via WebGPU): To handle offline game logic and interactions, I integrated Gemma 3n 2B via WebGPU, providing a fully localized LLM experience without network calls.
AI Pair Programming: I utilized GitHub Copilot extensively throughout the project to accelerate writing the Rust backend boilerplate and constructing the strict React state machine for the UI.
Unified Deployment (Render): Both the Vite PWA frontend and the Rust backend are hosted on Render. I chose Render because it seamlessly handles the static site hosting for the React frontend while simultaneously providing native, low-latency WebSocket support required for the Rust multiplayer state machine, keeping the entire infrastructure under one roof.
Database (MongoDB Atlas): Game state and player scores are securely persisted using MongoDB Atlas, which connects smoothly to the Render deployment.

Why Does Open Innovation Matter?

For an app meant to be played in parks and on hiking trails, relying on a closed API (like OpenAI Vision) is a critical point of failure. Deep in the woods, cellular service is notoriously unreliable.

Open-source AI and open-weight models allowed me to compile a highly accurate ResNet-50 object detection model into WebAssembly and run Gemma 3n 2B via WebGPU. Because of this open innovation, the AI runs entirely on the player's local hardware at zero latency and zero cost per inference. It completely eliminates the need for an internet connection to validate quests, making true outdoor exploration possible.

Prize Categories

Best Use of Open-Source / Open-Weight Models: Successfully running both detr-resnet-50 for real-time object detection and Gemma 3n 2B for logic entirely client-side via WebGPU and WebAssembly.
Best Use of GitHub Copilot: Leveraged Copilot to rapidly architect a fault-tolerant Rust WebSocket backend and state-gated React components.
Best Cloud Deployment (Render): Utilized Render's unified ecosystem to deploy a seamless full-stack application, pairing a static edge PWA with a high-performance Rust WebSocket server.
Best Use of MongoDB Atlas: The backend utilizes MongoDB Atlas to securely manage multiplayer game rooms and player scores, configured dynamically to accept connections from the Render environment.

DEV Community: tejas raj