This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
WILDCASE: The World Is the Case File 🔍🌳
What I Built
WILDCASE is an outdoor mystery-investigation game that turns a walk in the park into a detective case. You receive a classified dossier, put your phone in your pocket, and go outside. You hunt for real physical evidence (a metal gate, weathered bark, brick masonry, a circular fastener), take a 2-5 second scan to verify it, eliminate suspects, and finally name the culprit.
Most "get outside" apps pull you away from your phone and still end up glued to a screen. WILDCASE inverts that. The phone is an intermittent field instrument; the world is the game board.
How it gets people off the screen:
- Field Mode: an ultra-minimal dark HUD with a single breathing sonar indicator. It is built to be ignored, so the phone stays in your pocket.
- Away Ratio: using the Page Visibility API, WILDCASE measures the share of your session spent away from the screen and shows it in your final Field Report (for example, 87.4% Away).
- Real objects as clues: progress depends on noticing materials, textures and shapes in your actual surroundings.
Who it's for: students and commuters, families on weekend walks, casual mystery fans, and anyone who wants a reason to look up from their screen.
The Player Journey
┌──────────────┐ ┌───────────────┐ ┌────────────────┐ ┌───────────────────┐
│ CASE BRIEFING│──▶│ PHONE IN │──▶│ WALK OUTDOORS │──▶│ OBSERVE SURROUND- │
│ (dossier) │ │ POCKET │ │ (Field Mode) │ │ INGS │
└──────────────┘ └───────────────┘ └────────────────┘ └─────────┬─────────┘
│
▼
┌──────────────┐ ┌───────────────┐ ┌────────────────┐ ┌───────────────────┐
│ FIELD REPORT │◀──│ VERDICT │◀──│ ACCUSATION │◀──│ DISCOVER PHYSICAL │
│ (Away Ratio) │ │ │ │ │ │ EVIDENCE │
└──────────────┘ └───────────────┘ └───────▲────────┘ └─────────┬─────────┘
│ │
┌───────┴────────┐ ┌─────────▼─────────┐
│ NEW LEAD │◀──│ BRIEF SCAN (2-5s) │
│ UNLOCKED │ │ + AI INTERPRETATION│
└────────────────┘ └───────────────────┘
Demo
🔗 Live demo: https://wildcase-web.onrender.com
It is a PWA, so open it on your phone and add it to your home screen. It also works fully offline.
Code
Himanshu0250
/
WILDCASE
AI-powered outdoor mystery investigation game — the world is the case file.
WILDCASE
"The world is the case file."
An outdoor mystery investigation game engineered to get players off screens and into the physical world.
-blue){kind=icon}
Live Demo
[click here] https://wildcase-web.onrender.com
What is WILDCASE?
WILDCASE is an outdoor detective adventure where the player's physical environment is the game board and their mobile device is an intermittent field sensor instrument.
Instead of sitting in front of a screen, players receive a classified mystery dossier, put their phone away in their pocket, walk outdoors through parks or streets, observe real physical objects (metal gates, weathered bark, brick masonry, circular fasteners), verify them on-device, eliminate suspects, and deduce the true culprit.
CASE BRIEFING ➔ PUT PHONE IN POCKET ➔ WALK OUTDOORS ➔ OBSERVE SURROUNDINGS
➔ DISCOVER PHYSICAL EVIDENCE ➔ BRIEF SCAN (2-5s) ➔ AI INTERPRETATION
➔ UNLOCK NEW LEAD ➔ PHONE AWAY ➔ ACCUSATION ➔ VERDICT ➔ FIELD REPORT
The Architectural Invariant
The repo is a pnpm monorepo (apps/, packages/, docs/, fixtures/, scripts/), MIT licensed, with documentation covering the vision pipeline, evidence system, case design, privacy and offline behavior. Run it locally with pnpm install && pnpm dev. It needs no API keys to start.
How I Built It
WILDCASE is built on one architectural rule:
Device senses descriptors → Core engine matches predicates → AI interprets context.
| Layer | Responsibility | Tech |
|---|---|---|
| On-device vision | Extracts statistical descriptors (metal, rough, vertical) from a short camera scan. No photo ever leaves the device. |
HTML5 Canvas, Sobel gradients |
Deterministic core (@wildcase/core) |
Holds the immutable ground truth, unlocks clues progressively, and eliminates suspects by evaluating predicates. | TypeScript state machine |
| AI Game Master | Interprets verified evidence and writes atmospheric dispatches. It narrates; it never decides what is true. | Gemma 2 (open-weight) via Mastra workflows |
| API | Case delivery and orchestration. | Hono |
| Client | Installable, offline-first app with local persistence. | React 19, Dexie (IndexedDB), Service Worker |
| Audio | Procedural shutter clicks and stamp impacts, plus optional voiceover. | Web Audio, ElevenLabs / Web Speech |
System Architecture and Trust Boundaries
The key design decision is who is allowed to decide what. Pixels never leave the device, the engine alone decides truth, and the AI only writes the story.
PHYSICAL WORLD
│ camera frames (never uploaded)
▼
┌──────────────────────────── DEVICE (PWA / React 19) ────────────────────────────┐
│ │
│ ┌────────────────────────┐ ┌───────────────────────────┐ │
│ │ On-device Vision │ │ Page Visibility Tracker │ │
│ │ Canvas → color, │ │ screen time vs away time │ │
│ │ roughness, Sobel edges │ └─────────────┬─────────────┘ │
│ └───────────┬────────────┘ │ │
│ │ descriptors only │ │
│ ▼ ▼ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Investigation State Machine (@wildcase/core) │ │
│ │ predicate matching • clue unlocking • suspect alibis │ │
│ └───────────┬───────────────────────────────┬────────────────┘ │
│ ▼ ▼ │
│ ┌────────────────────────┐ ┌──────────────────────────┐ │
│ │ Dexie / IndexedDB │ │ Deterministic Fallback │ │
│ │ offline cache + sync │ │ Narrator (no network) │ │
│ └────────────────────────┘ └──────────────────────────┘ │
└──────────────────────────────────────┬──────────────────────────────────────────┘
│ verified facts + case text only (online)
▼
┌───────────────────────────┐
│ WILDCASE API (Hono) │
└─────────────┬─────────────┘
▼
┌───────────────────────────┐
│ Mastra Workflow Engine │
└─────────────┬─────────────┘
▼
┌───────────────────────────┐
│ Gemma 2 (open-weight) │
│ → dramatic dispatch text │
└───────────────────────────┘
Evidence Verification Flow
What happens in the 2-5 seconds when a player scans an object:
┌────────────────────┐
│ Player scans item │
└─────────┬──────────┘
▼
┌────────────────────┐ no ┌──────────────────────────┐
│ Frame quality OK? │────────▶│ "Hold steady / more light"│──┐
│ (blur, brightness) │ └──────────────────────────┘ │ retry
└─────────┬──────────┘ │
│ yes │
▼ │
┌────────────────────┐ │
│ Extract descriptors│◀──────────────────────────────────────┘
│ metal / rough / │
│ vertical ... │
└─────────┬──────────┘
▼
┌────────────────────┐ no ┌──────────────────────────┐
│ Predicate matches │────────▶│ Evidence rejected, │
│ a clue in the case?│ │ no state change │
└─────────┬──────────┘ └──────────────────────────┘
│ yes
▼
┌────────────────────┐
│ Core eliminates │
│ suspects, unlocks │
│ next lead │
└─────────┬──────────┘
▼
┌────────────────────┐ online ┌──────────────────────────┐
│ Narrate result │──────────▶│ Gemma 2 via Mastra │
│ │ └──────────────────────────┘
│ │ offline ┌──────────────────────────┐
│ │──────────▶│ Deterministic narrator │
└────────────────────┘ └──────────────────────────┘
Key Code
The snippets below are simplified illustrations of the pattern each layer follows, trimmed for readability. The full implementations live in the repo.
1. On-device descriptors: no photo leaves the phone
Each frame is reduced to a few numbers (color, roughness, edge direction) in memory, then discarded.
type Descriptor = "metal" | "rough" | "smooth" | "vertical" | "horizontal";
export function extractDescriptors(img: ImageData): Descriptor[] {
const { data, width: w, height: h } = img;
// 1. Grayscale
const gray = new Float32Array(w * h);
for (let i = 0; i < w * h; i++) {
gray[i] = 0.299 * data[i * 4] + 0.587 * data[i * 4 + 1] + 0.114 * data[i * 4 + 2];
}
// 2. Sobel gradients: edge strength (roughness) + dominant direction
let magSum = 0, gxSum = 0, gySum = 0;
for (let y = 1; y < h - 1; y++) {
for (let x = 1; x < w - 1; x++) {
const i = y * w + x;
const gx =
-gray[i - w - 1] + gray[i - w + 1] - 2 * gray[i - 1] + 2 * gray[i + 1] - gray[i + w - 1] + gray[i + w + 1];
const gy =
-gray[i - w - 1] - 2 * gray[i - w] - gray[i - w + 1] + gray[i + w - 1] + 2 * gray[i + w] + gray[i + w + 1];
magSum += Math.hypot(gx, gy);
gxSum += Math.abs(gx);
gySum += Math.abs(gy);
}
}
const roughness = magSum / ((w - 2) * (h - 2));
// 3. Map statistics to a small, shared vocabulary
const out: Descriptor[] = [roughness > 40 ? "rough" : "smooth"];
if (gxSum > gySum * 1.4) out.push("vertical"); // strong horizontal gradient = vertical edges
if (gySum > gxSum * 1.4) out.push("horizontal");
return out;
}
2. The deterministic engine: truth is code, not a prompt
Suspects are eliminated by evaluating predicates against immutable case data. The same input always gives the same result, which is what makes the game testable.
interface Suspect {
id: string;
/** Descriptors that must NOT be present at the scene if this suspect is guilty */
alibiContradicts: Descriptor[];
}
interface Clue {
id: string;
requires: Descriptor[]; // predicate: all must be observed
unlocks: string[]; // next clue ids
eliminates: string[]; // suspect ids ruled out by this evidence
}
export function applyEvidence(state: CaseState, observed: Descriptor[]): CaseState {
const matched = state.caseFile.clues.filter(
(c) => !state.found.has(c.id) && c.requires.every((d) => observed.includes(d)),
);
return matched.reduce<CaseState>(
(s, clue) => ({
...s,
found: new Set(s.found).add(clue.id),
suspects: s.suspects.filter((x) => !clue.eliminates.includes(x.id)),
available: [...s.available, ...clue.unlocks],
}),
state,
);
}
export const solved = (s: CaseState) => s.suspects.length === 1;
3. The AI narrates, with a safe fallback
The model receives only verified facts and is told to add atmosphere, not new evidence. If it is unavailable, the game keeps working.
export async function narrate(event: VerifiedEvent, ctx: CaseContext): Promise<string> {
try {
const res = await gameMaster.generate({
system:
"You are a noir detective narrator. Use ONLY the facts provided. " +
"Never invent clues, suspects or outcomes. Reply in 2-3 sentences.",
input: JSON.stringify({ facts: event.facts, tone: ctx.tone }),
signal: AbortSignal.timeout(4000),
});
return res.text;
} catch {
// Offline, rate-limited or timed out: deterministic template keeps the game playable
return fallbackNarrator(event, ctx);
}
}
4. Measuring the Away Ratio
The Page Visibility API turns "did you actually go outside?" into a number.
export function createAwayTracker() {
let awayMs = 0, screenMs = 0;
let last = performance.now();
let hidden = document.hidden;
const flush = () => {
const now = performance.now();
(hidden ? (awayMs += now - last) : (screenMs += now - last));
last = now;
};
document.addEventListener("visibilitychange", () => {
flush();
hidden = document.hidden;
});
return {
ratio: () => {
flush();
const total = awayMs + screenMs;
return total ? awayMs / total : 0; // e.g. 0.874 → "87.4% Away"
},
};
}
Why Does Open Innovation Matter?
An outdoor game has constraints a closed API handles poorly:
- Connectivity: parks, trails and old neighborhoods often have weak or no signal. Open-weight models and an offline-first architecture mean the experience doesn't depend on a live call to a vendor.
- Privacy: because the model only sees abstract descriptors and case text, never images or locations, we can make a real zero-surveillance promise. That matters for a camera-based app that kids and families might use.
- Cost and access: open weights let anyone self-host or run a small model without per-request fees, which matters for a free community game.
-
Community cases: the game's content is data. Because the engine, case format and docs are open (
CASE_DESIGN.md), anyone can author a mystery for their own town, school or campus. That is the natural Hacktoberfest contribution path.
A closed API could have written the narration. Open innovation is what lets this run offline, stay private, cost nothing to scale, and be extended by people who live where the game is played.
Prize Categories
WILDCASE enters the categories below. Each one is a load-bearing part of the architecture, not a bolt-on, and each is tied to a specific job in the system.
| Category | Prize tier | Its job in WILDCASE |
|---|---|---|
| Best Use of Gemma | Featured | The open-weight model behind the AI Game Master |
| Best Use of Render | Featured | Hosts the live PWA and the Hono API (render.yaml in the repo) |
| Best Use of Mastra | Partner | Workflow engine that orchestrates the narration pipeline |
| Best Use of ElevenLabs | Partner | Voice for the detective's field dispatches |
Use of Gemma
Gemma 2 is the AI Game Master: it turns verified evidence into atmospheric, noir-style dispatches. The design makes an open-weight model the right choice rather than just an acceptable one:
- It only sees abstract descriptors and case text, never photos or locations, so a self-hosted or locally served Gemma keeps the zero-surveillance promise intact.
- It narrates, it never judges. The deterministic core decides what is true; Gemma only writes the voice, which keeps the mystery fair and the model swappable.
- It can be replaced or run offline. Because the model is open-weight and sits behind a narrow interface, the game degrades to deterministic fallback narrators when there is no network.
Use of Render
The live demo runs on Render at https://wildcase-web.onrender.com, defined by a render.yaml blueprint in the repo, with the web app and the Hono API deployed as separate services. The deployment steps and environment reference are documented in docs/PRODUCTION_SETUP.md, so anyone can fork the repo and deploy their own copy.
Use of Mastra
The API hands every verified evidence event to a Mastra workflow, which assembles the grounded prompt, calls Gemma 2, and returns the narration. Using a workflow engine instead of an ad-hoc fetch gives the model call a clear, testable shape: verified facts in, constrained narration out, with a defined failure path to the offline narrator.
Best Use of ElevenLabs
Narration is delivered as voice dispatches, so a player can keep the phone in their pocket and listen to the investigation while walking, which is the core of the "touch grass" idea. When ElevenLabs is unavailable, the app falls back to the browser's Web Speech API, so the experience still works offline.
Top comments (0)