People Won't Go Outside so I made this

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

GoSeek is a mobile scavenger hunt that gets you outside by giving you a riddle instead of an instruction.

The problem I started with: if an app says "go outside!", nobody goes. People don't like being told what to do, but they love solving puzzles. So GoSeek hands you this instead:

"Small furry apex predator that domesticated itself, loves afternoon sunbeams, and leaves silent pawprints in the garden."

You can't solve that from the couch. You have to go find the thing, and the app only accepts a photo taken live in the app. No gallery uploads.

The loop:

1. Tap Touch Grass and get a riddle for a random everyday object (a bench, a pigeon, a manhole cover).
2. Go outside and work out what it is.
3. Open the camera and take a photo.
4. A vision model running on your phone checks whether it matches.
5. Match confirmed: confetti, points, and the photo joins your Trophy Gallery.

Harder objects are worth more points, and points level you up from Wanderer to Explorer to Seeker to Trailblazer.

Who is it for? Anyone, especially city dwellers. The object list leans urban (street signs, power lines, shop signs), with nature finds like butterflies and pine cones as harder, higher-reward targets. The screen is the shortest part: read, walk, snap, done.

Demo

🌿 Live app: goseek.ai.studio (best on your phone)

Code

0shuvo0 / goseek

GoSeek is a mobile-first outdoor scavenger hunt game. Each round gives you a riddle clue, and you must find the matching object in the real world by taking a photo with your camera or uploading an image.

GoSeek

GoSeek is a mobile-first outdoor scavenger hunt game. Each round gives you a riddle clue, and you must find the matching object in the real world by taking a photo with your camera or uploading an image.

Try it: goseek.ai.studio

How it works

• The app picks a random object from src/data/objects.json and shows a clue/riddle.
• You open the mission screen, then use the live camera or a file upload to capture a photo.
• src/services/aiModel.ts validates the image against the target object using a local Hugging Face vision pipeline and a fallback visual heuristic.
• If the object matches, the app awards points, stores the discovery, and updates your run stats.
• Progress is saved in the browser using IndexedDB (src/services/db.ts), so your streak, score, and discovery history persist locally.

Main flow

• src/App.tsx manages the dashboard, active mission, score updates, and success/reset modals.
• src/components/ActiveMission.tsx handles the camera, verification loop, save…

View on GitHub

How I Built It

1. Design: Google Stitch → Figma

I described the app to Google Stitch, imported the generated design into Figma, simplified it, and edited the copy.

Stitch prompt

APP: GoSeek
1.
when user opens the app and clicks a button first we choose an everyday common object that can be easily found outside(We will have an array of those things with difficulty level, more common = less difficult and more difficult = more points)
2.
app uses ai to describe the object as a mystery/riddle eg. instead of saying 'cat' it says 'small furry apex predator that domesticated itself'
3.
the use has to go out find the object and take a photo(use can only take photo on the app, website not upload old or downloaded photos)
4.
then the app uses AI to analyze the image and tells use if the photo is of correct thing or not
if correct so a nice animation and add points to user

design a mvp for it

Google stitch output:

After simplifying on figma:

2. The architecture problem

My first plan: keep a list of objects and have an on-device model write riddles on the fly. That fails. A model good enough for clever riddles is too big for most phones, and this is an outdoor game that has to run on the device you actually carry.

So I split the job and put each task where it fits:

Job	Model	Runs on	When
Write creative riddles	Gemma 3 4B (open-weight)	My computer, via Ollama	Once, ahead of time
Judge the photo	SmolVLM-500M (open-weight)	The user's phone, in the browser	Every play

The heavy creative model runs once and produces a static objects.json. The tiny vision model handles the one thing that must happen live.

3. Generating riddles with Gemma

I pulled gemma3:4b with Ollama and wrote a small script that asks for five riddles per object. The key is the prompt:

You write riddles for an outdoor scavenger hunt game for all ages.
Write 5 different riddles, each describing "${obj.name}" WITHOUT ever saying the word "${obj.name}" (or its plural, or any word in it).
Rules:
- Each riddle is 1-2 sentences, maximum 30 words.
- Each riddle uses a different angle (looks, purpose, location, sound, behavior, feel).
- Family-friendly, no mention of people or danger.
- Do not start with "I am" every time.

Output ONLY a JSON array of 5 strings. No explanation, no markdown.

Asking for a different angle per riddle keeps replays fresh, and the "never say the word" rule stops riddles from giving the answer away.

Full generation script (ai.ts + index.ts)

// ai.ts
async function askGemma(userPrompt: string): Promise<string> {
  const response = await fetch('http://localhost:11434/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model: 'gemma3:4b',
      messages: [{ role: 'user', content: userPrompt }],
      stream: false,
    }),
  });
  if (!response.ok) throw new Error(`Local AI request failed (${response.status}). Make sure Ollama is running.`);

  const data: { message?: { content?: string } } = await response.json();
  const answer = data.message?.content?.trim();
  if (!answer) throw new Error('Local AI returned an empty recommendation. Please try again.');
  return answer;
}

export { askGemma };

// index.ts
import { askGemma } from "./ai";
import * as fs from "fs/promises";
import * as path from "path";

export type Difficulty = 'easy' | 'medium' | 'hard';

export interface GameObject {
  name: string;
  difficulty: Difficulty;
}

const objects: GameObject[] = [
  { name: "Cat", difficulty: "easy" },
  { name: "Tree", difficulty: "medium" },
  // ...
];

const OUTPUT_FILE = path.join(process.cwd(), "objects.json");

async function run() {
  const results: any[] = [];

  for (const obj of objects) {
    console.log(`Generating "${obj.name}"...`);

    const prompt = `...the prompt above...`;

    const raw = await askGemma(prompt);
    const cleaned = raw.replace(/```
{% endraw %}
json|
{% raw %}
```/gi, "").trim();
    const riddles = JSON.parse(cleaned);

    results.push({ name: obj.name, difficulty: obj.difficulty, riddles });
  }

  await fs.writeFile(OUTPUT_FILE, JSON.stringify(results, null, 2), "utf8");
  console.log(`✅ Done. Saved ${results.length} objects to ${OUTPUT_FILE}`);
}

run().catch(console.error);

The output is a plain objects.json:

[
  {
    "name": "road",
    "difficulty": "easy",
    "riddles": [
      "A long ribbon of black stone that carries wheels from where you are to where you're going, never moving itself",
      "Painted with dashed lines and arrows, it tells travelers where to go, yet it has no voice"
    ]
  }
]

4. Building the app in Google AI Studio

I opened Google AI Studio's app builder, attached the design screenshots and objects.json, and gave it a single detailed prompt: game loop, data models, tech stack (React, TypeScript, Tailwind, Transformers.js, IndexedDB), and camera rules. A few follow-up prompts later the app worked, and one click on Publish made it live.

AI Studio prompt

Role & Goal: Act as an expert frontend developer. Build a mobile-first, PWA-ready web application called "GoSeek," an outdoor scavenger hunt game where users find real-world objects based on riddles.

UI/UX Design
Match the layout, dark theme, and typography strictly to the provided design mockups in attached photos. The app must be highly responsive on mobile devices.

Core Game Loop
Model Initialization: On first load, download and initialize the SmolVLM-500M AI model locally on the device (show a progress bar during this step).
Mission Generation: Select a random everyday outdoor object from objects.json. Display one of its associated riddles to the user (e.g., "small furry apex predator that domesticated itself" instead of "cat").
Capture: The user clicks "Open camera" to access the device's live camera. Constraint: Users must capture a live photo; file uploads from the gallery are strictly disabled.
Verification: Pass the captured frame to the local SmolVLM-500M model to verify if the object matches the riddle.
Success State: If matched, trigger a celebratory animation (e.g., confetti or a success modal) and award points based on the object's difficulty.
Gallery & State: Downscale the verified image and save it locally along with the object name, timestamp, and points earned. Display these in a scrollable "Trophy Gallery & Log" history section.

Data Structures & Logic
export type Difficulty = 'easy' | 'medium' | 'hard';
export interface GameObject {
  name: string;
  difficulty: Difficulty;
  riddles: string[];
}
export const points: Record<Difficulty, number> = { easy: 10, medium: 20, hard: 30 }
export const level: Record<string, number> = {
  wanderer: 0,      // 0-49 points
  explorer: 50,     // 50-99 points
  seeker: 100,      // 100-149 points
  trailblazer: 150  // 150+ points
}

Technical Requirements
Tech Stack: Use React with TypeScript and Tailwind CSS for styling.
AI Integration: Use Hugging Face transformers.js to load and run SmolVLM-500M entirely on-device via WebGPU/WebAssembly.
Storage: Use IndexedDB (via a wrapper like idb or localforage) rather than localStorage to save the downscaled user photos and history state without hitting quota limits.
Camera API: Use the HTML5 navigator.mediaDevices.getUserMedia API facing the environment (rear camera). Fall back to typical file upload if it fails.
Handle edge cases and figure stuff out if missing in description.

The stack

Layer	Tool
Design	Google Stitch → Figma
Riddle generation	Gemma 3 4B via Ollama
On-device vision	SmolVLM-500M via Transformers.js (WebGPU/WASM)
UI	React + TypeScript + Tailwind CSS
Storage	IndexedDB (photos + history)
Camera	getUserMedia, rear camera, live capture
Build + hosting	Google AI Studio

How It Went Outside

• So I played with it outsider for 20-0 minutes and it was a blast
• One of the riddle I got was: No back cushions, no screen, just a place to rest, feed the pigeons, or wait for someone and very interesting I must way
• The SmolVLM-500M is small but so far it has been working decent]
• It runs very smoothly on my iPhone 14

Why Does Open Innovation Matter?

Open models didn't just make GoSeek cheaper. They made the architecture possible.

• Photos never leave your phone. GoSeek points a camera at your street and neighborhood. Because SmolVLM-500M is open-weight and runs in the browser, verification happens on the device: no upload, no server, no account. A closed vision API would receive every photo.
• It works where the game happens. After the one-time model download, checking a photo doesn't need a network round-trip.
• It costs nothing per play. Every verification is free, where a hosted vision API charges per image. Running Gemma locally made generating all the riddles free too.
• Right model, right place. Open weights let me run a bigger model once for a creative task and a tiny one live on the phone. That split is the core design decision of the app.
• Everything is swappable. A better small vision model next month? Swap it in. Riddles in another language or tone? Regenerate objects.json with a new prompt.

Honest trade-offs: The first load need internet connection for downloading the AI model, but after that it can run completely offline.

Prize Categories

• Best Use of Gemma: Gemma 3 4B, run locally through Ollama, writes every riddle in the game.

Thanks for reading, and now go touch some grass. 🌿

Struggling to learn Git?

Learn Git by solving problems on GitMission.com