Touch Grass Bingo - offline nature bingo checked by an AI that lives on your phone

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Touch Grass Bingo is a nature bingo card that you play outside. You open it, get a seasonal card (a red leaf, an acorn, moss, a feather, a mushroom…), and put your phone away. When you find something, you take one photo. An open-weight vision model running on your phone checks it and marks the square.

The center square is never free. It is "Touch grass": a photo of your hand on real grass.

Who it is for:

• Families who want a reason to make a walk last longer.
• Run clubs and hiking groups. Everyone types the same card code (like FERN-42) and gets the same card. First to bingo wins. There are no accounts and no server.
• Anyone doomscrolling who needs a nudge. If you photograph your monitor, the model detects a screen and tells you that you cannot touch grass through a monitor.

The screen is the shortest part of the game: one glance at the card, then the phone goes in the pocket until you find something.

Demo

Play it: https://prateekiiitg56.github.io/touch-grass-bingo/

Open it on your phone. Tap Download photo checker once on Wi-Fi (about 150 MB). After that, it works in airplane mode.

Play the same card as a friend: https://prateekiiitg56.github.io/touch-grass-bingo/?code=FERN-42

Code

Prateekiiitg56 / touch-grass-bingo

Offline nature bingo. Snap what you find outside; open-weight CLIP checks it on your phone. No signal, no uploads.

Touch Grass Bingo

Offline nature bingo for walks, hikes and run clubs. The app deals you a seasonal bingo card (a red leaf, an acorn, moss, a feather…). You go outside and take a photo only when you find a square. An open-weight AI model running on your phone checks the photo. No signal is needed and no photo leaves the device.

• Open-weight model: CLIP ViT-B/32 (ONNX, 8-bit quantized)
• Open-source runtime: Transformers.js 3.7.6 on ONNX Runtime Web (WASM)
• No backend: static files only. Host on GitHub Pages, Netlify or any static host.

How it works

1. Field kit. On Wi-Fi, tap Download field kit. The model (about 150 MB) goes into the browser cache. A service worker caches the app shell and runtime. From then on the app works in airplane mode.
2. Deal a card. A card code (for example FERN-42) seeds a deterministic shuffle of the season's item…

View on GitHub

How I Built It

• Model: CLIP ViT-B/32, open weights, ONNX export, 8-bit quantized (about 150 MB).
• Runtime: Transformers.js on ONNX Runtime Web (WASM). Inference runs in the browser tab.
• App: plain HTML, CSS and JavaScript modules. No framework, no build step, no backend. Hosted free on GitHub Pages.
• Design: styled like a topographic map. Grid references (A1, B2…) run along the edge of the card, so a group can shout "got B2". A finished line is drawn as an orange trail-blaze route.
• Offline: a service worker caches the app shell and runtime. Transformers.js caches the model weights in the browser. After a one-time download on Wi-Fi, the app works in airplane mode.

The core trick is zero-shot classification. Each bingo square is only a text prompt, for example "a photo of an acorn". When you take a photo, CLIP scores it against every item in the season's pool plus a set of distractors (a screen, an indoor room, a selfie, a document, a city street). On a laptop, one photo takes about 100 ms:

• If the square you tapped wins, you found it.
• If a different unfound square on your card wins clearly, the app marks that one as a bonus ("Not running water, but it is a mushroom").
• If a seasonal item that is not on your card wins, you get "Good find, but it is not on this card."
• If a distractor wins, the find does not count, and you get a hint.

My first version scored photos only against the squares on the card. In testing, a red maple leaf was marked as the "yellow leaf" square with 95% confidence, because it was the closest option. Scoring against the full season pool fixed this: the same photo now comes back as "red leaf, not on this card."

Group play uses a seeded shuffle: the card code, season and size feed a hash, which seeds a small PRNG that shuffles the season's item pool. Every phone with the same code deals the same card, with no network.

Why Does Open Innovation Matter?

• It works where the grass is. Trails, parks and forests often have no signal. A closed vision API needs a network round trip for every photo. Open weights let the model live on the phone, so the game works in airplane mode.
• Photos stay private. Families take photos of kids' hands, backyards and favorite spots. With an on-device model, no photo is uploaded to a server that the player does not control.
• It costs nothing to run. No API key, no per-image bill, no rate limit. I can host it on GitHub Pages for free, and a whole run club can play without anyone paying.
• It is easy to change. Because CLIP is zero-shot and open, a new square is one line of text. A teacher can make a "schoolyard" card, a birding club can make a "local birds" card, and anyone can swap in a different open model (for example SigLIP) by changing one constant.

Prize Categories

Best Use of Gemma
Using Gemma, Google's open-weight model, in building my project