This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
Trail Bingo is a bingo game you play on a hike.
Before you leave home, you build a Card for your trail. The app asks iNaturalist what people have actually seen within 10 km of that spot at this time of year, and deals out a 3×3, 4×4 or 5×5 Card from it: a bur oak, Canada goldenrod, a splitgill mushroom, "a mammal", "a butterfly or moth". The middle Square is a Wildcard that anything living can fill.
On the trail, when you spot something on your Card, you tap one button and take a photo. An open model on your phone checks it against your Card, with no signal and nothing sent anywhere:
- Sure: the Square is marked Verified.
- Not sure: it shows its top three guesses, and the one you pick is marked Confirmed.
- Either way: you get a short fact card (a photo, two or three sentences from Wikipedia) and dismiss it with one tap.
Fill a row, column or diagonal and you get a Bingo; fill everything and you get a Blackout. Then the phone goes back in your pocket.
I thought it could be an tool to educate and help engage with nature. It's built for casual hikers and families, so the copy is written for kids to follow, and the animal Squares are broad on purpose: "a bird" counts whether or not you can tell a junco from a sparrow.
Demo
Play it: sparkly-selkie-a49b7b.netlify.app
It's made for Chrome on Android; desktop Chrome with a webcam should work too. The first launch downloads the photo check's model once (about 300 MB), so use Wi-Fi. After that, building a Card needs signal, and playing doesn't. You can install it to your home screen and play a whole game in airplane mode.
System requirements
- Browser: Chrome (or another Chromium browser) on Android, which is what it's tested on. iPhone and Safari aren't supported yet.
- Storage: about 450 MB free, for the model and a Card's photos and facts.
- Camera: every Sighting is a photo.
-
Best with WebGPU and 4 GB or more of memory. On my phone, a photo check takes about 1.3 seconds.
- Phones with 2–4 GB run a lighter mode that uses about half the memory, but checks more slowly.
- Without WebGPU, each check takes a few seconds, and building a Card can take several minutes.
- Phones with 1 GB or less can't run it.
You don't have to work any of this out yourself. The setup screen checks the phone before anything downloads, so a phone that can't run Trail Bingo is told why, instead of spending 300 MB to find out.
Work in progress
Some more field testing still needs to be done to tune the model for more accurate identification. See the Tune the confidence gap from real Sightings section for more info.
Code
The model export is published on Hugging Face: jsteinshouer/trail-bingo-bioclip-onnx.
How I Built It
The model: BioCLIP, in the browser
The photo check is BioCLIP (Imageomics Institute, MIT licence), a CLIP-style model trained on the tree of life to match photos of organisms to their taxon names. That's exactly the question the game asks: which of the Squares on this Card is this?
No browser-ready build of both BioCLIP encoders existed, so I exported them myself:
- Export: the image encoder and the text encoder, both to ONNX.
- Weights: stored as fp16 and computed in fp32. That halves the download with no accuracy loss. (Full fp16 math crashes ONNX Runtime's WebAssembly backend, and plain int8 quantization measurably hurt this ViT.)
- Parity: the export script fails if either encoder drifts below 0.99 cosine similarity from PyTorch. Both came out at 1.000.
In the app, ONNX Runtime Web runs the model in a Web Worker, on WebGPU where the phone has it and WebAssembly where it doesn't. Hugging Face's tokenizers.js tokenizes labels exactly as open_clip does.
The first thing I did was a one-day spike to find out whether this was even possible on my phone (Android 10, Chrome, PowerVR GPU). It was: WebGPU, about 6 seconds to load from storage, about 1.3 seconds per photo check.
A static app that works in airplane mode
There's no backend:
- Hosting: the app is a static site on Netlify.
- The model: it comes from my Hugging Face repo, stored in the browser's Cache API on first launch.
- The Card, marks and photos: kept in IndexedDB, so closing the app or restarting the phone mid-hike loses nothing.
- Offline: a service worker keeps the app shell, so the installed app opens with no signal.
Before the big download, a device check makes sure the phone can actually run it. The stack is Vite and TypeScript with no framework, and Vitest tests the game logic through fake adapters.
Why Does Open Innovation Matter?
Every reason Trail Bingo works comes down to the model being open.
- It works where hikes happen. Most of the trails I care about have no signal. A closed vision API needs a connection for every photo. Because BioCLIP's weights are open, I could export it, put it in the browser and run it on the phone itself. The trail is offline, so the AI has to be too.
- Your location and photos stay yours. A hosted API would see where you're hiking and every photo you take, often of your kids. Here, photos never leave the phone. The only thing that sees your (rounded) location is iNaturalist, when you build a Card. There's an opt-in hike log I added for tuning the photo check, and even that only leaves when you export it yourself.
- The right tool beats the biggest one. I considered a general model like Gemma, but its browser build is about 2.5 GB, more than a phone tab can bear. It's also a text generator, not a species matcher. BioCLIP is smaller, made for exactly this question, and its open training and evaluation told me what to expect. Open meant I could choose the specialist.
- It costs nothing to run. No inference bill, no API keys, no server. A static host and a free model repository serve it, and each player's phone does the work.
- Open goes both ways. Because the model and licence are open, I could adapt it, measure it against PyTorch, and publish the browser-ready export back to Hugging Face with credit to the BioCLIP team, so the next person doesn't have to.
Taking It Outside
More field testing, and better identification
So far I have only had time to do testing in my back yard and havent had time to do more extensive field testing. That automated testing used research-grade iNaturalist photos, the kind BioCLIP partly learned from, so real Sightings will probably do worse: taken one-handed, in shade or wind, often from too far away.
Even in the desk test, the misses had a pattern:
- Look-alikes: bluestem grasses, goldenrods, sunflowers and shelf fungi.
- "Another insect": the weakest animal group, right on only 7 of 12 photos.
Trail Bingo needs a lot more time outside before I'd call its identification reliable. Here's what I'll adjust as the field data comes in:
- Tune the confidence gap from real Sightings. It's 0.03 now, set from the desk test. The hike log records every Sighting's top matches and scores, so I can see how many wrong answers slipped through as Verified and how many right ones needlessly asked. Then I'll move the threshold, or set one per group.
- Treat look-alikes honestly. When the top matches are close relatives, the game should ask rather than guess, or let a Square accept a near-identical species from the same genus.
- Test the label wording per group. "Scientific name plus common name" did as well as BioCLIP's full-taxonomy format for plants, but insects and fungi may need different wording.
- Fine-tune on local photos. Because BioCLIP's weights are open, it could be fine-tuned on a region's own iNaturalist photos. That's a bigger step, and one a closed API wouldn't let me take at all.
What's Next
- Field test, then tune. More hikes with the hike log on, and the adjustments above.
- Shrink the download without hurting accuracy.
- Support iPhones, which aren't a target yet.
Trail Bingo was built with Claude Code as a coding partner, and this post was drafted with AI assistance and edited by me.
Top comments (0)