This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
We live surrounded by plants and animals we never look at. The robin on the fence, the heather on the hillside, the butterfly crossing the path. Kids walk past all of them with their eyes on a screen.
So I flipped it around and used the screen to send them outside. NaturDex is a Pokédex for real nature. You photograph a plant or an animal, and an AI model running on the phone tells you what it is. If it's new, it goes into your NaturDex with its number, its photo and the date you found it. The goal is to fill it, and the only way to do that is to go out and look.
The scenario I designed it for is a school field trip to the forest. Schools could hand out recycled phones (old phones everyone has in a drawer) with NaturDex installed, and the activity is to fill the NaturDex. That scenario set three hard rules:
- No signal. Forests don't have coverage, so everything has to work offline.
- Kids' privacy. No accounts, no uploads. Photos and GPS positions of children never leave the phone.
- Old hardware, zero cost. It has to run on a phone that's a few years old, with no app store and no API bill.
And, since it's for children outdoors, a fourth one: safety first. NaturDex warns before it entertains.
How a trip works:
- Before leaving, on wifi: tap Prepare trip. It downloads the model (87 MB) and the list of 1,944 species that live in the region (Asturias, in northern Spain).
- In the forest: tap the big camera button. In a few seconds NaturDex answers in one of three ways: "It's a European Robin!", "Which one is it?" (and the kid compares the photo with three options), or "Hmm, I'm not sure…" with a tip to take a better photo.
- Every species found gets its NaturDex number (No. 0046 for the robin) and a coloured badge for its kind: bird, plant, butterfly, fungus…
Why only Asturias? Because it's where I live, and where I could actually use it. The AI model knows living things from all over the world, but the species list and, above all, the safety rules have to be right for the place where the kids are. Which vipers live here, which plants are poisonous, which species are protected by law: I can only vouch for that in a region I know. So I chose to do one region well instead of the whole world badly. The last section explains how other regions could join.
If you don't live in Asturias, you can still try it: on a computer, the camera button opens a file picker, so you can give it any photo of a common European species, like a robin, a fly agaric or a peacock butterfly.
Demo
Try it: virien84.github.io/naturdex. Open it on a phone, tap Prepare trip on wifi, then go outside. You can also install it from the browser menu (Install app / Add to Home screen).
The design is a handheld field device: a red shell, a big blue lens that glows while the AI is thinking, a leaf-green LCD screen and chunky buttons that press down.
| Ready to go | A confident answer | Added to the NaturDex |
|---|---|---|
![]() |
![]() |
![]() |
That robin is a small bird far away, between branches, and NaturDex is still 100% sure. The shield means it's a protected species: look, but don't disturb it.
Safety comes first. Dangerous species get a black-and-yellow hazard warning above everything else. On the right, the model isn't sure which snake it is, but one of the candidates is a snake, and that's enough to warn.
| Deadly mushroom | "Could it be…?" |
|---|---|
![]() |
![]() |
When the model hesitates, the kid decides. It's a learning moment: compare your photo with the options. And when NaturDex really doesn't know, it says so instead of guessing.
| Which one is it? | Not sure | My NaturDex |
|---|---|---|
![]() |
![]() |
![]() |
At night the screen switches to a backlit LCD:
The photos in these screenshots are openly licensed iNaturalist observations from Spain (CC BY / CC0). Credits are in screenshots/CREDITS.md.
I took it outside
I installed it on my phone, a mid-range Android (vivo V50 Lite), and went out to photograph trees and plants. All three answers showed up: direct identifications, "Which one is it?" with the right species among the options, and "not sure". It worked with no errors.
The "not sure" one taught me something. I had photographed a whole tree standing among other trees. The model looks at a square crop of 224×224 pixels, and in that square there were several crowns and a lot of sky, so it couldn't tell which tree I meant. It was right not to guess. So now the camera says "For trees, photograph one leaf, flower or the bark", and the "not sure" screen repeats it.
The second lesson was about patience. After taking a photo, the app looked frozen. The message "Looking closely…" existed, but the screen never got the chance to show it (more on that below). Kids would tap again, or give up. Now the lens glows and a spinner turns from the very first instant.
Code
NaturDex
A nature "Pokémon Snap" for school field trips: photograph plants and animals and fill your NaturDex, fully offline on recycled phones.
Try it: https://virien84.github.io/naturdex/ (open it on a phone, tap Prepare trip on wifi, then go outside).
-
app/— the PWA (React + Vite + vite-plugin-pwa + onnxruntime-web). -
embeddings/— Python scripts (uv) that prepare the model and the species list on a laptop. -
SAFETY.md— how NaturDex keeps kids and protected species safe.
How it is hosted
| What | Where |
|---|---|
| The app | GitHub Pages, rebuilt on every push to main by .github/workflows/deploy.yml
|
| The BioCLIP model (87 MB, too big for GitHub) | Hugging Face: Virien/naturdex-bioclip-onnx |
| Species list + embeddings (~5 MB) | In this repo, app/public/species/
|
Everything runs on the phone: photos and locations never leave it.
Credits
- BioCLIP by the Imageomics Institute (MIT).
- Species occurrences from GBIF; test photos from iNaturalist.
-
app/: the PWA (React + Vite + vite-plugin-pwa + onnxruntime-web). -
embeddings/: Python scripts that prepare the model and the species list on a laptop. -
SAFETY.md: every safety rule and where the data behind it comes from.
MIT licensed. The model is on Hugging Face: Virien/naturdex-bioclip-onnx.
How I Built It
The trick that makes it fit on a phone is splitting the AI work in two: the heavy, one-off part runs once, on a laptop, and the phone only does what it has to do for each photo.
LAPTOP, once PHONE, every photo (offline)
GBIF ─► 1,944 species of Asturias Photo ─► BioCLIP image encoder
│ (ONNX int8, 87 MB, WASM)
▼ │
BioCLIP text encoder ─► 1 "fingerprint" ▼
per species (512 numbers) ──────────────────► compare with the 1,944
fingerprints ─► top 5 + %
1. The model: BioCLIP. BioCLIP is an open CLIP model (MIT) trained by the Imageomics Institute on the Tree of Life: millions of images of organisms with their full taxonomy. Like any CLIP model, it puts images and texts in the same space, so it can do zero-shot classification: no training on my side, just compare the photo with a text description of each species.
2. The species list, on the laptop. A script downloads from GBIF every species with at least 20 human observations in Asturias since 2000 (1,933), and a safety script adds 11 dangerous species that are too rare to make the cut, because the app can only warn about species it knows. Then BioCLIP's text encoder turns each species into a "fingerprint": its full taxonomy and common name, averaged over 15 photo prompts ("a photo of a…", "a blurry photo of a…"). That's a 4 MB file.
3. The model, on the phone. I exported only BioCLIP's image encoder to ONNX and quantized its weights to int8: from 345 MB to 87 MB. It's too big for GitHub Pages, so it lives on Hugging Face, which serves it with CORS. The phone runs it with onnxruntime-web (WebAssembly), compares the photo's embedding with the 1,944 fingerprints, and picks a verdict:
const top = candidates[0].probability
const verdict = top >= CONFIDENT ? 'confident' : top >= UNSURE ? 'maybe' : 'unsure'
return { candidates, verdict, warnings: precautionWarnings(candidates) }
Those thresholds (70% and 40%) aren't guesses. I evaluated the int8 model, running exactly as the phone does, on 244 research-grade iNaturalist photos from Spain (82 species, including every dangerous one):
| top-1 | top-3 | top-5 |
|---|---|---|
| 66% | 81% | 86% |
In the evaluation (an earlier run, with 232 of those photos), above 70% the top answer was right 84% of the time, so NaturDex says "It's a…!". Between 40% and 70% it was right only ~45%, but the right species was usually in the top 3, so NaturDex shows three and lets the kid choose. Below 40% it was right 7% of the time, so it says "not sure".
4. Safety rules (SAFETY.md). This is the part that matters most, because a wrong answer here isn't a bug, it's a child touching something.
- Look, don't touch. Every species can carry a warning: danger (vipers, the Asian hornet, foxglove, death cap…) or caution (wasps, nettles…). They come from a hand-curated list with a source for each, plus group rules: never eat wild mushrooms, don't touch amphibians' skin.
- Precaution rule: if any of the top 5 candidates with at least 5% probability is dangerous, the warning shows, even when it's not the top answer. That's the "Could it be a Viperine Snake?" screenshot. On the 124 test photos of hazardous species, 118 got a warning.
- Every snake is "danger". In the test, the model identified a venomous Baskian viper as a harmless viperine snake with 96% confidence. Kids can't tell them apart either, so NaturDex never relies on that difference.
- NaturDex never says that anything is edible.
- Hide where threatened species live. The 283 species protected in Spain or Asturias show a "look, but don't disturb it" note, and for threatened ones the saved GPS position is rounded to about 10 km, like iNaturalist does, so a shared album can't lead anyone to a nest.
5. Offline for real. It's a PWA: a service worker caches the app shell, the fonts and onnxruntime's WebAssembly; the model and species list go to Cache Storage when you tap Prepare trip; the album lives in IndexedDB. Once prepared, it works with no connection: I tested it by preparing the trip, cutting the network, reloading the page and identifying a butterfly. If you're building something similar, this post on offline PWA pitfalls by ohugonnot covers the Vite PWA and Workbox traps well.
Three bugs worth sharing
The app looked frozen, because the browser never got to paint. Decoding a full-size phone photo, checking whether it's blurry and running the model all happen on the page's single thread. React had already switched to the "thinking" screen, but the browser had no free moment to draw it. The fix is to wait for a paint before the heavy work:
// Waits until the browser has drawn the screen. Decoding the photo and running
// the model block the page, so without this the "thinking" screen never shows.
function nextPaint() {
return new Promise((resolve) => requestAnimationFrame(() => setTimeout(resolve, 0)))
}
The spinner and the glowing lens are CSS transform/opacity animations, which the GPU keeps running even while the model has the page busy. Measured inside the page with the CPU throttled 4×: the spinner is painted 0.05 s after the photo arrives, and the result comes ~13 s later.
Firefox got stuck at 99%. After downloading, the app asks the browser not to delete its data (navigator.storage.persist()). Chrome, Brave and Safari answer at once. Firefox asks the user first, and the app waited for that answer. If the kid didn't notice the prompt, the download looked stuck forever. Now it waits at most 3 seconds and moves on. I found it by running the full flow (download + identify) in Chromium, Brave, Firefox and WebKit with Playwright, and all four pass now.
"It's an European Robin!" In English it's a European. Small, but the app talks to kids.
I built it with Claude Code as my coding agent: it did the heavy lifting on the ONNX export, the evaluation script, the PWA plumbing and the design, while I made the decisions and tested it outside.
Why Does Open Innovation Matter?
For this project, open isn't a nice-to-have. A closed API would have broken every one of the rules:
- It works in the forest. A cloud model needs coverage. BioCLIP runs on the phone, in the browser, in airplane mode. That's only possible because the weights are open: I could export them to ONNX, quantize them to int8 and host the result myself.
- Children's photos and locations stay on the phone. With a cloud API, every photo a child takes (and where they took it) would go to a server the school doesn't control. Here there are no accounts, no uploads and no analytics. Nothing to consent to, nothing to leak.
- It costs nothing per photo. A class of 25 kids taking photos all morning would be thousands of API calls. With an open model running locally, the cost is zero, forever, and the school doesn't need a credit card or an API key.
- It runs on old phones. It's just a web page: no app store, no installation process, no minimum Android version beyond a modern browser. Recycled phones are enough.
- A specialist beats a generalist here. BioCLIP was trained by researchers specifically on the Tree of Life, with taxonomy. It knows thousands of European species, from beetles to orchids. And being open, I could measure exactly how good it is on photos from my region, and set the thresholds from that data instead of trusting a black box.
- It's adaptable. The model doesn't change from one place to another; only the species list and the safety rules do. Another region is a matter of re-running the Python scripts with another GBIF area. No one has to ask for permission.
The data is open too: species from GBIF, test photos from iNaturalist, protected species from public Spanish catalogues. Open models plus open data made a tool that a school can own.
What's next: regions made by the people who live there
Asturias is a starting point, not the limit. The final idea, and the reason it had to be open source, is that anyone could add their own region:
- A region pack is small: the species list and its fingerprints (about 5 MB, generated with the scripts in the repo from GBIF data), plus the region's safety files. The model stays the same for everyone.
- The safety files are the valuable part, and the part I can't write for other places: which species are dangerous there, which ones are protected, and where to hide locations. Today
safety.pyis built around Spanish and Asturian catalogues. The plan is to turn it into one folder per region (dangerous.csv,sensitive.csv, the local protected-species list), so the people who know each place are the ones who feed it: a biology teacher, a park ranger, a local naturalists' group. - New regions would arrive as pull requests to a shared repository, where others can review the sources before a pack goes live, and NaturDex would list them in Prepare trip so a school picks its own before the trip.
That's the kind of collaboration a closed app can't have: the AI is open, the data is open, and the knowledge that keeps kids safe comes from the community itself.
My Agent Session
This is the session from the last day: testing it outside on my phone, fixing the "frozen" screen, the Pokédex-style redesign, the install icons, the cross-browser test that caught the Firefox bug, and preparing this post (including swapping out a photo of a run-over fox for a living one). It's in Spanish, my native language.
Prize Categories
None. NaturDex doesn't use any partner technology, so I'm entering only the overall prize.









Top comments (1)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.