This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
Discover the world of flora with BioDex — an offline plant‑collector simulation game.
Capture real plants using your device camera and transform them into collectible cards with scientific names, rarity levels, and unique attributes.
🌱 Features:
• Camera-based plant capture for collectible cards
• Learn scientific names and common names of flora
• Rarity system (S‑rank, rare, common) for added challenge
• Offline gameplay — no internet required
• Educational yet fun: explore, collect, and build your own flora deck
• Safe and private: no personal data collected or shared
Perfect for nature enthusiasts, students, and casual gamers who love discovery.
Turn your surroundings into a living Pokédex of plants — all while playing offline.
Demo
A short screen recording shows BioDex in action:
Code
Everything lives in one repo: the Android app, the model export script, and the Gemma note generator.
BioDex
BioDex is an Android-first, offline plant-collector MVP built with Kotlin, Jetpack Compose, CameraX, and the official Imageomics BioCLIP 2.5 ViT-H/14 image encoder. Capture a plant photo, find its closest match in a 4,271-species field guide, generate a field card, save it locally, and share the PNG.
Run
Prepare the official model
The app uses the official checkpoint, not the smaller distilled student. The upstream OpenCLIP/PyTorch checkpoint is about 3.94 GB. Before building the app:
- Install Python 3.13 and create the project environment with
py -3.13 -m venv .venv. - Install the model-export dependencies:
.venv\Scripts\python.exe -m pip install -r scripts\requirements-bioclip-export.txt. - Run
.venv\Scripts\python.exe scripts\export-bioclip-25-onnx.py.
The exporter verifies the existing 4,271-label embedding table against its published SHA-256 (the model author identifies it as the teacher's text-embedding space), downloads the checkpoint at a pinned Hugging Face revision, exports an FP16 ONNX image encoder, and verifies its output against OpenCLIP and…
How I Built It
BioDex is a Kotlin app: Jetpack Compose for the UI, CameraX for the camera, ONNX Runtime for the model. The interesting part is what the model is and where each half of it runs.
BioCLIP, split in half
BioCLIP is a CLIP-style model for biology. It was trained to put photos of living things and their names into the same vector space, so "which species is this?" becomes "which name sits closest to this photo?" That framing is what makes the whole app possible on a phone.
I use the official BioCLIP 2.5 Huge (ViT-H/14) from the Imageomics Institute, MIT-licensed. Only one half of it runs on the phone: the image encoder. The text half had already done its job before the app existed, because the 4,271 plant names were turned into embeddings ahead of time and shipped as a table (I reused a published one, more on that below). At runtime the app crops and normalizes your photo to 224 x 224, runs one forward pass to get a vector, compares it against the table with cosine similarity, and takes the closest match. If nothing scores at least 0.45, BioDex says so instead of inventing a plant. The phone never has to read a word.
photo -> crop + normalize (224 x 224) -> image encoder (FP16 ONNX, on the phone) -> vector
vector vs. 4,271 precomputed name embeddings -> cosine similarity -> best match (>= 0.45) -> card
Getting a 4 GB checkpoint into an APK
The official checkpoint is 3.94 GB. A Python script in the repo downloads it at a pinned revision, exports just the image encoder to ONNX in FP16, and checks the result against OpenCLIP and ONNX Runtime before it goes anywhere near Android. What comes out is a 1.26 GB file, roughly 632 million weights at two bytes each. It sits in the app's assets and gets copied into private storage on first use, which is why the very first run is slow due to model download.
That's the honest price of open and offline: this is not a small model. The first version of BioDex used a tiny distilled student model from Nate Hamilton's BioCLIP 2.5 Mobile project. The current version runs the full teacher. The swap was possible because Nate published his 4,271-plant list together with text embeddings in the teacher's own embedding space, so the same table works behind either image encoder. I reused his species list and that table (verifying its SHA-256 along the way), left his student encoder out, and put the real thing behind it. That was a lot of work I didn't have to do, and I'm grateful for it.
Teaching 4,271 plants to talk
A card needs words, and I wanted them to read like a field guide rather than a database row. The obvious move, a language model on the phone, would have meant squeezing it in beside a 1.26 GB vision model. So I moved the language to build time. Gemma 4 (the E4B size, running locally through Ollama) wrote a common name and a short field note for every one of the 4,271 plants, once, with a resumable script in the repo. The results ship as a JSON file and the app just looks them up. No language model runs on the phone.
The trade-off is that the notes describe the species, not your photo. Specimen #0001 shows it: the note talks about small, pale flowers, and my photo is full of pink ones. The notes are AI-written and can be wrong, and the README says so plainly.
The game on top
I designed the look in Figma Make first: forest greens, DM Serif Display for names, an orange full stop after the logo, and a card that turns purple and gold the moment you capture something. Then I rebuilt it in Compose. XP depends on rarity (a duplicate is worth 25), every 1,000 XP is a level (level 1 is Seedling), and there's a daily streak, a daily field mission with bonus XP and badges, and a field rank. Cards and progress live in the app's private storage, and any card can be shared as a PNG through Android's share sheet.
Rough edges
- Plants only. Animals and fungi aren't in the field guide, and a plant that isn't on the list can't come back as an answer, only its closest cousin.
- The score isn't a confidence level. Cosine similarity says how close two vectors are, not how likely you are to be right. BioDex treats it as a suggestion and a minimum bar, never a verdict.
- It's heavy. You need a high-end Android phone with several GB of free RAM and about 3 GB of storage. Smaller phones may run out of memory or crawl.
- Never use it to decide whether a plant is edible, poisonous, or medicinal. Seriously.
Why Does Open Innovation Matter?
The prompt asks where the open approach beat a closed one, so here is exactly what BioDex gets from each open piece, and what it costs.
It works where there's no signal. The model is inside the APK and the app never touches the network. A hosted vision API would turn "what is that?" into a spinner on exactly the trails where you'd ask it.
Your photos stay on your phone. Pictures of your garden, your kids, your street: none of it gets uploaded, because the app never connects to anything. Photos and cards stay in private storage.
I could pin, verify, and swap. The weights are MIT-licensed, so I pinned an exact revision, verified the label table against its published SHA-256, recorded hashes for everything the exporter produced, and swapped a tiny student model for the full one without redoing any label work. An API that can change underneath you can't promise that.
Identifying a plant costs nothing. No per-photo fee, no rate limit, no API key to leak. The only bill was my own machine's time while Gemma wrote 4,271 field notes locally.
Where closed would have won. A hosted model would probably know far more than 4,271 plants, handle animals and fungi too, and wouldn't need 1.26 GB of my phone. Open cost me storage, RAM and one very long export script instead of money and privacy. For a game whose whole point is walking somewhere without signal, I'd make that trade again.
I also didn't have to invent any of the hard parts. The Imageomics Institute trained and open-sourced the model, Nate Hamilton published a plant list and embeddings I could build on, Google's Gemma wrote the notes, and ONNX Runtime ran the whole thing on a phone. I wired them together and put a card game on top. That's open innovation at hobby scale.
My BioDex started at 0 / 4,271. That's 4,271 reasons to look up from the screen.
Prize Categories
Best Use of Gemma. Gemma 4 (E4B), running locally through Ollama, wrote the common name and field note for all 4,271 plants in BioDex's field guide. The generator is a resumable script in the repo, and its output ships with the app as a plain JSON file.
Best Use of GitHub Copilot. GitHub Copilot became my silent co‑developer throughout BioDex’s creation — helping me transform a concept into a working MVP far ahead of schedule. It accelerated development by generating clean Kotlin snippets for Android UI components, data models, and coroutine logic, while also assisting in debugging complex camera and JSON parsing workflows.
Top comments (0)