Side Quests Grow on Trees (Leave Them Where They Grow) 🌿

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

TL;DR

• Wild Find is a native Android treasure hunt for kids 8 and up, built for (and tested by) my family, that sends kids after real plants common to their area.
• Two open-weight models run on the phone, BioCLIP 2.5 Mobile and TinyCLIP, and no images ever leave the device.
• Gemma 4 26B runs locally in the build pipeline to precompile hints and hazard warnings, and every win still comes down to deterministic code: 0 of 267 toxic test photos passed as a match.
• 🎵 I love assigning a theme song to my builds, just because it's fun. This one is Pinecones by Bug Hunter, and it's fabulous!
• 🎬 Watch the demo · 💻 Browse the repo · 📱 APK releases

What I Built

The idea for this game came from a family powwow, where every idea built off the last until I was satisfied that we had outlined something I could realistically build.

Wild Find is a treasure hunt game designed for kids 8 years and up to play with their parents. Even though my son is much older than that, I still had a lot of fun building and testing!

This is not a quick game. It's designed to keep kids, both young and old, outside looking for plants as long as they can stand it.

The app gives you three randomly selected plants for a region that is set by an adult user based on location or picked from a map.

The kid is then tasked to find one of three plants common to their specific area of the world. When they think they've found one, they capture a picture without picking the plant, or they can skip an extra hard one. The game is finished when you find all three plants from your list.

My son and sister both played the game without any instruction from me. We all learned that the previously deferred in-game hints were a necessity to prevent users from getting frustrated by the game. Also, just because my region contains those plants does not mean that my yard does. So I added in skip functionality for each plant to keep it kid-friendly.

If the app flags a potentially hazardous plant, it warns the user not to touch it — which is a general safety rule in the game, anyway. There are 85 total hazards: seven in North America that are always on, like poison ivy, oak, and sumac, and another 78 that are only on when iNaturalist says they're nearby.

I'm most proud of the latency numbers I was able to accomplish with two open-weight models: BioCLIP 2.5 Mobile, a plant-focused model distilled from the larger BioCLIP 2.5, and TinyCLIP, a tiny general-purpose model that checks whether there's a plant in the frame at all. Together, they take a median of 189 ms per frame on my S24 Ultra.

As always, I built this app to be accessible from the start: big touch targets, labels for TalkBack, no color-only signals, animations that follow the reduced-motion settings, and hints and warnings that are still there when text is set to 200% size.

The app rounds the user's rough coordinates to the nearest whole degree before sending them to iNaturalist, which returns a list of regional plants, each with a short description of its type and up to three hints prebuilt from Wikipedia and USDA PLANTS. No photos or saved gameplay progress leave the device. iNaturalist also sees the request's IP address, as disclosed on the grown-ups page.

Demo

🎵 Theme song: Pinecones by Bug Hunter

📱 Want to play it yourself? Grab the Android APK (Android 11+) from the v1.0.0 release.

Code

anchildress1 / wild-find

Kids' plant scavenger hunt for west Georgia. Gemma 4 + BioCLIP on-device; no photo leaves the phone.

🌿 Wild Find

Wild Find sends kids 8 and up outside to find and photograph plants that grow near them. Open-weight models on the phone check each photo. No photo ever leaves the device.

Look. Photograph. Leave it where it grows.

Table of contents

• About
• Features
• Tech stack
• Architecture
• Project structure
• Getting started
• Configuration
• Security and privacy
• How to contribute
• What's next
• License
• Credits
• Author

---

About

Wild Find is an Android scavenger hunt for kids, played with a parent. Each hunt picks three plants that people have actually recorded nearby this month on iNaturalist. The kid finds each one, points the camera, and taps Capture. Models on the phone check the photo, so it never leaves the device, and the app warns when a plant might hurt to touch.

It's built for the DEV Hacktoberfest 2026 "Touch Grass" challenge. Ashley's challenge entry, Side Quests Grow on Trees (Leave Them…

View on GitHub

⚖️ This project is licensed under MIT.

I build with AI, including Claude, Codex, and Copilot — which ran out of credits before I had finished the build. There are plenty of distinct design decisions you can see in the code throughout.

First, I don't want a toxic or hazardous plant to win a hunt. I guard against this with a "blocker," where every local toxic or hazard species is checked in the photo capture. The best match must beat them all by 0.034 when the full frame is recognized as a plant, or 0.048 when only the circle is recognized. A blocker also scores on the circle alone, so a toxic plant that only fills the circle can't hide behind the rest of the photo. I tested this until 0 of 267 toxic plant photos passed, while real finds went from 97 to 108 of 198.

override fun score(reticle: FloatArray, full: FloatArray?): GoalScore {
    fun s(row: Int) = if (full == null) {
        table.dot(row, reticle)
    } else {
        (table.dot(row, reticle) + table.dot(row, full)) / 2
    }
    val margin = if (full == null) MARGIN else FULL_MARGIN
    val scores = eligible.map(::s)
    val best = scores.max()
    val top = eligible[scores.indexOf(best)]

    // A toxic plant that fills only the circle must not be diluted by the full frame, so a blocker scores the
    // larger of its mean and its reticle cosine.
    fun guard(row: Int) = if (full == null) s(row) else maxOf(s(row), table.dot(row, reticle))
    val blocker = blockers.maxOfOrNull(::guard) ?: Double.NEGATIVE_INFINITY
    val score = s(target)
    // A tie for top-1 is no pass, whichever rows tie.
    val met = scores.count { it == best } == 1 && genus[top] == genus[target] && best - blocker >= margin
    return GoalScore(met, score, 1 + scores.count { it > score })
}

/**
 * The reticle-only margin, for a frame whose full frame isn't a plant: the smallest round margin that let 0 of
 * 180 local toxic photos pass on Oct 7 (a poison ivy photo read as beautyberry won by 0.0477).
 */
const val MARGIN = 0.048

/**
 * The margin on the reticle and full-frame mean (day-5 rule R1h): the 0.024 tune boundary plus 0.01 headroom.
 * With blockers floored at their reticle cosine it lets 0 of 267 toxic photos pass, with the closest toxic
 * photo 0.020 under the margin, and raises target passes from 97 to 108 of 198 (`blocker_guard.log`).
 */
const val FULL_MARGIN = 0.034

Second, according to BioCLIP's model card, it matches an image against a fixed table of known species, and it's strongest at the genus level. The same card says "Do not use this for toxicity or edibility decisions," which is why the toxicity flags come from Wikipedia and USDA text, not the model.

So the app takes the full local list, asks BioCLIP to rank the capture image against each local plant species, and then deterministic checks run to determine if a match was made or not.

The code uses BioCLIP's scores for each plant, averaged across the circle and the whole photo when TinyCLIP sees a plant in the whole photo (for example, how much does this look like water oak?), to determine the top species and whether that top species is in the target's genus (because this is only the easy level; future iterations would increase the difficulty).

The 0.034 margin originally came from an earlier rule's 0.024 tuning boundary plus 0.01 of headroom. After adding the circle-only blocker guard, I tested the rule again. Lowering the margin to 0.011 allowed two toxic photos through in the fresh test set, so I kept 0.034.

That costs some legitimate finds, but the closest toxic test photo now falls 0.020 below the margin instead of just 0.0028.

A single matching frame still isn't a win, either. The app needs three matching frames in a row before it counts a find.

The Day 5 results have every rule I tried and every photo.

fun of(sightings: List<Sighting>): LocalList {
    val floor = max(MIN_SIGHTINGS.toDouble(), SHARE * sightings.sumOf { it.count })
    val byRow = sightings.mapNotNull { s ->
        rowOf(s.scientific)?.let { it to s }
    }.groupBy({ it.first }, { it.second })
    val counted = byRow.mapValues { (_, names) -> names.sumOf { it.count } }.filterValues { it >= floor }
    val eligible = counted.mapNotNull { (row, count) ->
        val common = byRow.getValue(row).sortedByDescending { it.count }
            .firstNotNullOfOrNull { it.common?.trim()?.takeIf(::isKidName) }
        if (table[row].playable && common != null) Eligible(row, common, count) else null
    }.sortedWith(compareByDescending<Eligible> { it.count }.thenBy { it.row })
    // Any sighting blocks: the floor-only blockers let 87 of 180 toxic photos pass as some target on Oct 7.
    val blockers = byRow.keys.filterNot { table[it].playable }.sorted().toIntArray()
    // A hunt takes one target per genus, so three species of two genera still can't fill it; a row held back from
    // picks can't fill a slot either.
    val genera = eligible.filter { table[it.row].target }.distinctBy { table[it.row].genus }.size
    return LocalList(eligible, blockers, needsWiden = genera < MIN_TARGETS)
}

Also, there were four plants BioCLIP couldn't reliably recognize in testing: sweetgum, tulip tree, persimmon, and winged sumac. So I pulled them all out of the hunt. They will still count when the app compares plants, though, so nothing sneaks through as one of those.

There are plenty of other examples in the repo, too. Check out these pages, if you're curious.

File	What it does	Why it's code, not AI
Verify.kt	The verdict table: every message a kid sees after a capture, first matching row wins	One readable rule table decides the star, with no model judgment
HazardCheck.kt	Warns on 7 fixed hazards, including poison ivy, oak, and sumac, plus 78 regional ones, using local iNat sightings	The warning is a fixed cutoff that was tested on 52 hazard photos, not a model's opinion
toxicity.py	Flags toxic plants from Wikipedia sentences and USDA ratings	CLIP models scored toxicity at coin-flip odds (AUC 0.42–0.65), so text matching does the job
assets.py	Refuses to build if any plant is missing its toxicity flag, and never gives a toxic or hazard plant a hint	The build fails closed: a safety gap stops the APK, it doesn't ship it
RegionKey.kt	Rounds the location to a whole degree before anything leaves the phone	Privacy for kids comes from a single rounding function

I tested the hazard warning separately, too. It caught 48 of 52 hazard photos and only warned on 3 of 253 regular plants by mistake. Worldwide, using a fresh set of images, it caught 41 of 52, which isn't perfect. And it's exactly why the game never tells a kid that a plant is safe to touch (hazard results).

Also, this build includes 314 Kotlin unit tests, 350 Python tests, and another 120 tests that run on device (67 device and 53 end-to-end). The automated checks pass, and it survived several rounds of manual testing on different devices, as proven by my son playing barefoot at the edge of our property.

How I Built It

The app is separated into two distinct sections: the build pipeline and the app itself that runs on device.

Open-source AI	Type	Runs on	Job
BioCLIP 2.5 Mobile	Open-weight model	Phone	Scores each capture against the local plant list
TinyCLIP ViT-8M	Open-weight model (MIT)	Phone	Checks there's a plant in the frame at all
ONNX Runtime	Open-source inference runtime	Phone	Runs both models offline, with its telemetry switched off
BioCLIP 2.5 (ViT-H) + OpenCLIP	Open-weight model + framework	Laptop, build time	Writes the tutorial vectors and the hazard rows BioCLIP's table is missing
Gemma 4 26B	Open-weight model	Laptop, build time	Writes the hints and hazard warnings
Ollama	Local inference	Laptop, build time	Serves Gemma with no cloud API calls

I looked into training a new model based on BioCLIP that could identify non-plants or poisonous plants, and could return hints inside the game. Based on research and model documentation already published on Hugging Face (Gemma 4 E2B, BioCLIP 2.5 Mobile, BioCLIP 2), I decided that would probably waste a lot of time and still not get me the results that I wanted.

Early testing had already proved a larger model would slow down gameplay. Gemma 4 E2B is a 2.6 GB file next to BioCLIP Mobile's 47 MB, and it took 2.3 to 2.9 seconds per photo, while BioCLIP and TinyCLIP together take a median of 189 ms per frame.

Instead, I chose to precompile a list of 4,272 plants, including the 2,093 not flagged as toxic or hazardous, and build that list into the released APK. First the build pipeline runs Gemma 26B against plant Wikipedia articles to determine hints based on hint type.

Hint type	Plant	Hint	Source
Place	Blue mistflower	"Look in sandy woodlands and clearings."	Wikipedia
Light	Water oak	"It does not like shade."	Wikipedia
Ground	Christmas fern	"It grows in moist habitats."	Wikipedia
Nearby	American sweetgum	"It grows near willow oaks."	Wikipedia
Ground (USDA)	American beautyberry	"Look on ground that drains well, not soggy."	USDA PLANTS

After Gemma runs, deterministic checks confirm the quote is really in the article and the plant itself is never named. They also check for banned words like "safe," "harmless," or "touch," make sure no hint exceeds 20 words total, and throw out anything that sends a kid toward any road, body of water, or a giant cliff.

USDA PLANTS fills in the gaps using its shade and soil ratings, size, season, and common signs. After both lists are joined, the code drops anything without a source as a safeguard, and then all hints for a specific plant are ranked by aspect, source, and rarity.

On the first full run, Claude graded a random sample of 300 picked hints against their quotes to determine overreach and spot inaccuracies: 94% were supported and none were unsafe after a recheck.

Claude grading Gemma could only get me so far, so I read a sample of the generated hints and graded them myself. 13 were off: 10 unsupported, 3 overreaching, so I tightened the rules, reran Gemma, and graded a fresh set. Only 3 of 48 in that run were off — a total of 6%. The stricter rules did cost plant coverage in the game, but 1,849 of 2,093 playable plants — or 88% — still have at least one hint (hint results).

This doesn't make the game error-free, but it helps reduce the number of errors a user may find from any location. I tested it in West Georgia, and the game works better inside the United States; however, it supports regions worldwide where iNaturalist has enough sightings.

Why Does Open Innovation Matter?

I wanted the game to work offline, for campers and out in the backwoods. That means I can also use the app when I go back home to the mountains, where cell phone signal is unreliable at best.

Offline functionality depends on a cached lookup in the user's hunt region. Parents have the ability to pre-load a cached list for all 12 months of a selected region from the grown-ups page to prep for offline use ahead of time.

The only calls that leave the device when it's online go to iNaturalist.org without any user accounts or logins.

I caught one potentially fatal problem to my privacy statement during testing. ONNX Runtime quietly opens a telemetry connection to Microsoft unless you explicitly turn it off. After that, model startup ran with zero open connections and a fresh hunt only ever talks to iNaturalist (privacy log).

Having access to open-weight models meant I could test different approaches without paying for every photo, bundle the models directly in the APK, and keep the entire recognition process on the phone. I didn't have to choose between asking kids to upload their pictures or paying for a hosted API every time somebody wanted to find a tree.

Prize Categories

Best Use of Gemma

My original plan was to use Gemma E2B, since the E4B I know runs hot on my device from my last Gemma project, Vestige. The problem with that is that I ultimately needed Gemma to function as a knowledge base, which it can't do without grounding (asked to describe the 20 most-seen plants near me, it got 4 right and 7 wrong enough to mislead).

My next idea was to just let it identify the plant, which was before I found the TinyCLIP and BioCLIP models. It performed terribly in early rounds of testing (2.3 to 2.9 seconds per photo once warm, when a live camera needs an answer about every 200 ms). So I found open-weight models that could do this job, which turned out to be the only thing that saved this game.

The next job I wanted Gemma to do was to help provide hints during gameplay. My idea was if we gave Gemma E2B a picture and what it was looking for, then it could tell the user "look to your right." That round of testing produced the same hint five times over (across two outdoor runs, every one said some version of "look for a big tree near the woods edge," whatever was in the picture).

Then I was left with rewriting text alone from a precompiled list at the cost of 0.7 seconds per run. What I got in return was the exact same phrase I gave it, minus a few words here and there, stacked on top of a long list of hallucinations from previous test runs. So, I decided to cut Gemma from the phone entirely.

Even though there's no active Gemma running in the app itself, I did use a local Gemma instance through Ollama to help build both the hints displayed for each plant and the hazard warnings that appear in the game. I've wasted hundreds of dollars using GPT, Claude, and Gemini APIs to do this sort of proof-of-concept work before this challenge, and I really wasn't interested in adding to that number.

Gemma also decided which plants made the contact-hazard list in the first place. It had to quote the exact Wikipedia sentence it relied on, and anything it wasn't sure about came to me for review before it could ship.

I tested both Gemma 12B and 26B models. 12B made up three different source quotes. 26B made up none and it ran faster on my laptop, anyway. The decision made itself, really.

The results ship with the APK as data and are used for lookups later.

It might not be the best use case, but it's a real one. It also shows you exactly where the lines are drawn between AI capability and wishful thinking. AI is not meant to perform a job I can accomplish in the codebase, nor should it be kept for a challenge when the cost exceeds the benefits. I decided to enter this category anyway, so I could show you exactly why this was the right call.

Future Enhancements

• Increase the real-find pass rate to > 90% without letting a toxic plant win.
• Improve hint coverage to include all plants by sourcing information outside of Wikipedia and USDA PLANTS.
• Shrink the APK and make installing it easier for people who aren't developers.

Ultimately, I'd rather a kid hear "keep looking" than win with poison ivy, so that's the line I drew throughout to keep things honest.

Ashley ChildressFollow

Distributed backend specialist. Perfectly happy playing second fiddle—it means I get to chase fun ideas, dodge meetings, and break things no one told me to touch, all without anyone questioning it. 😇

---

🛡️ Picked by Me, Not the Plants

Claude, Codex, and Copilot all had a hand in this build, right up until Copilot ran out of credits and sat out the rest of the hunt. Leonardo painted the pictures, and Claude checked every number against the repo, dragged me back to the logs whenever my memory wandered off, and wrote this footer. The decisions, the 0.034 margin, and the barefoot test pilot are all mine.