This is my submission for the Hacktoberfest Open-Source AI Challenge, Week 1: Touch Grass.
I'm Arnav, from Bengaluru, and I'm new to coding. When the Touch Grass prompt arrived (build something with open-source AI at its core that gets people off their screens), I kept thinking of the one thing that makes people look up from a phone: collecting things in the real world.
So I built WildTrail: point your camera at a real bird, plant, insect, or animal, let an AI model running locally on your phone guess what it is, and collect it as a 3D stamp in a digital field journal. As you walk outside, an RPG-style fog-of-war map clears around you.
- Live app: https://wildtrail-chi.vercel.app
- Code (MIT): https://github.com/XiaoArnav/wildtrail
How it works
-
The AI. An open-weight CLIP model (
clip-vit-base-patch32) runs inside the browser through Transformers.js, pinned to version 3.8.1. It performs zero-shot image classification: I provide a curated dictionary of wildlife species and it scores how well your live photo matches each one via WebGPU or WebAssembly. No server ever sees your photo. - The collection. Confirmed finds are stored in the browser's IndexedDB, rendered as interactive tilting 3D stamps with gyroscope tilt physics, and can be exported as a vintage botanical postcard image. There are no accounts or paywalls.
- The map. Your walk is divided into 20 m squares that light up as you cover them, unmasking territory and tracking your total distance. You can also export your walk as a GPX file.
-
Offline. After the first load, the app shell, vendor libraries, and AI model weights (~90 MB) are cached by the Service Worker and Cache Storage. You can import your own
.pmtilesmap packs for offline map tiles.
How it gets you outside
I wanted the screen to be the shortest part of the walk. The app only asks for a few seconds: point, tap, and put the phone back in your pocket. Nothing happens while you're indoors, because the reward is the fog-of-war map clearing as you physically walk, and each new species needs you to find a real creature. There are no feeds, no notifications and no streaks. The scanning moment should take only a few seconds.
Why open source mattered here
I didn't choose open source for the ideology. The idea simply doesn't work without it:
- No signal on a trail. A cloud vision API fails the moment your cell bars drop in a park or ravine. An open-weight model downloaded once to device storage keeps running anywhere. A closed cloud API would have given better accuracy on a fast connection, but it would have failed the moment you needed it most, in the park with no signal.
- Location is personal. The AI inference, your photos, your finds, and your GPS track stay on your device. There is no backend, because the app has no use for one.
- No per-use cost. There are no metered API keys, token fees or per-photo bills, so nothing in the app costs money per use.
- I can swap things. The model and species taxonomy are plain code. Anyone can expand the wildlife list or swap in a specialized bird model through a pull request.
The one honest exception: the default basemap loads tiles from OpenFreeMap, which can see your IP address and the area of map you view. Importing or downloading an offline PMTiles pack avoids that.
What went wrong (the part I'd want to read)
I built this with AI coding assistants (Antigravity for most of the code, Claude as a reviewer and mentor, and GitHub Copilot on two issues). Because I'm a beginner, I couldn't trust any of it blindly, so I made a rule: read everything and verify the claims. That rule uncovered real problems:
- A fake download. An early "Download offline map" button ran a loop with a 120 ms timer that updated a progress bar and saved a 2 KB empty placeholder. It looked finished, but did nothing. I had it completely removed and replaced with a real PMTiles import and header verification pipeline.
-
A broken map loader. The first real offline loader called
pmtiles.BlobSource, an API that doesn't exist in PMTiles v3 (the actual class isFileSource). It also loaded a plain vendor script as if it were an ES module. The map silently failed to appear. Fixing it required opening the library's actual exports and reading the documentation. - Advertised species that didn't exist. Early demo screenshots showed a tiger, a penguin, and a razorbill, but none of them were in the dictionary the AI could actually classify. I had to audit the species array to make sure every advertised animal could actually be identified.
- Privacy claims that outran the code. The README proudly claimed "zero telemetry" and "100% client-side", even though OpenFreeMap tiles inevitably hit an external tile server, and an automatic pull request had added Vercel analytics scripts. I had them removed, rewrote the documentation, and added an honest privacy note to the app modal and docs.
- A blank map when offline. If you lost cell signal without an offline pack loaded, the online vector map stayed completely blank. I had to add an explicit offline canvas fallback so the fog-of-war, GPS trail, and specimen pins draw reliably even with zero network.
The lesson I'd give another beginner: AI is happy to write code that looks complete. Ask "is this actually doing the thing?" every single time.
🔍 Behind the Scenes: The Full Agent Sessions
To give the Hacktoberfest judges a transparent look at how WildTrail was built and reviewed, I've shared both public agent sessions. They document the implementation, debugging, and verification process behind the project.
- Antigravity — Building & Debugging — Building the app, debugging issues, and implementing fixes.
- Claude — Mentor & Code Reviewer — Reviewing AI-generated code and verifying technical claims.
What Copilot did
I opened two GitHub issues and assigned them to GitHub Copilot:
- GPX trail export (PR #7): export your walk as a standard GPX 1.1 file to use in apps like Strava or Garmin.
- High-contrast mode (PR #6): a toggle so the map and compass cone remain readable under direct mid-day sunlight.
I reviewed Copilot's two pull requests before merging them.
Taking it outside
Honest status: I haven't taken WildTrail on a long outdoor walk yet; I built and checked it at my desk, so the camera, GPS and airplane-mode behaviour on a real phone are still untested by me. What I did verify is the code and the live site: I had every change checked against the code and fixed the problems above. If you try it outdoors, I'd love to hear what breaks, and issues are open on the repo.
What it can't do yet
- It can only identify species from its built-in taxonomy list, and CLIP will sometimes guess wrong, so results should be treated as suggestions rather than definitive botany.
- Offline maps require importing or downloading a
.pmtilesarchive. The bundled demo pack covers a tiny area (~300 m) in Taipei. - The very first load requires internet connectivity to fetch the libraries and download the ~90 MB vision model weights into browser cache.
Try it
Open https://wildtrail-chi.vercel.app on your phone, add it to your home screen, and go outside. Pull requests and feedback are welcome. Adding a new regional species to the classification dictionary is a great first contribution!
Thanks to the Hacktoberfest team for the prompt. 🌱


Top comments (1)
Official Platform Update
Security protocols have been updated for all developer accounts.