WanderEar: Shazam for the Wild — open-weight AI that sends you outside

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

WanderEar is Shazam for the wild.

You stand in a forest, a creek bed, a garden at dusk. Shazam can't help you — it only knows music, and it needs a server. Field guides want Wi-Fi. The exact moment you most want to know what am I hearing is the moment you have no bars.

So WanderEar flips it. Record 5–10 seconds of the world around you — wind in pines, rain on leaves, cicadas, a creek, an owl — and get back a sound postcard: what you were actually hearing, written in words, alongside the audio and a spectrogram of the clip.

It's for hikers, birders, gardeners, and anyone who has ever stopped mid-trail and wondered what is that sound? The screen is meant to be the shortest part of the experience: press record, pocket the phone, listen to the world, and read the postcard later. The AI listens while you look up.

Everything runs on-device, with no signal needed. That isn't a nice-to-have — it's the whole reason the app exists. Nature doesn't have Wi-Fi.

Demo

WanderEar is deliberately software you take outside rather than a website you visit, so the honest demo is the two commands that install it and turn a sound into a postcard:

ollama pull gemma3:4b          # one-time: the open-weight model, locally
pip install -e ".[audio,plot]" # install WanderEar
wanderear listen --seconds 8   # record the world → get a sound postcard

There's also an interactive local web app (python web/server.py) for browser recording, playback, and offline "find similar" search, plus a small optional hosted gallery for browsing postcards across devices.

Code and full setup: https://github.com/DesaiVishal-16/wanderear

Code

The project is MIT-licensed and lives here — a Python package plus a tiny dependency-free web app:

https://github.com/DesaiVishal-16/wanderear

🎙 record 5–10s  ──►  🧠 classify (open audio model)  ──►  ✍️ narrate (local LLM)
on-device mic         YAMNet / PANNs / CLAP              Gemma via Ollama
                                                              │
                            📮 sound postcard (text + wav + spectrogram)

How I Built It

WanderEar is built around open-source AI rather than calling into it as a service. Every AI piece is open-weight and local:

• Narration — Gemma (gemma3:4b) via Ollama. After classification, a local Gemma model writes the postcard: a title and a short, evocative description of the soundscape. The prompt uses a bundled offline nature glossary (data/field_guide/) so the writing stays grounded in real phenology instead of hallucinating a rainforest in your backyard.
• Classification — open audio models (YAMNet / PANNs / CLAP). A pluggable classifier turns the clip into ranked sound labels with confidences. The classifiers load lazily and are optional, so the app still runs on a modest machine.
• Offline similarity search. Each postcard gets an embedding, and wanderear search ranks saved postcards by cosine similarity — a vector search with no server involved.
• A graceful floor. If no local model is available, WanderEar falls back to a deterministic template narrator, so the pipeline always produces a postcard. The LLM makes it beautiful; the app never breaks without it.

Around that core sits a small, honest local store (JSON-lines index + a media folder, the source of truth), a CLI (listen, analyze, watch, search, gallery, sync, doctor), and a web app that degrades to a read-only gallery when opened as a static file.

The optional cloud layer is opt-in and non-essential: wanderear sync pushes already-written postcards to a hosted gallery, which is a mirror, never the source of truth. I used Render (a partner in this challenge) for that gallery — a free Docker web service with a /healthz probe doing nothing but storing and displaying finished postcards and their media.

Why Does Open Innovation Matter?

WanderEar only works because its AI is open and local, in four ways that a closed API could not match:

1. It works where nature is. The weights run on the device, so there's no round trip and no connectivity requirement. A closed API is useless the moment you leave the trailhead — which is precisely where you need it.
2. Privacy by construction. What you record is often where you are. In WanderEar, your audio never leaves your device; there is no server that could log your backyard, your kid's voice, or your location.
3. Zero marginal cost, forever. Postcard #10,000 costs the same as postcard #1. There's no per-request bill that quietly decides how much of the outdoors you're allowed to ask about.
4. Swappable and hackable. Because the model and the classifiers are open and interchangeable, you can swap YAMNet for CLAP, drop in a bird-specific model, or fine-tune on your biome. You can change how the agent behaves instead of filing a feature request and waiting.

That last point is the heart of it. Openness isn't just cheaper inference — it means a birder in one valley and a gardener in another can each tune the same tool to the soundscape in front of them. AI belongs to everyone, including people standing somewhere with no signal.

Prize Categories

• Best Use of Gemma — Gemma runs locally via Ollama as the narrator, turning ranked sound labels into the written postcard, with an offline glossary grounding its writing in real seasonal detail.
• Best Use of Render — the optional hosted gallery is a Render Docker web service (free plan, health-checked) that receives synced postcards and serves them across devices, while the model itself stays on-device.