Trailside: an open-weight bird call identifier that talks back

What I Built

Trailside is a bird call identifier built to get you outside — and keep you outside. Open the PWA, tap record, stand still for 5–20 seconds, and it tells you what you heard out loud: the species, the confidence, and a short spoken field note. No account, no app-store install, no staring at a screen mid-trudge.

It's for day hikers, beginner birders, parents with curious kids, and anyone who walks past birds they'd like to name without stopping to fiddle with a phone.

Demo

Try it live: https://trailside-0hlr.onrender.com

Verified end-to-end from the deploy: a 34-second American Robin recording returned American Robin @ 90% (plus Song Sparrow and House Finch hints) with an ElevenLabs-narrated field note in ~15 seconds.

Code

Open source: https://github.com/madhavgupta07/trailside

• Python/FastAPI backend, plain-JS dependency-free PWA frontend (no build step)
• onnxruntime CPU inference — no TensorFlow, no GPU needed
• Async job flow so a 30-second clip never trips a slow gateway timeout
• 10 unit/integration tests, including a real-audio classification test (american_robin.mp3 → American Robin @ 90.1% locally)

How I Built It

The whole experience is built around open-weight AI — the species engine is the core, and the narrator is a second open-weight model on top:

Piece	Role	License
BirdNET v3.0 (Cornell / TU Chemnitz, via tphakala/BirdNET-v3.0-Models)	11,560-species bioacoustics classifier; I use the 800-class north-america-east regional ONNX slice (~150 MB)	CC BY-SA 4.0
Gemma 4 (Google)	writes the 2–3 sentence spoken field note; served via an OpenAI-compatible endpoint (OpenRouter, free google/gemma-4-26b-a4b-it:free by default)	Gemma Terms (open weights)
ElevenLabs	narrates the note back so the result is heard, not read	—
Render	Blueprint (render.yaml) deploys FastAPI; startCommand downloads and checksum-verifies the model at boot	—

Pipeline: mic → browser WAV encode (16-bit PCM) → FastAPI → BirdNET (32 kHz, 5 s windows, running max-aggregate) → top-5 species → Gemma field note → ElevenLabs MP3, played back in the page.

Resilience was a real constraint, not a checkbox: the free Gemma endpoint rate-limits hard under load, so narration retries with backoff and then degrades to a deterministic offline template note — the app never errors because a model is busy. There's the same graceful no-voice fallback if no TTS key is set. The API surfaces why (note_error) instead of hiding it.

Why Does Open Innovation Matter?

Trail use is exactly where closed APIs fail:

• It can work fully offline. The identical code path runs on a laptop with llama.cpp and no internet at all. A network drop mid-hike doesn't produce an error — it produces a gentler fallback note.
• Your recordings stay yours. Audio goes to a server you own; you can run it yourself and take the network out entirely.
• Swap anything at runtime. Regional model, Gemma size, voice, provider — all env vars, zero rewrites. Fine-tune a model and point a variable at it.
• It costs nothing at the core. Both core models are free weights; BirdNET inference is pure CPU on the cheapest deploy.

A closed API literally cannot offer a free, server-owned model that keeps working when the phone loses signal in the woods. That's the difference open weights made here.

My Agent Session

The full build transcript, as it happened:

Session 2026-10-09 08:14

You

Take a Touch Grass / open-source AI idea and build it: a bird call identifier that gets people outside, with open-weight models at its core, deployable on Render's free plan.

Agent

Architecture: ref — browser mic -> client-side WAV encode (no ffmpeg) -> FastAPI -> BirdNET v3.0 ONNX (open weights, regional north-america-east, 800 classes) -> Gemma writes a spoken field note via OpenRouter -> ElevenLabs TTS. Scaffolding FastAPI app, classifier service, config, requirements.

▸ write_file app/services/classifier.py (BirdNET ONNX loading: scipy resample_poly to 32kH...

Input

app/services/classifier.py (BirdNET ONNX loading: scipy resample_poly to 32kHz, 5s windowing, uncontrollable-max-aggregate, low-memory SessionOptions)

Output

Classifier service written; model auto-download + sha256 verify script added

You

Add tests and a real audio fixture, then download the model and verify classification locally.

Agent

Downloaded the 149MB regional model, grabbed an American Robin recording from Wikimedia Commons as a fixture, wired narrator (Gemma) + TTS (ElevenLabs) with graceful fallbacks, and wrote API + service tests with mocked HTTP.

▸ bash python -m pytest tests/ -q

Input

$ python -m pytest tests/ -q

Output

10 passed — classifier returns American Robin @ 90.1% on the 34s fixture

Deployed via render.yaml Blueprint; set OpenRouter + ElevenLabs keys in Render env; health endpoint reports model/gemma/elevenlabs readiness.

You

Gemma narration is silently falling back to the offline template on production. Find out why.

Agent

▸ bash curl /api/identify with american_robin.mp3; inspect note_error

Input

$ curl /api/identify with american_robin.mp3; inspect note_error

Output

note_error: 404 'No endpoints found for google/gemma-3n-e4b-it:free' — OpenRouter no longer serves gemma-3n

Swapped to google/gemma-4-26b-a4b-it:free (open-weight Gemma 4, actually serviced), updated docs and the DEV post, redeployed. Next prod run hit 429 rate limits — added 3 retries with backoff before the template fallback.

You

Long clips time out / 502 on the 512MB free instance. Make the API fit the free tier.

Agent

Diagnosed peak-memory & long-request issues: inference now runs windows one-at-a-time (batches stayed safe), threads=1, analysis windows capped; then made /api/identify fully async — POST returns a job_id in ~0ms and the page polls GET /api/jobs/{id}.

▸ edit_file app/jobs.py (ThreadPoolExecutor job store) + frontend app.js polling loop

Input

app/jobs.py (ThreadPoolExecutor job store) + frontend app.js polling loop

Output

Verified end-to-end on prod: 34s clip -> American Robin 90% + ElevenLabs narration in ~15s, no 502s

You

Wrap up the submission: structure the DEV post per the challenge template, fill verified facts, and prep the agent session embed.

Agent

Wrote DEV_SUBMISSION.md matching the official template (What I Built / Demo / Code / How I Built It / Why Open Innovation / My Agent Session / Prize Categories), filled in the live URL and verified results, and saved this session transcript for the {% agent_session %} embed.

10 of 10 messages

(If the embed doesn't render on your view, the direct link is https://dev.to/agent_sessions/session-2026-10-09-0814-tf0pln)

Prize Categories

• Best Use of Gemma (open-weight Gemma 4 writing the field note)
• Best Use of ElevenLabs (spoken narration keeps your eyes on the trees)
• Deploy/Run on Render (Blueprint deploy; the model is downloaded + checksummed at boot)

---

Built for Round 1 of the Hacktoberfest 2026 "Touch Grass" challenge. Open weights for the win — now go outside and listen.