Sit Spot: Listen First, Then Check 🌿

Tushar Singh — Sat, 10 Oct 2026 21:00:43 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Sit Spot is an open-source, screen-minimal outdoor listening companion built on a simple premise: "Listen first. Then check."

Most nature apps do the opposite of getting you outdoors: they encourage you to stare at a glowing screen, hunt for AR badges, or wave your camera at bushes. Sit Spot is designed to make the screen the shortest, quietest part of the experience:

Choose a duration: Pick a short outdoor listening session (30 seconds to 5 minutes).
True Dark Listening Mode: Put your phone face-down or in your pocket. The screen switches to a distraction-free dark interface with zero flashing graphics or notifications, recording ambient surroundings while you pay attention to the trees, wind, water, and wildlife around you.
Listen First, Guess First: When the session finishes, you write down what acoustic textures you noticed before looking at any automated output.
Local On-Device AI Analysis: Your recording is analyzed completely on your machine using an open-weight acoustic model—without sending a single byte of audio to the cloud.
Timeline Reveal & Snippet Playback: Compare your observations with timestamped sound events, inspect honest qualitative confidence bands, and tap individual detections to play back exact audio snippets.
Grounded Field Journal: Receive a deterministic reflection contrasting your notes with acoustic signals, add personal field notes, and save or export your private journal locally.

Who it's for: Naturalists, hikers, birders, campers, students, and anyone experiencing digital fatigue who wants an excuse to sit quietly outside and reconnect with sensory observation.

Demo

🎥 Full Video Recording: Watch sit-spot-demo.mp4 on GitHub

What the demo shows:

Screen-Off Nature Listening: Launching a session outdoors with the distraction-free dark screen timer.
Listen-First Reflection: Entering human field observations before revealing any AI predictions.
Local ONNX Inference: Offline classification and temporal event timeline generated on-device with YAMNet without sending audio to the cloud.
Field Journal & History: Comparing human notes with detections, playing timestamp snippets, and exporting the session log.

Code

The entire codebase is open source under the Apache-2.0 license:

tushaar-05 / TouchGrass

Sit Spot 🌲

"Listen first. Then check." An open-source, screen-minimal AI nature listening companion.

Sit Spot is a local-first nature listening application designed to encourage people to spend time outdoors paying attention to their natural surroundings. The user selects a listening duration (from 30 seconds up to 5 minutes), puts their device aside, and listens without screen distractions. Afterward, the user writes down what they observed before revealing a timestamped timeline of acoustic detections analyzed locally on-device.

Demo Walkthrough

Full demo recording (62s): assets/sit-spot-demo.mp4

What Sit Spot Does

Distraction-Free Outdoor Listening: A true dark screen mode minimizes screen glare and notifications while you sit quietly outdoors.
Listen-First Methodology: You record your observations first before viewing AI results to cultivate active sensory awareness.
100% Local On-Device AI: Powered by Google YAMNet (Apache-2.0) running natively on Apple Silicon via ONNX Runtime with ARM64 NEON optimization.
Honest Detections & Score…

View on GitHub

Repository: https://github.com/tushaar-05/TouchGrass

Full automated test suite (48 tests, 100% passing across audio validation, aggregation, and API routes).
Verified model weights provenance and SHA-256 checksums documented in MODELS.md.
Strict local-only privacy pledge in PRIVACY.md.

How I Built It

Sit Spot is built around a local open-source AI architecture that requires zero external APIs:

1. Open-Source AI Model: Google YAMNet (Apache-2.0)

Architecture: MobileNetV1 depthwise-separable convolutional backbone trained on the AudioSet ontology (521 acoustic event classes, including Bird, Bird vocalization, Wind, Rain, Stream, Water, Insects, Animals).
Local Inference Runtime: Converted to ONNX and executed natively via ONNX Runtime on Apple Silicon M4 using ARM64 NEON CPU vectorization.
Extreme Efficiency: Evaluates a 3-second recording in 10.3 ms (~200× faster than real time), with a model weight footprint of just 15.3 MB and <60 MB RAM usage.

2. Audio Preprocessing & Hardened Validation

Formats: Direct in-memory parsing for 16 kHz WAV via soundfile, plus sandboxed FFmpeg fallback (shell=False, 15s timeout) for browser-recorded WebM (Opus) and MP4/M4A.
Safety Bounds: Enforces a strict 25 MB file limit, 1.0s to 300.0s duration bounds, and header magic-byte verification (RIFF, 0x1A45DFA3, ftyp).
Acoustic Integrity: Normalizes to 16 kHz mono float32 in [-1.0, 1.0] without synthetic amplification (avoiding boosting sensor noise into false-positive nature detections). Flags silence ($\text{RMS} < -80\text{ dBFS}$) and performs clipping diagnostics (>5% warning, >30% distortion rejection).

3. Temporal Window Aggregation & Honest Calibration

YAMNet operates on 0.975s sliding windows with 50% overlap (0.4875s hop). Without aggregation, a bird call triggers redundant alerts across every frame.
Our custom aggregate_detections() engine clusters contiguous windows of the same label, preserves the peak activation score, and clamps timestamps strictly to the real recording duration—ensuring internal zero-padding is never presented as observed sound.
Rather than misleading users with pseudo-probabilities, raw activation scores are mapped to documented qualitative bands:
- STRONG_SIGNAL ($\ge 0.50$)
- MODERATE_SIGNAL ($\ge 0.25$)
- SUGGESTIVE_TRACE ($\ge 0.10$)

4. Grounded Reflection Engine

A deterministic rule-based reflection engine compares user guesses with detected sound classes. It highlights acoustic alignments and notes subtle sounds that fell below model thresholds—guaranteeing zero AI hallucinations of animals or weather.

5. Frontend

Built with React 18, Vite, and TypeScript, styled using semantic CSS design tokens inspired by natural history field notebooks (parchment surfaces, restrained moss-green accents, and a true dark listening mode).

Why Does Open Innovation Matter?

Open innovation is what made Sit Spot possible in ways a closed API model never could:

Backcountry Offline Independence: Nature doesn't have reliable 5G cell towers. If this app relied on OpenAI, Claude, or proprietary cloud inference, it would fail the moment you stepped onto a forest trail or deep into a national park. Because YAMNet runs on-device, Sit Spot functions identically with Wi-Fi disabled in airplane mode.
Absolute Privacy in Shared Spaces: Outdoor field recordings can inadvertently capture nearby human speech or private conversations. Uploading unvetted audio to commercial clouds poses severe privacy risks. In Sit Spot, audio stays in memory and on local disk; no telemetry or audio bytes ever leave your device.
Zero Marginal Cost: Cloud audio transcription and multimodal APIs carry per-minute billing that makes screen-free exploration expensive. Sit Spot costs $0.00 to run forever.
Honest Science Over Hype: Proprietary classification APIs often present black-box "99% confidence" figures that mislead users into thinking an exact bird species was positively identified. Open model cards and transparent AudioSet ontologies allowed us to document exact limitations and implement honest qualitative score bands.

My Agent Session

This project was built pair-programming with Google Antigravity on a MacBook Pro (Apple Silicon M4). The agent session navigated model provenance verification, audio buffer mathematics, and test-driven architecture across test suites:

Milestone 1: Scaffolding, Apache-2.0 licensing, and verified YAMNet model card SHA-256 checksums.
Milestone 2: Secure audio validation, format magic-byte sniffing, and sandboxed FFmpeg decoders.
Milestone 3: Concrete ONNX adapter, temporal window aggregation, and real offline inference integration.
Milestone 4 & Final: Complete React user journey, guess-first interaction, snippet audio playback, and field journal storage.

Prize Categories

Overall Winner (Hacktoberfest Week 1: Touch Grass)

Listen first. Then check. Go outside and sit in your spot today!

DEV Community: Tushar Singh