StepSynth flips the dynamic entirely: your footsteps compose the music.

Abhishek Kumar — Sun, 11 Oct 2026 18:39:12 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Most fitness and outdoor apps demand that you keep your eyes glued to a bright screen: checking route maps, watching GPS split times, or chasing gamified badges.

StepSynth flips the dynamic entirely: your footsteps compose the music.

StepSynth is a lightweight, mobile-first web app that turns physical walking into an evolving generative soundscape:

1 Step = 1 Note: Utilizing your smartphone's internal accelerometer, every physical footstep triggers an immediate, zero-latency synthesizer note.
Every 4 Steps = Harmonic Chord: An anchored bass harmony chord triggers on bar divisions, grounding strides into rhythmic 4/4 musical phrases.
Acoustic Thump: A transient 40Hz acoustic pitch-drop creates a tactile "foot-on-grass" thump with every stride.
Walking = Generative Symphony: As you maintain your pace, speed up, or slow down, an open-weight AI conductor analyzes your cadence, stride consistency, and walk duration every 16 steps to dynamically evolve the key, scale, chord progressions, and acoustic timbre.
AMOLED "Pocket Mode": Once you tap "Start Walk," the UI switches to an ultra-low glare, pure-black (#000000) display with oversized tactile controls and iOS safe-area inset support. Slip the phone into your pocket, plug in your earbuds, and step outside.
Zero Screen Engagement: You don't look at a screen; you listen to your environment and movement. If you stop walking, the music halts. If you pick up your pace, the harmony shifts into bright, driving progressions.
15-Second Walk Compression: When your walk finishes, the app generates a compressed 15-second acoustic replay that condenses your entire physical route into an accelerated musical recap.

Who is it for?

Developers & Students: Tired of desk-bound burnout who want acoustic dopamine to step away from their workstations and touch grass.
Casual Walkers & Commuters: Anyone who wants their stroll, nature walk, or campus transit to feel like an interactive, mindful adventure without staring at notifications.

Demo

Live Production App: https://stepsynth.vercel.app/
Vercel Deployment Build: https://vercel.com/abhishekapril93-sudos-projects/stepsynth/4ZUZxmZoPAqek2s3doWzGeDyELPz

Quick Demo Mode: When running on desktop browsers without an accelerometer, StepSynth includes a Manual Tap Pad (Spacebar) and an Auto-Walk Demo Simulator so judges can test the real-time audio synthesis and AI conductor triggers without walking outside.

Code

GitHub Repository: https://github.com/abhishekapril93/stepsynth

The codebase is built with React 18 + Vite and responsive Vanilla CSS, structured for rapid execution and sub-5ms audio reactivity:

src/audio/AudioEngine.js: Built on the native Web Audio API, managing polyphonic oscillators, envelope generators (ADSR), low-pass resonance filters, and the 40Hz kinetic turf transient.
src/hooks/useStepDetector.js: Connects to the browser's devicemotion / DeviceMotionEvent API, calculating dynamic threshold magnitude peaks ($\sqrt{x^2 + y^2 + z^2}$) with a 280ms refractory lockout to eliminate false double-steps.
src/services/aiConductor.js: Dispatches telemetry every 16 steps to the open-weight model API, parsing structured JSON musical parameters.
src/components/PocketMode.jsx: AMOLED black screen with oversized tactile tap surfaces for zero visual glare while moving outdoors.
src/components/RecapReplay.jsx: Handles the 15-second compressed playback recap of the walk's musical progression.

How I Built It

The Latency Paradox & Solution

If you ask a cloud AI model to generate a note on every step, network latency (300ms–1500ms) completely destroys the rhythmic feedback loop of walking.

To solve this, I designed a two-tier hybrid audio architecture:


text
[ Physical Movement ] ──> Accelerometer (DeviceMotionEvent)
                                 │
                     ┌───────────┴───────────┐
                     ▼                       ▼
            [ Local Web Audio ]      [ Cadence Buffer ]
             0ms Instant Note         Every 16 Steps
             Harmonic Feedback               │
                                             ▼
                                  [ Open-Weight Gemma 2 ]
                                    AI Music Conductor
                                             │
                                             ▼
                                  [ Harmonization JSON ]
                                  Updates Scale & Timbre

DEV Community: Abhishek Kumar