Footsteps: What If Your Audio Story Could Only Move When You Do?

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

What if your next audio thriller could only move forward when you did?

Footsteps is an experimental, location-aware audio mystery that turns an ordinary walk into an interactive story. Instead of staring at a screen to progress through a game, you put your phone in your pocket, wear your earbuds, and step outside.

The idea is simple: the world around you becomes part of the narrative.

Footsteps uses real nearby landmarks to ground its story in your surroundings. As you walk or jog, the story adapts to your pace, creating a mystery that unfolds along your route. The goal is to make movement part of the experience rather than another activity tracked on a screen.

It's designed for people who enjoy immersive storytelling but spend too much time looking at their devices. Your neighborhood becomes the setting, your movement becomes the trigger, and the next chapter is something you experience rather than watch.

The central principle is that the story should respond to the real world, not invent one.

Demo

The project is organized around a lightweight FastAPI backend and a mobile-first Progressive Web App. The backend handles location processing, pace detection, landmark selection, story generation, and session data. The frontend handles GPS updates, audio playback, and the session recap.

The code is designed to keep location geometry and story generation separate, making the system easier to test and the model easier to replace.

How I Built It

Footsteps combines conventional programming with open-source AI. Each component has a specific responsibility.

• Gemma through Ollama: Generates short mystery narration from structured session data.
• OpenStreetMap and Overpass API: Supplies real nearby landmarks without requiring a paid maps API.
• Python and FastAPI: Handle GPS pings, pace detection, geometry, story state, and backend endpoints.
• Piper / browser speech synthesis: Provides spoken narration, with browser speech synthesis as the fallback when server-side audio is unavailable.
• Tiger Data / TimescaleDB: Provides time-series storage for movement points and story beats, with SQL-based session summaries.
• Sentry: Provides tracing and debugging visibility across the story-generation pipeline.
• Render: Hosts the web application and backend.

One architectural decision matters more than the number of technologies involved: code finds the facts; the language model writes the words.

The backend calculates distances and bearings, filters nearby landmarks, and constructs a structured state object. Gemma receives that data and writes the next beat. Its response must pass validation before narration, with a retry and deterministic fallback for invalid output.

This separation makes the experience more grounded and gives the application explicit control over safety rules.

Why Does Open Innovation Matter?

Footsteps uses location and movement data to create a personal experience. That makes openness and control important design considerations, not just technology preferences.

Open-weight models make it possible to run inference on infrastructure I control rather than making a closed, hosted language model the central dependency. Ollama also makes the model configurable, so a compatible model can be changed through an environment variable.

OpenStreetMap provides a community-maintained source of real-world geographic data. Open-source speech tools offer another route to narration without requiring a proprietary voice-generation API.

Open infrastructure also makes the project easier to inspect, adapt, and extend. Developers can examine how landmarks are selected, how directions are calculated, how model output is validated, and how movement metrics are derived.

My goal is to keep the application transparent about what it knows, what the model generates, and what happens when a component fails.

Location privacy and safety remain essential: stored coordinates should be coarsened, sessions should be deletable, and narration must never encourage dangerous movement or pressure someone to continue exercising.

My Agent Session

[Optional: Add your DevRelay agent-session link or embed your saved session here.]

I used an AI coding agent to accelerate implementation and help assemble the project under hackathon time constraints. The engineering challenge is not simply generating code; it is verifying that the individual components work together and that the resulting experience behaves reliably outside a development environment.

Prize Categories

• Gemma: Open-weight language model for grounded story generation.
• Tiger Data: Time-series persistence and movement analytics.
• Sentry: Agent tracing across the narration pipeline.
• Render: Application hosting and deployment.