GhostRunner: Turning Every Street into a Story

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

GhostRunner: An open-source, local-first audio drama engine that turns the real streets under your feet into the set of a thriller.

What I Built

GhostRunner is a generative audio companion for runners and walkers. You pick a genre (zombie escape, cyberpunk courier, spy exfiltration), a distance and an intensity. GhostRunner plans a route over real streets, finds the real landmarks along it, and writes an audio drama in which your surroundings are the set:

"Turn right onto Oak Street. See that old brick building on your left? That's where the drop is hidden."

Then you put your phone away and run. The right line plays at the right place, in a radio-dispatcher voice, just before the turn it describes. If you speed up, the tone turns urgent and a pursuer closes in. If you flag, the story hands you a breather. If you take a wrong turn, the dispatcher notices, re-routes you and rewrites the rest of the mission.

Three things make it more than a prompt wrapped around a map:

• It is spatially exact. Every beat is anchored to a distance along the route, so a line about a turn plays before the turn, not after it.
• It is grounded. The narrator can only name landmarks that exist in OpenStreetMap along your route. A validator rejects anything else.
• It is fast, because nothing is generated while you run. The story is compiled and voiced ahead of you. Playing a line is a lookup, not an inference.

And it runs on a laptop with a 4 GB GPU.

Demo

Video Link

Code

Repository: https://github.com/Nishchal-29/GhostRunner

engine/     routing, chainage math, corridor POIs, tracker, trigger engine, pacing, safety
pipeline/   LangGraph compile_graph and runtime_graph
llm/        Ollama client (structured output)
audio/      TTS providers and clip cache
client/     mobile-first PWA (vanilla JS, Web Audio API, Leaflet)
tools/      LLM benchmark, GPX replayer, demo launcher

Run it locally:

conda activate agents-env
docker compose up -d                 # Valhalla (routing) + Postgres
python tools/setup_valhalla.py
ollama serve
python tools/demo.py                 # compiles a demo loop and starts the app
# open http://localhost:8000

How I Built It

The problem: latency and trust

A story that plays late is wrong, and a story that invents landmarks is worse. A runner moves roughly 3 m/s. An LLM on a small GPU takes seconds per line, and text-to-speech adds more. If you generate reactively, the line about the clocktower arrives after you have passed it. And small models hallucinate confidently.

So I split the system by when work happens:

COMPILE (before and during the run)
Plan → Valhalla route → Overpass corridor POIs → route model (chainage)
     → Director (LLM, 1 call) → Beat planner → Beat writer (LLM, 3 variants/beat)
     → Validator → TTS pre-render (rolling window)

RUN (every tick, no LLM on the happy path)
GPS → pace classifier (TabPFN) → locate on route → trigger check → variant select → play

Two LangGraph graphs: compile_graph carries the heavy work, runtime_graph is a handful of deterministic nodes that finish in milliseconds.

The route is a coordinate system

I use Valhalla with pedestrian costing for routing. It returns an encoded polyline plus maneuvers with street names. I decode it (precision 6), project into the local UTM zone with pyproj, and densify to one point every ~5 m.

Everything on the route is then addressed by chainage: meters along the path. A turn is at 1,240 m. A fountain is at 1,310 m, on the left. This one choice does most of the work in the project:

• Triggers are distance-based, so a pre-compiled beat stays valid at any pace. Pace only selects which variant plays.
• Side-of-route is a cross product in metric coordinates: with tangent T and landmark vector L, the sign of Tx·Ly − Ty·Lx says left or right.
• Runtime snapping searches only [last_chainage − 30 m, last_chainage + 200 m] of the densified path, so loops and out-and-back routes can't snap the runner onto the wrong leg.

Some details I designed around, because they fail silently:

• Valhalla's begin_shape_index is relative to each leg, so multi-leg routes need a running offset or every maneuver lands in the wrong place.
• Street detours make loops overshoot (a triangle at radius D/4 is ~0.93·D before detours). I generate three candidate loops from waypoints at 60°/180°/300°, score them by distance error and self-overlap, and recalibrate the radius once if the first pass is off by more than 15%.

Real landmarks, not imagined ones

One Overpass query along a simplified (≤150 point) buffer of the route pulls historic sites, artwork, fountains, places of worship, named buildings, parks, waterways, bridges and steps. I score each by kind and distance from the path, then apply greedy non-maximum suppression so no two story landmarks sit within 150 m of each other. Results are cached per H3 cell in Postgres, so a second run makes no Overpass calls.

OSM coverage varies. When a route is landmark-poor, the story degrades gracefully into street names, turns, parks and water, and a hand-curated demo_pois.json can add landmarks.

The LLM narrates; it doesn't orchestrate

Small models are unreliable at tool calling, and every tool-call round trip costs latency a runner can't spare. So all decisions are plain Python: which beat goes where, which landmark, what maneuver text, how far ahead to trigger. The LLM does two jobs only:

1. Director (one structured call): produces three acts with chainage ranges, 3-5 objective landmarks (drop, checkpoint, extraction), an antagonist premise, and a story bible (callsigns, motifs, tone).
2. Beat writer: for each beat, writes three variants (cruising, accelerating, exhausted) of at most 30 words, from a structured brief. Each brief carries the story bible, a rolling summary of recent beats, and the landmarks already passed, so lines can call back to earlier ones.

A validator enforces word count, an allowlist of landmark names, and safety rules (no crossing roads, no trespassing). On failure it retries once with the violation named, then falls back to a deterministic template.

On a 4 GB GPU I chose the model by benchmarking rather than reputation: Gemma 2 2B as the baseline against Gemma 4 E2B/E4B and Qwen3 4B, measuring time-to-first-token, tokens per second, and constraint adherence on ten fixed briefs. The winner is [model], at [x] tok/s with [y]% constraint adherence. [fill in from bench_llm.py]

Trigger timing

For each beat, the line should finish as the runner reaches the anchor. The engine dead-reckons position between 3-5 s GPS fixes and fires a beat when:

def due(beat, chainage_est, speed, variant):
    dur = beat.duration_s.get(variant) or est_duration(beat.variants[variant])
    lead_m = max(speed, 0.5) * (dur + 2.0)           # finish speaking as the runner arrives
    start_at = beat.anchor_chainage_m - lead_m
    return start_at <= chainage_est < beat.expires_chainage_m

Durations come from the measured length of the rendered audio clip, so the lead distance is accurate to the clip. A priority queue (navigation > checkpoint > landmark > challenge > tension) plays one beat at a time, enforces a minimum gap between non-navigation beats, and skips expired ones.

Pace awareness with TabPFN

I treat pace as a small tabular classification problem. Each tick builds features: rolling speed over 30 s / 2 min / 5 min, speed delta and variance, grade, time since last stop, and current speed relative to a personal baseline from the first two minutes. TabPFN classifies the runner as CRUISING, ACCELERATING, EXHAUSTED or STOPPED, with a 3-tick hysteresis filter so the story doesn't flip-flop. It runs on CPU, throttled to once every 5 s, leaving all VRAM to the LLM.

One honest caveat: there is no labeled dataset for this, so the training labels come from a transparent heuristic over simulated run profiles. TabPFN's value here is a fast, calibrated, retrain-in-seconds classifier. If it fails to load, a rules classifier takes over automatically.

Voice

ElevenLabs voices every line, rendered ahead of the runner through a rolling window of the next three triggers. Clips are content-addressed (sha256(voice + text)), so repeated lines are free. Reactive moments (off route, pick up the pace, checkpoint countdown) use a pre-rendered tier-C bank that plays from disk in under 50 ms while any dynamic line renders in the background. On the client, Web Audio adds a bandpass-and-distortion radio effect, and the ambient drones and sirens are synthesized procedurally, so there are no audio assets to ship.

Product details that make it feel real

A mobile-first PWA with a briefing screen (route preview, premise, objectives, safety acknowledgement, headphone check), a countdown, a run HUD, and a debrief with a shareable card. Every compiled mission is saved, so any mission replays with the LLM, router and TTS all switched off. Wi-Fi at a venue is the one thing a demo can't count on, so the whole experience can run without it.

Stack

Python, FastAPI, WebSockets, LangGraph (Postgres checkpointer), Valhalla, Overpass/OpenStreetMap, shapely, pyproj, H3, Ollama with Gemma, TabPFN, ElevenLabs, Sentry, Postgres, vanilla JS with the Web Audio API and Leaflet.

Why Does Open Innovation Matter?

GhostRunner is built almost entirely on open pieces, and that is not incidental. It is why the product can exist.

• Open data makes every street a stage. OpenStreetMap is a community-built map of the world. Valhalla and Overpass turn it into routes and landmarks. A proprietary map would have limited what I could query, cache and replay, and would have made the offline-replay mode impossible.
• Open weights make location privacy real. Your GPS trail is among the most sensitive data a phone produces. Because Gemma runs through Ollama on my own machine, the story is written locally, and the app doesn't send your coordinates to a model provider. Raw trails are deleted when a session ends.
• Open weights make the cost structure work. A run generates dozens of lines. With a local model, a 5 km mission costs electricity, not tokens. That is the difference between a toy and something you can offer to anyone.
• Small open models reward good engineering. A 2-4B model won't improvise a perfect story, which pushed me toward a design that is better for it: deterministic structure, grounded briefs, a validator and pre-generation. Constraints produced the architecture.
• Open tooling compounds. LangGraph, TabPFN, Ollama and Valhalla let one person assemble in days what would otherwise be a platform team's job, and anyone can fork GhostRunner to add a genre, a city, or a language.

Prize Categories

Best Use of ElevenLabs. Every line is voiced in a per-genre radio-dispatcher voice, with an optional second voice for the antagonist. Rolling pre-rendering, content-hash caching and measured clip durations are what let the audio land at the right place, and a pre-rendered reactive bank gives sub-50 ms responses to off-route and pace events.

Best Use of TabPFN. A TabPFN classifier turns streaming telemetry features into a runner state (cruising / accelerating / exhausted / stopped) that selects the story variant in real time, with hysteresis and a rules fallback.