Two to four tablets sit on plinths in a gallery. Each one is listening: a contact mic on the floor, an electromagnetic pickup near the wall, a shortwave radio scanning. A black-hole visualization fills the screen, slowly accreting whatever the room is making. Visitors swipe a circular slider and release short bursts back into the system. The bursts are fragments of the captured sound, transformed in flight by synthesizers and effects they unlock by spending time with the piece.
This is Past Tense, an idle game that's also an electroacoustic composition. (Or an electroacoustic composition that's also an idle game; the framing depends on who's asking.) I built it over the last two years on top of DragonRuby GTK, with Pure Data driving every audio sample, packaged as a single iOS app. It's been accepted to the ICMC 2026 installation showcase.
This is the first of six posts on building it. The artistic decisions came first; the engineering took six months. I want to write the engineering up because the stack is unusual, the documentation is sparse, and a few of the problems I solved would have been days shorter if someone else had written the answer down. So this is that writeup.
What it is, in one paragraph for the non-art people
A visitor walks up. The tablet is already running, has been running, will keep running. Ambient sound has been accumulating for hours. The visitor swipes the slider, releases a burst, sees and hears the burst stutter through the effects chain, and walks away. They've contributed a few seconds of intentional input to a process that runs whether they're there or not. The premise is closer to Alvin Lucier's Music for Solo Performer than to Guitar Hero: visitors nudge a self-running system rather than play it. The reference points on the game side are A Dark Room and Orb of Creation, idle games where progress accumulates with or without you, and Olli Tapio Leino's writing on "boring games."
In a racing game or a platformer, the gameness predominates: you're pulled into flow by tight feedback loops and reflexive rewards. In a boring game, the gameness recedes and what's left is the system's worldness. The absence of an objective becomes the invitation: you have to make sense of the world and decide what it means yourself. Past Tense is built on that invitation.
The technical stack
The whole thing is three runtimes glued together. DragonRuby GTK (mRuby) handles the game side: scenes, UI, sprite rendering, the per-tick game loop, the XP and tier-progression system. Pure Data, embedded via libpd, handles every audio sample: spectral analysis across four frequency bands, burst recording, the synthesis and effects chain, the feedback routing. A small custom C extension bridges the two via thread-safe ring buffers, with miniaudio doing cross-platform device I/O. The bridge is small because the contract between Ruby and Pd is small: numbers in, numbers out.
Three layers, each doing what it's best at: Ruby for state and scenes, Pd for samples, C for the bridge.
What a Pure Data patch looks like, in case you've never seen one
A Pd patch is a graph of operators ("objects") connected by wires. Audio-rate objects have a tilde in their name: osc~, dac~, *~. Control-rate objects don't. The smallest useful patch that talks to Ruby looks roughly like this:
The Ruby side talks to it like this:
$pd.send_float(to_receiver: "freq", value: 440)
That's the whole contract in one direction. Ruby pushes a named value into the patch. The patch routes it through the [r freq] (receive) object to whatever it's wired to. The other direction works the same way: tap a signal with [env~], send the result through [s level], and Ruby gets it back.
$pd.subscribe("level")
def receive_float(args)
receiver, value = args
puts "level: #{value}" if receiver == "level"
end
Most of Past Tense's ~500 lines of Pd graph is variations on this pattern at scale: one patch per band, four bands, dozens of named send/receive buses connecting Ruby's tier-progression state to live DSP parameters.
How the layers fit together
Each tick, Ruby decides what's happening (a slider moved, a tier was unlocked, a parameter needs a new value) and pushes a number into a named Pd receiver:
$pd.send_float(to_receiver: "envelope_type_256", value: 0.05)
$pd.send_float(to_receiver: "cutoff_256", value: 2000)
Pd performs the DSP at audio rate and pushes analysis results back through subscribed receivers:
$pd.subscribe("env_256") # output envelope per band
$pd.subscribe("input_env") # mic input level
$pd.subscribe("bursts_256") # [energy, index] when a burst fires
Ruby drains those each tick with $pd.poll. The receive_float and receive_list callbacks dispatch on the receiver name, the same way as the toy example above, just with a switch on the receiver prefix to route to game state. Ruby never touches a sample. Pd never knows about scenes or sprites. The C extension provides one thread-safe channel in each direction. Post 2 in this series goes deep on the dr-libpd internals: the FFI surface, the ring buffer between Pd's audio thread and Ruby's game loop, and how poll keeps the two clocks from running into each other.
Why Pure Data, and not the obvious alternatives
Before settling on libpd I briefly considered three alternatives.
SuperCollider has a longer DSP feature list and a more powerful language. The dealbreaker was deployment: scsynth is a separate process. Shipping a game app that has to spawn and supervise another OS process, on iOS, with sandboxing and lifecycle quirks on top, was more friction than I wanted. libpd, by contrast, runs embedded in the game process.
Csound is mature and famously precise, but the orchestra/score authoring model is verbose for live tuning. Pure Data's visual patches let me change a delay time, save the file, and hear the difference in the running app within seconds. That iteration speed mattered more than Csound's depth.
Faust was the closest call. It's a DSP language that compiles to C++ (or wasm, or LLVM), elegant, fast, and has reasonable live-coding paths via the faust2* scripts. The catch is its library catalog. Pure Data has decades of community-built abstractions for the kind of instrument-flavored building blocks this piece leans on. Faust has fewer of these prebuilt. The choice came down to "find or build what I need in Faust" vs "wire up things that already exist in Pd." Pd won.
One distinction matters for understanding why libpd specifically: in Past Tense, bursts are samples, but everything else is live. Recorded bursts are short WAV files written by Pd's soundfiler and read back during playback. Everything that processes them on the way out is generated in real time. Without live DSP, the piece would collapse into a sample player with effects, which is a different work. libpd is what makes the live half practical.
Where it actually runs
Each gallery station is a self-contained audio context: it accumulates and plays back its own sound material with no digital connection to the others. The only mixing happens acoustically, in the room, as separate stereo outputs overlap. A station needs a tablet, a class-compliant USB audio interface (one input with optional 48V phantom power, two outputs), a stereo loudspeaker pair pointed into the shared space, and one of the venue-dependent input sources (geophone, EM pickup, radio, or mic). The footprint is roughly 2-3 m² per station, with stations spatially separated rather than clustered. The composition runs continuously during opening hours; it has no fixed start or end.
What's coming in the rest of the series
Post 2 unpacks dr-libpd itself: ring-buffer architecture, FFI surface, and how send_float and receive_float cross the audio/game thread boundary safely.
Post 3 walks through the actual patch graph and the game-side accumulators that turn audio energy into XP and tier unlocks.
Post 4 is the audio-backend journey before miniaudio won: SDL2, OpenSL, and portaudio each lost in different ways, and DragonRuby 7 with SDL3 may rewrite the picture.
Post 5 covers the iOS port: a build that compiled, signed, launched, and silently failed to register its Ruby module, plus MA_NO_RUNTIME_LINKING, mic permissions, and audio session routing for speakers and Bluetooth.
Post 6 is the iOS sandbox vs Pure Data: writing files from a read-only bundle, plus a Pd subtlety that bit me twice about why msg set does not re-parse $n tokens at runtime.
Links go in as the posts go up.
For the artistic context behind the piece, the project page has the longer-form thinking.
Top comments (0)