I Built a Rhythm Game That Lives Above My IDE

#ai

1. Introduction

Every year, I pick up new hobbies. This year: drums.

When I was watching Claude Code flibbertigibbeting, I thought, "why don't I build a game to practice my rhythm skills?"

So I built a rhythm game in Electron that floats transparently above my IDE. When Claude Code is thinking, I hit F and J to play low and high hits against whatever song is loaded. When it responds, I go back to coding. No context switch, no separate window. The game is just there, above everything else, always.

2. What I Built

The game loads any audio file, analyzes it offline to find where the low and high hits land, then spawns hit targets that scroll toward two drum pads synced to the song's playback position. You hit F for low hits, J for high hits. Timing accuracy scores each hit.

Loads any audio file: MP3, WAV, OGG, M4A, AAC, FLAC
Analyzes beats offline: runs onset detection across the whole file before playback starts
Five visual themes: Lime, Classic, Forest, Neon, Dusk
BPM auto-detection: estimates tempo from detected low-hit intervals

3. How to Run

git clone https://github.com/asakohayase/drum-overlay.git
cd drum-overlay
npm install
npm start

Controls:

F: low hits
J: high hits
Space: play / pause song
Cmd+Shift+Q: quit

4. What is an Electron Overlay?

A browser tab can't float above other apps. It lives inside the browser window, so you'd have to alt-tab to use it, which defeats the whole point. You need OS-level window control.

Electron gives you that. It's normally used to build standalone desktop apps. VS Code, Claude Desktop, Slack are all Electron. Three window flags make the overlay possible:

transparent: true removes the default white window background Electron adds. Without it, it looks like this:

alwaysOnTop: true keeps the window above all other windows system-wide. It doesn't lose its position when you click on something else.

setIgnoreMouseEvents(true, { forward: true }) without this, you cannot click your IDE. The window covers the full screen, so it would intercept every click. This flag passes clicks through to whatever's underneath, while still telling the overlay where your cursor is. When it enters the panel, the overlay temporarily becomes clickable. When it leaves, clicks pass through again.

win = new BrowserWindow({
  transparent: true,
  frame: false,
  alwaysOnTop: true,
  webPreferences: { preload: path.join(__dirname, 'preload.js') }
});
win.setIgnoreMouseEvents(true, { forward: true });

The renderer toggles interactivity dynamically based on what the cursor is over:

document.addEventListener('mousemove', (e) => {
  const over = e.target.closest('.pad, .icon-btn, .play-btn, .progress-bar');
  ipcRenderer.send('set-ignore-mouse', !over);
});

5. Architecture

Web Audio API (built-in):

OfflineAudioContext: runs the full analysis pass before playback starts. Some libraries only offer real-time analysis, which is too late to pre-populate the note lane.
createBiquadFilter: applies lowpass/bandpass frequency filters
decodeAudioData: decodes MP3/WAV/etc into raw samples

Custom code built on top:

detectOnsets: finds low-hit and high-hit timestamps from the filtered audio
estimateBPM: estimates tempo from low-hit intervals
playKick / playSnare: synthesized drum sounds
drawFrame: game loop, note scrolling, hit detection, scoring

Audio file
    │
    ▼
decodeAudioData()
    │
    ├─ detectOnsets(lowpass,  100Hz)  → lowTimes[]
    └─ detectOnsets(bandpass, 2500Hz) → highTimes[]
    │
    ▼
estimateBPM(lowTimes) → bpm
    │
    ▼
Game loop (requestAnimationFrame)
    ├─ Spawn notes from lowTimes/highTimes ahead of currentTime
    ├─ Scroll notes toward hit zone
    └─ Score hit on keydown (F=kick, J=snare)
    │
    ▼
playKick() / playSnare() on hit

Onset detection

DSP (Digital Signal Processing) is math applied to audio signals: filtering frequencies, measuring energy, finding patterns in waveforms.

The naive approach is to threshold amplitude: find frames above a loudness cutoff. This fails on any real track because overall loudness varies constantly. A quiet verse and a loud chorus have completely different amplitude ranges.

The insight: drum hits are transients, sharp sudden attacks, not just loud frames. A low hit is a sudden spike in bass energy that decays in under half a second. What distinguishes it isn't loudness. It's a sharp increase in energy. So instead of thresholding energy, threshold the first difference of energy.

// RMS energy in 10ms windows, 5ms hop
const energy = new Float32Array(nFrames);
for (let i = 0; i < nFrames; i++) {
  const s = i * hop;
  let e = 0;
  for (let j = 0; j < win; j++) e += raw[s + j] ** 2;
  energy[i] = Math.sqrt(e / win);
}

// Half-wave rectified first difference: energy increases only
const strength = new Float32Array(nFrames);
for (let i = 1; i < nFrames; i++) {
  strength[i] = Math.max(0, energy[i] - energy[i - 1]);
}

Percentile threshold over mean+std: "the top 3% of energy spikes count as onsets." It adapts to each song automatically, regardless of the noise floor.

const positives = [...strength].filter(v => v > 0).sort((a, b) => a - b);
const threshold = positives[Math.floor(positives.length * 0.97)];

Local maxima above the threshold with a 220ms minimum gap prevent catching echoes. Without this, a hit's ring-out produces a secondary spike that gets detected as a second note. Low and high hits separate by frequency: lowpass at 100Hz captures bass-range hits (kick, tom, bass); bandpass at 2500Hz captures treble-range hits (hi-hat, cymbals, snare crack). OfflineAudioContext applies these filters and renders faster than real-time.

const [lowTimes, highTimes] = await Promise.all([
  detectOnsets(buf, 'lowpass',  100,  1.0, 0.22, 0.98),
  detectOnsets(buf, 'bandpass', 2500, 1.5, 0.22, 0.97),
]);

Sound synthesis

A kick is a sine wave sweeping from 160Hz down to near-zero over 450ms (the body) plus a 20ms square-wave burst at 900Hz (the click attack).

function playKick() {
  const c = ctx();
  const osc = c.createOscillator();
  osc.type = 'sine';
  osc.frequency.setValueAtTime(160, c.currentTime);
  osc.frequency.exponentialRampToValueAtTime(0.001, c.currentTime + 0.45);
  // gain envelope, connect to destination...
}

A snare is white noise (Math.random() into a buffer) filtered through a bandpass at 2200Hz plus a short triangle-wave tone sweep for the crack.

6. Key Learnings

OfflineAudioContext for pre-analysis. To render the note lane, all hit timestamps need to be known before playback starts. OfflineAudioContext runs the full analysis pass upfront. Real-time analysis would only surface hits as the song plays, too late to populate the lane.
Percentile threshold over mean+std. Tracks with heavy cymbal wash raise the noise floor and collapse mean+std thresholds. Percentile threshold only cares about relative spike height within the track.
Onset detection parameters needed tuning. The minimum gap between onsets and the percentile threshold both took a few iterations to feel right. Too permissive and you catch echoes; too strict and real hits get dropped.

7. Conclusion

AI thinking pauses are dead time by default. They don't have to be. Build something creative, build it for yourself, and you might end up with more than you expected.