DEV Community

leviyi
leviyi

Posted on

Syncing a rhythm game to a YouTube iframe (a metronome on someone else's player)

#webdev #javascript #audio #gamedev

I shipped a little browser game where the backing track is a YouTube embed and the gameplay has to land on the beat. Not "close to the beat". On it, within tens of milliseconds, or the whole thing feels broken.

That sounds simple until you try it. The YouTube IFrame API gives you a video player, not a clock. This post is what broke on the way, and the three small modules that came out of it. The code is all here: yt-beat-sync. No dependencies, take what you need.

Problem 1: whose clock do you trust?

My first version ran game logic on performance.now() and started the YouTube player at the same moment. Two clocks, started together. What could go wrong?

Quite a lot, it turns out:

  • The player buffers. Your clock keeps running. The beat grid slides half a second off the audio and never comes back.
  • The user seeks. Your clock doesn't know.
  • getCurrentTime() is coarse. It updates in chunks and freezes while buffering.

The fix felt backwards: the coarse, freezing, low-resolution player clock has to be the source of truth. Not because it's accurate, but because it's honest. When the video buffers, game time should freeze too, because that's what the player is actually hearing. Any extrapolation you layer on top drifts away from the audio and the beats visibly slide.

So the game reads one clock:

time() {
  if (this.useExternal) {
    const t = this.externalTime && this.externalTime(); // player.getCurrentTime()
    if (t != null) return Math.max(0, t - this.intro);
  }
  if (!this.wallStart) return 0; // not started yet, never return an epoch
  return Date.now() / 1000 - this.wallStart - this.pausedAccum + this.base;
}
Enter fullscreen mode Exit fullscreen mode

The intro offset shifts t=0 to the moment the beat actually drops in the video, so the game logic can think in "seconds since the drop" and ignore however much lead-in the video has.

And when the embed is blocked entirely (some networks and regions do this), the same clock falls back to wall time and the game keeps running. Degraded, but playable.

Problem 2: autoplay policy is a game-design problem

Browsers won't let you play audio without a user gesture. Fair enough. The nasty part is in the details.

An AudioContext starts suspended. You must call resume() inside a gesture handler. Fine. But creating a YouTube player costs a network round-trip: the API script, the iframe, the video. If you start all of that inside the click handler, the player materializes after the gesture window closes, and mobile Safari and Chrome block the sound.

The fix is to preload the API long before you need it:

// on page load, not in the click handler
window.addEventListener('pointerdown', () => YTEngine.preload(), { once: true });
Enter fullscreen mode Exit fullscreen mode

Then the play click only has to do new YT.Player(...) and playVideo(), which fits inside the gesture window.

Android has a secret third state. Granting microphone permission (my game uses the mic) can flip the context to 'interrupted'. Every state === 'suspended' check sails right past it. Your scheduler runs, notes get scheduled, and nothing is heard. Silently. I found this on a live Android test where the beat audio was just gone. Now a wake() covers both states, hooked to onstatechange:

wake() {
  if (this.ctx && this.ctx.state !== 'running') {
    try { const p = this.ctx.resume(); if (p && p.catch) p.catch(() => {}); } catch (e) {}
  }
}
Enter fullscreen mode Exit fullscreen mode

Problem 3: setInterval is not a metronome 

setInterval(playClick, 60000 / bpm); // do not do this
Enter fullscreen mode Exit fullscreen mode

Timers jitter by tens of milliseconds and get clamped to once per second in background tabs. For anything rhythm-critical that's unusable.

The standard answer is a lookahead scheduler: a short-interval timer (40ms) that schedules every beat falling inside a 150ms window, with sound placed on the AudioContext clock, which is sample-accurate:

this.loop = setInterval(() => {
  while (next < wallNow() + 0.15) {
    const at = this.ctx.currentTime + Math.max(0, next - wallNow());
    this.kick(at); // scheduled ahead, sample-accurate
    next += interval;
  }
}, 40);
Enter fullscreen mode Exit fullscreen mode

One wrinkle I didn't expect: the audio clock freezes while the context is suspended. If you drive visuals off the same schedule, the whole metronome starves. So visuals run on wall time. A setTimeout aligned to each beat fires the CSS pulse whether or not audio is currently allowed. Sound catches up when the context resumes; the grid never stops.

That separation ended up being the whole architecture: audio on the audio clock, visuals on wall time, game logic on the player's timeline.

The demo

github.com/dengyu123456/yt-beat-sync is a single page: paste a YouTube URL, set the BPM and the intro length, and a ball bounces on the beat. Buffer the video, seek around, background the tab. The grid follows the player, because everything reads the one honest clock.

Three files, no dependencies: yt-engine.js (API preload, singleton player with loadVideoById reuse, a 12s watchdog for dead embeds), beat-clock.js (the two-source clock), metronome.js (the lookahead scheduler and kick/hat synths).

If you're building anything that syncs to embedded media (karaoke, play-along tabs, rhythm games, video-synced quizzes), steal it.

I use this in production at The Rhyme Game, a free browser game where you practice freestyle rhymes over real beats. The same clock drives the bouncing ball and the bar-by-bar word drops there. If you try the demo on something weird (SoundCloud embeds, livestreams, 7/8 time), I'd be curious what breaks.

Top comments (0)