Grading piano timing in the browser with Web MIDI

I picked up piano again this year and wanted to connect my digital piano to my laptop, read real sheet music, and figure out which notes or beats I missed. But none of the apps I tested could do that. So I created Keystair, a browser-based piano course that listens to a USB MIDI keyboard. Most of it is standard React code. A few browser details took me longer than they should have, so here they are.

Reading the keyboard

Web MIDI is a simple API where you request access once, attach handlers to each input, and read three bytes per message. The tricky part is handling the note-off signal; many keyboards skip it and send a note-on with a velocity of 0 instead, so that has to count as releasing the key.

const access = await navigator.requestMIDIAccess({ sysex: false });

for (const input of access.inputs.values()) {
  input.onmidimessage = (e) => {
    if (!e.data || e.data.length < 2) return;
    const [status, a, b = 0] = e.data;
    const type = status & 0xf0;
    if (type === 0x90 || type === 0x80) {
      // Plenty of keyboards send note-on with velocity 0 instead of note-off.
      onNote({ pitch: a, on: type === 0x90 && b > 0, t: e.timeStamp });
    } else if (type === 0xb0 && a === 64) {
      // Control change 64 is the sustain pedal.
      onPedal(b >= 64, e.timeStamp);
    }
  };
}

The sustain pedal appears as control change 64, meaning it's down if its value is 64 or higher. The event's timeStamp runs on the same clock as performance.now(), which matters later on. Output works in reverse: MIDIOutput.send() takes a timestamp too, so the app can play an example at a precise moment on the student's own piano.

Permissions are worth mentioning briefly. If navigator.permissions.query({ name: 'midi' }) indicates that the site already has access, the app reconnects on the next visit without prompting. It also monitors this permission; if someone allows MIDI later via the address bar, they get connected immediately, no reload needed. Chrome, Edge, and Opera have Web MIDI built in, while Firefox requires accepting an add-on prompt. Safari and the browsers on iPhone and iPad don't support it, so there the app falls back to using the microphone.

One clock for everything

The metronome click comes from the Web Audio API, with a clock maintained by an AudioContext: currentTime, in seconds, with no fixed tie to performance.now(). If you schedule clicks based on one clock but judge key presses on another, each grade will be off by whatever delay the speakers introduce. I keep all times in performance.now() milliseconds and convert only when scheduling sounds:

// Audio clock time (s) at which a sound is heard, for a performance.now() time (ms).
function toCtx(ctx: AudioContext, tPerf: number): number {
  const ts = ctx.getOutputTimestamp?.();
  if (ts?.contextTime !== undefined && ts.performanceTime) {
    return ts.contextTime + (tPerf - ts.performanceTime) / 1000;
  }
  // Fallback: the current time, minus the latency the browser reports.
  const lag = (ctx.outputLatency || 0) + (ctx.baseLatency || 0);
  return ctx.currentTime + (tPerf - performance.now()) / 1000 - lag;
}

getOutputTimestamp() aligns the two clocks at the moment a sample actually plays through the speakers, so a click meant for beat 3 is heard right then, not slightly later. Not every browser supports it, which is why there's a fallback.

How close counts as on time

A beginner playing at 60 beats per minute and someone playing eighth notes at 120 can't share the same time window; these windows are determined by the tempo. A note counts if it's within half a beat, up to 400 milliseconds. It's considered on time within 18% of a beat, kept between 70 and 140 milliseconds. Faster notes reduce both, so one note's window never stretches into the next one's. At 60 BPM, that means 400 ms and 140 ms; for eighth notes at 120 BPM, the windows are 125 ms and 75 ms.

// stepBeats is the shortest gap between two notes in the piece, in beats.
function timingWindows(msPerBeat: number, stepBeats = 1) {
  const step = Math.min(1, stepBeats);
  const windowMs = Math.min(400, msPerBeat * 0.5 * step);
  const goodMs = Math.min(140, Math.max(70, msPerBeat * 0.18), windowMs * 0.6);
  return { goodMs, windowMs };
}

Each key press goes to the nearest unmatched note of the same pitch within its time frame, and anything left over counts as an extra note. A run passes if 90% of the notes are played and 70% are on time, allowing for about one extra note per ten notes. Afterward, the app highlights the bars where you rushed or dragged in both color and words, because "early in bar 3" is easier to act on than a red note head.

Measuring the delay

Every setup introduces a delay from pressing a key to the browser recognizing it, with speakers adding even more time. So rather than guess, the settings have a timing check where you tap along with 12 clicks at 80 BPM. The first four are only a count-in, so they're ignored. For the remaining eight, the app finds the closest tap to each click, measures the gap, and saves the median as an offset that later runs subtract. I chose the median so that one missed or double tap wouldn't skew the result.

The key press that doesn't count

This can be easily overlooked: browsers don't start an AudioContext until they receive a user gesture, like a mouse click or keyboard key press, but MIDI notes don't count. If the page just loaded and the only thing the student interacted with was the piano, the context remains suspended, and resume() keeps waiting for a click that doesn't come. No errors appear; there's just no sound.

So the start waits on resume() with a 400 ms timer next to it. If the audio hasn't started by the time the timer goes off, a little message asks the student to click or tap somewhere to turn the sound on. Then once the sound begins, this message disappears.

// The first click or key press anywhere on the page lets the audio start.
const wake = () => {
  void ctx.resume();
  window.removeEventListener('pointerdown', wake);
  window.removeEventListener('keydown', wake);
};
window.addEventListener('pointerdown', wake);
window.addEventListener('keydown', wake);

// Before a timed run: wait for the audio, and ask for a click if it's stuck.
async function ready() {
  if (ctx.state === 'running') return;
  const ask = setTimeout(() => {
    if (ctx.state !== 'running') showSoundNotice();
  }, 400);
  try {
    await ctx.resume();
  } catch {
    // Still blocked. wake() will start it on the next click or key press.
  } finally {
    clearTimeout(ask);
  }
}

ctx.addEventListener('statechange', () => {
  if (ctx.state === 'running') hideSoundNotice();
});

Moving tunes to new keys

In later lessons, students play the same melodies in different keys. Moving a MIDI note a few semitones is simple; what's important for someone reading the staff is how these notes are spelled: an E in C major turns into F# in D major, and it should never end up as Gb. Each note shifts by a number of semitones and a number of letter names at once, with the sharp or flat worked out from the difference:

const LETTERS = 'CDEFGAB';
const NATURAL = [0, 2, 4, 5, 7, 9, 11]; // semitones above C for each letter

// E4 moved up 2 semitones and 1 letter is F#4, never Gb4.
function transposeName(name: string, semitones: number, steps: number): string {
  const m = /^([A-G])(#|b)?(-?\d)$/.exec(name);
  if (!m) throw new Error(`Bad pitch name: ${name}`);
  const pitch = pitchFromName(name) + semitones; // MIDI number, C4 = 60
  const li = (((LETTERS.indexOf(m[1]) + steps) % 7) + 7) % 7;
  let acc = (((pitch - NATURAL[li]) % 12) + 12) % 12;
  if (acc > 6) acc -= 12;
  if (acc < -1 || acc > 1) throw new Error(`${name} would need a double sharp or flat`);
  const octave = (pitch - NATURAL[li] - acc) / 12 - 1;
  return `${LETTERS[li]}${acc === 1 ? '#' : acc === -1 ? 'b' : ''}${octave}`;
}

A note that would need a double sharp or double flat throws an error. The transposed melodies are built when the content module loads, so a bad key choice fails the tests instead of confusing students with unexpected notes.

Keystair is available at keystair.com. Stage 1 is free and you can test it out without signing up, in either English or Spanish. If you've worked on a project using Web MIDI, I'm curious how you dealt with latency.