Our reel has no licensed audio, because 130 lines of Node synthesise every sound in it

We built a 26 second vertical reel for Nakodo in Remotion, which means the whole video is a React component tree rendered frame by frame. The pictures were the part I expected to be hard. The sounds were the part that actually stopped me.

The reel needs almost nothing: our mascot is a woodpecker, and every sound in the finished video is one of his pecks. A knock when he hits wood, a click when he presses something on a screen. Nine knocks in the tree at the start, a knock and five clicks through the middle, two clicks at the end.

Licensing seventeen taps turned out to be more paperwork than writing them. Free sound libraries come with attribution requirements that do not fit in an Instagram caption, per-platform restrictions, and the standing question of whether "royalty free" covers an advert. So scripts/sfx.mjs makes them instead: 130 lines, no dependencies, writes 16-bit mono WAVs into public/sfx.

A WAV is a 44 byte header and some integers

There is no library here. A mono 16-bit PCM WAV is the RIFF header, then one signed little-endian integer per sample:

const pcm = Buffer.alloc(44 + data.length * 2);
pcm.write("RIFF", 0);
pcm.writeUInt32LE(36 + data.length * 2, 4);
pcm.write("WAVEfmt ", 8);
pcm.writeUInt32LE(16, 16);
pcm.writeUInt16LE(1, 20);   // PCM
pcm.writeUInt16LE(1, 22);   // mono
pcm.writeUInt32LE(RATE, 24);
pcm.writeUInt32LE(RATE * 2, 28);
pcm.writeUInt16LE(2, 32);
pcm.writeUInt16LE(16, 34);
pcm.write("data", 36);
pcm.writeUInt32LE(data.length * 2, 40);

knock.wav is 10,628 bytes, which is the header plus 0.12 seconds at 44.1 kHz times two bytes a sample. You can check that arithmetic against the file, which is a nice property for a build artefact to have.

The two details that matter in the writer are both about not sounding broken. Every file is peak normalised before the gain is applied, so one loud sound does not make the others inaudible, and every file gets a 2 ms fade at each end:

const fade = Math.round(0.002 * RATE);
data.forEach((v, i) => {
  const edge = Math.min(1, i / fade, (data.length - 1 - i) / fade);
  pcm.writeInt16LE(Math.round((v / peak) * gain * edge * 32767), 44 + i * 2);
});

A waveform that starts or ends at a non-zero sample is a step change, and a step change is a click. You get a free unwanted percussion hit on top of your deliberate one. Two milliseconds is inaudible and removes it.

A knock is two decaying sines and a scratch of noise

Struck and plucked sounds are mostly envelope. Pick a couple of frequencies, multiply by a falling exponential, add a very short burst of noise for the impact:

write("knock", render(0.12, (t) => {
  const body = Math.sin(tau * 820 * t) * Math.exp(-t / 0.018)
             + 0.6 * Math.sin(tau * 310 * t) * Math.exp(-t / 0.03);
  return body + noise() * Math.exp(-t / 0.004) * 0.8;
}));

820 Hz decaying in 18 ms over a 310 Hz body decaying in 30 ms reads as wood: bright, short, a bit hollow. The noise term lives for about 4 ms and is the sound of the two things touching. The UI click is the same shape moved up to 1900 Hz with a tiny 160 Hz thud under it, and 60 ms instead of 120.

Noise-based sounds need filtering, which is also arithmetic. A one-pole lowpass is one line in a loop, and a highpass is the signal minus its own lowpass:

const highpass = (data, cutoff) => {
  const low = lowpass(data, cutoff);
  return data.map((v, i) => v - low[i]);
};

A whoosh is noise swept up and back down through that lowpass with a sin² amplitude window. Rushing water is low noise that swells over half a second and decays. Both took about four lines each.

It regenerates byte for byte

The noise source is a seeded Lehmer generator, not Math.random:

let seed = 7;
const noise = () => {
  seed = (seed * 16807) % 2147483647;
  return (seed / 2147483647) * 2 - 1;
};

The eight files are committed, which usually means binary churn every time anyone touches the script. Here it does not, because the output is a pure function of the source. I checksummed the committed files, re-ran node scripts/sfx.mjs, and checksummed again:

$ shasum -a 256 public/sfx/*.wav > before.txt
$ node scripts/sfx.mjs
Wrote knock.wav, pop.wav, click.wav, tap.wav, whoosh.wav, rush.wav, thump.wav, chime.wav
$ shasum -a 256 public/sfx/*.wav | diff before.txt -
$

Identical, and git status stays clean. One caveat worth knowing if you copy this: there is a single generator shared by every file, consumed in call order, so the determinism depends on the files being written in the same order every time. Reordering the write calls changes the noise in all of them. That is a trade I took knowingly, because one seeded source is three lines and per-file seeds would be eight constants to keep straight.

The cues come from the animation, not from a timeline

This is the part that would be awkward in a video editor and is trivial here. The frames Alcy's bill actually strikes are already exported by the scenes that animate him:

export const PECKS_1 = [81, 85, 89];
export const PECKS_2 = [109, 112, 115, 118, 121, 124];

The soundtrack imports those arrays and maps each one to a <Sequence> holding an <Audio>:

const CUES: Cue[] = [
  ...PECKS_1.map((p, i): Cue => ({ at: p, sound: "knock", volume: 0.9, rate: 1 + i * 0.04 })),
  ...PECKS_2.map((p, i): Cue => ({ at: p, sound: "knock", volume: 0.85, rate: 1.05 + i * 0.03 })),
  ...
];

The same two arrays also drive the lean of his head and how many icons shake loose from the tree on each hit. Audio, pose and physics all read one source of truth, so moving a peck two frames later moves everything that belongs to it. There is no sound to re-nudge in a timeline, because there is no timeline.

rate is a playbackRate on the audio, which pitches each repeat a few percent up from the last. Three identical knocks sound like a loop; three knocks rising 4% at a time sound like a bird getting into it. One expression, and the thing people notice.

Eight sounds, two survived

The script makes knock, pop, click, tap, whoosh, rush, thump and chime. The finished reel uses two. The waterfall scene had the rushing water under it for an afternoon and was better silent: the picture already says water, and the noise made a 26 second video feel busy. The chime on the yes was worse than nothing for the same reason.

I kept all eight in the repo anyway. They are 316 KB of WAV against an 8.6 MB video, they cost one command to regenerate, and the next reel may want a thump. Deleting them would save nothing except the ability to change my mind.

The drawings in the reel are not stock either: they are the app's own illustration components, imported from src/components/illo, the same ones behind the bird on the landing page and the tree on the sign-in page. That part is the next post.