The home screen of XNeuronal, an Android memory assistant, is a brain. Every note, reminder or contact you dictate becomes a neuron placed inside it, linked to the others by thin curved fibres, drawn with React Native Skia and animated by Reanimated worklets. This week I needed that brain as a vertical video: empty at first, then filling up neuron by neuron, the way graph timelapses of note-taking apps grow, and full exactly on the drop of the music.
Three obvious routes, all wrong. A screen recording shows a real person's memory (out of the question), drops frames, and cannot time 320 births to a beat. A motion design tool means drawing a second brain by hand, which drifts from the app the first time someone moves a lobe. Porting the drawing code to a video library means two copies of the geometry to keep in sync forever.
What shipped is a Node script that imports the app's own geometry, redraws it on a canvas and streams frames to ffmpeg. A 21 second video (630 frames at 1080x1920) renders in about 45 seconds on a laptop. Here is how it is wired, and the details that took the most thought. Every excerpt is copied from the repository and lightly trimmed.
The seam that already existed: geometry with no Skia in it
The brain on the phone lives in two kinds of files. brainModel.ts knows where things are: the shape of the brain as a dozen ellipsoids, where each neuron sits, the camera, the projection. brainDraw.ts knows how to paint: Skia paths, drawAtlas stamps, paints, blend modes. Only the second one imports @shopify/react-native-skia.
That split was made a week earlier for a different reason: tests. The model's header says it is "pure and free of Skia, so jest pins the geometry", and the specs do exactly that ("is stable: the same neuron always lands on the same spot", "keeps the whole brain inside the square", "gives a depth between 0 (far) and 1 (near)"). A neuron's spot comes from a hash of its id, so nothing is simulated and nothing moves between two openings of the app.
The projection is typical of what lives on the pure side:
// frontend/src/components/orb/brainModel.ts
export function projectInto(out: Float32Array, offset: number, x: number, y: number, z: number, cam: BrainCamera): void {
'worklet';
const x0 = x - BRAIN_CENTER[0];
const y0 = y - BRAIN_CENTER[1];
const z0 = z - BRAIN_CENTER[2];
const x1 = x0 * cam.cy + z0 * cam.sy;
const z1 = -x0 * cam.sy + z0 * cam.cy;
const y2 = y0 * cam.cp - z1 * cam.sp;
const z2 = y0 * cam.sp + z1 * cam.cp;
const persp = cam.d / (cam.d - z2);
out[offset] = cam.ox + x1 * persp * cam.scale;
out[offset + 1] = cam.oy + y2 * persp * cam.scale;
out[offset + 2] = persp;
out[offset + 3] = Math.min(1, Math.max(0, (z2 / BRAIN_EXTENT + 1.2) / 2.4));
}
Two things make it portable. It takes numbers and writes numbers into a buffer, with no Skia type in sight. And the 'worklet' directive is only a string literal: the Reanimated Babel plugin reads it to ship the function to the UI thread, and every other runtime evaluates it as an expression statement that does nothing. The same function runs in a worklet on the phone and in plain Node.
Bundling app code for Node: one entry, one alias
The video tool lives in a marketing folder, outside the app, and copies nothing. An entry file re-exports what it needs straight from the frontend sources, and esbuild bundles it with react-native aliased to a stub:
// brain/entry.ts (trimmed)
export { brainTargetOf, buildCortex, brainCamera, projectInto } from '../../../../frontend/src/components/orb/brainModel';
export { shapeCodeFor, edgeColorFor, nodeAnimAt } from '../../../../frontend/src/components/orb/constellationScene';
export { palettes } from '../../../../frontend/src/theme/colors';
# brain/build-model.sh (trimmed)
esbuild entry.ts --bundle --platform=node --format=esm \
--outfile=.build/brain.mjs --alias:react-native=./rn-stub.mjs
// brain/rn-stub.mjs
export const Appearance = { getColorScheme: () => 'dark', addChangeListener: () => ({ remove() {} }) };
export const NativeModules = {};
export const Platform = { OS: 'android', select: (o) => o.android ?? o.default };
export default { Appearance, NativeModules, Platform };
Why a stub at all, when the geometry is pure? Because of the palette. theme/colors.ts does work at import time: it imports Appearance and NativeModules from react-native, asks a native module which theme the user picked, and computes isDarkTheme before anyone calls anything. Node cannot load react-native, so without the alias the bundle dies on the first color. With it, the native theme module reads as absent, the mode falls back to "system", the system answers "dark", and the module evaluates cleanly. The renderer then asks for palettes.dark explicitly anyway: the video is the night version of the screen.
esbuild only follows what the entry imports, so the output is a 14 KB ESM file built in about a second. The renderer rebuilds it when it is missing, and the tool's README says to rerun it after any change to the brain in the app. Nothing in the video tool knows the coordinates of a lobe.
Same primitives, different draw calls
The video does not reuse brainDraw.ts, and should not. It is written around Skia objects and around a frame budget the video does not have. On the phone, the neuron cores are stamped from one sprite sheet with two drawAtlas calls for the whole brain, because drawing shapes one by one made turning the brain stutter. Offline, a frame can take 70 ms and nobody notices, so the renderer draws each neuron with plain canvas paths (@napi-rs/canvas).
What is shared is the numbers. The size of a silhouette against its white bead comes from the app's shapeRatio, the color of a fibre from edgeColorFor, the silhouette of each kind of note from shapeCodeFor, the color of each sphere of life (family, work, friends) from sphereColor. And a fibre bends by the same formula on both sides:
// app, brainDraw.ts (Skia)
f.path.quadTo((p[a] + p[b]) / 2 - dy * f.bends[e], (p[a + 1] + p[b + 1]) / 2 + dx * f.bends[e], p[b], p[b + 1]);
// video, render-brain.mjs (canvas)
ctx.quadraticCurveTo((a[0] + b[0]) / 2 - dy * bend, (a[1] + b[1]) / 2 + dx * bend, b[0], b[1]);
One difference is deliberate: the dust that outlines the cortex is drawn a little larger and brighter in the video. H.264 at a sane bitrate smears one-pixel dots into the background, and the comment in the renderer says so plainly: "a video is compressed, the dust must survive it".
Birth: a damped spring from the parent neuron
On the phone, the brain is already full when you look at it. The video had to invent the moment a neuron appears. The memory it shows is fictional (a seeded generator, no real person's data), and each newborn gets a "mother": the older neuron it links to first. It appears on her and springs to its own place, pulling its fibre behind it.
// motion/lib.mjs
export function spring(t, { freq = 2.2, damp = 6 } = {}) {
if (t <= 0) return 0;
return 1 - Math.exp(-damp * t) * Math.cos(2 * Math.PI * freq * t);
}
// brain/render-brain.mjs, inside posAt(i, t, cache)
const u = (t - born[i]) / FLIGHT;
if (u < 1.6) {
const from = posAt(m, t, cache);
const s = spring(Math.max(0, u) * 1.25, { freq: 1.05, damp: 4.2 });
p = [lerp(from[0], T[0], s), lerp(from[1], T[1], s), lerp(from[2], T[2], s)];
}
The recursion is the point. Births accelerate as the memory grows (the birth times follow (i / (N - 1)) ** 0.62), so a child is often born while its own mother is still in flight. It leaves from where she is on this frame, not from where she will land, and three generations unfold like a cell dividing twice. The cache is per frame, so each position is computed once however deep the chain. The spring overshoots once and settles, which reads as alive in a way an ease-out curve does not.
The camera that never zooms back in
The video opens close on the first two neurons and draws back as the memory spreads. Each frame, the camera fits the neurons born so far: their centroid, the farthest one from it, a zoom between 1 and 3.2.
zoom = clamp((BOX * 0.3) / Math.max(reach, 1), 1, 3.2);
// Once the memory spreads, the whole brain comes into view, centred.
const w = clamp((3.2 - zoom) / 1.6);
const wantX = -cx * zoom * (1 - w);
const wantY = -cy * zoom * (1 - w);
const dt = camAt < 0 ? 1 : Math.min(1, (t - camAt) / 0.9);
camZoom = camAt < 0 ? zoom : lerp(camZoom, Math.min(camZoom, zoom), dt); // never zooms back in
Without that Math.min, the camera breathes. When a burst of neurons lands in one dense region, the centroid slides toward it, the farthest neuron can end up closer to the new centroid than the old one was, and the fitted zoom goes up for a few frames: on screen, the brain lunges at you. A growing memory should only ever widen the view, so the zoom is a ratchet. It eases out, never in. The pan still follows freely, and as the zoom nears 1 the weight w pulls it back to the center.
The ratchet has a price: the camera is state, eased from one frame to the next, so frames must be rendered in order. The contact-sheet mode, which checks a timing in half a minute instead of encoding a video, still runs every frame and keeps a dozen. Everything else in the scene is a pure function of time. The impulses that travel along the fibres come straight from the app's brainPulses.ts, whose header says it is "stateless on purpose": which link a pulse takes and how bright it shines depend only on the clock. That is the property an offline renderer wants, and the camera is the one place I traded it away knowingly.
Landing on the beat
The brain should be full on the drop, then flare on every beat. Asking for "the drop at 15 s" is not precise enough: the drop is located on 0.25 s energy windows, a beat at a house tempo lasts under half a second, and a flash between two beats looks late.
motion/beats.mjs finds the grid. ffmpeg decodes the cut to mono floats at 11,025 Hz; the script takes a log energy every 256 samples (about 23 ms), keeps only the rises (an onset envelope), scores every tempo from 70 to 170 BPM by autocorrelation, then slides a phase across one period to find the offset that collects the most onsets. The renderer then snaps the requested drop to that grid:
const { bpm, beats } = analyseBeats(CUT);
const asked = DROP;
DROP = beats.reduce((best, b) => (Math.abs(b - asked) < Math.abs(best - asked) ? b : best), beats[0]);
BEATS = beats.filter((b) => b >= DROP - 0.01);
The last birth is scheduled 0.1 s before the snapped drop. On the drop the whole brain lights up and fades with a 0.55 s time constant; on each later beat, neurons and fibres swell with Math.exp(-(t - b) / 0.16). No keyframes anywhere, just functions of t.
Frames out: rawvideo through a pipe
No PNG sequence on disk. Each frame's RGBA buffer goes straight into ffmpeg's stdin:
const ff = spawn("ffmpeg", [
"-f", "rawvideo", "-pix_fmt", "rgba", "-s", `${W}x${H}`, "-r", String(FPS), "-i", "-",
"-vf", "scale=out_color_matrix=bt709:out_range=tv,format=yuv420p",
"-c:v", "libx264", "-preset", "slow", "-crf", "19", "-maxrate", "14M", "-bufsize", "28M",
"-colorspace", "bt709", "-color_primaries", "bt709", "-color_trc", "bt709",
"-movflags", "+faststart", silent,
], { stdio: ["pipe", "inherit", "inherit"] });
for (let f = 0; f < frames; f++) {
drawFrame(ctx, f / FPS);
const raw = canvas.data();
if (!ff.stdin.write(Buffer.from(raw.buffer, raw.byteOffset, raw.byteLength))) await new Promise((r) => ff.stdin.once("drain", r));
}
Three details. Waiting for drain is the backpressure: without it, Node queues 8 MB frames in memory faster than x264 on the slow preset can eat them. The scale filter converts RGBA to yuv420p with an explicit BT.709 matrix, and the three color flags tag the file the same way, so a player never has to guess the color space and the app's night blue stays that blue. And the audio is a second pass: the video is encoded silent, then muxed with the music cut, faded and normalized to -14 LUFS, the video stream copied rather than re-encoded.
What I would keep from this
The video was cheap because the drawing code had already been split, for tests, into geometry and painting. Positions, projection and time-based animation as pure functions of their inputs; every platform call in one painting layer. Any renderer can host the first half: Skia on a phone, a canvas in Node, a test runner.
The one trap was a module that did work at import time. A palette that asks a native module for the user's theme the moment it is imported is fine in an app and fatal in Node. Five lines of stub fixed it, but the better habit is to keep import time boring in any file you might one day want to run somewhere else.
The brain this video grows is the one on the home screen of the Android app, at xneuronal.com.
Top comments (0)