From animated HTML+CSS to video frames, in the browser

I am building a captions editor that runs in the browser. Captions are HTML elements styled with CSS. The browser renders that in the live preview for free.

The export is the hard part. Producing a video file means producing pixels for every frame, with the captions rendered into them. Export runs client-side (project constraint), so a server-side renderer is not an option. And there is no browser API that screenshots a DOM node.

There is one supported path: put the markup inside an SVG <foreignObject>, load the SVG as an image, and draw that image onto a canvas. This post walks through it, measuring each step against a small fixture. All numbers come from headless Chromium on my Linux dev machine.

The fixture

A dummy transcript with per-word timings. Sampling it at t = 0.5s produces this caption state:

<div class="captions">
  <span class="word active">RENDERING</span>
  <span class="word">CAPTIONS</span>
</div>

The stylesheet uses Bungee (a Google Font) for the display face, a thick outline, a drop shadow, and a color change on the active word:

.captions {
  position: absolute;
  left: 0; right: 0; bottom: 18%;
  text-align: center;
  font-family: 'Bungee', sans-serif;
  font-size: 64px;
  color: #ffffff;
  -webkit-text-stroke: 10px #000000;
  paint-order: stroke fill;
  text-shadow: 0 6px 24px rgba(0, 0, 0, 0.55);
}
.word {
  display: inline-block;
  margin: 0 0.12em;
}
.word.active {
  color: #ffd400;
  transform: scale(1.12);
}

Step 1: rasterize captions across a clip

Render 5 seconds at 24 fps: 120 output frames. For each timestamp, seek the video, draw the frame, sample the caption state at that time, rasterize it, draw on top:

for (const t of timestamps) {
  await seekVideo(video, t);
  ctx.drawImage(video, 0, 0, WIDTH, HEIGHT);

  const svg = buildSvg(captionHtmlAt(t), captionCss);
  const img = new Image();
  img.src = 'data:image/svg+xml;charset=utf-8,' + encodeURIComponent(svg);
  await img.decode();
  ctx.drawImage(img, 0, 0);
}

Result: 106 ms of caption rasterization for the 120 frames (about 0.9 ms per frame).

Bungee falls back to the default sans-serif on every frame (no @font-face), but the pipeline works end to end. Every following step measures caption rasterization on this same 120-frame clip.

Reported times cover caption rasterization only. The video seek and canvas composite are separate costs that do not change with the optimizations in this post.

Step 2: reference the font by URL

Add a @font-face rule with a URL that points to the Google Fonts CDN:

@font-face {
  font-family: 'Bungee';
  src: url('https://fonts.gstatic.com/s/bungee/v17/N0bU2SZBIuF2PU_ECg.ttf') format('truetype');
}

Result: the output is byte-identical to Step 1 across the whole clip. The browser does not send the request, and does not log anything.

The reason is documented on MDN's page about SVG as an image. Under the list of restrictions that apply when SVG is used as an image:

External resources (e.g., images, stylesheets) cannot be loaded, though they can be used if inlined through data: URLs.

The url() in the @font-face rule falls in that category, so the browser skips it.

Step 3: inline the font as a data URI

Fetch the font once, convert to base64, and inline it into the CSS:

async function toDataUri(url) {
  const blob = await (await fetch(url)).blob();
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.onloadend = () => resolve(reader.result);
    reader.onerror = reject;
    reader.readAsDataURL(blob);
  });
}

const fontDataUri = await toDataUri('/assets/bungee.ttf');
const css = `@font-face { font-family: 'Bungee'; src: url('${fontDataUri}'); }` + baseCss;

Bungee now renders on every frame. The SVG payload grows from 787 bytes to 148 KB per frame, and caption rasterization for the clip goes from 106 ms to 339 ms.

The fetch and base64 conversion happen once per export, not per frame. The inlined CSS is reused for every raster from that point.

Step 4: sprite sheets

Each call to rasterize pays fixed overhead: the browser parses the SVG, applies the font, and decodes the image. If this happens once per frame, that overhead dominates the total render time.

Reducing the number of calls means placing several frames inside one SVG as tiles. Each <foreignObject> covers its own area of the sheet, one decode returns the whole sheet as a bitmap, and individual frames are sliced out later with drawImage.

const tiles = tileHtmls.map((html, i) =>
  `<foreignObject x="${i * width}" y="0" width="${width}" height="${height}">` +
    `<div xmlns="http://www.w3.org/1999/xhtml" style="...">${html}</div>` +
  `</foreignObject>`
).join('');
const svg = `<svg xmlns="..." width="${width * tileHtmls.length}" height="${height}">` +
  `<style>${css}</style>${tiles}</svg>`;

With 16 tiles per sheet, the 120 frames become 7 decodes and 26 ms. About 13x faster than Step 3, with identical output.

The SVG payload grows very little because the font is shared across all tiles. Step 3's SVG is 148,154 bytes. Step 4's 16-tile sheet is 152,285 bytes, only ~4 KB more (+2.8%).

The batch size has a ceiling. Browsers limit how large an image they will decode and draw, and no browser API reports the effective limit. It is not MAX_TEXTURE_SIZE, and it varies by browser and platform. Past the limit you get an opaque decode error, or worse, silently wrong output. One way to handle that is to probe the limit empirically at startup and cap the sheet size accordingly.

Step 5: deduplicate identical caption states

Most caption frames look the same as their neighbours. In this clip the picture only changes when the highlighted word changes. Across 120 frames there are 10 unique caption states.

The fix is to identify each state by its content and render each one only once. In this fixture the state key is a small string like "segment-2:word-1". In the general case it is the full set of CSS classes in the caption subtree at that time.

const tileOfState = new Map();
const perFrame = timestamps.map((t) => {
  const key = stateKeyAt(t);
  if (!tileOfState.has(key)) tileOfState.set(key, tileOfState.size);
  return tileOfState.get(key);
});

const sheet = await rasterizeSheet(
  [...tileOfState.keys()].map(captionHtmlFor),
  css, width, height
);

perFrame.forEach((tileIndex, frameIndex) => {
  ctx.drawImage(sheet, tileIndex * width, 0, width, height, 0, 0, width, height);
});

Result: 1 decode, 10 tiles, 9 ms for 120 frames. About 38x faster than Step 3.

Deduplication is only valid while no CSS animation is running. During an animation every frame is a unique state, and consecutive frames cannot share a tile. One way to handle that is to probe the caption subtree for active animations at each timestamp, using getComputedStyle to read animation-duration and animation-delay, and disable deduplication inside those windows.

Summary

Strategy	SVG decodes	Time
One raster per frame (Step 3)	108	339 ms
Sprite sheets, 16 tiles (Step 4)	7	26 ms
Sprite sheets + state dedup (Step 5)	1	9 ms

A note on img.decode(). The code above uses it instead of img.onload. For SVGs that include @font-face rules, onload is not always enough. It can fire before the fonts have applied. decode() waits until the image is paint-ready.

Even decode() has limits. On Firefox, some styles (in particular gradients clipped to text with background-clip: text) can resolve early and the resulting frame does not match what a live browser would paint.

Context

This pipeline is the export path of tscaps, a browser-based captions editor I am building. The preview shows the caption DOM over the video; the export rebuilds the same CSS states per timestamp, rasterizes them as described here, composites them with the video frames through WebCodecs, and muxes the result with Mediabunny. Everything runs client-side.

The rendering core is open source as @tscaps/engine (source). It contains the production versions of the ideas above, plus the parts that did not fit in this post.

Anything that improves this approach is welcome: a trick that avoids the foreignObject and data URL pipeline entirely, another optimization along the lines of tiling or state dedup, or a cleaner answer to the image-size limit than empirical probing.