Warping an image across a deformable mesh in plain Canvas 2D (no WebGL)

I recently built a browser tool that turns any photo into a paper cut-out that
flutters and floats — Paper Animator.
No upload, no server, no WebGL required to get the first frame on screen. The
whole thing renders in the browser and exports the exact same animation to
GIF/WebM/MP4.

The interesting part isn't the paper aesthetic. It's a problem that comes up
whenever you want to bend, ripple, or warp an image on the web without reaching
for a 3D engine: Canvas 2D has no primitive for texture-mapping an image onto
an arbitrary triangle. drawImage can translate, scale, and (with
setTransform) shear — but it can't natively stretch a rectangular bitmap onto
three arbitrary corner points.

This post walks through the technique I landed on, and the two systems around it
that make it fast and export-safe. Everything here is framework-agnostic
vanilla TypeScript.

The mental model: animation as a pure function t → pixels

Before any rendering, one decision shapes everything else: a frame is a pure
function of time.

function renderFrame(target: HTMLCanvasElement, source, settings, timeSeconds: number): void

No hidden requestAnimationFrame state, no physics world.step() accumulating
between frames. Given the same timeSeconds, you get the same pixels — every
time.

This sounds academic until you try to export. If your preview loop mutates state
frame-to-frame, your exported video will drift, stutter, or depend on the
machine's frame rate. When the frame is pure, preview and export share one code
path:

// Preview: sample at the wall clock
const t = (performance.now() - start) / 1000;
renderFrame(canvas, source, settings, t);

// Export: sample at fixed timestamps, guaranteed identical output
for (let frame = 0; frame < totalFrames; frame++) {
  renderFrame(canvas, source, settings, frame / fps);
  recorder.captureFrame();
}

Keep this constraint in your head — it's why the motion below is expressed as
Math.sin(phase) and never as position += velocity.

Step 1: build a deformable mesh

Warping a flat image means we can't treat it as one rectangle. We slice it into
a grid of vertices, then move those vertices. Each vertex carries its (u, v)
texture coordinate (where it samples from the source, 0..1) and its (x, y)
screen position (where it lands after deformation).

export type PaperVertex = { x: number; y: number; u: number; v: number };

An 18×12 grid is plenty for preview (432 vertices, 396 triangles); I bump it to
22×12 for export where the extra smoothness is worth the cost.

Here's the "flutter" motion — a high-frequency, small-amplitude wiggle that
reads as stiff paper vibrating. The key is that displacement is a sum of
harmonics, which gives a snappier, less sine-wavey feel than a single
frequency:

// Snappy wave: primary + 3rd + 5th harmonic (more percussive than a plain sine)
const snap = (p: number) =>
  Math.sin(p) * 0.72 + Math.sin(p * 3) * 0.18 + Math.sin(p * 5) * 0.1;

for (let row = 0; row <= rows; row++) {
  for (let column = 0; column <= columns; column++) {
    const u = column / columns;
    const v = row / rows;

    // Local ripple that propagates quickly across the sheet (stiff paper)
    const localPhase = basePhase + u * 1.2 + v * 0.8;
    const localX = snap(localPhase * 1.7) * localAmount * 0.4;
    const localY = snap(localPhase * 2.3) * localAmount * 0.25;

    // Edges and corners flex a little more than the center
    const edgeFlex =
      0.6 + 0.4 * (Math.abs(u - 0.5) * 2 + Math.abs(v - 0.5) * 2) * 0.5;

    const baseX = u * width + localX * edgeFlex;
    const baseY = v * height + localY * edgeFlex;

    // Then apply a global micro-rotation + horizontal shake to the whole sheet
    const dx = baseX - centerX;
    const dy = baseY - centerY;
    const finalX = dx * cosR - dy * sinR + centerX + shakeX;
    const finalY = dx * sinR + dy * cosR + centerY + shakeY;

    vertices.push({ u, v, x: finalX, y: finalY });
  }
}

Two things make this feel like a physical material rather than a screensaver:

• Phase offset per vertex (u * 1.2 + v * 0.8) so the wave travels across the sheet instead of every point moving in lockstep.
• Edge flex so the interior stays relatively rigid while corners flap — real paper is stiffer in the middle of a held sheet.

Step 2: the actual trick — affine texture mapping per triangle

Now we have deformed vertex positions. How do we draw the source image stretched
across them? Split each grid quad into two triangles, and for each triangle
compute the affine transform that maps its source corners (from the
undeformed image) onto its destination corners (the deformed positions),
then clip to the triangle and drawImage.

This is the load-bearing function. It solves for the 2×3 affine matrix
[m11, m12, m21, m22, dx, dy] that sends source triangle (a, b, c) to the
destination triangle:

function drawTriangle(ctx, layer, a, b, c, offsetX, offsetY) {
  // Source-space corners: where this vertex samples FROM in the image
  const ax = a.u * layer.width,  ay = a.v * layer.height;
  const bx = b.u * layer.width,  by = b.v * layer.height;
  const cx = c.u * layer.width,  cy = c.v * layer.height;

  // Edge vectors in source space
  const sbx = bx - ax, sby = by - ay;
  const scx = cx - ax, scy = cy - ay;
  const denom = sbx * scy - sby * scx;
  if (Math.abs(denom) < 0.0001) return; // degenerate triangle, skip

  // Edge vectors in destination (deformed) space
  const dbx = b.x - a.x, dby = b.y - a.y;
  const dcx = c.x - a.x, dcy = c.y - a.y;

  // Solve the 2x2 linear map (source edges -> destination edges)
  const m11 = (dbx * scy - dcx * sby) / denom;
  const m12 = (dcx * sbx - dbx * scx) / denom;
  const m21 = (dby * scy - dcy * sby) / denom;
  const m22 = (dcy * sbx - dby * scx) / denom;
  // Translation so source point `a` lands exactly on destination `a`
  const dx = a.x - m11 * ax - m12 * ay;
  const dy = a.y - m21 * ax - m22 * ay;

  ctx.save();
  // Clip to the destination triangle so we only paint inside it
  ctx.beginPath();
  ctx.moveTo(a.x + offsetX, a.y + offsetY);
  ctx.lineTo(b.x + offsetX, b.y + offsetY);
  ctx.lineTo(c.x + offsetX, c.y + offsetY);
  ctx.closePath();
  ctx.clip();
  // Apply the affine map and blit the whole source layer through the clip
  ctx.setTransform(m11, m21, m12, m22, dx + offsetX, dy + offsetY);
  ctx.drawImage(layer, 0, 0);
  ctx.restore();
}

The math is a plain change-of-basis: express the destination edge vectors in
terms of the source edge vectors, and you get a linear transform; add a
translation to pin one shared corner. Because it's affine (not perspective),
each triangle stays visually correct as long as triangles are small — which is
exactly why we use a mesh instead of one big quad.

Drawing the whole sheet is just iterating the grid, two triangles per cell:

for (let row = 0; row < rows; row++) {
  for (let column = 0; column < columns; column++) {
    const tl = vertices[row * (columns + 1) + column];
    const tr = vertices[row * (columns + 1) + column + 1];
    const bl = vertices[(row + 1) * (columns + 1) + column];
    const br = vertices[(row + 1) * (columns + 1) + column + 1];
    drawTriangle(ctx, layer, tl, tr, bl, offsetX, offsetY);
    drawTriangle(ctx, layer, tr, br, bl, offsetX, offsetY);
  }
}

One caveat worth knowing: clip() in Canvas 2D isn't anti-aliased against the
triangle you're compositing next to, so you can occasionally see hairline seams
between triangles on some browsers. In practice, at these mesh densities with
imageSmoothingEnabled = true it's invisible, but it's the reason the WebGL2
path (below) exists for anyone who wants pixel-perfect edges.

Step 3: don't recompute what didn't change (static-layer cache)

Here's the performance insight that made preview smooth. Look at what actually
changes per frame versus what's constant across an animation:

• Per frame: vertex positions (cheap — a few hundred Math.sin calls).
• Constant: the paper texture, the cut-out composite, edge roughness, drop shadow base, fiber grain. Regenerating those every frame is the expensive part — and it's completely wasted work, because they only depend on the source image and the material settings, not on time.

So I pre-render the time-independent layers once and cache them, keyed by a
signature of everything that would invalidate them:

const signature = [
  settings.layout,
  settings.paperColor,
  settings.paperMargin,
  settings.edgeThickness,
  settings.edgeRoughness,
  settings.textureStrength,
  settings.creaseStrength,
  source.image.width,
  source.image.height,
].join(':');

const cached = layerCache.get(target);
if (
  cached &&
  cached.source === source.image &&
  cached.sourceRevision === source.sourceRevision &&
  cached.signature === signature
) {
  return cached; // reuse — no repaint
}
// otherwise rebuild the texture/cutout layers and cache them

Notice what's not in that signature: timeSeconds, movement, bend,
speed. Those only move mesh vertices — they must never bust the texture cache.
This is the single most common place to accidentally kill your frame rate:
recomputing a texture because a value that only affects geometry changed. The
layerCache itself is a WeakMap<HTMLCanvasElement, …> so entries are
collected automatically when the canvas goes away.

Step 4: progressive enhancement to WebGL2

The Canvas 2D path guarantees something renders in any browser. But for a large
export, iterating hundreds of clip() + drawImage() calls per frame adds up.
So the same mesh feeds an optional WebGL2 backend that uploads the layer as a
texture and draws all triangles in one pass. Backend selection is lazy, cached,
and — critically — falls back safely if a GPU context is lost mid-session:

try {
  renderWithBackend(backend, /* ...mesh, layers... */);
} catch (error) {
  if (backend.kind !== 'webgl2') throw error;
  backend.dispose();                     // WebGL2 died (context loss, etc.)
  const fallback = createCanvas2dBackend();
  backendCache.set(target, fallback);
  renderWithBackend(fallback, /* ...same args... */); // seamless retry
}

Because both backends consume the identical vertex array and layer canvases, the
fallback is genuinely transparent — the user never sees a broken frame, just a
slightly cheaper one. This is the payoff of keeping the mesh and the rendering
strictly separate.

Putting it together

The full per-frame pipeline is small:

1. Compute workspace bounds and (optionally) run a browser-side segmenter to isolate the subject.
2. getStaticLayers() → cached texture + cut-out composite.
3. getPaperMesh(t) → deformed vertices for this instant.
4. renderWithBackend() → WebGL2, or Canvas 2D triangle mapping as fallback.

Every step is a pure function of (source, settings, t), which is what lets the
export sample the same function at fixed timestamps and get a deterministic
video out.

Takeaways you can reuse

• Model animation as t → pixels. It costs you nothing up front and buys you deterministic export for free.
• Canvas 2D can texture-map — you just do it per triangle with a computed affine transform and a clip. A subdivided mesh keeps affine artifacts invisible.
• Split constant work from per-frame work and cache the constant part behind a signature. Be ruthless about what belongs in that signature.
• Layer a WebGL2 fast path on top, sharing the exact same mesh, so the 2D path stays your safety net instead of your only option.

If you want to see the finished version — flutter/float motion, paper texture,
subject cut-out, and export — it's live and free at
purupurumaker.io/paper-animator. I'd
genuinely love feedback on the motion feel from anyone who's done mesh warping
before.

Happy warping.