Realistic Fabric Wrinkles in Real-Time: Building Displacement Maps with Fabric.js

How to make designs conform to t-shirt folds without Photoshop or 3D engines

The Challenge

When you upload a design to a t-shirt mockup, it looks flat and fake. Real fabric has wrinkles, folds, and shadows. The design should bend around these contours—but how do you achieve this effect in real-time, entirely in the browser?

The answer: displacement mapping, a computer graphics technique that warps images based on depth information.

What is Displacement Mapping?
Displacement mapping uses a grayscale "height map" to determine how pixels should move:

• White areas (255) = peaks (pixels move outward)
• Black areas (0) = valleys (pixels move inward)
• Gray areas (128) = neutral (no movement)

When applied to a design on fabric:

• Design pixels over wrinkles/folds get compressed
• Design pixels over stretched areas expand
• The result: your design appears to wrap around the fabric's topology

The Approach: Three-Stage Pipeline

Stage 1: Generate Displacement Map from Mockup
Stage 2: Apply Mesh-Based Pixel Displacement
Stage 3: Blend Shading for Realistic Lighting

Let's break down each stage with code.

Stage 1: Extracting Depth from the Mockup

The t-shirt mockup already contains depth information encoded in its brightness values. Dark areas = shadows/folds, light areas = highlights/peaks.

const generateDisplacementMap = (mockupImg: HTMLImageElement): HTMLImageElement => {
  const canvas = document.createElement("canvas");
  canvas.width = mockupImg.naturalWidth;
  canvas.height = mockupImg.naturalHeight;
  const ctx = canvas.getContext("2d")!;

  // Draw the mockup
  ctx.drawImage(mockupImg, 0, 0);
  const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
  const data = imageData.data;

  // Step 1: Find luminance range
  let minLum = 255, maxLum = 0;

  for (let i = 0; i < data.length; i += 4) {
    if (data[i + 3] < 10) continue; // Skip transparent pixels

    // Calculate perceived brightness (ITU-R BT.601 standard)
    const lum = 0.299 * data[i] +      // Red weight
                0.587 * data[i + 1] +  // Green weight (human eye most sensitive)
                0.114 * data[i + 2];   // Blue weight

    minLum = Math.min(minLum, lum);
    maxLum = Math.max(maxLum, lum);
  }

  const range = maxLum - minLum || 1;

  // Step 2: Normalize to full 0-255 range for maximum displacement contrast
  for (let i = 0; i < data.length; i += 4) {
    if (data[i + 3] < 10) {
      data[i] = data[i + 1] = data[i + 2] = 128; // Neutral gray
      continue;
    }

    const lum = 0.299 * data[i] + 0.587 * data[i + 1] + 0.114 * data[i + 2];

    // Stretch luminance to full range
    const stretched = ((lum - minLum) / range) * 255;

    // Convert to grayscale displacement map
    data[i] = data[i + 1] = data[i + 2] = stretched;
  }

  ctx.putImageData(imageData, 0, 0);

  const resultImg = new Image();
  resultImg.src = canvas.toDataURL("image/png");
  return resultImg;
};

Why This Works:

• Luminance Calculation: Uses scientifically accurate weights (0.299, 0.587, 0.114) because human eyes are most sensitive to green light
• Normalization: Stretches the range to 0-255 to maximize displacement effect
• Transparent Handling: Sets transparent areas to neutral gray (128) so they don't displace
• Visual Result: A grayscale map where t-shirt folds are dark and flat areas are light.

Stage 2: The Displacement Engine

Now comes the complex part: using the displacement map to actually warp the design's pixels.

The Technique: Bilinear Interpolation
Instead of simply offsetting pixels, we use bilinear interpolation to sample colors smoothly between four neighboring pixels. This prevents jagged edges.

private displace(
  designImage: HTMLImageElement,
  dispMap: Float32Array,      // Normalized 0-1 displacement values
  shadeMap: Float32Array,      // Normalized 0-1 shading values
  strength: number,            // How much to displace (45 = strong effect)
  blendFactor: number,         // How much shading to apply (0.6 = 60%)
  rotation: number             // Design rotation in radians
): string {

  const designWidth = dispMap.length / shadeMap.length; // Assuming square
  const designHeight = shadeMap.length / designWidth;

  // Draw design to temporary canvas
  const designCanvas = document.createElement('canvas');
  designCanvas.width = designWidth;
  designCanvas.height = designHeight;
  const designCtx = designCanvas.getContext('2d')!;
  designCtx.drawImage(designImage, 0, 0, designWidth, designHeight);
  const designData = designCtx.getImageData(0, 0, designWidth, designHeight);

  // Create output canvas
  const outputData = designCtx.createImageData(designWidth, designHeight);

  // Pre-calculate rotation matrix for performance
  const cosR = Math.cos(rotation);
  const sinR = Math.sin(rotation);
  const centerX = designWidth / 2;
  const centerY = designHeight / 2;

  for (let y = 0; y < designHeight; y++) {
    for (let x = 0; x < designWidth; x++) {
      const i = y * designWidth + x;
      const idx = i * 4;

      const alpha = designData.data[idx + 3];
      if (alpha < 5) {
        // Transparent pixel - skip
        outputData.data[idx + 3] = 0;
        continue;
      }

      // Get displacement value (0-1 range)
      const disp = dispMap[i];

      // Calculate displacement offset
      // Higher strength = more displacement
      const offsetX = (disp - 0.5) * strength;
      const offsetY = (disp - 0.5) * strength;

      // Apply rotation to displacement vector
      const dx = offsetX * cosR - offsetY * sinR;
      const dy = offsetX * sinR + offsetY * cosR;

      // Source coordinates (where to sample from)
      let srcX = x + dx;
      let srcY = y + dy;

      // Clamp to valid range
      srcX = Math.max(0, Math.min(designWidth - 1, srcX));
      srcY = Math.max(0, Math.min(designHeight - 1, srcY));

      // Bilinear interpolation for smooth sampling
      const x0 = Math.floor(srcX);
      const x1 = Math.min(x0 + 1, designWidth - 1);
      const y0 = Math.floor(srcY);
      const y1 = Math.min(y0 + 1, designHeight - 1);

      const xf = srcX - x0;  // Fractional part
      const yf = srcY - y0;

      // Sample four neighboring pixels
      const idx00 = (y0 * designWidth + x0) * 4;
      const idx10 = (y0 * designWidth + x1) * 4;
      const idx01 = (y1 * designWidth + x0) * 4;
      const idx11 = (y1 * designWidth + x1) * 4;

      // Interpolate each color channel
      const lerp = (a: number, b: number, t: number) => a + (b - a) * t;

      const r = lerp(
        lerp(designData.data[idx00], designData.data[idx10], xf),
        lerp(designData.data[idx01], designData.data[idx11], xf),
        yf
      );
      const g = lerp(
        lerp(designData.data[idx00 + 1], designData.data[idx10 + 1], xf),
        lerp(designData.data[idx01 + 1], designData.data[idx11 + 1], xf),
        yf
      );
      const b = lerp(
        lerp(designData.data[idx00 + 2], designData.data[idx10 + 2], xf),
        lerp(designData.data[idx01 + 2], designData.data[idx11 + 2], xf),
        yf
      );

      // Apply shading from shade map
      const shade = 1 - (1 - shadeMap[i]) * blendFactor;

      // Write final pixel with shading
      outputData.data[idx] = Math.min(255, Math.max(0, Math.round(r * shade)));
      outputData.data[idx + 1] = Math.min(255, Math.max(0, Math.round(g * shade)));
      outputData.data[idx + 2] = Math.min(255, Math.max(0, Math.round(b * shade)));
      outputData.data[idx + 3] = alpha;
    }
  }

  designCtx.putImageData(outputData, 0, 0);
  return designCanvas.toDataURL('image/png');
}

Breaking Down the Math:

1. Displacement Offset Calculation:

const offsetX = (disp - 0.5) * strength;

• disp ranges from 0 to 1
• Subtracting 0.5 centers it around 0 (-0.5 to +0.5)
• Multiplying by strength (45) gives ±22.5 pixel displacement range

2. Rotation Handling:

const dx = offsetX * cosR - offsetY * sinR;
const dy = offsetX * sinR + offsetY * cosR;

• Standard 2D rotation matrix
• Ensures displacement direction rotates with the design
• Without this, rotated designs would displace in wrong directions

3. Bilinear Interpolation:

const r = lerp(
  lerp(designData.data[idx00], designData.data[idx10], xf),
  lerp(designData.data[idx01], designData.data[idx11], xf),
  yf
);

• Samples 4 pixels around the source coordinate
• Blends them based on fractional position (xf, yf)
• Result: smooth color transitions instead of blocky artifacts

Stage 3: Shading Integration

The shade map adds realism by darkening areas in fabric folds.

// Apply shading from shade map
const shade = 1 - (1 - shadeMap[i]) * blendFactor;

outputData.data[idx] = Math.round(r * shade);      // Red channel
outputData.data[idx + 1] = Math.round(g * shade);  // Green channel
outputData.data[idx + 2] = Math.round(b * shade);  // Blue channel

How Shading Works:

• shadeMap[i] = 0 (dark fold) → shade = 1 - (1 - 0) * 0.6 = 0.4 → pixel becomes 40% brightness
• shadeMap[i] = 1 (light area) → shade = 1 - (1 - 1) * 0.6 = 1.0 → full brightness
• blendFactor (0.6) controls intensity: higher = more dramatic shadows

Integrating with Fabric.js

Here's how to apply this to a Fabric.js canvas object:

const applyDisplacementToDesign = async (designObj: any) => {
  const canvas = fabricCanvasRef.current;
  if (!canvas || !mockupImageRef.current) return;

  const mockupUrl = originalMockupUrlRef.current;

  // Load images
  const [designImg, mockupImg] = await Promise.all([
    loadImage(designObj._element.src),
    loadImage(mockupUrl)
  ]);

  // Generate displacement map
  const displacementMapImg = generateDisplacementMap(mockupImg);
  await new Promise(resolve => {
    displacementMapImg.onload = resolve;
  });

  // Get design bounds and rotation
  const angle = designObj.angle || 0;
  const angleRad = angle * (Math.PI / 180);

  const designBounds = {
    left: designObj.left - (designObj.width * designObj.scaleX) / 2,
    top: designObj.top - (designObj.height * designObj.scaleY) / 2,
    width: designObj.width * designObj.scaleX,
    height: designObj.height * designObj.scaleY
  };

  // Apply displacement
  const renderer = getDisplacementRenderer();
  const displacedDataUrl = renderer.displaceDesignWithBounds(
    designImg,
    displacementMapImg,
    designBounds,
    mockupBounds,
    { 
      strength: 45,        // Strong displacement
      blendFactor: 0.6,    // 60% shading
      rotation: angleRad 
    }
  );

  // Update Fabric.js object
  const displacedImg = await loadImage(displacedDataUrl);
  designObj.setElement(displacedImg);
  designObj.setCoords();
  canvas.renderAll();
};

Performance Optimizations

1. Mobile Detection

const isMobileDevice = window.innerWidth < 768;
const strength = isMobileDevice ? 0 : 45; // Disable on mobile

1. Blur the Displacement Map

dispCtx.filter = 'blur(20px)';  // Smoother displacement
dispCtx.drawImage(textureCanvas, 0, 0);

Reduces pixel-level noise and creates organic warping.

1. Only Reapply on Transform End

canvas.on("object:modified", async (e) => {
  await applyDisplacementToDesign(e.target);
});

Don't recalculate during drag—only when user releases.

Results

Before Displacement:

• Flat design overlay
• No depth perception
• Looks like a sticker

After Displacement:

• Design conforms to fabric topology
• Shadows in folds darken the design
• Realistic 3D appearance

Performance:

• ~80ms processing time on modern browsers
• Zero server calls (all client-side)
• Works with rotated designs (rotation matrix integration)

Key Takeaways

• Luminance extraction creates the displacement map from mockup brightness
• Bilinear interpolation ensures smooth pixel sampling without artifacts
• Rotation matrices allow displacement to work with transformed designs
• Shade maps add realistic lighting to complete the effect
• Canvas API makes this possible entirely in the browser—no WebGL needed

This technique can be applied to any fabric mockup: hoodies, bags, hats, or even non-fabric surfaces like wood grain or textured paper.

Try It Live

Virtual Threads IO: virtualthreads.io/2d-mockups

Have questions about the implementation? Drop them in the comments below!