Mack

Posted on Feb 19

Build a Visual Regression Testing Pipeline in Under 50 Lines

#testing #javascript #webdev #tutorial

Visual regression testing catches UI bugs that unit tests miss. A button that shifted 3px left. A font that didn't load. A CSS change that broke your checkout page.

Most visual testing tools are expensive, complex, or both. Here's how to build a lightweight pipeline with just a screenshot API and a bit of Node.js.

The Architecture

1. Capture baseline screenshots of key pages
2. On each PR/deploy, capture new screenshots
3. Compare pixel-by-pixel
4. Alert on differences above threshold

Simple. No Selenium grid. No browser farm. Just HTTP calls.

Step 1: Define Your Pages

// pages.json
[
  { "name": "homepage", "url": "https://yourapp.com" },
  { "name": "pricing", "url": "https://yourapp.com/pricing" },
  { "name": "login", "url": "https://yourapp.com/login" },
  { "name": "dashboard", "url": "https://yourapp.com/dashboard" }
]

Step 2: Capture Screenshots

Using a screenshot API keeps things simple — no local browser dependencies, consistent rendering, and it works in CI.

const fs = require("fs");
const pages = require("./pages.json");

async function captureAll(outputDir) {
  fs.mkdirSync(outputDir, { recursive: true });

  for (const page of pages) {
    const response = await fetch(
      `https://rendly-api.fly.dev/api/v1/screenshots`,
      {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${process.env.RENDLY_API_KEY}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          url: page.url,
          width: 1280,
          height: 800,
          full_page: true,
          format: "png"
        })
      }
    );

    const data = await response.json();
    const img = Buffer.from(data.image, "base64");
    fs.writeFileSync(`${outputDir}/${page.name}.png`, img);
    console.log(`✓ ${page.name}`);
  }
}

Step 3: Compare Images

The pixelmatch library is perfect for this — fast, zero dependencies, battle-tested.

const { PNG } = require("pngjs");
const pixelmatch = require("pixelmatch");

function compareImages(baselinePath, currentPath) {
  const baseline = PNG.sync.read(fs.readFileSync(baselinePath));
  const current = PNG.sync.read(fs.readFileSync(currentPath));

  const { width, height } = baseline;
  const diff = new PNG({ width, height });

  const mismatchedPixels = pixelmatch(
    baseline.data, current.data, diff.data,
    width, height,
    { threshold: 0.1 }
  );

  const totalPixels = width * height;
  const diffPercent = (mismatchedPixels / totalPixels) * 100;

  return { diffPercent, diffImage: PNG.sync.write(diff) };
}

Step 4: Put It Together

async function visualRegression() {
  await captureAll("./screenshots/current");

  const pages = require("./pages.json");
  const results = [];

  for (const page of pages) {
    const baselinePath = `./screenshots/baseline/${page.name}.png`;
    const currentPath = `./screenshots/current/${page.name}.png`;

    if (!fs.existsSync(baselinePath)) {
      console.log(`⚠ No baseline for ${page.name}, creating one`);
      fs.copyFileSync(currentPath, baselinePath);
      continue;
    }

    const { diffPercent, diffImage } = compareImages(baselinePath, currentPath);

    results.push({
      page: page.name,
      diffPercent: diffPercent.toFixed(2),
      passed: diffPercent < 0.5
    });

    if (diffPercent >= 0.5) {
      fs.writeFileSync(`./screenshots/diff/${page.name}.png`, diffImage);
    }
  }

  const failed = results.filter(r => !r.passed);
  if (failed.length > 0) {
    console.error("\n❌ Visual regression detected:");
    failed.forEach(f => console.error(`  ${f.page}: ${f.diffPercent}% different`));
    process.exit(1);
  }

  console.log("\n✅ All pages match baseline");
}

visualRegression();

That's ~45 lines of actual logic. The entire pipeline.

Add It to CI

# .github/workflows/visual-regression.yml
name: Visual Regression
on: [pull_request]

jobs:
  visual-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm install pixelmatch pngjs
      - run: node visual-regression.js
        env:
          RENDLY_API_KEY: ${{ secrets.RENDLY_API_KEY }}
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: visual-diffs
          path: screenshots/diff/

Now every PR gets automatically checked for visual changes. Diff images are uploaded as artifacts when something breaks.

Why This Works

No browser management — the screenshot API handles rendering
Consistent results — same rendering environment every time
Fast — parallel API calls, no Selenium startup time
CI-friendly — just Node.js and HTTP calls
Cheap — a few screenshots per PR is minimal usage

When to Update Baselines

When a visual change is intentional:

# Regenerate baselines
node visual-regression.js --update-baseline

Add a flag to your script that copies current screenshots to baseline instead of comparing.

Going Further

Responsive testing: capture at multiple widths (375, 768, 1280)
Dark mode: toggle color scheme in the screenshot request
Slack alerts: post diff images to a channel on failure
Scheduled monitoring: run nightly against production

Visual regression doesn't need to be complicated. A screenshot API, a comparison library, and 50 lines of code. Ship it.

Built with Rendly — screenshot and image generation API for developers.

DEV Community