Stitching a handful of overlapping screenshots into one tall capture sounds easy until you try to detect where screenshot A ends and screenshot B begins.
Your first instinct is usually to slide the bottom of image A over the top of image B and pick the overlap with the lowest pixel difference. That works great on dense pages, and fails completely the moment two screenshots share a white margin, a flat dark background, or a solid color band. Two blank strips have a pixel difference of zero, so a naive matcher happily glues the images together in the wrong spot.
While building the screenshot stitcher for Imagier, I put together a lightweight matcher in plain TypeScript with zero dependencies. Here is how it works and why a simple luminance variance threshold solved almost every bad join.
1. Downsample into a narrow strip first
Comparing full 2x Retina screenshots (often 2800+ pixels wide) across hundreds of candidate overlap offsets is way too slow on the main thread. Instead, draw each image onto a tiny canvas at a shared strip width of 96 pixels (or smaller for ultra-tall captures so the strip height stays bounded):
export function sampleStrip(
image: HTMLImageElement | ImageBitmap,
horizontal: boolean,
breadth = 96
): PixelStrip {
const width = 'naturalWidth' in image ? image.naturalWidth : image.width;
const height = 'naturalHeight' in image ? image.naturalHeight : image.height;
const scale = breadth / (horizontal ? height : width);
const length = Math.max(1, Math.round((horizontal ? width : height) * scale));
const canvas = document.createElement('canvas');
canvas.width = breadth;
canvas.height = length;
const ctx = canvas.getContext('2d', { willReadFrequently: true });
if (!ctx) throw new Error('Canvas is unavailable.');
if (horizontal) {
ctx.translate(breadth, 0);
ctx.rotate(Math.PI / 2);
ctx.drawImage(image, 0, 0, length, breadth);
} else {
ctx.drawImage(image, 0, 0, breadth, length);
}
return { width: breadth, height: length, data: ctx.getImageData(0, 0, breadth, length).data };
}
One nice trick here: if the user switches to horizontal stitching, we just rotate the strip 90 degrees during sampling. That way the overlap detection function only ever has to think about vertical strips.
2. Reject flat margins with luminance variance
Now we sweep through candidate overlaps from 8 pixels up to the shorter strip height. For each candidate overlap, we sample rows and columns, track the mean absolute RGB error, and simultaneously compute the luminance variance of the overlapping slice:
export function detectOverlap(a: PixelStrip, b: PixelStrip) {
const max = Math.min(a.height, b.height) - 8;
let best = Infinity;
let bestOverlap = 0;
let bestVariance = 0;
for (let overlap = 8; overlap <= max; overlap++) {
let error = 0;
let sum = 0;
let squared = 0;
let count = 0;
const step = Math.max(1, Math.floor(overlap / 36));
for (let y = 0; y < overlap; y += step) {
for (let x = 0; x < a.width; x += 3) {
const ai = ((a.height - overlap + y) * a.width + x) * 4;
const bi = (y * b.width + x) * 4;
for (let c = 0; c < 3; c++) {
error += Math.abs(a.data[ai + c] - b.data[bi + c]);
}
const luminance = (a.data[ai] * 3 + a.data[ai + 1] * 6 + a.data[ai + 2]) / 10;
sum += luminance;
squared += luminance * luminance;
count++;
}
}
const variance = squared / count - (sum / count) ** 2;
const score = error / (count * 3);
// Ignore flat margins where almost every pixel looks identical.
if (variance > 100 && score < best) {
best = score;
bestOverlap = overlap;
bestVariance = variance;
}
}
return best < 12 && bestVariance > 100
? { pixels: bestOverlap, confidence: Math.max(0, 1 - best / 12) }
: { pixels: 0, confidence: 0 };
}
That variance > 100 guard is what makes the matcher practical on real UI screenshots. Solid backgrounds, empty padding, and blank gutters have near-zero luminance variance, so they are skipped automatically. Only regions with actual text, borders, or UI elements can win.
3. Keep manual trims intact when users reorder captures
Auto-detection gets most joins right, but sticky headers or animated banners still need occasional manual nudging. One small UX detail that saves frustration: if a user drags screenshot #3 to a new position, do not wipe out the trims on joins that did not change. Track each sheet's predecessor ID, and only reset overlap back to 0 on pairs whose neighbor actually changed.
Test it in your browser
You can try the auto-matcher on your own scrolling screenshots (or with the built-in sample captures) at imagier.app/tools/screenshot-stitcher. Everything stays on your device, and if you want to mark up the stitched image with arrows or numbered callouts afterward, you can send it straight into the screenshot annotator.
Top comments (0)