DEV Community

Jay
Jay

Posted on Edited on

Color contamination after background removal: measuring fringe pixels

In my last post I measured cutout edge geometry: how wide the alpha ramp is, how rough the outline is, whether thin details like a mug handle survive. All three metrics read the alpha channel. I ended with an explicit limitation: they say nothing about color contamination — a green fringe left by the original background that alpha alone cannot see.

This post closes that gap. It gives you three metrics that read the RGB channels, code to compute them, and real numbers from six versions of the same cutout. Two of the cases are ones where the metrics disagree — and the disagreement is the useful part.

Full disclosure, same as before: I work on PicViva, an image tool. The measurement code here is framework-free and works on anyone's cutout. I'm not comparing products in this post.

The setup: a cutout where I know the truth

Measuring a metric is easy. Knowing whether it's telling you the truth is harder. So instead of grabbing an arbitrary cutout, I built one where every pixel's origin is known.

  • One photo: a white ceramic mug on a soft gray backdrop, 1200×800, Unsplash-licensed.
  • One real matte: Apple's Vision VNGenerateForegroundInstanceMaskRequest, run locally on macOS. It produced a proper alpha: 9.44% opaque, 89.52% clear, 1.04% partial. The handle has a see-through gap — exactly the detail color spill loves to hide in.
  • One clean foreground F: I extrapolated the mug's opaque-core colors into the whole subject with nearest-neighbor fill. This is the "correct" RGB — no background mixed in.
  • One known source background G: flat green, RGB (46, 204, 113).

From F, alpha, and G I derived six variants. Every one shares the same alpha matte; only the stored edge RGB (or, in two cases, the alpha itself) changes. That means any difference in the numbers is caused by that one change and nothing else.

Variant What it simulates
A clean correct straight RGB, no spill
B classic spill edge pixels keep the green-screen color
C half despill half the green removed
D over-despill despill pushed too far, toward magenta
E alpha only clean RGB, alpha grown by 2 px
F threshold spill alpha binarized; spilled RGB kept on opaque pixels

Variant B is the interesting ground truth. Its edge pixel is literally alpha·F + (1−alpha)·G, so the true fraction of background color in the partial band is mean(1 − alpha) = 0.498. A good metric should land on that number without being told it.

Metric 1 — spill fraction: how much background is still in the edge

The clean foreground F and the source background G are both known here, so I can ask a direct question: for each edge pixel, what fraction of its color is background?

If a pixel is F·(1−t) + G·t, then projecting the difference (pixel − F) onto the direction (G − F) recovers t:

t = dot(pixel − F, G − F) / ||G − F||²
Enter fullscreen mode Exit fullscreen mode

Clean edge → t ≈ 0. Fully spilled edge → t ≈ (1−alpha). Negative t means the color was pushed away from the background — usually a despill that went too far.

def spill_fraction(stored, F, G, band):
    d = stored[band] - F[band]
    g = G - F[band]
    t = (d * g).sum(axis=1) / np.maximum((g * g).sum(axis=1), 1e-9)
    return t.mean()
Enter fullscreen mode Exit fullscreen mode

This is a targeted metric: it needs to know G, the color the cutout came off. On a green screen or a white sweep, you usually do.

Metric 2 — edge color deviation: a background-agnostic check

Spill fraction has a blind spot — it only looks in G's direction. A magenta fringe from an over-aggressive despill points somewhere else and slips past it. For that I use a metric that needs no knowledge of the background at all.

Compare every partial-alpha pixel to the nearest clean foreground color, and take the mean distance. Call it ECD (edge color deviation):

ECD = mean over band of || stored_pixel − F_pixel ||
Enter fullscreen mode Exit fullscreen mode

Measured in 8-bit RGB levels. A clean edge sits near zero. Any color that doesn't belong to the subject — green, magenta, or a stray highlight — pushes it up.

def edge_color_deviation(stored, F, band):
    d = stored[band] - F[band]
    return np.sqrt((d * d).sum(axis=1)).mean()
Enter fullscreen mode Exit fullscreen mode

The catch: ECD needs a reference foreground F. Here I have one because I built it. On a real cutout you'd approximate it the same way — extrapolate from the opaque core, or compare against the source image's interior colors.

Metric 3 — inner-ring spill: contamination that survives thresholding

A common fix for fringing is to threshold alpha: everything above 128 becomes 255, everything below becomes 0. The partial band disappears, and with it both metrics above — they have no pixels to read.

But the spilled color doesn't disappear. It moves into pixels that are now fully opaque. So I define an inner ring: opaque pixels within 8 px of the boundary, and run the spill fraction there too.

opaque = alpha >= 250
inner = opaque & ~ndi.binary_erosion(opaque, iterations=8)
inner_spill = spill_fraction(stored, F, G, inner)
Enter fullscreen mode Exit fullscreen mode

This is the metric that catches "looks fixed" cutouts.

Results

All numbers below come from running the code in the last section. The shared matte gives 9,959 partial-band pixels and 11,961 inner-ring pixels.

Variant Spill t (band) ECD (band) Inner spill Inner ECD
A clean 0.000 0.000 0.000 0.000
B classic spill 0.498 95.955 0.000 0.000
C half despill 0.249 47.978 0.000 0.000
D over-despill −0.249 47.978 0.000 0.000
E alpha only n/a n/a 0.000 0.000
F threshold spill n/a n/a 0.085 16.526

The figure shows all six composited on dark charcoal so the fringe is easy to see. A and C look progressively cleaner; B has a visible green rim, D a pink one.

Six cutout variants composited on dark charcoal. A clean, B classic spill, C half despill, D over-despill (magenta cast), E alpha grown 2 px, F alpha thresholded with spill hidden in opaque pixels.

What the numbers actually told me

Spill fraction recovered the ground truth exactly. Variant B measured 0.498; the true value from construction was 0.498. That's the check that makes the metric trustworthy — it isn't just moving when something changes, it's reporting the actual amount. Half despill landed at 0.249, precisely half. These are linear by construction, which is the point: a metric you can predict on a known input is one you can interpret on an unknown one.

The negative number is the interesting one. Variant D scored −0.249. If you were watching only spill fraction and treating it as "lower is better," you'd call D better than clean — negative looks like you removed more background than there was. But ECD for D is 47.978, identical to the half-despilled C. The color is still wrong; it's just wrong in the opposite direction. A single directional metric cannot distinguish "fixed" from "over-fixed." Read spill fraction with ECD, or let the sign itself warn you.

The n/a rows are not missing data — they're a result. Variant E has clean color and a 2-px-fatter alpha. Every color metric returns n/a because there are no partial pixels to measure. That's correct: color metrics do not cover geometry. This cutout would pass all three tests here and still look wrong at the edges. You need the alpha-geometry metrics from the previous post to catch it. The two posts measure disjoint failure modes — neither substitutes for the other.

Thresholding hid the problem instead of fixing it. Variant F has no partial band, so band spill and ECD are both n/a — the same blind spot as E. But inner spill jumped to 0.085 and inner ECD to 16.526. Hardening alpha pushed the contaminated color into pixels labeled "fully opaque," where a band-only check would never look. If your pipeline thresholds alpha, the inner ring is not optional.

One caution on ECD: it's in absolute 8-bit levels, so it scales with image brightness and with how you build F. The zero baseline here is exact because F was constructed from the same core. On a real image your clean baseline will be small but nonzero — measure it on a known-good cutout first and compare deltas, not absolutes.

Where these metrics stop

They measure color at the edge, nothing else. They say nothing about:

  • Semantic correctness — a clean-colored cutout of the wrong object passes all three.
  • Interior color accuracy — a recolored product body with perfect edges is invisible here.
  • Shadow handling — whether a soft contact shadow survived is a judgment call, not a channel measurement.
  • Perceptual impact — ECD in RGB is not ΔE in Lab. A 48-level green shift and a 48-level gray shift are not equally visible, but this metric treats them the same.

Run these as regression gates on your matting pipeline, not as a verdict on whether an image looks good.

Two practical notes if you adopt them:

  1. Fix G from the actual source background, not from what it looks like. "White" sweeps are rarely 255; a wrong G tilts every spill number.
  2. Report the band and inner ring separately. Collapsing them into one average is exactly how variant F slips through.

Reproducing this

The whole thing is one Vision segmentation call plus three numpy functions. The matte comes from VNGenerateForegroundInstanceMaskRequest; F is a nearest-neighbor fill from the eroded opaque core; G is set once at (46, 204, 113). Drop your own RGBA cutout in as variant A and the table becomes a report on your pipeline.

Change CORE_ERODE (6 px here) and INNER_RING (8 px) and the absolute numbers shift — write both down with the result, the same way you'd record a threshold. The definitions are the measurement.

If you extend this, the missing piece is per-region reporting: sample the handle, the thin outlines, and the broad body separately. A single global number hides a dead handle behind a great silhouette — a lesson that carried over directly from the geometry post.

When you despill a cutout, how do you catch the case where you removed too much?


Method note: all numbers collected November 2026 from one Unsplash-licensed photo and one Vision-generated matte. Variants are synthetic degradations derived from that matte, not outputs of any production tool. The metric code is standalone and does not depend on PicViva.

Top comments (0)