This is part 8 of the ClearPix engineering series, and it is the story of a bug that lived in the last 40 lines of the pipeline: the step that pastes the inpainted result back into the image. Users kept telling us some photos "look processed" — a soft, smeared halo around where the watermark used to be. We spent a while suspecting the model's fill quality. The model was mostly innocent. Our blending band was taking the model's full-crop reconstruction — which is not the original image outside the hole — and smearing it into pixels that had been perfectly fine. And the metric we used to accept changes was structurally incapable of seeing it, because it averaged the hole and its surroundings into one number.
If you take one thing home, take this: before optimizing a generative pipeline, split your quality metric by region — inside the mask, outside the mask — and treat your feathering band as a first-class suspect. The fixes that fell out of that one measurement took our restored-region PSNR up, our video end-to-end score up 5dB, and dropped the damage to clean pixels to exactly zero — while a plausible-sounding optimization we tried along the way got fully refuted and reverted in a single bench run.
Where watermark removal artifacts actually come from
Our image inpainting pipeline (part 5 covers the model stack) ends with a paste step. The algebra of the old implementation looks harmless:
out = original + a · (inpaintedCrop − original) a = max(binary mask, blur(mask, 6–22px))
The trap is inside inpaintedCrop. MI-GAN (and LaMa) reconstruct the entire crop, not just the hole. Outside the mask, the model's output is a reconstruction of the original — close, but never identical (codec-level drift, color shift, softened texture). So wherever a > 0, we are mixing model error into pixels that were clean to begin with. And because a includes a Gaussian-blurred feather band that extends 6–22px beyond the mask, every watermark we removed came with a rim of synthetic blur.
Here is the part that made this bug invisible: for the thin-stroke text watermarks that users complain about most, that rim is enormous relative to the watermark itself. We measured area(dilate(mask, 12px)) / area(mask) across our 13-case golden set:
| Case family | Ring band area vs. mask area |
|---|---|
| Small/faint text watermarks | 2.8 – 6.5× |
| Large text | 1.8 – 2.9× |
| Dark badges | 1.2 – 2.0× |
| Solid logos | 0.4 – 0.8× |
For a small text watermark, the majority of "the area we touched" was never watermark at all — it was the transition band. That is the processed look, quantified.
It also retroactively explained a contradiction in our own tuning log: dilating masks by 2–4px had measured −0.5dB on our golden set, which we had recorded as "expansion is bad." The real mechanism was the opposite of what we concluded — dilation itself is fine, but every pixel you add to the mask becomes a pixel the blend can pollute. The same measurement that punished dilation would reward removing the pollution.
Metrics that cannot see direction
Our acceptance metric, psnrRegion, computed PSNR inside dilate(groundTruth, 12) — the hole and its ring averaged together. A pipeline that damages the ring and a pipeline that heals the hole can post the same number. We added four split metrics before changing any pipeline code (all pure observation — the 12 shared cases re-ran bit-identical):
- PSNR-in / PSNR-out — restoration error inside the mask vs. in the outer ring, measured separately;
-
ringDelta —
mean |result − input|in the outer ring; 0 means clean pixels were untouched; - colorShift — Lab ΔE between the filled hole and the surrounding ring;
- SSIM (region) — because PSNR and human eyes disagree in both directions.
Plus a new golden-set case, halo-flat-text: thin-stroke text on a perfectly flat gradient with a tight ground-truth mask. On a flat background there is no texture to hide in — any out-of-band contamination shows up in the numbers with nothing to blame.
And running the "harmless" check first paid for itself immediately: the current snapshot scored 23.384dB against the archived 23.824 baseline. That looked like a regression — but tracing it landed on a commit from two days earlier where we had fixed real watermark bleed-through (the blend used to put model output at ~50% alpha at thin-stroke centers, leaking half the watermark back). The fix legitimately trades metric for correctness. Knowing that before touching anything is the difference between chasing ghosts and measuring progress.
The fix: stop blending model pixels into clean ones
The change (A4 in our plan) is small enough to state in one formula. The alpha stays max(soft mask, hard threshold) — inside the mask we keep 100% model output, which is non-negotiable (the earlier fix that killed bleed-through). Outside the mask, the artificial feather band is deleted; the only blending left is the mask's own upscale ramp, a pixel or two wide:
Same image, same detection masks, same model — only the paste step differs. In the before panel, look at the ledge line and the background around the removed text: a soft band of reconstructed non-information. In the after panel the background is bit-identical to the input wherever the mask wasn't.
The split metrics moved exactly the way the diagnosis said they should:
- ringDelta: 5.63 average → 0.00, on every case. Not "less visible" — zero. Nothing outside the mask is touched anymore.
- PSNR-out: 30.8 → 84.3 dB (most cases hit the 99dB ceiling, meaning bit-identical to the input).
- The halo case: 37.0 → 49.9 dB region PSNR.
- Overall restored-region PSNR on the 12 shared cases: 23.4 → 23.9 dB — modest, exactly as our P0 analysis predicted for cases where false-positive repainting inside the mask dominates.
- Video end-to-end (which shares the paste code): 33.9 → 38.9 dB masked-region PSNR, a +5dB jump from deleting a blending band.
The optimization that lied to us: when reference statistics are wrong
With the composite layer fixed, the next target was inner color blocks — cases where the fill's average color doesn't match its surroundings (up to ΔE 14.4 on our set). The textbook move is statistical color matching: shift the model output toward the mean of the surrounding ring before pasting. It sounds airtight. It failed, and the failure is the most useful thing we learned this round.
We built a scatter plot of the two references — the color difference against the ring (what the alignment sees) versus the color difference against the clean ground truth (what actually matters):
The ring lies in both directions:
- tiled-med scored ΔE 14.4 against its ring but only 0.3 against ground truth — the fill's color was essentially perfect; the watermark band simply sits across content that genuinely differs from its surroundings. Aligning it to the ring dragged a near-perfect fill away from the truth: inner PSNR fell 24.9 → 21.5.
- logo-br was the opposite: a real color bias of 10.3 hidden behind a ring score of 1.3 — the ring-based alignment could never even see it.
At runtime you cannot distinguish "biased fill" from "content that legitimately differs from its surroundings." Our bench run of the alignment confirmed the split — some cases improved (text-bl's ΔE 11.9 → 2.6 for nearly free), the tiled cases degraded past our per-case gate. We reverted it the same day, per the rule we set in part 7: decide the stop line first, and treat a tripped gate as a bug report, not an opinion.
What the attempt left behind is permanent: a new metric, ΔE-true (fill vs. clean ground truth — computable offline where ground truth exists), which is now the acceptance basis for any future color-alignment work. Baseline: average 3.6, worst case 10.3. The residual color-block problem is real; it will need a mechanism that respects content boundaries (gradient-domain blending aligns boundary continuity rather than forcing hole statistics), not a stronger version of the same lie.
Snap the mask to the real edge
The last piece targets the upstream cause of ragged edges: glyph masks are born at OCR probability-map resolution — roughly 1/4 to 1/16 of the image — so after upscaling, the boundary position is inherently off by 4–16 pixels. We added a guided filter (He et al., 2010) inside the glyph-mask builder: the detection image, resampled to the probability grid, acts as the guide, and the mask's 0.5 level set snaps to the visible watermark edge instead of the flood fill's incidental boundary.
Zero plumbing changes — the product path and the bench path both consume the refined bitmap as-is, the op contract and coordinate contracts untouched, and the filter runs on the quarter-scale grid so detection time doesn't move. The perceptual metric moved most where the complaint lived:
- SSIM: 0.739 → 0.753 average, with the largest gains exactly on the perceptually worst cases: text-br-large 0.708 → 0.770, text-huge 0.471 → 0.524, dark-text-br 0.577 → 0.615.
One honest caveat for anyone copying this: precision against our ground-truth masks dropped slightly (47.4 → 47.0), and that is not a quality signal — the ground truth is looser than the real edge, and a mask that hugs the real edge loses points against a loose GT while covering more of what actually matters. Trust the region PSNR and SSIM pair, and always eyeball the worst case.
The scorecard
Across the 13-case suite (12 shared + the new halo case), one week of composite-and-mask work, two shipped fixes and one refuted experiment:
| Metric | Before | After |
|---|---|---|
| ringDelta (damage to clean pixels) | 5.63 avg, >0 everywhere | 0.00 everywhere |
| PSNR-out (outer ring) | 30.8 dB | 84.3 dB (bit-identical in practice) |
| Restored-region PSNR (12 shared) | 23.4 dB | 24.2 dB |
| SSIM (region) | 0.710 | 0.753 |
| Video e2e, masked region | 33.9 dB | 38.9 dB |
| Halo case (flat background) | 37.0 dB | 49.9 dB |
No case regressed more than 1dB; all acceptance gates held.
Three rules we kept, one we learned
- Localize damage before you localize the fix. A single mixed PSNR number hid a real bug for weeks and misexplained a tuning result for months. The in/out split cost one bench run to build and paid for itself on the first look.
- The blending band is a first-class suspect. Generative pipelines end with a paste, and the paste is where "the model's version of your image" meets "your image." Any alpha > 0 outside the hole is a place the model can be wrong.
- Validate your reference, not just your method. Statistical alignment toward a ring assumes the ring represents what the hole should look like. Sometimes it doesn't — and the failure is invisible to the very metric that motivated the method. Measure against ground truth when you have it.
- Revert cheap, keep the instrument. The refuted alignment cost one bench run and left behind ΔE-true, which will judge every future attempt. That is a good trade.
All numbers in this post come from our public bench snapshots (run-2026-10-01*.json), the quality dashboard at clearpix.org/quality, and the commits they were measured on — the same gates that block our releases.
Part 8 of the ClearPix engineering series. Earlier parts: in-browser architecture (1), WebCodecs vs ffmpeg.wasm (2), fp16 quantization (3), video frame skipping + EMA (4), LaMa and Real-ESRGAN in the browser (5), automatic watermark detection (6), the anchor-based frame-skip redesign (7).
Top comments (0)