DEV Community

yue xing
yue xing

Posted on

Why a matting model shouldn't see your full-resolution photo: mask median 0.975 to 0.223

The ONNX export of ISNet, a background-removal model, declares its input height and width as dynamic. That raises a fair question for anyone learning this stuff: if the network accepts any size, why would a pipeline squash every photo to 1024×1024 and stretch the mask back up afterwards? A 2048 portrait loses three quarters of its pixels on the way in, and hair looks like exactly the place where those pixels should matter. A test on 2026-11-13 fed the full 2048 to the same model. It runs, and the mask stops making sense.

The pipeline in question is the background removal in ImgIng https://imging.ai/ , which runs the model inside a desktop browser on an Apple-silicon Mac; the model settings below come from the config its live page loads. The fast tier is ISNet INT8 (44,229,662 bytes) and the pro tier is BEN2 FP16, both at 1024×1024. The top tier switches to a different model, BiRefNet HR-Matting FP16, with a 2048×2048 input. The photo is stretched into a square without keeping its aspect ratio, and the mask is stretched back to the original size with canvas smoothing set to high. The loaded code shows no edge refinement step after that.

The same model accepted a 2048 input

The fast tier's model file (same SHA-256 as the live config) was run locally with ONNX Runtime on the CPU, with the preprocessing copied over: resize to a square, then scale each value as (v − 128) / 256. The sample was a CC0 portrait from Wikimedia Commons with long, windblown hair, cropped to a 2048 square. Both runs returned a mask of the expected shape, and both maxima sat right next to 1. The median told a different story. At 1024 the median alpha was 0.975, which is what a sane mask looks like on a crop where the person fills most of the frame. At 2048 it was 0.223, so the typical pixel went from clearly foreground to probably background. Dynamic axes tell you the graph accepts the shape, not that the weights know what to do with it. Side note for anyone thresholding model output: the minimum was −1.19e-7, and float output can stray a hair outside 0 to 1, so leave some margin in any check on it.

Training resolution in plain words

This part is an explanation, not a measurement. A convolution filter looks at a small fixed patch, and the network stacks many of them with downsampling in between, so the deepest layers cover big chunks of the picture. Trained on 1024 inputs, the model learned how many pixels a face or a clump of hair usually spans on that grid. Double the side and every object is twice as wide, so each deep feature now covers a quarter of the area it was trained to reason about, a bit like reading a poster with your nose on the paper. That is a plausible reading of the 0.223, but the intermediate layers were not probed, so which part breaks first is still open.

Shrinking first and stretching back costs very little

The next worry is how much hair the stretch back loses. There was no ground truth, so the reference is the top tier's output on the 2048 photo: a higher-resolution model's result, not the truth. The portrait was area-resized into smaller "originals", the reference mask was squashed to 1024 and stretched back, and the score is mean absolute alpha error on a 0–255 scale inside the hair band (where the reference is partly transparent, 0.02 < α < 0.98, plus roughly 2 px around it). At a 1.41× stretch bilinear gave 1.51 and bicubic 0.73; at 1.88× they gave 1.43 and 0.79. The fast model's own mask against the same reference was off by 51.57 to 55.32 across sizes from 768 to 1920, and that number didn't climb with photo size. So the lazy-looking step costs about 1 in 255, and most of the gap is the model's judgment. Even the 2048 reference shows flyaway strands as one soft clump, so these numbers describe the outline of the hair mass, not single hairs. Scope check: a single image on a single machine, each setting run once, and no original above 1920 tested.

What to check before feeding a bigger image

Look up the training input size first, in the preprocessing code or the model card, and treat it as part of the model rather than a default you can raise. For more detail at the edges, the option this data points to is a model trained at the larger size, the way the top tier moves to a 2048 model, instead of handing more pixels to a 1024 one. Before spending an evening on fancy mask upsampling, run the round trip on one image; if it comes out around 1 in 255, the upsampler has little left to give on that image. Where the cliff sits between 1024 and 2048 is unknown, since only those two input sizes were tried.

The portrait in this post: "Windblown" by Kaci Baum, CC0, via Wikimedia Commons.

Top comments (0)