Most "remove the background" features ship a segmentation model to the browser or call an API. For one specific job, cleaning up product photos for an interior mood board, we got away with neither. Here is what we did, and where it falls apart.
The constraint: retail product photos are boring in a useful way
When someone drops a sofa or a lamp onto a mood board, the image almost always comes from a retailer's listing. Those photos share a trait: the object sits on a white or near-white studio background that touches every edge of the frame.
That means you do not need to understand what a sofa is. You need to find the background, and the background is the connected region of near-white pixels that starts at the border.
Flood fill from the edges
The approach is a plain flood fill run on a canvas:
- Draw the image to an offscreen canvas and read the pixel buffer with
getImageData. - Seed a queue with every pixel on the four edges whose colour is close to white (we compare each channel against a threshold rather than requiring pure
#fff, because JPEG noise and soft shadows are never pure). - Pop pixels, set their alpha to 0, and push their neighbours if they are also near-white and not visited yet.
- Put the buffer back with
putImageDataand export.
Because the fill only spreads through connected near-white pixels starting from the border, a white cushion in the middle of a grey sofa survives: it is not connected to the edge through other white pixels.
One small thing I like: the function reports how many pixels it removed. If that number is tiny, the photo probably was not on a white background, and the UI can say so honestly instead of pretending it worked.
Recolouring without a model either
The same board lets you slide a colour and watch a chair change. That is also pixel arithmetic on a canvas, no network call. The tricky part is deciding which pixels are "wood" and which are "fabric". We use a colour heuristic: warmer, more saturated, darker pixels are treated as wood, and the remaining opaque pixels are treated as fabric. It is wrong sometimes. It is also instant, deterministic and free.
Material swatches (wood grain, marble, plaster) are drawn in code too, using SVG feTurbulence. The same spec always renders the same swatch, which makes tests boring in the best way.
Where it breaks
- Products photographed in a room scene: there is no white border, the fill removes almost nothing, and the reported pixel count tells you that.
- White products on white backgrounds: a white lamp shade touching a white edge leaks into the background. Thresholds help a little; nothing fixes it fully.
- Heavy drop shadows: they are grey, not white, so they stay. Some people actually prefer that.
Why bother
Shipping a segmentation model for this would have added megabytes to the page and a lot of latency on cheap phones. For the 80% case, white-background retail shots, a flood fill is good enough, runs in milliseconds, and never phones home.
If you want to see it in context, it is the cut-out step in the mood board in HausDraft. Drop in a product photo and watch the edge go transparent.
I would be curious how others handle the white-on-white case without reaching for a model.
Top comments (1)
The removed-pixel count catches the almost-nothing case, but a white lampshade leaking into the edge-connected region can look like a highly successful removal. I'd keep that count as a diagnostic, not a pass score, and show a before/after toggle against both light and dark backgrounds before accepting the cut-out. A white-on-white fixture where the object touches only one border would be a useful regression case alongside your room-scene photo.