DEV Community

ai startup
ai startup

Posted on AI-assisted

Remove image backgrounds in the browser with ONNX Runtime Web (no server)

#javascript #webdev #machinelearning #tutorial

Disclosure: I built this project with a lot of help from an AI assistant (Claude). I tested the result myself on real photos, and the notes below are what I actually ran into.

I wanted a background remover that never uploads the photo. Most online tools send your image to a server. That is a privacy cost, and a hosting cost for whoever runs it. It turns out you can run a segmentation model entirely in the browser with ONNX Runtime Web and WebAssembly.

Here is the approach, and the problems I hit.

The pieces

  • Model: IS-Net (general use), exported to ONNX and quantised to int8. The fp32 file was about 178 MB; the int8 file is about 46 MB.
  • Runtime: onnxruntime-web with the WASM backend.
  • Everything else: plain canvas.

Loading and running the model 

import * as ort from 'onnxruntime-web';
ort.env.wasm.wasmPaths = '/js/vendor/ort/';
ort.env.wasm.numThreads = 1;   // multi-thread needs cross-origin isolation headers

const session = await ort.InferenceSession.create(modelBytes, {
  executionProviders: ['wasm']
});
Enter fullscreen mode Exit fullscreen mode

numThreads = 1 is a deliberate choice. Multi-threaded WASM needs SharedArrayBuffer, which needs special COOP/COEP headers. On a static host that is more trouble than it is worth for a free tool.

Pre-processing

The model expects a 1024×1024 input. I resize the photo on a canvas, then convert it to a float tensor in NCHW order:

const S = 1024;
const c = document.createElement('canvas'); c.width = c.height = S;
const ctx = c.getContext('2d');
ctx.drawImage(img, 0, 0, S, S);
const { data } = ctx.getImageData(0, 0, S, S);

const input = new Float32Array(3 * S * S);
for (let i = 0; i < S * S; i++) {
  input[i]             = data[i * 4]     / 255;
  input[i + S * S]     = data[i * 4 + 1] / 255;
  input[i + 2 * S * S] = data[i * 4 + 2] / 255;
}
const tensor = new ort.Tensor('float32', input, [1, 3, S, S]);
Enter fullscreen mode Exit fullscreen mode

Check the model's expected mean and std. This one needed different normalisation than the small U2-Net model I used as a fallback, and wrong values silently give poor masks.

Post-processing is where quality comes from

The raw output is a soft mask. Two things made a big difference:

  1. Normalise the mask to the 0..1 range (min-max), because the output range differs per image.
  2. Refine the edges with a guided filter using the original photo (grey) as the guide. It snaps the mask edge to real edges in the photo, which removes the blurry halo you get from just upscaling the mask.

Then a smoothstep curve on the alpha gives a cleaner outline.

Problems I hit

  • Hosting a 46 MB file. Cloudflare Pages has a 25 MB per-file limit. I split the model into 8 MB parts, hosted them on a separate Pages project, and concatenate them in the browser. Dedicated caching headers make repeat visits fast.
  • Stale cache. After updating the JS, users kept running the old file. Adding a ?v=NN query to script tags fixed it.
  • Hard images. Hair, glass and shiny surfaces still leave rough edges. A small in-browser model will not match big paid cloud tools there. I added Restore and Erase brushes so people can fix details by hand.

Result

The tool is at picfitter.online. It is free, has no signup, and the photo stays on the device. For clear products on plain backgrounds it works well; for hard cases expect to touch up.

If you have tips on faster WASM inference or better refinement, I would like to hear them in the comments.

Top comments (0)