DEV Community

Cover image for I built a background remover that never sees your photo (and it does GIFs and videos too)!
Rishab Dugar Jain
Rishab Dugar Jain

Posted on AI-assisted

I built a background remover that never sees your photo (and it does GIFs and videos too)!

#webdev #ai #javascript #opensource

When you’re the only engineer in the family, you automatically become the unpaid IT department, video producer, and resident graphic designer, because to relatives, "computers" is just one big subject (and that includes everything to do with mobile phones too... somehow!). So whenever someone needed a transparent PNG or a quick cutout, it fell on me, and I’d get stuck in the same annoying loop. I'd hunt down a "free" background remover (whatever showed up first on Google), upload the photo, wait, and then get held hostage by a "Buy Credits" button. My only options were settling for a blurry, postage-stamp-sized preview, or making a whole new account if I was desperate enough for the high-res version. Worse, somewhere in that mess, a family photo would get shipped off to a sketchy server I know nothing about.

For a product shot, that's annoying. For a passport photo, a signature, or a client's photo, it's a bit more than annoying.

So I added a background remover to Fizzdoc, my open-source toolkit where every tool runs inside the browser tab. It now does:

You can keep the background clear, or put white, any colour, a blur or another photo behind. It's free, there's no sign-up, no watermark, and no limit on how many photos you do. The size limit comes from your device's memory, not from me.

And the photo never leaves your device. Not "we delete it after an hour". It just never goes anywhere.

Below is how it works, and what broke along the way.

The rule: nothing gets uploaded, and you can check

The Remove Background page. The badge under the title says what happens to your photo: 0 bytes uploaded.

Fizzdoc is a static site on Cloudflare Pages. There's no backend, so there's nothing that could receive a file. The page also ships a Content Security Policy with connect-src 'self', which means the browser itself blocks the page from talking to any other website.

That rule shaped everything below. No CDN for the model, no hosted inference API, no "just this one request". Open DevTools, go to the Network tab, remove the background, and you'll see the model files coming from fizzdoc.com and nothing going out.

Picking a model

I wanted three things:

  1. Good edges on hair and fur, because that's where background removers usually show their seams.
  2. Small enough to download once into a browser.
  3. A licence that lets anyone use the result, including for work.

I went with BiRefNet (MIT licence), the lite variant, using the 512 × 512 ONNX export by studioludens. I store its big weight tensors as 16-bit floats and cast them back to 32-bit when the model loads, which halves the download to about 100 MB without changing the maths it runs.

For phones there's a second model, U²-Net small (Apache-2.0), just 4.6 MB. I learned this one the hard way: someone sent me a screenshot of an early version on a 6 GB Android phone, where Chrome killed the tab at 5% ("Aw, Snap!"). A browser tab on a phone gets far less memory than the phone has, and when it runs out there's no error to catch, the tab just dies. So now phones always use the small model, and the page says so ("for the cleanest edges on hair and fur, use a laptop"). If the big model ever crashes a tab on a laptop too, the page remembers it for 30 days and uses the small one there.

And for laptops with a graphics chip there's the opposite: a Sharper edges (hair, fur) button. It reruns the photo through a 1024 × 1024 export of the same BiRefNet-lite model (onnx-community, MIT). It's a one-time 135 MB download and needs WebGPU, so it's never offered on phones. If the device can't manage it, you keep the result you already had.

Running it: WebGPU when it can, CPU when it can't

The model runs with ONNX Runtime Web inside a Web Worker, so the page never freezes.

  • If the device has a graphics chip with WebGPU, the model runs there. This is the fast path.
  • If WebGPU isn't there (or the driver says no), the same model runs on the CPU through WebAssembly.
  • If that fails too, usually because the device ran out of memory, the small U²-Net model takes over.

So the chain is GPU, then the same model on CPU, then the small model. Nobody gets an error just because their laptop is old.

Two fun problems with "self-host everything"

Cloudflare Pages has a 25 MiB limit per file. The model is about 100 MB and ONNX Runtime's WebGPU build is a 27 MB .wasm. So both live on the server as 20 MiB parts: the models are split by a small script when I add them, and the runtime is split at build time. The worker downloads the parts, joins them, and keeps the result in the browser's Cache Storage, so the next time it starts straight from there, even with the internet off. The build also fails if any file it writes goes over 25 MiB, after one deploy failed on it.

A runtime that exists in pieces. ONNX Runtime normally fetches its .wasm as one file. Mine exists just as parts on the server, so the worker joins them from Cache Storage and hands the bytes to the runtime directly with ort.env.wasm.wasmBinary. It needs no extra hosting and no change to the security policy.

The part I'm proudest of: full resolution from a 512 px model

Top: the page loading the model once, and the options panel. Bottom: the original photo, the cut-out, and a new background in one tap.

The model looks at your photo squeezed to 512 × 512 and returns a soft mask. If you stretch that mask back over a 4000 × 3000 photo, the edges look like they were cut with a butter knife.

So the mask goes through two classic image-processing filters, both plain arithmetic on typed arrays in the worker:

  1. Fast guided filter (He and Sun, 2015). It uses the photo itself as a guide to snap the blurry mask onto the real edges, so hair follows the actual strands. The fit is computed on a smaller copy (up to 1280 px) and then applied to every pixel of the full photo in one pass.
  2. Blur-fusion foreground estimation (Forte, 2021). Edge pixels are a mix of your subject and the old background. Put a cut-out on a new colour without fixing that, and you get a faint halo of the old background. This step estimates the subject's true colour at the edges, so the new background looks like it was always there.

By default the result is saved at your photo's own size: your own pixels, with a cleaner mask.

When the AI gets it almost right

It's an AI model, so a busy background or a subject that blends into it can confuse it. For photos there's a small set of touch-ups:

  • Erase / Restore brush, with Undo, Redo and Zoom for careful work on a phone.
  • Tap to remove and Tap to keep: one tap picks the whole connected area of a similar colour and removes it, or brings it back.
  • Shadow and a white Outline, if you want the cut-out to stand out.
  • Crop: the whole photo, fit to the subject, or a square around the subject for online shop listings.
  • Hold to see the original, to compare before and after.

GIFs: every frame, same rhythm

Animated GIFs are sneaky. Many frames aren't full pictures, just small patches drawn over the previous frame, with rules ("disposal methods") about what to clear afterwards.

  • gifuct-js (MIT) decodes the frames, and Fizzdoc composes each one into a full picture the way a browser would show it.
  • Each frame goes through the same model (in a faster mode, since there can be dozens).
  • gifenc (MIT) writes the new GIF, keeping every frame's delay and the loop.

One catch: a GIF pixel is either fully see-through or fully solid. So on a clear background the edges are a little sharper than in a PNG. Pick a colour or a photo behind and the soft edges are back.

Videos: every frame, sound kept

Remove Video Background: every frame gets a new background, and the sound is kept.

For video, Mediabunny (MPL-2.0) reads the file and the browser's own WebCodecs encoder writes the new one. Every frame goes through the model and gets its new background, and the soundtrack is carried over.

The output is MP4 (H.264) if the browser can handle it; otherwise, it's WebM (VP9). Before you start, the page shows a preview frame and an estimate of how long the whole clip will take on that device.

What does "no background" mean for a video? Truly see-through video would need the encoder library to start helper workers from blob: URLs, which my page's security policy blocks, and most players can't show it anyway. So "None" gives you a green screen, which any video editor removes in one click. I chose to keep the policy strict rather than loosen it for one feature.

Stickers: words, emoji, and the last step in WhatsApp

The WhatsApp Sticker Maker turns any photo into a 512 × 512 sticker with a white outline, under 100 KB.

The WhatsApp Sticker Maker starts from the same cut-out. It squares it to 512 × 512, adds a white outline, and keeps the WebP under 100 KB by lowering the quality step by step until it fits.

On top of that, you can add words in five styles (Funny, Cute, Cozy, BIG, and Serious) and emojis. Each one is a layer you drag, pinch, resize and rotate, on a phone or with a mouse. The fonts are self-hosted like everything else: Bangers, Kalam, Teko and Inter, with Baloo 2 covering Hindi letters. Emoji are drawn with your device's own emoji font.

One fun bug on the way: Vite inlines small files as data: URLs, and a few of the font files were small enough. My own CSP (default-src 'self') refused them, and the end-to-end test that fails on any CSP violation caught it before it shipped. Fonts are now always separate files.

One limit, though. A web page can't add a sticker to WhatsApp, and a sticker shared from the browser arrives as a normal photo. So instead of a "Share to WhatsApp" button that doesn't do what it says, the page shows WhatsApp's own steps: save the sticker, then in any chat tap Stickers › Create and pick it. On iPhone there's a "Save to Photos" button, because downloads go to Files and WhatsApp picks stickers from Photos.

What it's like on low-end phones

The same tool on a phone, in dark mode. Phones use the small 4.6 MB model so the tab doesn't crash.

I care about this a lot because many people open Fizzdoc on Android phones.

  • Phones always get the small model, so it works rather than crashes. Edges are a bit softer than on a laptop, which is a trade I'll take every time.
  • The worker keeps the page responsive while it thinks.
  • The model downloads once (the page shows the progress in MB). After that it starts right away, and it keeps working with Wi-Fi and mobile data off, which is also the easiest way to check that nothing is uploaded.
  • Videos give you a time estimate first: short clips are fine, long ones are a laptop job.

Things it doesn't do (yet)

  • Truly transparent video (see above, green screen instead).
  • Many photos in one go: the background remover takes one photo at a time.
  • Several photos in one sticker. That's next.

Credits

None of this would exist without these people and projects. Thank you:

Project What it does in Fizzdoc Licence
BiRefNet by Peng Zheng and co-authors The background model MIT
birefnet-lite-512 by studioludens The 512 px ONNX export MIT
BiRefNet_lite-ONNX by onnx-community The 1024 px export for Sharper edges MIT
U²-Net by Xuebin Qin and co-authors The small model for phones Apache-2.0
ONNX Runtime Web by Microsoft Runs the models on WebGPU or WebAssembly MIT
gifuct-js by Matt Way Reads animated GIFs MIT
gifenc by Matt DesLauriers Writes animated GIFs MIT
Mediabunny by Vanilagy Reads and writes videos MPL-2.0
Kaiming He and Jian Sun The (fast) guided filter Paper
Marco Forte Blur-fusion foreground estimation Paper

The full licence texts are in Fizzdoc's third-party licences.

Try it, break it, tell me

It's part of a bigger toolkit of 68 free tools (PDF, Word, Excel, PowerPoint, images and audio), all running in your browser, in 16 languages. If something looks off on your device, there's a "Report it" link on every tool. And if you like it, a star on GitHub genuinely makes my day.

Top comments (0)