Transparent PNGs: the 3 background-removal algorithms, why JPG can't do it, and the color-key gotcha

Every "make this background transparent" tool is doing one of three things under the hood, and picking the wrong one for your picture is why a logo comes back with a hole in it, or the background is only half gone.

Transparency lives in the alpha channel

A pixel needs a fourth value beyond R, G, B: alpha, how see-through it is. Only formats that define an alpha channel can store it - PNG and WebP per-pixel, GIF as one fully-transparent color, never JPEG.

This isn't an implementation detail, it's in the spec. The HTML Living Standard says it outright for canvas export:

For image types that do not support an alpha component, the serialized image must be the bitmap image composited onto an opaque black background using the source-over compositing operator.

So canvas.toBlob(cb, 'image/jpeg') on a transparent canvas doesn't warn you, it silently paints black behind everything and hands you that. Pillow is less quiet about it - trying to save an RGBA image as JPEG raises OSError: cannot write mode RGBA as JPEG outright.

The three removal methods

AI subject detection - segments the main subject (person, product, animal) from everything else. Handles busy backgrounds, one click, but check fine edges (hair, glass) before you trust it. This is what Windows 11 Paint's Remove Background button and the Photos app do (Paint since the update Microsoft announced in Sept 2023, version 11.2306.30).

Connected area (flood fill / magic wand) - starts at a point you click and grows outward through similar, touching colors. GIMP's Fuzzy Select and macOS Preview's Instant Alpha both work this way. If the background is split into several disconnected regions by the subject, you click each region.

Color key - removes every pixel close to one chosen color, anywhere in the image. GIMP's Color to Alpha and PowerPoint's Set Transparent Color use this. Fast on a flat background, but if that color also shows up inside the subject, it goes transparent too.

I actually ran both non-AI methods on the same test image with Pillow - a blue circle logo on white, with a white square cut out in the middle (standing in for white text or a hole in a badge):

from PIL import Image, ImageDraw

# color key: kill every near-white pixel in the whole image
def color_key(im, key=(255, 255, 255), tol=30):
    im = im.convert("RGBA")
    px = im.load()
    for y in range(im.height):
        for x in range(im.width):
            r, g, b, a = px[x, y]
            if abs(r-key[0])<=tol and abs(g-key[1])<=tol and abs(b-key[2])<=tol:
                px[x, y] = (r, g, b, 0)
    return im

# connected area: flood fill from a known-background pixel
img = Image.open("logo.png").convert("RGBA")
ImageDraw.floodfill(img, (0, 0), (255, 255, 255, 0), thresh=30)

Color key made the inner white square transparent along with the background - a hole in the logo. Flood-fill from the corner only removed the background that's actually connected to it, leaving the inner square intact. Same tolerance, same image, opposite result, because they're answering different questions ("is this pixel near white?" vs "is this pixel reachable from here through near-white pixels?").

Rule of thumb: if the background color repeats inside the subject (white lettering on a white-background logo, a stamp with a white rim), color key gives you a hole - use flood fill or AI instead.

Doing it in bulk: rembg

For more than a couple of images, rembg (MIT license, Python 3.11-3.13) wraps several AI segmentation models behind one CLI:

pip install "rembg[cpu,cli]"

rembg i photo.jpg photo-cutout.png        # one file
rembg p input_folder output_folder        # a whole folder

It downloads the model on first use (cached under ~/.rembg/models/) and runs locally after that - nothing gets uploaded, which matters if the images aren't meant to leave your machine. The many online "remove.bg"-style tools are AI too, but the file goes to their server; fine for anything you'd publish anyway, not for scanned IDs or internal docs.

In the browser, without a server round-trip

If you're building this into a web page rather than a script, the color-key approach is a <canvas> pixel loop plus a distance check - no upload, no dependency:

function knockoutBackground(imageData, [kr, kg, kb], tolerance) {
  const d = imageData.data;
  for (let i = 0; i < d.length; i += 4) {
    const dist = Math.hypot(d[i]-kr, d[i+1]-kg, d[i+2]-kb);
    if (dist <= tolerance) d[i+3] = 0;
  }
  return imageData;
}

I use this approach (plus a small feather at the tolerance edge, to soften jagged outlines) in a small image converter tool - drop an image, eyedropper the background, adjust tolerance, export as PNG or WebP. Same color-key limitation applies: same-color pixels inside the subject go transparent too.

---

Full write-up with the side-by-side comparison image, Paint/Photos/PowerPoint/Preview steps, and a troubleshooting list: https://www.coding-now.com/en/guides/transparent-background-png?utm_source=devto

What's your go-to for batch background removal - rembg, something cloud-based, or hand-rolled?