What I learned building an AI tool to remove people from photos

景阳 — Tue, 10 Nov 2026 08:08:28 +0000

I'm a solo developer, and my second side project is a small web tool that lets you remove a person from a photo: you paint over a photobomber, tap Remove, and the AI fills the area with background. It sounds like a solved problem. It mostly is, until someone uploads a photo from a restaurant.

Version 1: LaMa, and why it is so good on beaches

The first version uses LaMa, the classic inpainting model. You send an image and a black-and-white mask (white = remove), and it fills the hole using what it sees around it. On Replicate it runs in about 1.4 seconds and costs about $0.0003 per image, which is cheap enough to offer three free removals a day without thinking twice.

On simple backgrounds such as sand, sea, sky, grass or a plain wall, the result is genuinely hard to tell from the original.

Where it breaks

My first real test was a dinner photo: six people around a round table, a carved wooden chair behind each of them, an ink-painting mural on the wall. I painted over the woman on the right and got… a smear. The chair turned into a brown cloud, the mural into grey mush, and the arm of the woman next to her, which was reaching out to toast, bent into something unnatural.

That is not a bug. LaMa does not know what a chair is. It continues textures. When the hole is large and the background has structure, there is nothing to continue.

Trying four models on the same photo

I ran the same photo and the same mask through four models:

Model	Result	Cost
LaMa	Smeared chair and wall	~$0.0003
Bria Eraser	Clean chair and wall, but the neighbour still toasts an empty seat	~$0.04
FLUX Fill Pro (with a prompt)	Good, but it added a glass of whisky	$0.05
FLUX Kontext Pro (text instruction)	The most natural: chair rebuilt, the neighbour's arm moved back	$0.04

Kontext won because it understands the scene, not just the pixels. It even fixed the toast. But there was a catch.

The catch: instruction editors don't read masks

Kontext takes an image and a sentence like "remove the woman in the dark navy top on the far right". It ignores masks. My users paint; they do not type descriptions. When I sent Kontext the photo with the red paint on it and asked it to "remove whatever is under the red paint", it removed the red paint and kept the woman.

The fix: let a vision model write the sentence

So the Pro mode is a two-step pipeline:

Send a cheap vision model (Gemini 2.5 Flash) two images: the clean photo and the same photo with the user's red paint. Ask it to name what is under the paint in under 15 words. Giving it the clean photo matters: with only the red version it described a green trash bin as "dark brown".
Send the clean photo and "Remove {description}. Fill that area naturally… Keep everything else exactly the same." to Kontext.

On four test scenes (a beach photobomber, a street bin, a power pole by a lake, the dinner photo) the descriptions were all right, for example "the large dark green wheeled trash bin on the right", and the removals looked natural. Total cost is about $0.04 per image and 6 to 8 seconds.

How it is offered

Standard (LaMa): free, 3 a day, no sign-up.
Pro (describe + Kontext): one free try a day, then one-time credit packs.

After a Standard result there is a "Try Pro on this photo" button that reruns the same painted area with Pro. Seeing the two side by side explains the difference better than any pricing page.

Small things that mattered

Grow the mask by 2 to 3 pixels. Users never paint the edge exactly, and the leftover outline is the most visible artifact.
Replicate throttles small accounts. Mine was limited to "6 predictions per minute with a burst of 1", so the second call in the Pro pipeline failed. The fix: read Retry-After, wait and retry, and if it still fails, give the user's credit back.
Daily limits need to be atomic. "Read the count, check it, write it back" in a key-value store lets parallel requests slip through. A Cloudflare Durable Object per day handles one request at a time, so 20 parallel requests from one IP get exactly 3 through.

I open-sourced the brush that paints the mask (zero dependencies, touch-friendly, exports the black-and-white PNG): mask-brush on GitHub.

If you want to try it, it is at swiftremover.com. I would love to hear about photos where it fails.

DEV Community: 景阳