We put an ONNX background-removal container behind a Next.js Worker on Cloudflare

Disclosure: I build CleanWhiteBG, the tool described here. This post is about the infrastructure, not a pitch — every number below comes from the running code.

Background removal looks like a one-liner in a demo: load an image, run a segmentation model, get a mask. In production it turns into a queueing problem, a cold-start problem, and a "why is my Worker crashing with error 1101" problem.

Here is what our stack actually looks like after a few rounds of getting it wrong.

The constraint that shapes everything: 30 seconds of CPU

Cloudflare Workers give you a hard CPU budget per invocation. We set ours to 30,000 ms in wrangler.jsonc:

"limits": { "cpu_ms": 30000 }

On top of that, waitUntil does not buy you a background thread that can run for a minute. Work that outlives the response is not reliable work. So you cannot simply accept an upload, run a 10-second inference in the background, and return 202 Accepted with a promise to email the user later.

Our answer was to split the pipeline so that the expensive part runs inside a request the client is still holding open, and the client drives the state machine.

Submit, then poll

POST /api/tool/background-to-white does almost nothing:

1. Check the per-IP rate limit.
2. Parse the multipart body.
3. Validate the MIME type (JPEG, PNG, WebP) and a 10 MB size ceiling.
4. Write the original to R2.
5. Create a job row in D1 and return the jobId.

That round trip is about a second. The actual inference happens later, when the client polls GET with that job id. The first poll that "claims" a queued job is the request that actually runs the container inference, with a long-poll budget of 20 seconds and a 3-second client interval.

It is a slightly unusual shape — the work happens in the GET, not the POST — but it means every request stays inside its CPU budget, and a dropped client connection just leaves the job to be reclaimed later.

Cold starts: loading a model while the health check is watching

The "Fast Cloud AI" path runs BiRefNet-lite exported to ONNX, served by ONNX Runtime on CPU inside a Cloudflare Container.

The container is a python:3.12-slim image with the model baked in at build time:

RUN pip install --no-cache-dir onnxruntime==1.21.1 Pillow==11.0.0 numpy==2.2.6 flask==3.1.1 gunicorn==23.0.0
RUN pip install --no-cache-dir huggingface_hub==0.34.4 && \
    python3 -c "from huggingface_hub import hf_hub_download; hf_hub_download(repo_id='onnx-community/BiRefNet_lite-ONNX', filename='onnx/model.onnx', local_dir='/app/models')"

Two decisions here matter more than they look.

First, the model is in the image, not fetched at boot. Pulling it from Hugging Face on every cold start adds seconds you cannot control and a failure mode you cannot fix at 2 a.m.

Second, gunicorn binds the port before the model finishes loading. If you load the model in the module body, the HTTP port stays closed while ONNX Runtime reads weights, and the platform's health check fails before your service ever answers. Instead we load it in a background thread and have requests wait:

session = None

def load_model_async():
    global session
    session = ort.InferenceSession("/app/models/birefnet_lite.onnx",
                                   providers=["CPUExecutionProvider"])

def ensure_model(timeout: float = 60.0):
    # block until the background load finishes, retry synchronously once on timeout
    ...

The port is ready in milliseconds, health checks pass, and the first real request just waits a little longer.

Even so, a genuinely cold container means pulling the image, starting the process, and loading the model — we measured that in the low tens of seconds. That is far too slow to sit behind a user's click.

So we hide it: the tool page pings GET /api/warm when it mounts and when the user starts choosing a file. That call is idempotent — it asks the Durable Object to start the container and returns immediately if it is already running. The cold start happens during the several seconds a human spends browsing for a photo, not after they hit submit.

The landmine: do not name an R2 binding IMAGES

This one cost us a production incident, so it is worth repeating.

OpenNext's /_next/image handler treats env.IMAGES as a Cloudflare Images binding and calls .input() on it. If you also have an R2 bucket bound as IMAGES, the runtime gets a bucket where it expects an image service, and the Worker dies with error 1101 — a crash, not a caught exception.

Our binding is therefore called R2_UPLOADS, with a comment in wrangler.jsonc so nobody "tidies" the name later:

{ "binding": "R2_UPLOADS", "bucket_name": "cleanwhitebg-images" },
{ "binding": "IMAGES" }   // Cloudflare Images binding for /_next/image — fixed name

The general lesson: binding names are part of the framework's API surface, not your namespace.

Not every edge needs the same model

A small ONNX model is fast and cheap and handles most photos. It is also visibly worse on the cases people actually care about — flyaway hair, jewellery, fur, anything translucent with a busy background.

Rather than run one big model for everything and pay for it on every request, we route on the quality parameter the user picked:

• standard → the self-hosted BiRefNet-lite container, CPU, cheap, fast.
• quality → pre-process only when needed (downscale beyond 2K, pad when the aspect ratio is extreme), then hand off to a larger hosted model and poll its progress.

Same API, same job table, two very different cost profiles. The user chooses the tradeoff, and we do not pay the expensive path for a passport photo that did not need it.

Cleanup is a cron, not a best-effort delete

Uploaded images are stored in R2 and the public promise is that they are gone within two hours. We do not rely on the request path to guarantee that. A Cron Trigger sweeps expired uploads every 15 minutes, and a second cron (every minute) resumes orphaned batch jobs:

"triggers": { "crons": ["*/15 * * * *", "* * * * *"] }

If cleanup only ran when a request happened to finish successfully, a crash would leave user images sitting in a bucket indefinitely. A cron makes the retention promise independent of traffic.

What I would tell someone starting today

• Decide early whether your work fits in one request. If it does not, design the job table and the polling contract before you write the model code.
• Load models in a background thread so health checks pass. A container that never becomes healthy is worse than a slow one.
• Bake weights into the image. Network fetches at boot are a reliability tax with no upside.
• Read the framework's binding names. IMAGES is taken.
• Hydrate the expensive resource before the user needs it, during a gap where they are already waiting on something else.
• Make retention a scheduled job, not a side effect.

The whole thing is a Next.js 16 app running through OpenNext on Workers, with D1 for job state, R2 for objects, and one Python container doing the segmentation. It is several moving parts — but each one is doing a job the others genuinely cannot.

FAQ

Why not run the ONNX model inside the Worker itself?
Workers do not ship a general-purpose CPU inference runtime for arbitrary ONNX models, and even if you got one working, the 30-second CPU ceiling and memory limits make it a poor fit for a 1024×1024 segmentation pass. A container gives us the real runtime and lets the Worker stay a router.

Why does the client poll instead of the server pushing?
Because the work is tied to a request's lifetime. Long-polling keeps the inference inside a request that is still open, which is the only place we can safely spend that CPU. A push model would need the work to outlive the response, which is exactly what we cannot rely on.

Is the inference running on a GPU?
No — CPU only, ONNX Runtime with CPUExecutionProvider. That is why the model is the "lite" variant and why the expensive edge cases are routed to a hosted model instead.

What happens if a user closes the tab mid-job?
The job stays in D1 in a claimed state. A stale-claim timeout lets a later poll or the batch cron reclaim it, so abandoned work gets cleaned up rather than stuck forever.