Marketing loves a single millisecond. Real browser-side matting does not.
We run BiRefNet locally in the visitor's browser so pixels never leave the device. That choice makes latency a property of their machine: CPU class, WASM thread count, cold versus warm model cache. Quoting one lab laptop as "the" speed was the wrong default.
The wrong default
Early pages said something like "about two seconds." That figure came from one desktop class on one model tier. Readers with fewer cores, cold caches, or mobile devices saw something else and assumed the product was broken. A single hero number hides the spread.
We also tried enabling WebGPU as the "fast path." On ORT-web with our quantized weights, WebGPU did not merely run slower. It computed wrong masks. So USE_WEBGPU stays false. The live path is WASM-only, fixed square input at the current tier, lite fp16 gzip around seventy-eight megabytes. Correctness first.
What we ship instead
We opened a public benchmarks page built from opted-in telemetry. Last ninety days hold about five hundred samples. Each row is a phase (download, cache hit, session create, first inference), backend, device class, and CPU bucket.
Rules we follow on that page:
- Show p50 and p95, not a marketing minimum.
- Buckets with too few samples stay listed but greyed so absence of data is visible, not silently dropped.
- JSON and CSV are downloadable so anyone can recompute.
One solid slice from the older mid-resolution quantized model (not the current larger fp16 tier): first inference on desktop WASM in the common eight-to-eleven-core bucket, with dozens of samples, sits near a two-second median. The same phase's p95 stretches past ten seconds.
A nearby twelve-plus-core bucket shows a higher median around three and a half seconds. Cache hits land in tens of milliseconds; cold downloads sit in the multi-second range. Session create for those classes is roughly two seconds at the median.
Those rows are historical mid-tier samples. The live product now serves the larger fp16 WASM build. We keep the older distribution public so the method stays honest while newer tiers accumulate past the display floor.
Cost of this approach
Publishing a distribution costs more words than a badge. It also costs the discipline not to cherry-pick. We accept grey cells and model-version labels instead of inventing a WebGPU "about one-point-seven seconds" story that our code does not run.
The upside is boring and useful: when someone asks "how fast is it?", we can point at buckets that match their machine class, and they can download the same numbers we use.
Live tables, greying rules, and downloads: https://koutuxia.com/en/benchmarks?utm_source=devto. I built this tool (Koutuxia); the benchmarks page is our own telemetry, not a third-party lab.
Top comments (0)