DEV Community

Cover image for Running ML models in the browser with ONNX Runtime Web: a practical guide
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

Running ML models in the browser with ONNX Runtime Web: a practical guide

#onnx #browserai #computervision #guide

FaceVision does face detection, recognition and liveness checks without sending a single frame to a server, because every model runs in the browser through ONNX Runtime Web. That architecture is the reason its privacy story is simple. It is also the part students find hardest to get working, so here is the path that worked.

Export, then verify

Export the model to ONNX from its training framework and immediately run the same input through both the original and the exported graph. Compare outputs numerically. Every model in FaceVision was verified against its actual ONNX graph and, where possible, the reference implementation — not assumed from documentation. Half the "the model is broken in the browser" reports I have seen were pre-processing mismatches that a five-minute comparison would have caught.

Choose the execution provider deliberately

  • WebGPU — fastest where available; check support and fall back gracefully.
  • WASM with SIMD and threads — the reliable default; needs the right headers for multi-threading.
  • WebGL — legacy; avoid for new work.

Pre- and post-processing is where bugs live

Models expect a specific channel order, normalisation and input size. Browser image data arrives as RGBA bytes in a different layout. Write the conversion once, test it against a known image, and keep the output tensors' shapes in a comment next to the code. Post-processing — decoding anchors for a detector, normalising an embedding before cosine similarity — deserves the same care.

Performance that feels live

  1. Warm up the session with one dummy inference on page load; the first run is always slow.
  2. Run inference in a Web Worker so the camera preview never stutters.
  3. Downscale frames before detection; run recognition only on detected crops.
  4. Skip frames when the queue is full rather than letting latency grow.

What the backend does when the model does not

In FaceVision the FastAPI and PostgreSQL backend stores only vector embeddings and matches them at enrolment and verification. Because no images are ever persisted, the privacy claim is easy to explain and audit. I wrote about the design reasoning in face recognition that never uploads a face.

Browser-side inference is not a gimmick. For anything involving faces, documents or health data, it is often the architecture that makes the product acceptable to the people using it.

About Pranjul Rathour

Pranjul Rathour in front of an Integral Startup Foundation hackathon backdrop
At an Integral Startup Foundation hackathon

Pranjul Rathour in a checked shirt inside a packed college auditorium
In a packed college auditorium

Pranjul Rathour in a suit and tie with a lanyard at a formal campus event
At a formal campus event

Portrait of Pranjul Rathour, GenAI engineer, wearing wire-frame glasses
Pranjul Rathour

Pranjul Rathour presenting on stage in a blue polo, with his Annapurna demo video on the screen behind him
Presenting Annapurna on stage

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-06.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)