DEV Community

Cover image for Your photo never leaves the browser — building a client-side face analysis app with MediaPipe
PENG YONG
PENG YONG

Posted on

Your photo never leaves the browser — building a client-side face analysis app with MediaPipe

FaceRuler is a free face shape detector and attractiveness test. You drop in one photo, and a few seconds later you get your face shape (one of seven), a score on the PSL 0–8 scale, a casual 1–10 rating, a percentile, and a breakdown of eight facial metrics — all computed on your device. The photo is never uploaded. Not "we delete it after processing" — it never leaves the browser tab.

That single constraint shapes almost every engineering decision in the app. Here's the pipeline, and the parts that were actually hard.

Why on-device at all

Most "rate my face" tools send your selfie to a server and run a model there. That's a big ask: it's a photo of your face, tied to your identity, going to infrastructure you can't inspect. And for this specific product category — attractiveness scoring — the trust problem is the product. If the privacy claim isn't literally, verifiably true, nothing else matters.

So inference runs in the browser with MediaPipe's FaceLandmarker (478 landmarks) compiled to WASM. The model (~3.6 MB) and the WASM runtime are served from our own origin, so there's no third-party request at inference time either.

The pipeline: detect → measure → map

  1. Detect. MediaPipe returns 478 normalized landmarks. We draw them over a canvas as a visual receipt.
  2. Measure. Pure geometry converts landmarks into ratios — facial thirds, FWHR, canthal tilt, jaw ratio, and so on.
  3. Map. A deterministic function turns those ratios into 0–100 sub-scores, then into the PSL 0–8 scale.

No LLM. No "AI vibes". The same landmarks always produce the same report. That's a product decision as much as a technical one: a score you can't reproduce is a score you can't argue with — and this audience argues.

The scoring engine is just math (that's the point)

Every sub-metric is scored with a Gaussian falloff around a reference target:

export const targetScore = (value: number, target: number, tol: number): number => {
  const z = (value - target) / tol;
  return 100 * Math.exp(-0.5 * z * z);
};
Enter fullscreen mode Exit fullscreen mode

Eight sub-metrics roll up into five dimensions with fixed weights — Symmetry 0.2, Proportions 0.25, Structure 0.25, Eyes 0.15, Harmony 0.15. The weighted overall (0–100) maps to PSL with a logistic curve anchored so a perfectly average profile lands at exactly 4.0:

// overall 50 -> exactly 4.0 on the 0-8 PSL scale
export const overallToPsl = (overall: number): number =>
  8 / (1 + Math.exp(-(overall - 50) / 13));
Enter fullscreen mode Exit fullscreen mode

Median at 4.0, not 5. The PSL scale is deliberately stricter than a casual rating. Percentile uses a normal distribution with mean 4.0 and SD 0.9. Because it's all pure functions, the whole engine is unit-tested (12/12 green) and recalibration is a one-line change, not a retrain.

Four things that were genuinely hard

1. Head roll. A tilted photo shouldn't change your score. We measure the midline angle from forehead-top to chin, then rotate every landmark into a roll-corrected frame before computing anything. Vertical distances — like the FWHR numerator — are especially sensitive to this.

2. Aspect ratio silently shrinking the face index. MediaPipe returns normalized coordinates: x divided by image width, y divided by image height. Compute a height/width ratio naively and a 3:4 portrait reads about 25% "shorter" than the same face on a square canvas. We correct by scaling the y component back into pixel-equivalent space.

3. FWHR doesn't mean what the papers say. The academic facial width-to-height ratio (1.85–1.9) is measured on physical photographs. In normalized landmark space the nasion-to-upper-lip distance runs ~36% of bizygomatic width, so the numbers simply aren't comparable. We calibrated our own center (2.75) from landmark batches and documented the mismatch instead of pretending the paper value applied.

4. Harmony is not the average. Averaging sub-scores rewards a face that's mediocre everywhere. We wanted the opposite — consistency. So "Harmony" is derived from the standard deviation of the sub-scores: one dragging feature lowers it.

Front-end lessons that cost the most hours

React 19's synthetic onChange doesn't fire for programmatically dispatched change events. Our upload component has to work both for real users and for automated tests that inject a File via DataTransfer. The synthetic handler didn't fire in the injected path. Fix: drop the synthetic handler and wire a native addEventListener through a ref. Bonus — it's more robust for real users too.

Never put a ref-dependent DOM node inside a conditional render. Our result <canvas> lived inside report && (...). Before the first report, canvasRef.current is null, so the draw function's guard silently returned — no error, no output, just nothing. We hit this exact bug twice, in two different components. The rule we now enforce: ref-dependent nodes are always mounted and toggled with hidden.

A single dependency mismatch can 500 your entire content layer. All MDX pages broke with TypeError: files.map is not a function. Root cause: the lockfile resolved fumadocs-core to 15.8.5 while fumadocs-mdx 11.10.1 expected the 15.7.x interface. Pinning fumadocs-core@15.7.13 fixed it.

Node's undici fetch ignores your system proxy. Server-side OAuth silently timed out and surfaced as a misleading invalid_code. A ProxyAgent global dispatcher for dev fixed it — and no-ops in production on Workers.

Stack

Next.js 16 (App Router), deployed on Cloudflare Workers via OpenNext, with D1 for the license/entitlement tables and a hosted checkout for the paid deep report. MediaPipe runs entirely client-side; the server never sees an image.

Try it

It's live, free, and no sign-up: https://faceruler.com — one photo, on-device, and you get the full breakdown including the exact formulas behind every number.

If you're building something similar — client-side ML, or a privacy-sensitive tool where "we don't upload it" is the whole pitch — I'd love to hear how you handled the trust story.

Top comments (0)