Measuring faces in the browser with MediaPipe: three things that broke

Jeff Li — Sun, 01 Nov 2026 11:45:49 +0000

I wanted a face-measurement tool with three properties most "face rating" sites don't have: the photo never leaves the browser, the same photo always gives the same numbers, and every formula is public. The result is FacialRatio, and its engine is open source. This post is about the parts that did not work the first time.

The setup

Landmarks: Google's MediaPipe Face Landmarker, loaded as WebAssembly only after the user picks a photo, so it never slows the first page load. It returns up to 478 points with a rough depth value for each.
Engine: one pure TypeScript function, analyze(input). No DOM, no randomness, no network. It takes landmark coordinates, not pixels, which makes it easy to unit-test against MediaPipe's canonical face mesh.
Hosting: static files. There is no server endpoint that could receive an image, which is a simpler privacy promise than "we delete it after processing".

Every length is divided by the pupil distance, so photo resolution doesn't matter. From there the engine computes 58 measurements and four scored parts: symmetry, facial thirds, facial fifths and a few golden-ratio comparisons. Each part is 100 − k × deviation, clamped to 0–100, and the overall score is a weighted sum (35 / 25 / 25 / 15). All the constants live in one file.

Problem 1: level heads looked tilted

I started with a pitch gate of ±8°: tilt the head more than that and the tool asks for a retake. On 13 ordinary front-facing portraits, 7 were rejected. When I looked at the numbers, MediaPipe read heads that were visibly level at a median pitch of +8.2°. One plainly level photo came out at 14.6°.

The fix was boring: centre the window on the offset the model actually has. The gate is now −8° to +16°. The lesson is that a landmark model's pose output has its own bias, and you only see it if you test on everyday photos rather than the canonical mesh.

Problem 2: a slight head turn read as asymmetry

Symmetry is measured by mirroring each landmark across a fitted midline and averaging the distance to its partner. On a perfectly frontal mesh that works. On a real photo with the head turned 5–8°, the far side of the face is foreshortened, and the engine reported it as asymmetry. That's exactly the kind of result that feels unfair to a user.

Since MediaPipe gives a depth estimate per point, the engine now estimates yaw, rotates the points back to frontal, and only then measures. On the same 13 portraits, the median mirror error dropped from 7.31% to 2.28% of pupil distance. Turns beyond ±10° still trigger a retake instead of being corrected.

Problem 3: the model can't see the hairline

Facial thirds are usually measured hairline → brow → nose base → chin. MediaPipe has no hairline point. The highest point it offers sits on the upper forehead. So the upper third and the face length are estimates, and the tool says so in the result and in the report instead of pretending. The reference I use (18 / 41 / 41%) is defined from that same upper-forehead point, so at least the comparison is like-for-like.

Why the golden ratio only gets 15%

It's the most-searched part of the topic and the weakest evidence. Holland (2008) found that φ-based masks don't describe a wide range of faces, and Pallett, Link and Lee (2010) found preferred proportions that differ from φ. The tool still shows each ratio and its deviation from 1.618, because people want to see them, but it carries the lowest weight. Nothing in the tool is validated against human ratings, and the methodology page says that plainly.

Counting without analytics code

I wanted to know how many people finish an analysis without adding an analytics SDK. When an analysis completes, the page requests an empty static file such as /e/done-golden-ratio-face.txt. The CDN's request log counts those requests. The request carries no image, score, cookie or identifier, only which step happened on which page.

What I'd do differently

Collect a small set of everyday test photos on day one. The canonical mesh hides every real-world bias.
Write down every constant in one config file from the start. It made the methodology page a direct render of the code, so the two can't drift apart.

The engine, its tests and the canonical-face fixture are on GitHub. The full method, including known limits, is on the FacialRatio methodology page. Feedback on the measurement approach is very welcome.

DEV Community: Jeff Li