I Built an AI Face-Analysis Suite From Scratch — Here's What I Learned

Ahsan Fayyaz — Fri, 17 Jul 2026 22:21:37 +0000

The Problem With "Which Celebrity Do I Look Like?" Tools

If you've ever searched for a celebrity look-alike tool, you've probably noticed most of them fall into one of two camps: quiz-style pages that ask you five vague questions about your own face ("is your jaw pointed or round?"), or apps that quietly outsource the heavy lifting to a third-party face recognition API and slap a UI on top.

Neither approach sat right with me. Self-assessment is unreliable — people are bad at judging their own proportions, especially in a mirror where everything is flipped and inconsistently lit. And outsourcing the model means you're limited by whatever a generic library was trained to do.

So I built StarDoppel, a suite of AI-powered face-analysis tools, with the matching engine written entirely in-house in Python rather than wrapped around an off-the-shelf face recognition library.

How the Core Matching Engine Works

The idea is simple in concept, harder in execution: read a photo, extract measurable structure, and turn that structure into a comparable score.

Landmark detection — When a photo is uploaded, the model maps facial landmarks: eyes, nose, jawline, lips, and their relative positions.
Proportion extraction — Those landmarks get converted into a set of proportional measurements (not raw pixel distances, since photos vary wildly in scale and angle).
Distance scoring — Each face gets compared against a reference database using a raw distance calculation. The smaller the distance between your measurements and a given reference face, the higher the similarity percentage.
Ranking — For the celebrity matcher specifically, the five closest matches (by distance) are returned, sorted from closest to furthest. Because the scoring is distance-based rather than a fixed threshold or bucket, small variations — a different angle, a different expression — can shift percentages enough to change which result lands in first place. That's a deliberate tradeoff: fixed-threshold classification is easier to explain but throws away information, while distance-based scoring preserves nuance at the cost of being slightly less "clean" to talk about.

Beyond Celebrity Matching: A Full Face-Analysis Suite

Once the landmark-detection pipeline existed, extending it to other structural questions was a natural next step. StarDoppel now includes:

Face Shape Detector — classifies into oval, round, square, heart, diamond, or oblong based on measured width/length/jaw ratios rather than self-reported quiz answers
Face Symmetry Test — compares the left and right sides of the face across multiple landmarks and returns a symmetry score
Golden Ratio Face — measures facial proportions against φ (1.618) across several ratios (eye spacing, nose-to-chin, face height) and produces an overall harmony score
Face Age Test — estimates age from facial structure and texture markers
Eye Shape, Lip Shape, and Nose Shape Tests — each reads a specific set of landmarks (eyelid crease and depth for eyes; fullness, symmetry, and cupid's bow curvature for lips; bridge curvature, tip angle, and nostril width for the nose) to classify that feature into recognized categories Each tool shares the same underlying landmark-detection foundation, just pointed at a different subset of facial geometry.

Design Decisions Worth Talking About

A few choices that came up repeatedly during development:

No account requirement. Every tool works with zero signup. This meant designing the backend to be fully stateless per-request — no user session tied to a stored photo, no persistence layer for images at all.

Photos are deleted immediately after processing. Each image exists on the server only long enough to generate a result. This shaped a lot of the architecture: no caching of uploaded images, no logging of raw photo data, and processing pipelines that explicitly discard the file reference once a result object is returned.

Handling ambiguous input gracefully. If more than one face is detected in a frame, the system doesn't try to guess which one the user meant — it returns an error and asks for a new photo. Same behavior when no face is detected at all (blur, extreme angle, poor lighting). Silent wrong guesses are worse than an explicit "please try again."

What doesn't break the model. Interestingly, makeup and facial hair don't meaningfully affect results, since the underlying bone/cartilage structure is what's being measured rather than surface texture. Masks are a different story — they remove too many reference points for a reliable read.

What's Next

The celebrity database currently reflects the actor set it was built with, sourced from IMDb — expanding and refreshing that dataset is an ongoing task, along with extending coverage beyond actors. If you're interested in the technical side of facial landmark detection, distance-based similarity scoring, or just want to see the tools in action, you can try them at stardoppel.com.