KYC Process: Why the Bank Selfie Maps 68 Face Points

Deconstructing the 68-point facial landmark pipeline behind modern identity verification reveals a critical architectural lesson for computer vision engineers: facial comparison and presentation attack detection (PAD) are two fundamentally distinct layers that must never be treated as a single inference step.

For developers building identity, onboarding, or automated verification workflows, it is easy to assume that extracting facial landmarks and computing vector similarity solves authentication. In practice, running a standard 68-point landmark detector (such as an ensemble of regression trees or a lightweight CNN) is only the initialization phase.

[Raw Frame] 
    │
    ├──> [68-Point Alignment] ──> [Feature Extraction] ──> [512-d Embedding] ──> [Euclidean / Cosine Match]
    │
    └──> [PAD Pipeline] ───────> [Frequency / Texture Analysis + Challenge-Response] ──> [Liveness Score]

The Mathematics of Comparison vs. Verification

In a typical computer vision pipeline, extracting 68 landmark points allows your system to perform affine transformations—aligning the eyes, nose bridge, and jawline across varying pitch, yaw, and roll. Once normalized, the aligned crop passes into an embedding model (like an ArcFace or ResNet backbone) to generate a high-dimensional feature vector (typically 128-d or 512-d).

Evaluating similarity comes down to metric learning:

$$d(u, v) = \sqrt{\sum_{i=1}^n (u_i - v_i)^2}$$

If the Euclidean distance between the ID document embedding and the live capture falls below a predefined threshold $\tau$, the system registers a mathematical match.

However, a low Euclidean distance indicates geometric and textural resemblance—not presence. A printed color photo, a 4K replay attack on an OLED screen, or a synthetic deepfake mask can yield an identical embedding vector.

Why Liveness Requires a Separate Inference Pipeline

To satisfy ISO/IEC 30107-3 compliance for Presentation Attack Detection, engineering teams must decouple facial comparison from anti-spoofing heuristics:

1. Active Liveness (State Machines): Prompting client-side interactions (e.g., yaw rotation, randomized blink sequences) evaluated via optical flow and frame-by-frame landmark tracking. Pre-recorded synthetic video often fails dynamic, server-generated challenge-response routines.
2. Passive Liveness (Frequency & Texture Analysis): High-frequency spatial domain filters and CNN-based binary classifiers that detect flat specular reflection, missing sub-surface light scattering, pixel grid artifacts, and unnatural skin smoothing typical of GAN-generated outputs.

When designing biometric pipelines for case analysis, fraud detection, or user onboarding, treating face matching as proof of live presence introduces severe architectural vulnerabilities. Robust computer vision systems isolate Euclidean distance analysis—measuring structural consistency across source imagery—from the runtime integrity checks required to validate sensor input.

If you are implementing facial analysis or identity validation in your stack, how are you handling presentation attack detection—are you relying on edge-based passive texture classification or centralized multi-frame challenge-response microservices?