A bounding box tells you where a face is. What happens next depends on the application. A photo search tool needs a representation it can compare. A portrait editor needs the boundary between skin, hair, and background. A video workflow may need to follow a face across frames or obscure it before the footage is shared.
UniFace is an open-source Python library for face detection, recognition, facial geometry, segmentation, and tracking. Most neural models run through ONNX Runtime. Developers can use the modules individually or combine them into a pipeline.
This article examines how detection results feed recognition, geometry, and image processing. The images are selected examples from the project; they illustrate outputs rather than measure accuracy, robustness, or speed.
See nine tasks running on video in the UniFace animated demo: detection, landmarks, face mesh, tracking, recognition, attributes, head pose, face parsing, and anonymization.
Detection is the beginning of a pipeline
In UniFace, a detection becomes a structured Face object containing a bounding box, a confidence score, and, for compatible detectors, alignment landmarks. That gives subsequent stages a common starting point.
Alignment matters because a recognition model should compare faces in a consistent pose and crop. The landmarks around the eyes, nose, and mouth provide reference points for that transformation. Small differences in preprocessing can affect the embedding that comes out, even when the underlying recognition model is unchanged.
UniFace handles this connection between detection and recognition. The detector locates the face; the recognizer uses its landmarks to align the image before extracting a vector. Developers can configure the detector and recognizer separately, provided the detector supplies the alignment landmarks required by the recognizer.
That separation is also useful when recognition has no role in the application. A face-blurring workflow can stop at detection and pass the results directly to anonymization.
A face has more than one useful representation
An embedding is useful when the question is whether two face images are similar. A recognition model maps an aligned face to a numerical vector that can be compared with another vector or searched in a collection. UniFace supports recognizers including ArcFace, AdaFace, and EdgeFace, along with an optional FAISS-backed store for embedding search.
Those scores still need interpretation. A threshold suitable for organizing a personal photo collection may be unsuitable for a verification workflow. Model choice, image quality, and the cost of false matches all influence that decision; a demo score cannot settle it for every application.
Geometry answers a different set of questions. A few alignment landmarks describe the broad arrangement of a face, while dense landmarks describe its contours in much greater detail. UniFace includes 68-, 98-, and 106-point landmark options, as well as a 468- or 478-point face mesh with relative depth.
For an overlay or an effect that follows the shape of a face, those points are often the more useful output. Head pose and gaze estimation extend this spatial view with estimates of head orientation and viewing direction.
The choice of representation follows the job: vectors for comparison, landmarks for geometry, and masks when the next operation happens at the pixel level.
From analyzing an image to changing it
Face parsing predicts a semantic label for each pixel. UniFace's BiSeNet models use 19 classes, including regions such as skin, hair, eyes, and lips. The resulting mask can select a component for a downstream edit.
A portrait editor, for example, could use the predicted skin region to select where to apply a color adjustment. The quality of that selection depends on the segmentation result. A face mask from XSeg serves a broader selection task. Portrait matting with MODNet produces an alpha matte for separating a person from the background, including partially transparent boundaries.
These outputs have different meanings, even though they may look similar in a demonstration. A semantic label identifies a region; an alpha matte describes how foreground and background mix. The downstream operation needs to account for that distinction.
Anonymization is another practical destination for a detection. UniFace can apply pixelation, blur, or a solid fill to detected face regions. This is useful for preparing images or video for sharing, provided the output is reviewed for missed faces and adequate coverage.
The architecture follows the task
UniFace exposes individual modules alongside a higher-level FaceAnalyzer. The modules give developers control over each stage; the analyzer connects detection, alignment, recognition, and optional attribute predictors in one call.
A small example shows that composition. After installing pip install "uniface[cpu]", the following uses RetinaFace for detection and ArcFace for embeddings:
import cv2
from uniface import ArcFace, FaceAnalyzer, RetinaFace
image = cv2.imread('portrait.jpg')
if image is None:
raise FileNotFoundError('Could not read portrait.jpg')
analyzer = FaceAnalyzer(
detector=RetinaFace(),
recognizer=ArcFace(),
)
for face in analyzer.analyze(image):
print('Face bounds:', face.bbox)
if face.embedding is not None:
print('Embedding dimensions:', face.embedding.shape)
Additional predictors can enrich each face with selected attributes. Other workflows can use the modules directly: a video application might combine detection with tracking, while a face search system might assess image quality before retaining an embedding. There is no need to run every available model on every image.
ONNX Runtime provides a common inference layer for most of these components, with CPU execution and acceleration through supported CUDA or CoreML environments. Execution-provider availability and model compatibility depend on the runtime build and environment. Latency and memory use need to be measured on the target machine. The emotion module is an exception that uses optional PyTorch.
Model weights download on first use, undergo SHA-256 verification, and are cached locally. They can also be prepared ahead of time for offline use. Offline execution requires the selected weights and runtime dependencies to be available on the target machine.
Evaluation and limitations
The examples here do not establish a deployment accuracy or performance level. Recognition thresholds need evaluation on representative data, including false matches and missed matches. Attribute predictions can be inaccurate and biased. Tracking IDs indicate continuity across frames, not verified identity. Face obscuring requires checking for missed detections and insufficient coverage.
The same care applies to model selection. UniFace's code is MIT-licensed, while individual pretrained weights have their own terms. The licence and attribution guide documents those distinctions.
UniFace provides model wrappers, preprocessing, and pipeline composition. Whether a particular configuration meets an application's requirements depends on the selected weights, input data, hardware, and evaluation criteria.
Explore the examples in the interactive demo, find implementation details in the documentation, or browse the models and source on GitHub.
Top comments (0)