DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

Elliptic Envelope: robust-covariance outlier detection with Mahalanobis distance and a chi-square cutoff

#python #datascience #statistics #machinelearning

The Elliptic Envelope is outlier detection with one clean modelling bet: assume the inliers are drawn from a single Gaussian. Under that assumption every contour of constant density is an ellipse, and "how far out is this point?" has a precise, scale-and-correlation-aware answer. Flag whatever falls outside one ellipse around "normal." The whole method is a center, a covariance, a distance, and a statistical cutoff — plus the one trick that keeps outliers from sabotaging the fit.

Mahalanobis distance — distance in the data's own units

Ordinary Euclidean distance treats every direction equally. That is wrong for correlated data: a point two standard deviations out along a thin axis should count as "just as far" as one two σ out along a fat one. The Mahalanobis distance stretches and rotates distance by the covariance to do exactly that:

$$d^2(x) = (x - \mu)^\top \Sigma^{-1} (x - \mu)$$

For 2D points, inverting the 2×2 covariance is a one-liner via the adjugate over the determinant — but a nearly-collinear cloud makes Σ singular, so add a whisker of ridge and guard the determinant so it never divides by zero.

function maha2(dx, dy, inv){                 // = 0 at the mean μ
  return inv.i00*dx*dx + 2*inv.i01*dx*dy + inv.i11*dy*dy;
}
Enter fullscreen mode Exit fullscreen mode

The chi-square cutoff — contamination becomes a boundary

If the inliers really are Gaussian, their squared Mahalanobis distances follow a chi-square law with p degrees of freedom. So "flag the outer c fraction" becomes "flag every d² above the χ²ₚ quantile at 1 − c." For p = 2 the chi-square CDF is the clean F(x) = 1 − e^{−x/2}, so that quantile is just −2·ln(c) — a closed form, no tables. Sliding the contamination directly resizes the decision ellipse.

function chiCut2(contamination){             // p = 2 dimensions
  const c = Math.min(0.999, Math.max(1e-4, contamination));
  return -2 * Math.log(c);                   // c=0.10 -> 4.605 ; c=0.50 -> 1.386
}
Enter fullscreen mode Exit fullscreen mode

Why plain covariance fails: masking

Here is the catch, and the whole reason this is a robust-statistics method. The plain empirical covariance averages over all points, so a single gross outlier does two bad things at once: it drags the mean μ toward itself, and it inflates Σ in its own direction. That bloated ellipse then gives the outlier a small Mahalanobis distance — the point hides inside its own influence. This is masking: the outlier ends up inside the ellipse it should have been flagged by.

The fix: robust MCD covariance

The Minimum Covariance Determinant estimate fits only the tight majority of points and never lets the outlier vote. It looks for the subset of h points whose covariance has the smallest determinant — the densest core — via concentration steps: from a current subset compute μ, Σ, score all points, keep the h with the smallest distance, refit. Each step provably shrinks the determinant; a few random starts avoid local minima, and a final rescale by the median distance keeps Σ consistent for Gaussian data so the chi-square cutoff stays honest.

With a robust Σ, the whole envelope is five calls:

function fitEnvelope(pts, contamination, robust){
  const n = pts.length;
  const est = (robust && n >= 6) ? mcd(pts, contamination)
                                 : meanCov(pts, [...Array(n).keys()]);
  const inv = inv2(est.cxx, est.cyy, est.cxy);
  const cut = chiCut2(contamination);
  const d2  = pts.map(p => maha2(p.x - est.mx, p.y - est.my, inv));
  const flags = d2.map(d => d > cut);        // outside the ellipse -> flagged
  return { est, inv, cut, d2, flags };
}
Enter fullscreen mode Exit fullscreen mode

Drop one gross outlier into a clean cloud and the difference is stark: the plain ellipse lunges and bloats toward it until it masks it, while the robust ellipse barely flinches and flags it cleanly. The classic distance-distance plot (classical Mahalanobis on x, robust on y) exposes the masked points that sit low-left but high-up.

Where it sits

In practice you reach for scikit-learn's EllipticEnvelope, which fits a robust MinCovDet and flags by the chi-square cutoff at your contamination — its one assumption is the same one above: the inliers are a single Gaussian blob. For multi-modal or non-elliptical normal data, use a density- or distance-based detector instead. It is the covariance-flavoured member of the robust-stats family alongside RANSAC, Huber and Theil–Sen — here the thing made robust is the covariance itself.

Fit it live on points you place yourself at https://dev48v.infy.uk/ml/day56-elliptic-envelope.html

Top comments (0)