Silver Kernel

Posted on Oct 30 Edited on Oct 30

Spatial Qualimetry: Extracting 3D Depth Profiles in Pure NumPy

#computervision #python #numpy #math

When building desktop or edge AI applications, the default instinct for many engineers is to run pip install opencv-python.

However, as your architecture matures toward lightweight, self-contained edge software (such as a PyQt6 GUI app running ONNX Runtime on CPU), third-party C++ wrappers introduce real operational friction:

Deployment Bloat: 40–50 MB binary wheels (opencv-python-headless on Linux/Windows x64) that expand desktop installers.
Cold-Start Latency: Shared C++ library initialization delays during application startup.
Dependency Conflicts: Threading and event-loop clashes with native Qt backends.

In our project — Hawk Eye, an industrial AI edge application for monocular tire tread depth assessment — we enforced a strict rule: Zero CV-specific external dependencies (no OpenCV, SciPy, or Scikit-Image). The pipeline relies strictly on PyQt6, ONNX Runtime, PIL, and pure NumPy.

In this article, we look at how we solve camera perspective tilt in monocular depth maps using simple vector operations in NumPy.

The Problem: Relative Depth and Perspective Distortion

Monocular depth estimation models (like U-Net architectures optimized for edge CPU) map 2D pixels to depth predictions Z_raw in range [0.0, 1.0].

In field diagnostics, handheld devices or smartphone cameras are almost never held perfectly parallel to the tire surface. A 5-degree perspective tilt creates an artificial linear gradient across the depth map, causing the model to mistake perspective for actual physical tread wear.

To fix this, we detrend the baseline reference plane using Ordinary Least Squares (OLS) fitted via SVD.

The Mathematics of Plane Fitting: Why SVD over Normal Equations?

We model the background surface using a linear 3D equation:

Z(x, y) = a * x + b * y + c

Where (x, y) are spatial pixel coordinates, a, b are surface slopes, and c is the global baseline offset.

Given an ROI matrix Z_raw of size H x W, we set up an overdetermined linear system A * x = b.

Numerical Stability: SVD vs. Normal Equations

In textbook linear algebra, the analytical solution is written via Normal Equations:

x_opt = inv(A^T * A) * A^T * b

In production, forming A^T * A squares the dynamic range of pixel coordinates. For a 224x224 depth grid (N = 50,176 pixels, max coordinate x_max = 223), the condition number of A^T * A reaches 10^4 - 10^5. Explicit matrix inversion leads to numerical instability on low-precision CPU runtimes.

Instead, we use Singular Value Decomposition (SVD) via np.linalg.lstsq. SVD solves the system directly without forming A^T * A, avoiding matrix inversion instability.

Production Python Implementation

Below is the self-contained implementation written purely in NumPy. It fits a least-squares plane, subtracts perspective tilt, and uses percentile masking to prevent deep grooves from corrupting the surface fit.

 python
import numpy as np

def detrend_and_scale_depth(
    depth_raw: np.ndarray, 
    scale_3d: float = 1.80, 
    bias_mm: float = 0.0,
    groove_mask_percentile: float = 40.0
) -> np.ndarray:
    """
    Removes camera perspective tilt using SVD Least Squares Plane Fitting
    and converts raw relative depth predictions into millimeter profile maps.

    Convention: Assumes Z_groove < Z_surface (lower relative height in depth_raw).

    Parameters
    ----------
    depth_raw : np.ndarray
        2D floating-point array (H, W) of relative predictions from ONNX U-Net.
    scale_3d : float
        Optics scale factor (mm per relative depth unit).
    bias_mm : float
        Systemic bias correction in millimeters.
    groove_mask_percentile : float
        Empirical threshold (40%) masking lower depth values (grooves) 
        during plane fitting to prevent fit sags.

    Returns
    -------
    np.ndarray
        2D array (H, W) containing detrended physical tread depth in millimeters.
    """
    h, w = depth_raw.shape

    # 1. Generate 2D pixel coordinate grids
    grid_y, grid_x = np.indices((h, w), dtype=np.float32)

    x_flat = grid_x.ravel()
    y_flat = grid_y.ravel()
    z_flat = depth_raw.ravel()

    # Design matrix A: Shape (N, 3) where N = H * W
    A = np.column_stack([x_flat, y_flat, np.ones_like(x_flat)])

    # 2. Trimmed OLS: Mask out deep grooves to fit plane ONLY on top tread blocks
    threshold_val = np.percentile(z_flat, groove_mask_percentile)
    mask = z_flat >= threshold_val

    # 3. Solve for plane parameters [a, b, c] via SVD (lstsq)
    plane_params, _, _, _ = np.linalg.lstsq(A[mask], z_flat[mask], rcond=None)
    a, b, c = plane_params

    # 4. Reconstruct background trend plane across full ROI
    z_plane = (a * grid_x + b * grid_y + c)

    # 5. Detrend surface (Subtract baseline slope)
    z_detrended = depth_raw - z_plane

    # 6. Robust baseline shift (P05 percentile instead of min to reject noise)
    baseline_floor = np.percentile(z_detrended, 5)
    z_detrended -= baseline_floor

    # 7. Convert relative depth delta to physical millimeters
    depth_mm = (z_detrended * scale_3d) + bias_mm

    return depth_mm


if __name__ == "__main__":
    # Simulate 100x100 tile with camera tilt + 2.0 unit groove (20% of area)
    y, x = np.ogrid[:100, :100]
    simulated_tilt = 0.05 * x + 0.02 * y
    simulated_groove = np.where((x > 40) & (x < 60), -2.0, 0.0)
    raw_input = simulated_tilt + simulated_groove

    # Process depth map
    depth_mm = detrend_and_scale_depth(raw_input, scale_3d=2.5)

    # Evaluation using P95 - P05 percentiles (Assumes groove area >= 5-10% of ROI)
    groove_depth = np.percentile(depth_mm, 95) - np.percentile(depth_mm, 5)

    print(f"Input Range (Tilted): [{raw_input.min():.2f}, {raw_input.max():.2f}]")
    print(f"Corrected Depth Range: [{depth_mm.min():.2f}, {depth_mm.max():.2f}] mm")
    print(f"Measured Groove Depth: {groove_depth:.2f} mm (Expected Synthetic: 5.00 mm)")

Expected Output

Input Range (Tilted): [0.00, 6.93]
Corrected Depth Range: [0.00, 5.00] mm
Measured Groove Depth: 5.00 mm (Expected Synthetic: 5.00 mm)

Production Considerations & Limitations

While linear plane detrending is fast, edge deployment has specific constraints:

Groove Fitting Bias (Trimmed OLS): Unmasked OLS fits the plane to all pixels. If an ROI contains deep grooves, the fitted plane sags downward. Our implementation uses a 40% threshold mask (groove_mask_percentile=40) to fit strictly to elevated tread blocks (valid for typical 15–35% groove occupancy).
Extreme Camera Tilt: If perspective tilt across the ROI exceeds the total groove depth, global percentile thresholding bleeds across regions. For severe angles (>30 degrees), local block-based RANSAC or quadratic fitting z = a*x^2 + b*y^2 + c*x*y + d*x + e*y + f should replace global OLS.
Local vs. Global Slope Removal: OLS removes all linear trends. This works well for measuring local groove depth relative to adjacent tread blocks, but intentional tire shoulder curvature will also be flattened.

Benchmark Results

Replacing external CV packages with vectorized NumPy operations yielded clear benefits in our PyQt6 runtime:

Execution Latency: 0.38 ms on a 224x224 depth ROI (median of 100 runs after 10 warm-up iterations on Intel Core i5-1135G7 @ 2.40GHz, OMP_NUM_THREADS=1, NumPy 1.26.4).
Package Footprint: Saved ~45 MB on Linux/Windows x64 installers by removing opencv-python-headless.
Thread Safety: Pure NumPy array operations run inside PyQt6 worker threads without GUI loop blocking.

Conclusion

Linear algebra operations in pure NumPy are often enough for edge post-processing without bundling full C++ vision frameworks.

While synthetic tests verify the math, production calibration against real physical depth gauges (e.g., Mitutoyo 571-201-30) remains necessary to establish the optics scale factor (scale_3d) for different smartphone camera modules.

By combining SVD plane fitting with percentile masking, we achieved stable profile extraction for Hawk Eye — completely independent of external computer vision packages.

Interactive Diagnostic: You can check your current wear pattern using our AI Tire Scanner right now. No app installation required.

DEV Community