Running the Lightweight ZipDepth Model on Apple Silicon

Hello, everyone.

Depth information is useful when we want to distinguish the foreground from the background in a single image. It can support background blur, 3D effects, and perception for robots.

Today, I test the lightweight ZipDepth model on Apple Silicon and compare PyTorch CPU, PyTorch MPS, and ONNX Runtime CPU.

The short result is that MPS was the fastest option on an Apple M1 Max, averaging 15.34 ms. It was about five times faster than PyTorch CPU with the same standard model. The model also produced clear near-to-far relationships for three different images.

What I tested

Depth estimation predicts how near or far each pixel is from the camera. This test uses monocular depth estimation, which needs only one RGB image.

• Whether ZipDepth runs on the Apple Silicon CPU and GPU
• Whether the NPU-compatible model can be converted to ONNX and executed
• Inference time and output differences between backends
• Qualitative results for a street, tabletop objects, and a crowd

This is the street image used for the benchmark.

Lab: kiarina/labs/2026/07/15/zipdepth-apple-silicon

Reproducing the test

You need mise, uv, an Apple Silicon Mac, and an internet connection for the first model and image download.

git clone --depth 1 --filter=blob:none --sparse \
  https://github.com/kiarina/labs.git
cd labs
git sparse-checkout set .gitignore .mise/tasks Makefile mise.toml 2026/07/15/zipdepth-apple-silicon
mise -C 2026/07/15/zipdepth-apple-silicon run

On the first run, the task downloads the checkpoints, verifies their SHA-256 hashes, and exports the NPU-compatible model to ONNX opset 18. Benchmarking and result image generation are included.

What is ZipDepth?

ZipDepth is a monocular depth estimation model whose paper was released in July 2026 and presented at ECCV 2026. It has 6.1 million parameters and requires 3.0 GMACs for a 384x384 input. It targets devices ranging from server GPUs to mobile hardware.

Its output is relative inverse depth. In the images in this article, brighter pixels are nearer and darker pixels are farther away. It does not return metric distances such as three meters.

The model was distilled on about 14.07 million images across 17 domains, using pseudo-depth produced by Depth Anything V2 Large. Knowledge distillation trains a small model using predictions from a larger teacher model.

The data flow is short:

Training performed by the ZipDepth authors
RGB image -> Depth Anything V2 Large -> pseudo-depth -> train ZipDepth

Inference in this lab
RGB image -> resize short side to 384 -> ZipDepth -> inverse depth map
                                            -> color visualization

Depth Anything V2 Large was the training-time teacher. It is not executed by this lab.

Models and licenses

ZipDepth provides two checkpoints that use different methods to restore the output resolution.

Checkpoint	Role	Runtime
zipdepth_base.pth	Standard model with Unfold-based upsampling	PyTorch CPU / MPS
zipdepth_base_npu.pth	Conversion-friendly, unfold-free model	PyTorch CPU / ONNX Runtime CPU

MPS is the PyTorch backend for running on the Apple Silicon GPU. Despite the NPU-compatible name, this test runs that model on the CPU, not the Apple Neural Engine.

• ZipDepth code and checkpoints: MIT License
• Commit: a302e5437bc58f15c4efd41d3e8222bf24f7d470
• Standard SHA-256: a55910bb0b99c8c5e641cb9206e810b269690ad94e8a2ef08c827c4679391a65
• NPU-compatible SHA-256: 627c04fda584133ead4310074884a4a037061b4c01ba86e73e492ea30fab570d

The Depth Anything V2 Large teacher is licensed under CC BY-NC 4.0. The official project lists Small under Apache-2.0 and Base, Large, and Giant under CC BY-NC 4.0. This lab downloads and runs the MIT-licensed ZipDepth checkpoints; it does not download Depth Anything V2 Large.

Method

Each image keeps its aspect ratio, with its shorter side resized to 384 pixels. I benchmarked the 768x384 street image for ten runs after three warm-up runs. A warm-up avoids including one-time initialization work in the measurement.

Model loading, image loading, preprocessing, visualization, and file saving are excluded from inference time. I visually inspected the output for a street, tabletop objects, and a crowd.

machine: MacBook Pro (Apple M1 Max, 64 GB, arm64)
OS: macOS 26.5.2
Python: 3.12.10
PyTorch: 2.13.0
ONNX Runtime: 1.27.0
input: FP32, batch 1, 768x384 for the benchmark
warm-up: 3 runs
measurement: 10 runs

Results

The input is on the left and the ZipDepth result is on the right. The color range is normalized independently for each image, so colors cannot be compared directly between rows.

For the street, the nearby road and cars are bright, while distant buildings and the vanishing point are dark. On the tabletop, the desk, books, laptop, and background form separate depth levels. In the crowd, foreground people are brighter. Some small distant people merge into smooth regions instead of remaining individually separated.

These are visual observations, not an accuracy test against ground-truth depth.

Inference speed

Backend / model	Mean	Median	Min	Max	Std. dev.
PyTorch CPU / standard	77.78 ms	77.49 ms	75.75 ms	80.89 ms	1.65 ms
PyTorch MPS / standard	15.34 ms	15.56 ms	14.36 ms	15.89 ms	0.55 ms
PyTorch CPU / NPU-compatible	101.49 ms	100.47 ms	97.44 ms	109.55 ms	3.26 ms
ONNX Runtime CPU / NPU-compatible	47.08 ms	47.17 ms	46.54 ms	47.46 ms	0.33 ms

By median time, MPS was 4.98 times faster than PyTorch CPU with the same standard model. ONNX Runtime CPU was 2.13 times faster than PyTorch CPU with the same NPU-compatible model.

The MPS and ONNX Runtime outputs were also nearly identical by eye.

Numerical output differences

Relative depth can describe the same geometry with a different scale and offset. I therefore aligned scale and shift with least squares before measuring error.

Comparison	Aligned MAE	Aligned RMSE
Standard: PyTorch CPU vs MPS	0.00000002	0.00000002
NPU-compatible: PyTorch vs ONNX Runtime CPU	0.00000001	0.00000002
PyTorch CPU: standard vs NPU-compatible	0.00044436	0.00086735

Differences between backends using the same checkpoint were at the level of FP32 rounding. The standard and NPU-compatible models were not exactly identical, but the difference was not visible in the street result.

ONNX export issues

The first ONNX export failed because onnxscript was missing. PyTorch 2.13.0's exporter requires it, so adding the dependency fixed the error.

Requesting opset 17 also failed during conversion from the internally generated opset 18 model. The generated opset 18 model passed the ONNX checker and ran with ONNX Runtime, so the lab uses the actual opset 18 output.

Interpretation

The detailed data is above, but there are three main points.

1. MPS reached roughly 65 FPS of model inference

The model averaged 15.34 ms without any compatibility patch. That is promising for local image-processing features on Apple Silicon.

1. ONNX Runtime roughly halved CPU inference time

With the same NPU-compatible model, ONNX Runtime took 47.08 ms versus 101.49 ms for PyTorch CPU. The numerical output difference was minimal.

1. The direct MPS-to-ONNX comparison needs care

They use checkpoints with different upsampling methods. The roughly three-times speed difference cannot be attributed to the backend alone.

This test is limited to three generated images, one benchmark image, and one M1 Max. It does not test metric distance accuracy, official benchmarks, temporal stability in video, Core ML, the Apple Neural Engine, quantization, power use, or peak memory.

Final thoughts

For a model with only 6.1 million parameters, ZipDepth preserved useful object boundaries while producing a clear near-to-far structure. The roughly 15 ms MPS result looks fast enough for experiments such as local background blur or simple 3D effects.

The output is relative depth, so it cannot directly support applications that require real-world distance. Small distant people also tend to merge, making the model unsuitable as the only perception component in safety-critical systems. Even so, combined with object detection and tracking, it could provide useful context for an LLM-controlled agent to understand the relative position of obstacles or follow a person in real time.