This is a simplified guide to an AI model called Depth-Anything-V3-Metric-Large maintained by David20321. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter.
Overview
depth-anything-v3-metric-large is a specialized metric depth estimation model developed by david20321 that predicts real-world scaled depth maps from single images. Built on Depth Anything 3 architecture, it uses a plain transformer backbone (DINO-based) with a DualDPT head to output 16-bit PNG depth maps with metric scale metadata, enabling applications requiring absolute depth measurements rather than relative geometry. The model uses a depth-ray representation for improved generalization and is trained exclusively on public academic datasets. The critical distinction from relative depth models is that outputs represent actual metric distances in world space, making this suitable for robotics, autonomous systems, and 3D reconstruction requiring calibrated scale.
Best use cases
Robotics and autonomous systems requiring metric depth — Applications like robotic manipulation, navigation, and obstacle avoidance depend on knowing absolute distances in world coordinates. This model outputs metric-scale depth directly, eliminating the need for separate scaling or calibration steps that relative depth models require. The 16-bit output provides sufficient precision for servo control and path planning calculations.
3D reconstruction with metric accuracy — When building 3D scenes or point clouds for architectural documentation, surveying, or digital twins, metric depth is essential. This model pairs with structure-from-motion pipelines to generate metrically accurate reconstructions without requiring secondary depth-to-metric conversion, reducing accumulated error in multi-stage pipelines.
Autonomous vehicle perception — Self-driving systems need calibrated depth for distance estimation to obstacles, lane boundaries, and other vehicles. Metric depth output integrates directly with sensor fusion and decision-making systems that expect world-space measurements rather than relative estimates.
AR/VR applications with real-world anchoring — Augmented reality requires placing virtual objects at correct metric distances from the camera to maintain realism. This model provides the necessary metric scale to properly occlude virtual content behind real obstacles and position effects at correct depths in 3D space.
Industrial depth sensing replacement — Manufacturing inspection and quality control often rely on expensive structured-light or stereo systems. This model can replace those sensors for applications where metric depth accuracy matters more than sub-millimeter precision, at significantly lower cost.
Limitations
This model outputs only single-image monocular depth estimates and cannot perform multi-view depth fusion or camera pose estimation — use the full DA3 foundation models for those capabilities. The metric scale depends entirely on camera intrinsics; if focal length is not provided or accurately estimated via the heuristic, depth values will be incorrect. The default focal length uses a 60-degree horizontal field-of-view assumption, which fails for wide-angle or telephoto lenses requiring focal_length_px parameter override.
Output resolution is capped by the max_process_res parameter (server default applies if set to 0), potentially limiting performance on high-resolution inputs requiring preservation of fine detail. The 16-bit PNG output format may lose precision for very close distances with small depth variations. Processing speed and memory requirements depend on input resolution but are not published; users should expect typical transformer inference costs. The model is fine-tuned from the general DA3-Large foundation model, so it may not match DA3-Large performance on tasks outside metric monocular depth. License is Apache 2.0, permitting commercial use, but the model card notes it is specialized for monocular metric settings specifically.
How it compares
vs depth-anything-v3-metric — Both models target metric depth estimation. The distinction depends on which implementation/backend the vufinder version uses; this model explicitly outputs 16-bit PNG with metric metadata, making it clearer for downstream applications requiring calibrated scale. Choose this version if you need guaranteed metric output format and don't need multi-view capabilities.
vs depth-pro — Depth Pro emphasizes speed (under one second inference) and sharp output for single images. This model offers the foundation model's broader capabilities through the DA3 architecture but trades pure speed for potentially more reliable metric scaling across diverse scenes. Pick this for applications prioritizing accuracy and metric consistency over sub-second latency.
vs depth-anything-v3-mono — The mono variant produces relative depth only, which requires external metric scaling. Use this metric-large model if you need absolute world-space distances directly; use the mono model only if relative geometry is sufficient and you can scale externally.
vs depth-anything-v2 — DA2 produces relative depth with faster inference on older hardware. DA3-Metric-Large outputs metric depth with improved geometric accuracy from the newer architecture, at the cost of higher compute. Choose this version for metric requirements; choose DA2 if you need maximum speed or relative depth is acceptable.
vs ml-depth-pro — Another fast metric depth model. The ml-depth-pro is optimized for speed and closed-source. This model is open-source (Apache 2.0), trained on public datasets, and part of the broader DA3 ecosystem, making it more suitable for research or customization while offering comparable metric output.
Technical specifications
The model is based on Depth Anything 3 architecture with a plain vision transformer (DINO) encoder backbone and DualDPT decoder head producing two output channels (depth value and confidence). It is part of the DA3 metric series specifically fine-tuned for monocular metric depth, distinct from the multi-view capable DA3-Giant and DA3-Large any-view models. The model was trained exclusively on public academic datasets.
Output format is 16-bit PNG depth maps with metric metadata encoded. Camera intrinsics are handled via the focal_length_px parameter; if not provided, a heuristic assumes 60-degree horizontal field-of-view. The model can optionally output raw float32 metric depth in compressed NPZ format alongside the PNG.
- Architecture: Vision transformer (DINO) encoder + DualDPT head with depth-ray representation
- Training data: Public academic datasets only
- License: Apache 2.0
- Output format: 16-bit PNG (default) or optionally NPZ with float32 raw depth
- Metric scale: Dependent on accurate focal length input; default assumes 60° HFOV
- Inference backend: Cog version 0.16.11 on Replicate
Model inputs and outputs
Inputs
- image (string, URI format, required) — Input image as URL or file path
- focal_length_px (number, default 0) — Focal length in pixels for metric calibration; 0 uses heuristic based on 60-degree horizontal FOV
- max_process_res (integer, default 0) — Optional per-request cap on processing resolution; 0 uses server default
- return_raw_depth (boolean, default false) — Include float32 metric depth map in compressed NPZ output alongside PNG
- include_base64 (boolean, default true) — Include depth PNG as base64 string for compatibility
Outputs
- depth_map (16-bit PNG) — Primary output containing metric depth values with metadata
- depth_map_base64 (string, if include_base64=true) — Base64-encoded version of depth PNG for API compatibility
- raw_depth (float32 array in NPZ, if return_raw_depth=true) — Uncompressed metric depth values for downstream processing
- confidence (float32 array, if return_raw_depth=true) — Per-pixel confidence scores in NPZ format
Getting started
import replicate
client = replicate.Replicate()
output = client.run(
"david20321/depth-anything-v3-metric-large:e3523ab17a5e6f0e279933a6afdde67efe130bb9e7753cafc52a4b082257f46b",
input={
"image": "https://example.com/photo.jpg",
"focal_length_px": 1000, # Override heuristic for known lens
"max_process_res": 2048, # Cap processing resolution
"return_raw_depth": True, # Get float32 depth for analysis
"include_base64": True
}
)
# output contains depth_map_base64 and optionally raw_depth in NPZ
print(output)
Frequently asked questions
Q: How does focal_length_px affect the metric depth output?
A: Focal length in pixels is essential for converting pixel coordinates to world space; it scales the entire depth map to real-world meters. If you know your camera's focal length (from calibration or EXIF), provide it to get accurate metric depth; if 0 (default), the model assumes a 60-degree horizontal field-of-view heuristic, which will produce incorrect absolute scale for wide-angle or telephoto lenses.
Q: Can I use this model for multi-view depth estimation or pose recovery?
A: No, this is a monocular metric model. For multi-view depth fusion, camera pose estimation, or any-view geometry, use the full DA3 foundation models (DA3-Large, DA3-Giant) or the nested series that combines those with this metric model.
Q: What is the difference between the 16-bit PNG output and the optional float32 NPZ output?
A: The 16-bit PNG is the standard output format with metadata baked in, suitable for visualization and basic downstream use. The float32 NPZ output (enabled via return_raw_depth=true) provides full floating-point precision for numerical analysis, point cloud generation, or operations where 16-bit quantization would introduce unacceptable error.
Q: Is this model suitable for production use in autonomous systems?
A: Yes, it is designed for real-world metric depth. However, you must validate performance on your specific camera hardware (verify focal length calibration) and environmental conditions. The model is trained on public datasets which may not cover all edge cases, so testing on representative data before deployment is essential.
Q: What resolution can this model handle and what is the inference speed?
A: Maximum processing resolution is controlled by max_process_res (0 uses server default); specific speed/resolution tradeoffs are not documented. For metric applications, err on the side of testing with your target resolution to confirm acceptable latency and accuracy before deployment.
Q: How does this compare to Depth Anything V2 for metric depth?
A: DA2 outputs relative depth only and requires external metric scaling. This metric-large model from the newer DA3 architecture outputs metric depth directly with improved geometric accuracy per the DA3 paper, eliminating the separate scaling step and reducing error accumulation.
Q: Can I run this locally or only on Replicate?
A: This Replicate version runs on Replicate's infrastructure. The underlying Depth Anything 3 codebase is open-source on GitHub, so you can download weights and run inference locally with appropriate GPU hardware, but this specific Replicate endpoint runs on their servers.
Q: What camera types does this work well with?
A: The model works with any camera if you provide accurate focal length calibration. It assumes a pinhole camera model. It has been tested on diverse imagery from public datasets but focal length accuracy is critical — verify with your camera specs or calibration before relying on output scale.
Click here to read the full guide to Depth-Anything-V3-Metric-Large
Top comments (0)