DEV Community: Yuji Kobayashi

Creating Depth GeoTIFFs from Satellite and Aerial Imagery with Depth Anything 3: From Environment Setup to Monocular Depth Inference

Yuji Kobayashi — Tue, 13 Oct 2026 03:33:16 +0000

Introduction

This article uses Depth Anything 3 (DA3) to perform monocular depth estimation on a single RGB GeoTIFF. No additional training is performed; we use a publicly available pretrained model.

Depth estimated from a single satellite image can be used to generate building models with geometry comparable to LOD2. For an overview of monocular depth estimation and our work on generating 3D building data from satellite imagery, see this explanatory page.

As the first step in that workflow, this article covers environment setup on Ubuntu with an NVIDIA GPU, model downloading, depth inference, GeoTIFF export, and inspection in QGIS. Building polygons, a DEM, and camera exterior orientation data are not used as inputs.

For the satellite imagery example, we used an archived WorldView-2 image with a resolution of 50 cm. Because lossy compression noise made building boundaries unclear in the original image, we first applied 2× super-resolution and image enhancement.

The resulting GeoTIFF is 2048×1955 pixels. When we performed monocular depth estimation on the entire image, the predicted boundaries appeared blurred, possibly because the whole image was downscaled internally. We therefore tried splitting the image into 300-pixel tiles, running inference on each tile, and blending the overlapping predictions.

This approach was inspired by our experience with Depth Anything 2 a little over a year earlier, when depth estimation for the same area produced a map that appeared tilted across the entire scene.

For our image, tiling produced visually sharper building boundaries, as it had with Depth Anything 2. However, this is a visual comparison, not a validation of building heights or roof slopes against measurements.

The output is relative depth. It does not directly provide building heights in meters, a DSM, or a DEM. Generating 3D building models and placing them on a DEM are separate postprocessing steps.

1. Model and Processing Pipeline

We use depth-anything/DA3MONO-LARGE. Its official model card describes it as a model for monocular relative depth estimation. It is a different model from DA3-LARGE and DA3METRIC-LARGE.

The workflow is: read the RGB GeoTIFF → run DA3 inference → resize the depth prediction to the original pixel grid → save as GeoTIFF. The tiled version splits the image before inference and blends the depth tiles before saving.

We use the official API for DA3 itself. GeoTIFF input/output and tile blending are implemented in the additional scripts provided in this article. This is not an official DA3 geospatial tiled inference feature.

2. Environment

The environment used in this article is listed below. All commands are intended for Bash on Ubuntu.

Item	Configuration
OS	Ubuntu 24.04
GPU	NVIDIA RTX PRO 6000 Blackwell Max-Q
Python	3.10 / dedicated conda environment
PyTorch	2.9.1, CUDA 12.8 wheel
torchvision	0.24.1
xFormers	0.0.33.post2
NumPy	1.26.4
OpenCV	opencv-python 4.10.0.84
MoviePy	1.0.3
Model	DA3MONO-LARGE
GeoTIFF input/output	Rasterio

The memory capacity of this GPU is not a requirement for using the workflow. GPU memory requirements depend on input resolution and the environment. This article does not evaluate minimum memory requirements or processing speed on other GPUs.

For the PyTorch version combination, see the official previous versions page. For xFormers compatibility, see the 0.0.33.post2 release notes. This procedure pins a combination of major packages rather than upgrading everything to the latest version.

The “CUDA Version” shown by nvidia-smi, the Toolkit version shown by nvcc --version, and the CUDA version used by PyTorch, shown by torch.version.cuda, have different meanings. This procedure uses the CUDA 12.8 PyTorch wheel and does not replace the system CUDA Toolkit. Install a suitable NVIDIA GPU driver beforehand.

3. Create the Working Directory and Conda Environment

We assume that Miniconda or Anaconda is already available.

conda create -n da3_bw python=3.10 pip -y
conda activate da3_bw

mkdir -p ~/da3_geotiff/input ~/da3_geotiff/output ~/da3_geotiff/logs
cd ~/da3_geotiff

nvidia-smi
git --version
tmux -V

If git or tmux is unavailable, install it through Ubuntu's package manager.

sudo apt-get update
sudo apt-get install -y git tmux

Save all files listed in “Appendix: Scripts” at the end of this article directly under ~/da3_geotiff/. Copy the code from each expandable section and save it using the filename in its heading. No separate custom ZIP download or private repository is required.

File	Purpose
`constraints.txt`	Pin major package versions
`setup.sh`	Install official code and dependencies
`check_env.py`	Check CUDA and imports
`download_model.py`	Download official model weights
`infer_geotiff.py`	Run inference on the whole image
`da3_tile_io.py`	Handle GeoTIFF input/output for tiled inference
`infer_geotiff_tiled.py`	Run tiled inference and blend predictions

4. Install DA3 and Its Dependencies

conda activate da3_bw
cd ~/da3_geotiff
bash setup.sh

setup.sh performs the following steps:

Install the CUDA 12.8 versions of PyTorch, torchvision, and xFormers.
Clone the official repository into vendor/Depth-Anything-3.
Install the basic DA3 dependencies, Rasterio, and other required packages.
Run pip check, check imports, and test matrix multiplication on CUDA.
Save the Git commit ID and pip freeze output in logs/.

The official code is obtained with the following command:

# This is executed by setup.sh; you do not need to run it again manually.
git clone https://github.com/ByteDance-Seed/Depth-Anything-3.git vendor/Depth-Anything-3

Only the basic dependencies are installed; optional components for Gradio and 3D Gaussian output are not installed. Depth inference uses infer_gs=False.

To check the environment again, run:

python -m pip check
python check_env.py

Check that the output includes CUDA runtime: 12.8, the GPU name, and Imports and CUDA matrix multiplication: OK. These checks alone do not establish that inference with the model weights succeeds.

Although major packages are pinned, this is not a fully locked environment with every dependency and repository commit fixed in advance. If future changes cause dependency conflicts, inspect the error log and downloaded commit before removing the constraints and upgrading packages.

5. Download the DA3MONO-LARGE Weights

python download_model.py

The model is downloaded from the official Hugging Face repository, depth-anything/DA3MONO-LARGE, into models/DA3MONO-LARGE/. The script resolves a revision, downloads that specific revision, and records its ID in download_revision.txt.

cat models/DA3MONO-LARGE/download_revision.txt
cat logs/da3_git_commit.txt

These files record the versions retrieved at download time. They do not guarantee that a new download on another day will resolve to the same revision. Retain the model directory and environment records for reproducibility.

Inference loads the locally saved weights. The scripts do not upload input imagery to Hugging Face.

6. Prepare the Input GeoTIFF

The examples use input/area_rgb.tif. Place your own image at that path, or change the input path in the commands below.

The input requirements are:

An RGB source image with an 8-bit uint8 pixel type
A GeoTIFF with a CRS and affine transform
Bands 1, 2, and 3 used as R, G, and B by default
No previously estimated depth or height image as input

For 16-bit satellite imagery, first convert it to 8-bit RGB using a consistent method across the entire image. Avoid a simple astype(uint8) conversion or independent contrast adjustments for each tile. The script stops on 16-bit imagery rather than converting it automatically.

Example input inspection:

python - <<'PY'
import rasterio
with rasterio.open('input/area_rgb.tif') as src:
    print('size:', src.width, src.height)
    print('bands:', src.count)
    print('dtype:', src.dtypes)
    print('CRS:', src.crs)
    print('transform:', src.transform)
    print('color interpretation:', src.colorinterp)
PY

If the RGB band order differs, add --bands R_number G_number B_number when running inference. For example, use --bands 3 2 1 if R, G, and B correspond to bands 3, 2, and 1.

An additional band in a four-band image is not passed to the model. However, alpha and NoData masks recognized by Rasterio are used to identify valid pixels. Missing pixels are filled with the RGB median during inference and restored to NoData in the output. This does not eliminate their influence on predictions near missing regions.

7. Start with Whole-Image Inference

Use tmux for long-running processing.

tmux new -s da3

Run the following commands inside the session:

conda activate da3_bw
cd ~/da3_geotiff
mkdir -p output logs
set -o pipefail

python -u infer_geotiff.py \
  --input input/area_rgb.tif \
  --output-prefix output/area_da3_full_r1008 \
  --process-res 1008 \
  2>&1 | tee logs/area_da3_full_r1008.log

--process-res 1008 specifies the approximate longest side of the image passed to the model. It does not split the original image into 1008-pixel tiles. Preprocessing resizes the image without cropping and adjusts the resolution to the model's patch size.

The output depth is resized to the original image dimensions using bilinear interpolation before saving. Therefore, matching the input and output pixel dimensions does not mean that details lost during downscaled inference have been recovered.

Press Ctrl+B, then D, to detach from tmux. Reattach with:

tmux attach -t da3

8. Use 300-Pixel Tiles to Inspect Building Details

Instead of downscaling the whole image, run inference on small regions of the original image. Here we use 300-pixel crops, 100-pixel overlaps, and a model input size of 518 pixels.

python -u infer_geotiff_tiled.py \
  --input input/area_rgb.tif \
  --output-prefix output/area_da3_t300_o100_r518_shift \
  --tile-size 300 \
  --overlap 100 \
  --process-res 518 \
  --alignment shift \
  2>&1 | tee logs/area_da3_t300_o100_r518_shift.log

Argument	Meaning
`--tile-size 300`	Crop size in the original image, normally 300×300 pixels
`--overlap 100`	Overlap between adjacent tiles; the normal stride is 200 pixels
`--process-res 518`	Input size used when passing a 300-pixel RGB tile to the model
`--alignment shift`	Correct only the offset using depth differences in overlapping regions
`--output-prefix`	Common filename prefix for the output files

518 is a multiple of 14. This wrapper requires process-res to be a multiple of 14. Upscaling a 300-pixel tile to 518 pixels is a model input preparation step; it does not improve the actual ground resolution of the imagery.

For example, a 300-pixel tile covers 75×75 m in an image with a resolution of 0.25 m/pixel. At 0.5 m/pixel, it covers 150×150 m. The proportion occupied by buildings and the surrounding context therefore differ even with the same 300-pixel tile size. Equivalent accuracy is not guaranteed for other regions or image resolutions.

At image boundaries, crop positions are shifted back to keep the tiles inside the image, so the final overlap may exceed 100 pixels. If the image width or height is less than 300 pixels, the image is padded using edge pixel values. A 2048×1955 image produces 10×10 tiles, or 100 tiles in total.

The model is loaded once at the start, and tiles are processed sequentially. Each prediction is resized back to the original 300-pixel tile grid and blended into the mosaic.

Blending Modes: shift, affine, and none

Mode	Correction	Characteristics
`shift`	Offset only	Does not rescale depth differences within each tile
`affine`	Scale and offset	Aligns scales in overlapping regions, but also changes the magnitude of local relief
`none`	No value correction	Takes a weighted average of the original predictions; differences between tiles may remain

In this workflow, we explicitly use shift to avoid the effects of scale correction on local height differences during comparison. The supplied code defaults to affine, so do not omit this argument.

For shift alignment, let $d$ be the current tile's depth and $m$ the depth in the overlapping region of the existing mosaic. For valid overlapping pixels, apply:

b = median (m - d), d^{'} = d + b

For the first tile, or a tile without valid overlap, the offset is set to zero. The corrected tiles are then blended with a weighted average that assigns greater weights toward each tile's center.

D (x, y) = \frac{\sum _{i} w _{i} ( x , y ) d _{i'} ( x , y )}{\sum _{i} w _{i} ( x , y )}

“Alignment” here does not geometrically transform the image. It aligns the reference levels of depth values predicted at the same pixel locations.

The tiles are not independently normalized to 0–1 using their own minimum and maximum values before blending. Nevertheless, shift alignment cannot resolve true scale differences. Scan-order-dependent error accumulation and blurring caused by averaging overlapping regions may remain.

9. Run Inference on Another Area from Scratch

Change the input path and output name, then run the same command. Depth results or tiles from the previous area are not reused.

python -u infer_geotiff_tiled.py \
  --input input/another_area_rgb.tif \
  --output-prefix output/another_area_da3_t300_o100_r518_shift \
  --tile-size 300 --overlap 100 --process-res 518 \
  --alignment shift \
  2>&1 | tee logs/another_area_da3_t300_o100_r518_shift.log

The scripts prevent overwriting existing outputs. When rerunning inference on the same image, specify an unused --output-prefix. The DA3 scripts in this article do not support resuming an interrupted run.

10. Understanding the Output Files

Filename suffix	Contents	Main use
`_depth.tif`	Float32 relative depth; larger values indicate greater distance	Numerical analysis and downstream processing
`_near01.tif`	Display GeoTIFF normalized to 0–1, with nearer regions set toward 1	Visualization in QGIS
`_near.png`	8-bit PNG using the same display representation	Image comparisons and article illustrations
`_metadata.json`	Inference settings, GPU, image dimensions, tile corrections, and other information	Checking execution conditions

The GeoTIFF outputs retain the input CRS, affine transform, width, and height. The output NoData value is -9999. Invalid pixel locations are preserved, but arbitrary TIFF tags and the original NoData value are not all copied unchanged.

For tiled inference, _depth.tif contains relative depth after blending corrections, not the uncorrected predictions for individual tiles. Shift or affine corrections can produce negative values. The script does not save unblended tile predictions as separate files.

The near01 Display Is Not Building Height

Let $P_{1}, P_{99}$ be the 1st and 99th percentiles of the valid depth values across the blended mosaic. The display image is computed as:

N = clip (\frac{P _{99} - D}{P _{99} - P _{1}}, 0, 1)

Nearer regions appear white and farther regions black. This is contrast adjustment and a linear reversal of the depth direction, not inverse depth $1/ D$ or a conversion to height in meters.

Because the display range is determined independently for each image, the same brightness in near01 outputs from different regions does not represent the same depth or height. Use _depth.tif for numerical processing.

11. Inspect the Results in QGIS

Load the original RGB GeoTIFF and _near01.tif into QGIS.
Display _near01.tif using “Singleband gray.”
Set the minimum to 0, maximum to 1, and color ramp from black to white.
At the same location and scale, inspect building boundaries, roof ridges, roof surfaces, and their relationship to roads.

When displaying _depth.tif, stretch it according to its actual value range. A fixed 0–1 range may cause many pixels to appear solid white or black. A white image alone does not establish that inference failed.

Example checks for coordinates and output values:

python - <<'PY'
import numpy as np
import rasterio
source='input/area_rgb.tif'
result='output/area_da3_t300_o100_r518_shift_depth.tif'
with rasterio.open(source) as src, rasterio.open(result) as dst:
    assert src.crs == dst.crs
    assert src.transform == dst.transform
    assert (src.width, src.height) == (dst.width, dst.height)
    source_valid=np.all(src.read_masks([1,2,3]) > 0, axis=0)
    depth=dst.read(1, masked=True)
    assert np.array_equal(source_valid, ~np.ma.getmaskarray(depth))
    values=depth.compressed()
    assert values.size and np.isfinite(values).all()
    print('CRS, pixel grid, and valid pixel locations: OK')
    print('NoData:', dst.nodata)
    print('min / median / max:', values.min(), np.median(values), values.max())
PY

If you change the RGB bands, update [1,2,3] in the checking code accordingly. These checks validate data positioning and format; they do not evaluate depth accuracy.

Example Inference Results

The image below shows the input and the monocular depth estimation result from Depth Anything 3. The input is an archived WorldView-2 image processed with 2× super-resolution.

Input image and Depth Anything 3 monocular depth estimation result:

Left: input image after 2× super-resolution. Right: estimated relative depth displayed with nearer regions brighter.

Differences between buildings and the surrounding ground, as well as relief on some roof surfaces, appear as variations in brightness. However, broad brightness variations are also visible across the image. These may include terrain effects and errors from estimation or tile blending; this image alone cannot distinguish their causes.

Roof brightness should therefore not be interpreted directly as building height. Relative differences from the surrounding ground and agreement with measured values must be checked separately.

12. Common Problems

Symptom	Checks and actions
`CUDA GPU required` / CUDA is not detected	Check `nvidia-smi`, `python check_env.py`, and the active conda environment
CUDA out of memory	Lower `process-res` for whole-image inference or use tiled inference. For tiled inference, compare 518 with another multiple of 14, such as 392.
`xFormers` version mismatch	Check the PyTorch/xFormers combination. Do not upgrade only one of them in the environment.
`Model directory missing`	Complete `python download_model.py`. If the model is stored elsewhere, specify `--model`.
`Input must be RGB uint8`	Check that the input is the source RGB image and that band selection and data type are correct
Output appears entirely white or black	Check the display range. Start by displaying near01 over 0–1 and inspecting depth minimum and maximum values.
Depth discontinuities at tile boundaries	Compare overlap widths and blending modes. Distinguish actual height differences from blending errors.
Buildings appear too high or too low overall	Check whether relative depth is being interpreted as height in meters
`Output exists`	Run from scratch with an unused output prefix

This implementation tiles model inputs, but it loads the original RGB image and mosaic arrays into CPU memory. It is not a streaming implementation that processes arbitrarily large GeoTIFFs with constant memory usage. Crop an AOI when working with large-area imagery.

13. Separate the Evaluation Questions

Visually sharp building boundaries, correct local depth geometry, and building heights that agree with measurements are separate evaluation questions.

For comparisons, use the same source image, crop extent, and display scale when examining whole-image versus tiled inference and different blending modes. Comparisons with DA2 or other models also require attention to internal input resolution and the representation of output values.

GeoTIFF export in this workflow preserves image georeferencing. It does not calibrate depth to ground-relative height or elevation. To proceed to a DSM or 3D building models, depth-to-height relationships, ground treatment, and validation against measured values must be addressed separately.

References

Appendix: Scripts

The following scripts are the GeoTIFF wrappers used in this article. Save each code block directly in the working directory using the indicated filename. The official model implementation and weights are not included here; obtain them from the official sources using the steps above.

constraints.txt

Full contents of constraints.txt

torch==2.9.1
torchvision==0.24.1
xformers==0.0.33.post2
numpy==1.26.4
opencv-python==4.10.0.84
moviepy==1.0.3

setup.sh

Full contents of setup.sh

#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")"
if [[ "${CONDA_DEFAULT_ENV:-}" != "da3_bw" ]]; then
  echo 'Activate the dedicated environment first: conda activate da3_bw' &gt;&amp;2
  exit 1
fi
mkdir -p vendor models output logs
python -m pip install --upgrade pip
python -m pip install -c constraints.txt torch torchvision xformers \
  --index-url https://download.pytorch.org/whl/cu128
if [[ ! -d vendor/Depth-Anything-3 ]]; then
  git clone https://github.com/ByteDance-Seed/Depth-Anything-3.git vendor/Depth-Anything-3
fi
python -m pip install -c constraints.txt -e ./vendor/Depth-Anything-3
python -m pip install -c constraints.txt rasterio addict
python -m pip check
git -C vendor/Depth-Anything-3 rev-parse HEAD &gt; logs/da3_git_commit.txt
python -m pip freeze &gt; logs/environment_freeze.txt
python check_env.py

check_env.py

Full contents of check_env.py

import torch
import torchvision
import xformers
import rasterio
from depth_anything_3.api import DepthAnything3

print('torch:', torch.__version__)
print('torchvision:', torchvision.__version__)
print('xformers:', xformers.__version__)
print('CUDA runtime:', torch.version.cuda)
print('rasterio:', rasterio.__version__)
assert torch.cuda.is_available(), 'CUDA GPU is unavailable'
print('GPU:', torch.cuda.get_device_name(0))
print('Capability:', torch.cuda.get_device_capability(0))
x = torch.randn(512, 512, device='cuda')
y = x @ x
torch.cuda.synchronize()
assert torch.isfinite(y).all()
print('Imports and CUDA matrix multiplication: OK (model inference not yet tested)')

download_model.py

Full contents of download_model.py

from pathlib import Path
from huggingface_hub import HfApi, snapshot_download

root = Path(__file__).resolve().parent
repo = 'depth-anything/DA3MONO-LARGE'
info = HfApi().model_info(repo)
path = snapshot_download(
    repo_id=repo, revision=info.sha,
    local_dir=str(root / 'models' / 'DA3MONO-LARGE'),
    allow_patterns=['*.json', '*.safetensors', '*.md', 'LICENSE*'],
)
(root / 'models' / 'DA3MONO-LARGE' / 'download_revision.txt').write_text(
    f'{repo}\n{info.sha}\n', encoding='utf-8')
print('Model downloaded:', path)

infer_geotiff.py

Full contents of infer_geotiff.py

"""Single-image DA3MONO inference. Relative depth, never metric height.

Whole-image inference avoids independent tile scales. The API resizes, without
cropping; the depth is bilinearly restored to the source pixel grid. Original
CRS/affine and the intersection of RGB valid masks are retained. Unknown extra
bands are ignored; rasterio-recognized alpha masks are honored.
"""
from pathlib import Path
import argparse
import json
import time
import numpy as np
import rasterio
from PIL import Image

NODATA = -9999.0


def read_rgb(path, bands):
    with rasterio.open(path) as src:
        if min(bands) &lt; 1 or max(bands) &gt; src.count:
            raise ValueError(f'Invalid RGB bands {bands}; source has {src.count} bands')
        rgb = src.read(bands)
        if rgb.dtype != np.uint8:
            raise ValueError('Input must be RGB uint8. Convert 16-bit/float data explicitly first.')
        if src.crs is None:
            raise ValueError('Input has no CRS; supply a georeferenced RGB GeoTIFF')
        valid = np.all(src.read_masks(bands) &gt; 0, axis=0)
        if not valid.any():
            raise ValueError('Input has no valid RGB pixels')
        profile = dict(driver='GTiff', width=src.width, height=src.height,
                       count=1, dtype='float32', crs=src.crs, transform=src.transform,
                       nodata=NODATA, compress='deflate', predictor=3, tiled=True,
                       blockxsize=256, blockysize=256, BIGTIFF='IF_SAFER')
    rgb = np.moveaxis(rgb, 0, -1).copy()
    # Fill masked pixels only for inference; their outputs remain NoData.
    if not valid.all():
        rgb[~valid] = np.median(rgb[valid], axis=0).astype(np.uint8)
        print('Warning: masked areas can affect nearby predictions.', flush=True)
    return rgb, valid, profile


def restore_depth(depth, height, width):
    depth = np.asarray(depth, dtype=np.float32)
    if depth.ndim != 2 or not np.isfinite(depth).all():
        raise ValueError('Model depth must be a finite 2-D array')
    if depth.shape != (height, width):
        depth = np.asarray(Image.fromarray(depth).resize(
            (width, height), resample=Image.Resampling.BILINEAR), dtype=np.float32)
    return depth.copy()


def write_raster(path, data, valid, profile, description, tags):
    arr = np.where(valid, data, NODATA).astype(np.float32)
    with rasterio.open(path, 'w', **profile) as dst:
        dst.write(arr, 1)
        dst.set_band_description(1, description)
        dst.update_tags(**tags)
        v = arr[valid].astype(np.float64)
        dst.update_tags(1, STATISTICS_MINIMUM=str(v.min()),
                        STATISTICS_MAXIMUM=str(v.max()),
                        STATISTICS_MEAN=str(v.mean()),
                        STATISTICS_STDDEV=str(v.std()),
                        STATISTICS_VALID_PERCENT=str(100 * valid.mean()))


def save_results(prefix, depth, valid, profile, meta):
    vals = depth[valid]
    if not np.isfinite(vals).all():
        raise ValueError('Non-finite predictions found')
    lo, hi = np.percentile(vals, [1, 99])
    if hi &lt;= lo:
        raise ValueError('Depth map is effectively constant; do not treat it as a valid result')
    # Display only: linear sign reversal, NOT inverse-depth or metric height.
    near = np.clip((hi - depth) / (hi - lo), 0, 1).astype(np.float32)
    tags = dict(model='DA3MONO-LARGE', units='relative_arbitrary_units',
                inference='whole_image', process_res=str(meta['process_res']),
                native_prediction_shape=str(meta['native_prediction_shape']))
    write_raster(str(prefix) + '_depth.tif', depth, valid, profile,
                 'Relative depth (larger = farther); NOT height in meters', tags)
    write_raster(str(prefix) + '_near01.tif', near, valid, profile,
                 'Display only: near=1 far=0; clipped P1-P99; NOT height',
                 dict(tags, units='normalized_display_only',
                      display_formula='clip((P99-depth)/(P99-P1),0,1)',
                      display_p1=str(lo), display_p99=str(hi)))
    preview = np.zeros(depth.shape, dtype=np.uint8)
    preview[valid] = np.rint(near[valid] * 255).astype(np.uint8)
    Image.fromarray(preview).save(str(prefix) + '_near.png')
    meta.update(depth_statistics={
        'min': float(vals.min()), 'max': float(vals.max()),
        'p1': float(lo), 'median': float(np.median(vals)), 'p99': float(hi)},
        valid_pixels=int(valid.sum()), nodata=NODATA,
        depth_units='arbitrary; not meters',
        preview='near=white; P1/P99 clipping; invalid=black',
        output_resampling='bilinear onto original pixel grid; no new detail added')
    Path(str(prefix) + '_metadata.json').write_text(
        json.dumps(meta, ensure_ascii=False, indent=2), encoding='utf-8')
    print(json.dumps(meta['depth_statistics'], indent=2), flush=True)


def main():
    p = argparse.ArgumentParser(description=__doc__)
    p.add_argument('--input', required=True)
    p.add_argument('--output-prefix', required=True)
    p.add_argument('--model', default=str(Path(__file__).resolve().parent / 'models/DA3MONO-LARGE'))
    p.add_argument('--process-res', type=int, default=1008,
                   help='Inference longest side, divisible by 14; try 1008 then 2016')
    p.add_argument('--bands', nargs=3, type=int, default=[1, 2, 3], metavar=('R', 'G', 'B'))
    p.add_argument('--device', default='cuda:0')
    args = p.parse_args()
    if args.process_res &lt; 28 or args.process_res % 14:
        p.error('--process-res must be &gt;=28 and divisible by 14')
    inp = Path(args.input).expanduser().resolve()
    prefix = Path(args.output_prefix).expanduser().resolve()
    if not Path(args.model).is_dir():
        p.error('Local model directory missing; run download_model.py first')
    for suffix in ['_depth.tif', '_near01.tif', '_near.png', '_metadata.json']:
        if Path(str(prefix) + suffix).exists():
            p.error(f'Output exists: {prefix}{suffix}; use a new output prefix')
    prefix.parent.mkdir(parents=True, exist_ok=True)
    rgb, valid, profile = read_rgb(inp, args.bands)
    height, width = valid.shape
    if min(height, width) / max(height, width) * args.process_res &lt; 28:
        p.error('Image aspect ratio is too extreme at this process resolution')
    print(f'Input: {width}x{height}, RGB bands {args.bands}; process-res={args.process_res}', flush=True)
    import torch
    from depth_anything_3.api import DepthAnything3
    if not args.device.startswith('cuda') or not torch.cuda.is_available():
        p.error('This workflow requires a CUDA GPU')
    torch.cuda.set_device(torch.device(args.device))
    print(f'GPU: {torch.cuda.get_device_name()}', flush=True)
    model = DepthAnything3.from_pretrained(args.model).to(args.device).eval()
    if model.model_name != 'da3mono-large':
        raise ValueError(f'This workflow expects da3mono-large, got {model.model_name}')
    start = time.time()
    with torch.inference_mode():
        prediction = model.inference(
            [rgb], process_res=args.process_res,
            process_res_method='upper_bound_resize',
            infer_gs=False, export_dir=None,
        )
    native = np.asarray(prediction.depth[0], dtype=np.float32)
    print('Native depth shape:', native.shape, flush=True)
    depth = restore_depth(native, height, width)
    meta = dict(input=str(inp), model=str(Path(args.model).resolve()),
                model_name=model.model_name, process_res=args.process_res,
                native_prediction_shape=list(native.shape), output_shape=[height, width],
                crs=str(profile['crs']), transform=list(profile['transform']),
                rgb_bands=args.bands, torch_version=torch.__version__,
                cuda_runtime=torch.version.cuda, gpu=torch.cuda.get_device_name(),
                inference_seconds=time.time() - start)
    save_results(prefix, depth, valid, profile, meta)
    print('Done:', prefix, flush=True)


if __name__ == '__main__':
    main()

da3_tile_io.py

Full contents of da3_tile_io.py

"""Raster IO for experimental tiled relative depth mosaics."""
from pathlib import Path
import argparse
import json
import time
import numpy as np
import rasterio
from PIL import Image

NODATA = -9999.0


def read_rgb(path, bands):
    with rasterio.open(path) as src:
        if min(bands) &lt; 1 or max(bands) &gt; src.count:
            raise ValueError(f'Invalid RGB bands {bands}; source has {src.count} bands')
        rgb = src.read(bands)
        if rgb.dtype != np.uint8:
            raise ValueError('Input must be RGB uint8. Convert 16-bit/float data explicitly first.')
        if src.crs is None:
            raise ValueError('Input has no CRS; supply a georeferenced RGB GeoTIFF')
        valid = np.all(src.read_masks(bands) &gt; 0, axis=0)
        if not valid.any():
            raise ValueError('Input has no valid RGB pixels')
        profile = dict(driver='GTiff', width=src.width, height=src.height,
                       count=1, dtype='float32', crs=src.crs, transform=src.transform,
                       nodata=NODATA, compress='deflate', predictor=3, tiled=True,
                       blockxsize=256, blockysize=256, BIGTIFF='IF_SAFER')
    rgb = np.moveaxis(rgb, 0, -1).copy()
    # Fill masked pixels only for inference; their outputs remain NoData.
    if not valid.all():
        rgb[~valid] = np.median(rgb[valid], axis=0).astype(np.uint8)
        print('Warning: masked areas can affect nearby predictions.', flush=True)
    return rgb, valid, profile


def restore_depth(depth, height, width):
    depth = np.asarray(depth, dtype=np.float32)
    if depth.ndim != 2 or not np.isfinite(depth).all():
        raise ValueError('Model depth must be a finite 2-D array')
    if depth.shape != (height, width):
        depth = np.asarray(Image.fromarray(depth).resize(
            (width, height), resample=Image.Resampling.BILINEAR), dtype=np.float32)
    return depth.copy()


def write_raster(path, data, valid, profile, description, tags):
    arr = np.where(valid, data, NODATA).astype(np.float32)
    with rasterio.open(path, 'w', **profile) as dst:
        dst.write(arr, 1)
        dst.set_band_description(1, description)
        dst.update_tags(**tags)
        v = arr[valid].astype(np.float64)
        dst.update_tags(1, STATISTICS_MINIMUM=str(v.min()),
                        STATISTICS_MAXIMUM=str(v.max()),
                        STATISTICS_MEAN=str(v.mean()),
                        STATISTICS_STDDEV=str(v.std()),
                        STATISTICS_VALID_PERCENT=str(100 * valid.mean()))


def save_results(prefix, depth, valid, profile, meta):
    vals = depth[valid]
    if not np.isfinite(vals).all():
        raise ValueError('Non-finite predictions found')
    lo, hi = np.percentile(vals, [1, 99])
    if hi &lt;= lo:
        raise ValueError('Depth map is effectively constant; do not treat it as a valid result')
    # Display only: linear sign reversal, NOT inverse-depth or metric height.
    near = np.clip((hi - depth) / (hi - lo), 0, 1).astype(np.float32)
    tags = dict(model='DA3MONO-LARGE', units='relative_arbitrary_units',
                inference=meta['inference'], alignment=meta['alignment'], tile_size=str(meta['tile_size']), overlap=str(meta['overlap']), process_res=str(meta['process_res']),
                native_prediction_shape=str(meta['native_prediction_shape']))
    write_raster(str(prefix) + '_depth.tif', depth, valid, profile,
                 'Relative depth (larger = farther); NOT height in meters', tags)
    write_raster(str(prefix) + '_near01.tif', near, valid, profile,
                 'Display only: near=1 far=0; clipped P1-P99; NOT height',
                 dict(tags, units='normalized_display_only',
                      display_formula='clip((P99-depth)/(P99-P1),0,1)',
                      display_p1=str(lo), display_p99=str(hi)))
    preview = np.zeros(depth.shape, dtype=np.uint8)
    preview[valid] = np.rint(near[valid] * 255).astype(np.uint8)
    Image.fromarray(preview).save(str(prefix) + '_near.png')
    meta.update(depth_statistics={
        'min': float(vals.min()), 'max': float(vals.max()),
        'p1': float(lo), 'median': float(np.median(vals)), 'p99': float(hi)},
        valid_pixels=int(valid.sum()), nodata=NODATA,
        depth_units='arbitrary; not meters',
        preview='near=white; P1/P99 clipping; invalid=black',
        output_resampling='bilinear onto original pixel grid; no new detail added')
    Path(str(prefix) + '_metadata.json').write_text(
        json.dumps(meta, ensure_ascii=False, indent=2), encoding='utf-8')
    print(json.dumps(meta['depth_statistics'], indent=2), flush=True)

infer_geotiff_tiled.py

Full contents of infer_geotiff_tiled.py

"""DA3MONO independent overlapping tile inference; experimental relative-depth mosaic."""
import argparse
from pathlib import Path
import time
import numpy as np
from da3_tile_io import read_rgb, restore_depth, save_results


def starts(length, tile, overlap):
    if length &lt;= tile:
        return [0]
    pos = list(range(0, length - tile + 1, tile - overlap))
    if pos[-1] != length - tile:
        pos.append(length - tile)
    return pos


def fit_overlap(x, y, mode):
    """Robust positive affine fit; reject poorly constrained scale estimates."""
    x, y = np.asarray(x, dtype=np.float64), np.asarray(y, dtype=np.float64)
    if not len(x):
        return 1., 0., 'unanchored'
    step = max(1, len(x) // 20000)
    x, y = x[::step], y[::step]
    shift = float(np.median(y - x))
    if mode == 'shift' or len(x) &lt; 128:
        return 1., shift, 'shift'
    keep = np.ones(len(x), dtype=bool)
    a, b = 1., shift
    for _ in range(4):
        xx, yy = x[keep], y[keep]
        if len(xx) &lt; 128 or np.std(xx) &lt; 1e-6 * max(1., abs(np.mean(xx))):
            return 1., shift, 'fallback_flat_overlap'
        dx, dy = xx - xx.mean(), yy - yy.mean()
        corr = np.dot(dx, dy) / max(np.linalg.norm(dx) * np.linalg.norm(dy), 1e-20)
        a = float(np.dot(dx, dy) / np.dot(dx, dx))
        if corr &lt; .5 or not .25 &lt;= a &lt;= 4.:
            return 1., shift, 'fallback_unstable_scale'
        b = float(np.median(yy - a * xx))
        residual = y - (a * x + b)
        med = np.median(residual)
        mad = np.median(np.abs(residual - med))
        keep = np.abs(residual - med) &lt;= max(3 * 1.4826 * mad, 1e-6)
    return a, b, 'affine'


def blend_weight(height, width):
    # Positive edge weights retain image boundary pixels.
    wy = np.maximum(np.sin(np.pi * (np.arange(height) + .5) / height)**2, .02)
    wx = np.maximum(np.sin(np.pi * (np.arange(width) + .5) / width)**2, .02)
    return wy[:, None] * wx[None, :]


def mosaic(rgb, valid, predict, tile, overlap, alignment):
    h, w = valid.shape
    total = np.zeros((h, w), dtype=np.float64)
    weights = np.zeros((h, w), dtype=np.float64)
    records = []
    ys, xs = starts(h, tile, overlap), starts(w, tile, overlap)
    for y in ys:
        for x in xs:
            sl = np.s_[y:min(y+tile, h), x:min(x+tile, w)]
            mask = valid[sl]
            if not mask.any():
                continue
            patch = rgb[sl]
            ph, pw = mask.shape
            # Small images padded; normal edge tiles remain exactly tile x tile.
            padded = np.pad(patch, ((0, tile-ph), (0, tile-pw), (0, 0)), mode='edge')
            native = predict(padded)
            depth = restore_depth(native, tile, tile)[:ph, :pw]
            old_w, old_sum = weights[sl], total[sl]
            common = mask &amp; (old_w &gt; 0)
            a, b, status = 1., 0., 'none'
            if alignment != 'none':
                a, b, status = fit_overlap(depth[common], old_sum[common]/old_w[common], alignment)
            corrected = a * depth + b
            weight = blend_weight(ph, pw) * mask
            old_sum += corrected * weight
            old_w += weight
            records.append(dict(x=x, y=y, width=pw, height=ph,
                                native_shape=list(np.shape(native)), scale=a, shift=b,
                                alignment_status=status, overlap_pixels=int(common.sum())))
            print(f'Tile {len(records)}/{len(ys)*len(xs)} ({x},{y}) {status}: a={a:.4g}, b={b:.4g}', flush=True)
    if np.any(valid &amp; (weights &lt;= 0)):
        raise RuntimeError('Valid pixels not covered')
    output = np.zeros((h, w), dtype=np.float32)
    output[valid] = (total[valid] / weights[valid]).astype(np.float32)
    return output, records


def main():
    p = argparse.ArgumentParser(description=__doc__)
    p.add_argument('--input', required=True)
    p.add_argument('--output-prefix', required=True)
    p.add_argument('--model', default=str(Path(__file__).resolve().parent/'models/DA3MONO-LARGE'))
    p.add_argument('--tile-size', type=int, default=300)
    p.add_argument('--overlap', type=int, default=100, help='Overlap pixels, NOT stride')
    p.add_argument('--process-res', type=int, default=518, help='Model input longest side, multiple of 14')
    p.add_argument('--alignment', choices=['affine', 'shift', 'none'], default='affine')
    p.add_argument('--bands', nargs=3, type=int, default=[1,2,3])
    p.add_argument('--device', default='cuda:0')
    args = p.parse_args()
    if args.tile_size &lt; 28 or not 0 &lt;= args.overlap &lt; args.tile_size:
        p.error('Require tile-size &gt;=28 and 0 &lt;= overlap &lt; tile-size')
    if args.alignment != 'none' and args.overlap == 0:
        p.error('Alignment needs overlap &gt;0; use --alignment none for nonoverlapping tiles')
    if args.process_res &lt; 28 or args.process_res % 14:
        p.error('process-res must be &gt;=28 and divisible by 14')
    if not Path(args.model).is_dir():
        p.error('Model directory missing; specify existing --model path')
    prefix = Path(args.output_prefix).expanduser().resolve()
    for suffix in ['_depth.tif','_near01.tif','_near.png','_metadata.json']:
        if Path(str(prefix)+suffix).exists():
            p.error(f'Output exists: {prefix}{suffix}; use a new prefix')
    rgb, valid, profile = read_rgb(args.input, args.bands)
    import torch
    from depth_anything_3.api import DepthAnything3
    if not args.device.startswith('cuda') or not torch.cuda.is_available():
        p.error('CUDA GPU required')
    torch.cuda.set_device(torch.device(args.device))
    model = DepthAnything3.from_pretrained(args.model).to(args.device).eval()
    if model.model_name != 'da3mono-large':
        p.error('Expected DA3MONO-LARGE')
    def predict(patch):
        with torch.inference_mode():
            result = model.inference([patch], process_res=args.process_res,
                                     process_res_method='upper_bound_resize',
                                     infer_gs=False, export_dir=None)
        return np.asarray(result.depth[0], dtype=np.float32)
    start = time.time()
    depth, records = mosaic(rgb, valid, predict, args.tile_size, args.overlap, args.alignment)
    meta = dict(input=str(Path(args.input).resolve()), model=str(Path(args.model).resolve()),
                model_name=model.model_name, process_res=args.process_res,
                native_prediction_shape=records[0]['native_shape'], output_shape=list(valid.shape),
                crs=str(profile['crs']), transform=list(profile['transform']), rgb_bands=args.bands,
                inference='independent_overlapping_tiles', tile_size=args.tile_size,
                overlap=args.overlap, stride=args.tile_size-args.overlap, alignment=args.alignment,
                alignment_warning='Experimental relative mosaic; scan-order drift and disconnected scales possible; not metric depth or height',
                tiles=records, torch_version=torch.__version__, cuda_runtime=torch.version.cuda,
                gpu=torch.cuda.get_device_name(), inference_seconds=time.time()-start)
    prefix.parent.mkdir(parents=True, exist_ok=True)
    save_results(prefix, depth, valid, profile, meta)
    print('Done:', prefix, flush=True)


if __name__ == '__main__':
    main()

DINO-EdgeQuery: Learning Building Edges and Their Connections for GIS-Ready Polygons

Yuji Kobayashi — Tue, 13 Oct 2026 01:11:50 +0000

Introduction

When extracting buildings from satellite or aerial imagery, predicting building regions through segmentation still leaves another task: converting those regions into building polygons that are practical to use in GIS.

Directly vectorizing mask contours can produce jagged boundaries and unnecessary vertices. Aggressive simplification, however, can remove important corners and small geometric details. Recognizing building regions accurately and producing geometry suitable for editing and inspection are two problems that need to be addressed separately.

To address this, we developed DINO-EdgeQuery, which learns the edges that form a building and the connections between them.

The model predicts candidate edges from the image features of each building and learns which edge follows which. The final vertices are computed as intersections of the supporting lines of the ordered edges.

This article introduces the architecture, training procedure, and evaluation results based on our paper and presentation at the Academic Track of FOSS4G Hiroshima 2026. The paper is available in the ISPRS Archives.

Although the intended application is building extraction from high-resolution satellite imagery, the training and quantitative evaluation reported here use aerial imagery from the WHU Building Dataset. These results do not demonstrate equivalent performance on satellite imagery.

This article provides a concise overview of the paper accepted for the FOSS4G Hiroshima 2026 Academic Track and the presentation delivered at the conference.

1. Why Predict Edges?

Mask Accuracy and Polygon Quality

A common building extraction workflow consists of building mask prediction, contour tracing, and polygon simplification. This approach can combine existing segmentation models with GIS processing, but errors in mask boundaries affect the final geometry.

Building polygons used in GIS need more than good region overlap. They should also meet requirements such as:

Representing straight boundaries with few vertices
Placing building corners appropriately
Avoiding self-intersections and degenerate geometry
Avoiding duplicate outputs for the same building
Carrying geographic coordinates so they can be inspected and edited in GIS

Many building outlines consist of long straight segments and a relatively small number of corners. DINO-EdgeQuery reflects this property by using edges as the primary prediction unit.

Positioning Among Existing Approaches

The related work discussed in the paper can be organized by its primary representation:

Approach	Main representation or idea
Mask contour tracing and simplification	Construct polygons from predicted raster boundaries
Frame Field Learning	Learn a boundary orientation field alongside masks to assist polygonization
HiSup	Use hierarchical supervision of vertices, line segments, and regions to learn masks suited to polygonization
PolyWorld / Re:PolyWorld	Represent vertices and their connections as a graph
BoundaryFormer	Predict object boundaries as sequences of polygon points
DINO-EdgeQuery	Predict edges and their successor relations, then construct vertices from supporting-line intersections

This table compares representations. The experiments presented in this article are not a performance comparison against these methods under identical conditions.

2. Overall Architecture: From Image Recognition to Geometry

DINO-EdgeQuery combines neural network predictions with deterministic polygon construction.

flowchart TD
    I[Input image] --> F[DINOv3 and Adapter Neck]
    F --> D[Instance Decoder]
    F --> R[ROI Tokenizer]
    D -->|Predicted masks and boxes| R
    R --> E[EdgeQuery Decoder]
    E --> H[Edge attributes and successor relations]
    H --> G[Cycle selection and line intersections]
    D -->|Predicted masks| P[Quality checks and duplicate suppression]
    G --> P
    P --> O[Coordinate transformation and GeoJSON export]

The responsibilities are divided as follows:

Component	Method	Role
DINOv3	Neural network with frozen pretrained weights	Extract image features
Adapter Neck and Instance Decoder	Neural networks	Predict masks, boxes, and other information for each building
ROI Tokenizer, EdgeQuery, and prediction heads	Neural networks	Predict building edges and successor relations
Cycle selection and line intersections	Deterministic geometric processing	Order edges and construct vertices
Quality Gate P1 / P2	Deterministic checks and filtering	Remove invalid geometry and duplicate candidates
Coordinate transformation and export	Geospatial processing	Connect the output to GIS through GeoJSON

Both edge prediction and successor relation prediction in EdgeQuery are neural network operations. Geometric algorithms are used for polygon construction and quality checks after these predictions have been produced.

3. DINOv3 and Instance Segmentation Provide Building-Level Cues

DINOv3 and the Adapter Neck

We use DINOv3 with frozen weights to extract features from the input image. The Adapter Neck converts its intermediate features into multiscale features for the downstream components.

The final implementation primarily uses feature maps with strides of 8, 16, and 32 relative to the input. The paper denotes these as $F_{8}$ , $F_{16}$ , and $F_{32}$ .

To supplement boundary information, features corresponding to F4 are extracted from the RGB image by a shallow CNN and added to the F8 path as a lightweight residual. The operation corresponding to Equation (2) in the paper is:

F_{8'} = F_{8} + ϕ_{F_{4} \to F_{8}} (ϕ_{stem} (I))

The F4-equivalent features are not used directly as a separate mask input or ROI input. The downstream components use the corrected $F_{8'}$ together with $F_{16}$ and $F_{32}$ .

Masks Provide Cues About Building Location and Extent

The Mask2Former-style Instance Decoder predicts a class, box, mask, and query embedding for each building candidate.

These masks provide visual priors for identifying the target building and determining which feature region to access. They are also used to check the consistency of the generated polygons.

The final polygon boundary is constructed from the edges predicted by EdgeQuery, rather than adopting the mask contour directly. However, EdgeQuery depends on instance segmentation and cannot recover buildings missed by the preceding stage.

4. EdgeQuery: Predicting Candidate Edges for Each Building

ROI Tokenizer

ROI stands for Region of Interest. The ROI Tokenizer uses each building's mask and box to generate local tokens from the corresponding multiscale features.

In the final configuration presented in the slides, the ROI size is $24 \times 24$ and the feature dimension is 256. EdgeQuery predicts edges using this building-specific context.

A Query Is a Slot for a Candidate Edge

EdgeQuery here does not mean asking an image questions in natural language. It refers to query slots used to predict the edges forming a building boundary.

The final configuration uses $Q = 96$ building candidates per tile and $E = 48$ edge slots per building candidate. This does not mean that every building has 48 edges. The edges used in the output are selected based on activity, successor relations, and other information.

The edge predictions include the following:

Information	Meaning
Activity	A score used to determine whether an edge slot is active
Center	The center position of the edge
Direction	The direction of the edge
Half-length	Half the edge length; the full segment length is twice this value
Mode and dominant axis	Information about the edge mode and the building's dominant axis
Relation embedding	An embedding used to predict successor relations between edges

The outputs provided by the prediction heads must be distinguished from the loss terms active in the final training configuration. In the final StageG2, the directly supervised quantities are activity, center, direction, half-length, and successor relations, as described below.

Successor Relations: Learning Which Edge Comes Next

Even after candidate edges have been predicted, the set alone does not specify the order in which to trace the exterior boundary. The Successor Relation Head predicts directed relations indicating which edge follows a given edge.

Let $S_{q}$ be the successor score matrix for building candidate $q$ . With $E$ edge slots, this represents $E \times E$ relations. For a four-edge building, for example, the selected edges might form a closed sequence such as 1→3→2→4→1.

The neural network predicts edge attributes and successor scores. Selecting the actual cycle from these outputs is a subsequent deterministic operation.

5. Constructing Vertices from Supporting-Line Intersections

A supporting line is the infinite line obtained by extending a predicted segment in both directions. We define each supporting line from the edge center and direction, then use the intersections of adjacent supporting lines along the cycle as vertices.

The following is Equation (4) from the paper, where $q$ denotes a building candidate and $k$ an edge candidate:

ParseError: KaTeX parse error: Expected '}', got '\right' at position 288: …\hat{\rho}{q,k}\̲r̲i̲g̲h̲t̲},\ \hat{\mathb…

$c^$ is the predicted center and $d^$ the predicted direction. $R_{π /2}$ rotates the direction by 90 degrees to obtain the line normal $n^$ .

By arranging the supporting lines in the order of the selected cycle $C_{q} = (k_{1}, \dots, k_{V})$ , corners are determined by intersections of adjacent lines. This explicitly represents the long straight boundaries and corners of a building.

Computing intersections alone does not always produce the correct building geometry. Errors in edge prediction or cycle selection can produce unnatural outlines or invalid polygons, so the following quality checks are applied.

6. Quality Gate P1 / P2: Checking Geometry and Duplicates

P1: Checking Geometry and Consistency with the Predicted Mask

P1 checks the polygon constructed from line intersections using its IoU with the predicted mask, coverage, outside ratio, area ratio, and geometric validity. Self-intersecting and empty polygons are also excluded.

The main thresholds reported in the paper are a mask IoU of at least 0.15, coverage of at least 0.35, and an outside ratio of at most 0.65. These are quality-check thresholds used during inference, separate from the IoU thresholds for evaluation against ground-truth polygons described later.

P2: Suppressing Duplicate Polygons Based on Containment

P2 adds containment-based duplicate suppression to P1.

With conventional IoU-based NMS, a small polygon inside a large polygon may remain because a substantial difference in area results in a low IoU.

For example, if a polygon of area 10 lies entirely inside a polygon of area 100, their IoU is 0.1. Yet 100% of the smaller polygon is covered. This numerical example illustrates the difference between containment coverage and IoU.

P2 removes $P_{a}$ when another polygon $P_{b}$ satisfies all of the following conditions, corresponding to Equation (5) in the paper:

\frac{∣ P _{a} \cap P _{b} ∣}{∣ P _{a} ∣} \geq 0.50, ∣ P_{b} ∣ \geq ∣ P_{a} ∣, centroid (P_{a}) \in P_{b} .

In other words, at least 50% of $P_{a}$ is covered, the other polygon has an equal or greater area, and the centroid of $P_{a}$ lies inside it.

This check uses predictions only; it does not access ground-truth data. Geometric validity checks also do not guarantee agreement with actual buildings. False detections and missed buildings must be evaluated separately.

7. Two Training Stages, One Inference Pipeline

From Joint Optimization to Staged Training

The presentation also explained our initial plan to jointly optimize instance segmentation, edge prediction, and polygon construction.

In practice, changes in the initial masks and boxes made ROI conditions unstable. Mask and geometry losses converged at different rates, and gradients were difficult to handle through discrete cycle selection and validity checks. GPU memory needed to retain intermediate states for backpropagation through the full system was another constraint.

This memory issue concerns attempting to treat instance segmentation, EdgeQuery, and polygon construction as a single large training graph. It does not mean that the final staged training procedure or inference pipeline necessarily exhausts 96 GB of memory.

The final implementation trains in two stages and then combines their weights:

Item	StageG1	StageG2
Trainable components	Adapter Neck and Instance Decoder	ROI Tokenizer, EdgeQuery Decoder, and heads for edge attributes and successor relations
Main frozen components	DINOv3 and the EdgeQuery branch	DINOv3, Adapter Neck, and Instance Decoder
Training images	All training tiles, including tiles without buildings	Tiles containing buildings
Main supervision	Classes, boxes, masks, and boundaries	Edge activity, center, direction, half-length, and successor relations
ROI conditions	—	Ground-truth boxes and masks

P1 and P2 are polygon-output quality-check profiles, not additional neural network training stages following G1 and G2.

StageG1: Boundary-Aware Instance Segmentation

In addition to classification, box, and mask losses, we use a BCE loss on a thin band around the ground-truth boundary. This explicitly accounts for boundary errors that would otherwise contribute relatively little compared with the larger building interiors.

Equation (6) in the paper combines classification, box, GIoU, mask BCE, mask Dice, and boundary BCE losses using weighted sums.

StageG2: Learning Edges and Connections with Stable ROIs

StageG2 uses oracle ROIs defined by ground-truth masks and boxes. This stabilizes the local coordinates and target regions during edge learning.

The active losses in the final model are given by Equation (7) in the paper:

L_{EdgeQuery} = λ_{act} L_{act} + λ_{center} L_{center} + λ_{dir} L_{dir} + λ_{len} L_{len} + λ_{succ} L_{succ} .

Losses related to vertices, rasterization, topology, regularity, and cycle consistency, which were included in the initial design, are not active in the final StageG2. Terms retained for compatibility or future extensions are masked to zero and excluded from the reported training objective.

After training, the weights of G1 and G2 are combined. Inference uses neither ground-truth boxes nor ground-truth masks: the entire pipeline operates only on information predicted from the input image.

8. Preparing the Dataset for an Edge Representation

We use aerial imagery and building annotations from the WHU Building Dataset. The 0.2 m resolution data in COCO format used in this study were prepared for EdgeQuery.

COCO polygons cannot be used directly as edge supervision. Their exterior ring orientation and edge counts need to be standardized.

Split each original 1024×1024 image into four non-overlapping 512×512 tiles.
Clip building polygons at tile boundaries and convert them to tile-local coordinates.
Remove tiny fragments and unstable geometries resulting from clipping, and repair invalid geometry where possible.
Remove duplicate vertices and standardize the exterior ring orientation in image coordinates.
Simplify exterior rings with more than 48 edges while preserving topology where possible. Exclude polygons that collapse during this process.
Generate edge attributes and ordered successor relations from the normalized exterior rings.

The current implementation handles one exterior cycle per instance. MultiPolygons are split into individual polygon instances, while holes and nested structures are removed, flattened, or otherwise processed to fit this representation.

This preprocessing is more than an image-format conversion. It also constrains the ground-truth geometry to match the edge count and exterior structure that the model can represent. Since the same preprocessing is applied to the training and validation data, the evaluation results must be interpreted under these conditions.

Final Capacity Configuration

The presentation reported the following configuration for stable operation:

Item	Setting or measured value
Input image	512×512
Microbatch size	8
Gradient accumulation	2 steps; effective batch size of 16
Building queries	$Q = 96$ per tile
Edge slots	$E = 48$ per building candidate
ROI	24×24, 256 channels
Peak VRAM	49,470 MiB, approximately 48.3 GiB

In particular, the successor relation matrix grows quadratically with the number of edge slots. Increasing the edge capacity therefore increases computation and memory requirements for this component. The memory value above was measured for the adopted configuration and does not specify the requirements for every possible input or setting.

9. Experimental Results: What Changes When P2 Reduces Duplicates?

Inference Configuration

The paper reports the following inference profile:

Parameter	Value
Instance confidence threshold	0.60
Mask binarization threshold	0.50
Mask-stage NMS	0.75
Polygon-stage NMS	0.75
Edge activity threshold	0.15
Edge candidates retained for polygon construction	Up to 32 per instance
Boundary support	Enabled
Collinear merging	Disabled

The 48 edge slots used during training and the maximum of 32 edge candidates retained during inference are separate settings.

Results on Validation Tiles Containing Buildings

The following values are from Table 2 in the paper. P1 and P2 are quality-check profiles applied to the outputs of the same model.

Metric	P1	P2
Predicted polygons	21,554	19,203
Union IoU	0.7107	0.7064
Boundary IoU, 2 px	0.3746	0.3710
Boundary IoU, 3 px	0.4535	0.4496
Recall@75	0.6459	0.6213
Precision@75	0.5486	0.5924
Recall@80	0.5442	0.5258
Precision@80	0.4622	0.5013
Predictions / GT	1.1774	1.0489
FP / GT	0.2997	0.2295
Self-intersections / collapses to triangles	0 / 0	0 / 0

Precision@75 and Recall@75 are precision and recall when predicted polygons are matched to ground-truth instances at an IoU threshold of 0.75. The @80 metrics use 0.80. These are not AP metrics. FP / GT is the ratio of false positives to the number of ground-truth instances.

With P2, the number of predicted polygons decreases by 2,351. Precision@75 improves by approximately 4.38 percentage points, and Precision@80 by approximately 3.91 percentage points. Meanwhile, Recall@75 decreases by approximately 2.46 percentage points, and Union IoU decreases by 0.0043.

These results show that P2 improves precision at high IoU thresholds and reduces excessive outputs, with a trade-off in recall. They should not be interpreted as a uniform improvement in IoU.

Both P1 and P2 produced zero self-intersections and zero collapses to triangles in this evaluation. These are results for the outputs after quality checks; they do not guarantee correct geometry or positioning for every input.

Validation Tiles Without Buildings

Table 3 in the paper reports results for 980 tiles without buildings:

Metric	P1	P2
Predicted polygons	90	81
Mean predictions per tile	0.0918	0.0827

The number of predictions also decreases on background-only tiles, but false detections are not eliminated. The value 0.0827 is the mean number of predictions per tile, not an image-level rate indicating that 8.27% of images contain false detections.

Outputs During Validation

Validation outputs: input image, ground-truth mask, predicted mask, candidate edges, selected cycle, and final polygon

Figure 1. DINO-EdgeQuery outputs for a validation image. Top row, left to right: input image, ground-truth mask, and predicted instance masks. Bottom row, left to right: unselected candidate edges, edges in the selected cycle, and final polygons constructed from line intersections. Source: Figure 2 of the original paper.

Figure 2 in the paper shows the input image, ground-truth mask, predicted instances, unselected EdgeQuery edges, selected cycle edges, and final polygons in sequence.

This visualization traces how masks identify the target regions for individual buildings, how predicted edge candidates are selected, and how their intersections form the final outlines. The ground-truth mask in the top-center panel is shown for comparison only and is not used as an inference input.

10. Exporting Image-Space Polygons to GIS as GeoJSON

Polygon vertices are initially obtained in image coordinates. To place them on a map, we transform them using the affine transform of the input GeoTIFF.

The vertex transformation corresponding to Equation (9) in the paper is:

g_{i, t} = A ⎣ ⎡ x_{i, t} y_{i, t} 1 ⎦ ⎤

$A$ is the GeoTIFF affine transform, $(x_{i, t}, y_{i, t})$ are vertex coordinates in the image, and $g_{i, t}$ is the resulting map coordinate.

The export pipeline reads the affine transform and CRS using Rasterio/GDAL and, where necessary, transforms coordinates to a target CRS such as EPSG:4326 using pyproj/PROJ. The polygons are then exported as GeoJSON together with attributes such as confidence scores and quality metrics.

Viewing Inference Results in QGIS

Inference results: three examples of exported GeoJSON building polygons overlaid in QGIS

Figure 2. Final P2 polygons for test images, transformed into geographic coordinates and loaded into QGIS as GeoJSON. Red outlines show the exported building polygons. Source: Figure 3 of the original paper. Basemap © OpenStreetMap contributors.

The exported red polygons can be seen overlaid on the building outlines in the imagery. These test-image examples demonstrate the connection to GIS and are separate from the quantitative evaluation on the validation data in the preceding section.

Here, “GIS-ready” means that the output can be delivered to GIS as vector data with geographic coordinates and checked geometric validity. It does not guarantee survey-grade accuracy or map updates without visual inspection.

11. Current Limitations and Future Development

The current implementation has the following limitations:

Issue	Current limitation or future direction
Dependence on instance segmentation	Buildings missed by the preceding stage cannot be extracted. A more robust preceding model is needed for different regions and sensors.
Holes, courtyards, and multiple exterior rings	The representation assumes one exterior cycle. Explicit support for multiple cycles and holes is needed.
Maximum edge count	Fixed capacity and simplification of ground-truth geometry limit the representation of complex buildings.
Quality-check thresholds	The thresholds are manually designed and need adaptation to different resolutions and building forms.
Precision and recall	Duplicate suppression may also remove valid predictions.
Evaluation scope	In addition to aerial imagery, evaluation on satellite imagery, other regions, and other resolutions is needed.

The paper identifies a learned polygon-quality prediction head, differentiable feedback from geometric construction to edge prediction, and support for multiple cycles and holes as future directions.

The Adapter Neck is also designed as an interface that converts intermediate DINOv3 features into a common downstream representation. This is intended to make the downstream EdgeQuery architecture easier to reuse when adopting future vision foundation models. An actual replacement would still require feature integration, training, and evaluation.

The presentation also introduced integration with Spatial Arrow as future work: generating model inputs at runtime from georeferenced raster and vector data, then connecting them to DINO-EdgeQuery's vector outputs. The evaluation reported here does not demonstrate that integration.

DINO-EdgeQuery aims to turn building recognition results into structured edges and inspectable geometry. By learning edge positions, directions, and successor relations and constructing exterior boundaries through geometric processing, it connects image recognition with practical use in GIS.

Paper and Presentation

Kobayashi, Y. and Lai, Y. C. (2026): DINO-EdgeQuery: Edge-First Polygon Decoding for Building Footprint Extraction from Satellite Imagery. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, L-4/W1-2026, 119–126. Paper / DOI
FOSS4G Hiroshima 2026 presentation (Speaker Deck)

The equations reproduced in this article are the relevant portions of Equations (2), (4), (5), (7), and (9) in the paper. The quantitative results are from Tables 2 and 3. The GPU memory measurements, capacity settings, and explanation of the transition from joint optimization to staged training are based on the presentation slides.