DINO-EdgeQuery: Learning Building Edges and Their Connections for GIS-Ready Polygons

Introduction

When extracting buildings from satellite or aerial imagery, predicting building regions through segmentation still leaves another task: converting those regions into building polygons that are practical to use in GIS.

Directly vectorizing mask contours can produce jagged boundaries and unnecessary vertices. Aggressive simplification, however, can remove important corners and small geometric details. Recognizing building regions accurately and producing geometry suitable for editing and inspection are two problems that need to be addressed separately.

To address this, we developed DINO-EdgeQuery, which learns the edges that form a building and the connections between them.

The model predicts candidate edges from the image features of each building and learns which edge follows which. The final vertices are computed as intersections of the supporting lines of the ordered edges.

This article introduces the architecture, training procedure, and evaluation results based on our paper and presentation at the Academic Track of FOSS4G Hiroshima 2026. The paper is available in the ISPRS Archives.

Although the intended application is building extraction from high-resolution satellite imagery, the training and quantitative evaluation reported here use aerial imagery from the WHU Building Dataset. These results do not demonstrate equivalent performance on satellite imagery.

This article provides a concise overview of the paper accepted for the FOSS4G Hiroshima 2026 Academic Track and the presentation delivered at the conference.

1. Why Predict Edges?

Mask Accuracy and Polygon Quality

A common building extraction workflow consists of building mask prediction, contour tracing, and polygon simplification. This approach can combine existing segmentation models with GIS processing, but errors in mask boundaries affect the final geometry.

Building polygons used in GIS need more than good region overlap. They should also meet requirements such as:

• Representing straight boundaries with few vertices
• Placing building corners appropriately
• Avoiding self-intersections and degenerate geometry
• Avoiding duplicate outputs for the same building
• Carrying geographic coordinates so they can be inspected and edited in GIS

Many building outlines consist of long straight segments and a relatively small number of corners. DINO-EdgeQuery reflects this property by using edges as the primary prediction unit.

Positioning Among Existing Approaches

The related work discussed in the paper can be organized by its primary representation:

Approach	Main representation or idea
Mask contour tracing and simplification	Construct polygons from predicted raster boundaries
Frame Field Learning	Learn a boundary orientation field alongside masks to assist polygonization
HiSup	Use hierarchical supervision of vertices, line segments, and regions to learn masks suited to polygonization
PolyWorld / Re:PolyWorld	Represent vertices and their connections as a graph
BoundaryFormer	Predict object boundaries as sequences of polygon points
DINO-EdgeQuery	Predict edges and their successor relations, then construct vertices from supporting-line intersections

This table compares representations. The experiments presented in this article are not a performance comparison against these methods under identical conditions.

2. Overall Architecture: From Image Recognition to Geometry

DINO-EdgeQuery combines neural network predictions with deterministic polygon construction.

flowchart TD
    I[Input image] --> F[DINOv3 and Adapter Neck]
    F --> D[Instance Decoder]
    F --> R[ROI Tokenizer]
    D -->|Predicted masks and boxes| R
    R --> E[EdgeQuery Decoder]
    E --> H[Edge attributes and successor relations]
    H --> G[Cycle selection and line intersections]
    D -->|Predicted masks| P[Quality checks and duplicate suppression]
    G --> P
    P --> O[Coordinate transformation and GeoJSON export]

The responsibilities are divided as follows:

Component	Method	Role
DINOv3	Neural network with frozen pretrained weights	Extract image features
Adapter Neck and Instance Decoder	Neural networks	Predict masks, boxes, and other information for each building
ROI Tokenizer, EdgeQuery, and prediction heads	Neural networks	Predict building edges and successor relations
Cycle selection and line intersections	Deterministic geometric processing	Order edges and construct vertices
Quality Gate P1 / P2	Deterministic checks and filtering	Remove invalid geometry and duplicate candidates
Coordinate transformation and export	Geospatial processing	Connect the output to GIS through GeoJSON

Both edge prediction and successor relation prediction in EdgeQuery are neural network operations. Geometric algorithms are used for polygon construction and quality checks after these predictions have been produced.

3. DINOv3 and Instance Segmentation Provide Building-Level Cues

DINOv3 and the Adapter Neck

We use DINOv3 with frozen weights to extract features from the input image. The Adapter Neck converts its intermediate features into multiscale features for the downstream components.

The final implementation primarily uses feature maps with strides of 8, 16, and 32 relative to the input. The paper denotes these as $F_8$ , $F_{16}$ , and $F_{32}$ .

To supplement boundary information, features corresponding to F4 are extracted from the RGB image by a shallow CNN and added to the F8 path as a lightweight residual. The operation corresponding to Equation (2) in the paper is:

$$F_8' = F_8 + \phi_{F_4\rightarrow F_8}\bigl(\phi_{\mathrm{stem}}(I)\bigr)$$

The F4-equivalent features are not used directly as a separate mask input or ROI input. The downstream components use the corrected $F_8'$ together with $F_{16}$ and $F_{32}$ .

Masks Provide Cues About Building Location and Extent

The Mask2Former-style Instance Decoder predicts a class, box, mask, and query embedding for each building candidate.

These masks provide visual priors for identifying the target building and determining which feature region to access. They are also used to check the consistency of the generated polygons.

The final polygon boundary is constructed from the edges predicted by EdgeQuery, rather than adopting the mask contour directly. However, EdgeQuery depends on instance segmentation and cannot recover buildings missed by the preceding stage.

4. EdgeQuery: Predicting Candidate Edges for Each Building

ROI Tokenizer

ROI stands for Region of Interest. The ROI Tokenizer uses each building's mask and box to generate local tokens from the corresponding multiscale features.

In the final configuration presented in the slides, the ROI size is $24\times24$ and the feature dimension is 256. EdgeQuery predicts edges using this building-specific context.

A Query Is a Slot for a Candidate Edge

EdgeQuery here does not mean asking an image questions in natural language. It refers to query slots used to predict the edges forming a building boundary.

The final configuration uses $Q=96$ building candidates per tile and $E=48$ edge slots per building candidate. This does not mean that every building has 48 edges. The edges used in the output are selected based on activity, successor relations, and other information.

The edge predictions include the following:

Information	Meaning
Activity	A score used to determine whether an edge slot is active
Center	The center position of the edge
Direction	The direction of the edge
Half-length	Half the edge length; the full segment length is twice this value
Mode and dominant axis	Information about the edge mode and the building's dominant axis
Relation embedding	An embedding used to predict successor relations between edges

The outputs provided by the prediction heads must be distinguished from the loss terms active in the final training configuration. In the final StageG2, the directly supervised quantities are activity, center, direction, half-length, and successor relations, as described below.

Successor Relations: Learning Which Edge Comes Next

Even after candidate edges have been predicted, the set alone does not specify the order in which to trace the exterior boundary. The Successor Relation Head predicts directed relations indicating which edge follows a given edge.

Let $S_q$ be the successor score matrix for building candidate $q$ . With $E$ edge slots, this represents $E\times E$ relations. For a four-edge building, for example, the selected edges might form a closed sequence such as 1→3→2→4→1.

The neural network predicts edge attributes and successor scores. Selecting the actual cycle from these outputs is a subsequent deterministic operation.

5. Constructing Vertices from Supporting-Line Intersections

A supporting line is the infinite line obtained by extending a predicted segment in both directions. We define each supporting line from the edge center and direction, then use the intersections of adjacent supporting lines along the cycle as vertices.

The following is Equation (4) from the paper, where $q$ denotes a building candidate and $k$ an edge candidate:

ParseError: KaTeX parse error: Expected '}', got '\right' at position 288: …\hat{\rho}{q,k}\̲r̲i̲g̲h̲t̲},\ \hat{\mathb…

$\hat{\mathbf c}$ is the predicted center and $\hat{\mathbf d}$ the predicted direction. $R_{\pi/2}$ rotates the direction by 90 degrees to obtain the line normal $\hat{\mathbf n}$ .

By arranging the supporting lines in the order of the selected cycle $C_q=(k_1,\ldots,k_V)$ , corners are determined by intersections of adjacent lines. This explicitly represents the long straight boundaries and corners of a building.

Computing intersections alone does not always produce the correct building geometry. Errors in edge prediction or cycle selection can produce unnatural outlines or invalid polygons, so the following quality checks are applied.

6. Quality Gate P1 / P2: Checking Geometry and Duplicates

P1: Checking Geometry and Consistency with the Predicted Mask

P1 checks the polygon constructed from line intersections using its IoU with the predicted mask, coverage, outside ratio, area ratio, and geometric validity. Self-intersecting and empty polygons are also excluded.

The main thresholds reported in the paper are a mask IoU of at least 0.15, coverage of at least 0.35, and an outside ratio of at most 0.65. These are quality-check thresholds used during inference, separate from the IoU thresholds for evaluation against ground-truth polygons described later.

P2: Suppressing Duplicate Polygons Based on Containment

P2 adds containment-based duplicate suppression to P1.

With conventional IoU-based NMS, a small polygon inside a large polygon may remain because a substantial difference in area results in a low IoU.

For example, if a polygon of area 10 lies entirely inside a polygon of area 100, their IoU is 0.1. Yet 100% of the smaller polygon is covered. This numerical example illustrates the difference between containment coverage and IoU.

P2 removes $P_a$ when another polygon $P_b$ satisfies all of the following conditions, corresponding to Equation (5) in the paper:

$$\frac{|P_a\cap P_b|}{|P_a|}\geq0.50, \qquad |P_b|\geq|P_a|, \qquad \operatorname{centroid}(P_a)\in P_b.$$

In other words, at least 50% of $P_a$ is covered, the other polygon has an equal or greater area, and the centroid of $P_a$ lies inside it.

This check uses predictions only; it does not access ground-truth data. Geometric validity checks also do not guarantee agreement with actual buildings. False detections and missed buildings must be evaluated separately.

7. Two Training Stages, One Inference Pipeline

From Joint Optimization to Staged Training

The presentation also explained our initial plan to jointly optimize instance segmentation, edge prediction, and polygon construction.

In practice, changes in the initial masks and boxes made ROI conditions unstable. Mask and geometry losses converged at different rates, and gradients were difficult to handle through discrete cycle selection and validity checks. GPU memory needed to retain intermediate states for backpropagation through the full system was another constraint.

This memory issue concerns attempting to treat instance segmentation, EdgeQuery, and polygon construction as a single large training graph. It does not mean that the final staged training procedure or inference pipeline necessarily exhausts 96 GB of memory.

The final implementation trains in two stages and then combines their weights:

Item	StageG1	StageG2
Trainable components	Adapter Neck and Instance Decoder	ROI Tokenizer, EdgeQuery Decoder, and heads for edge attributes and successor relations
Main frozen components	DINOv3 and the EdgeQuery branch	DINOv3, Adapter Neck, and Instance Decoder
Training images	All training tiles, including tiles without buildings	Tiles containing buildings
Main supervision	Classes, boxes, masks, and boundaries	Edge activity, center, direction, half-length, and successor relations
ROI conditions	—	Ground-truth boxes and masks

P1 and P2 are polygon-output quality-check profiles, not additional neural network training stages following G1 and G2.

StageG1: Boundary-Aware Instance Segmentation

In addition to classification, box, and mask losses, we use a BCE loss on a thin band around the ground-truth boundary. This explicitly accounts for boundary errors that would otherwise contribute relatively little compared with the larger building interiors.

Equation (6) in the paper combines classification, box, GIoU, mask BCE, mask Dice, and boundary BCE losses using weighted sums.

StageG2: Learning Edges and Connections with Stable ROIs

StageG2 uses oracle ROIs defined by ground-truth masks and boxes. This stabilizes the local coordinates and target regions during edge learning.

The active losses in the final model are given by Equation (7) in the paper:

$$\begin{aligned} L_{\mathrm{EdgeQuery}}={}& \lambda_{\mathrm{act}}L_{\mathrm{act}} +\lambda_{\mathrm{center}}L_{\mathrm{center}} +\lambda_{\mathrm{dir}}L_{\mathrm{dir}}\ &+\lambda_{\mathrm{len}}L_{\mathrm{len}} +\lambda_{\mathrm{succ}}L_{\mathrm{succ}}. \end{aligned}$$

Losses related to vertices, rasterization, topology, regularity, and cycle consistency, which were included in the initial design, are not active in the final StageG2. Terms retained for compatibility or future extensions are masked to zero and excluded from the reported training objective.

After training, the weights of G1 and G2 are combined. Inference uses neither ground-truth boxes nor ground-truth masks: the entire pipeline operates only on information predicted from the input image.

8. Preparing the Dataset for an Edge Representation

We use aerial imagery and building annotations from the WHU Building Dataset. The 0.2 m resolution data in COCO format used in this study were prepared for EdgeQuery.

COCO polygons cannot be used directly as edge supervision. Their exterior ring orientation and edge counts need to be standardized.

1. Split each original 1024×1024 image into four non-overlapping 512×512 tiles.
2. Clip building polygons at tile boundaries and convert them to tile-local coordinates.
3. Remove tiny fragments and unstable geometries resulting from clipping, and repair invalid geometry where possible.
4. Remove duplicate vertices and standardize the exterior ring orientation in image coordinates.
5. Simplify exterior rings with more than 48 edges while preserving topology where possible. Exclude polygons that collapse during this process.
6. Generate edge attributes and ordered successor relations from the normalized exterior rings.

The current implementation handles one exterior cycle per instance. MultiPolygons are split into individual polygon instances, while holes and nested structures are removed, flattened, or otherwise processed to fit this representation.

This preprocessing is more than an image-format conversion. It also constrains the ground-truth geometry to match the edge count and exterior structure that the model can represent. Since the same preprocessing is applied to the training and validation data, the evaluation results must be interpreted under these conditions.

Final Capacity Configuration

The presentation reported the following configuration for stable operation:

Item	Setting or measured value
Input image	512×512
Microbatch size	8
Gradient accumulation	2 steps; effective batch size of 16
Building queries	$Q=96$ per tile
Edge slots	$E=48$ per building candidate
ROI	24×24, 256 channels
Peak VRAM	49,470 MiB, approximately 48.3 GiB

In particular, the successor relation matrix grows quadratically with the number of edge slots. Increasing the edge capacity therefore increases computation and memory requirements for this component. The memory value above was measured for the adopted configuration and does not specify the requirements for every possible input or setting.

9. Experimental Results: What Changes When P2 Reduces Duplicates?

Inference Configuration

The paper reports the following inference profile:

Parameter	Value
Instance confidence threshold	0.60
Mask binarization threshold	0.50
Mask-stage NMS	0.75
Polygon-stage NMS	0.75
Edge activity threshold	0.15
Edge candidates retained for polygon construction	Up to 32 per instance
Boundary support	Enabled
Collinear merging	Disabled

The 48 edge slots used during training and the maximum of 32 edge candidates retained during inference are separate settings.

Results on Validation Tiles Containing Buildings

The following values are from Table 2 in the paper. P1 and P2 are quality-check profiles applied to the outputs of the same model.

Metric	P1	P2
Predicted polygons	21,554	19,203
Union IoU	0.7107	0.7064
Boundary IoU, 2 px	0.3746	0.3710
Boundary IoU, 3 px	0.4535	0.4496
Recall@75	0.6459	0.6213
Precision@75	0.5486	0.5924
Recall@80	0.5442	0.5258
Precision@80	0.4622	0.5013
Predictions / GT	1.1774	1.0489
FP / GT	0.2997	0.2295
Self-intersections / collapses to triangles	0 / 0	0 / 0

Precision@75 and Recall@75 are precision and recall when predicted polygons are matched to ground-truth instances at an IoU threshold of 0.75. The @80 metrics use 0.80. These are not AP metrics. FP / GT is the ratio of false positives to the number of ground-truth instances.

With P2, the number of predicted polygons decreases by 2,351. Precision@75 improves by approximately 4.38 percentage points, and Precision@80 by approximately 3.91 percentage points. Meanwhile, Recall@75 decreases by approximately 2.46 percentage points, and Union IoU decreases by 0.0043.

These results show that P2 improves precision at high IoU thresholds and reduces excessive outputs, with a trade-off in recall. They should not be interpreted as a uniform improvement in IoU.

Both P1 and P2 produced zero self-intersections and zero collapses to triangles in this evaluation. These are results for the outputs after quality checks; they do not guarantee correct geometry or positioning for every input.

Validation Tiles Without Buildings

Table 3 in the paper reports results for 980 tiles without buildings:

Metric	P1	P2
Predicted polygons	90	81
Mean predictions per tile	0.0918	0.0827

The number of predictions also decreases on background-only tiles, but false detections are not eliminated. The value 0.0827 is the mean number of predictions per tile, not an image-level rate indicating that 8.27% of images contain false detections.

Outputs During Validation

Validation outputs: input image, ground-truth mask, predicted mask, candidate edges, selected cycle, and final polygon

Figure 1. DINO-EdgeQuery outputs for a validation image. Top row, left to right: input image, ground-truth mask, and predicted instance masks. Bottom row, left to right: unselected candidate edges, edges in the selected cycle, and final polygons constructed from line intersections. Source: Figure 2 of the original paper.

Figure 2 in the paper shows the input image, ground-truth mask, predicted instances, unselected EdgeQuery edges, selected cycle edges, and final polygons in sequence.

This visualization traces how masks identify the target regions for individual buildings, how predicted edge candidates are selected, and how their intersections form the final outlines. The ground-truth mask in the top-center panel is shown for comparison only and is not used as an inference input.

10. Exporting Image-Space Polygons to GIS as GeoJSON

Polygon vertices are initially obtained in image coordinates. To place them on a map, we transform them using the affine transform of the input GeoTIFF.

The vertex transformation corresponding to Equation (9) in the paper is:

$$\mathbf g_{i,t} =A\begin{bmatrix} x_{i,t}\cr y_{i,t}\cr 1 \end{bmatrix}$$

$A$ is the GeoTIFF affine transform, $(x_{i,t},y_{i,t})$ are vertex coordinates in the image, and $\mathbf g_{i,t}$ is the resulting map coordinate.

The export pipeline reads the affine transform and CRS using Rasterio/GDAL and, where necessary, transforms coordinates to a target CRS such as EPSG:4326 using pyproj/PROJ. The polygons are then exported as GeoJSON together with attributes such as confidence scores and quality metrics.

Viewing Inference Results in QGIS

Inference results: three examples of exported GeoJSON building polygons overlaid in QGIS

Figure 2. Final P2 polygons for test images, transformed into geographic coordinates and loaded into QGIS as GeoJSON. Red outlines show the exported building polygons. Source: Figure 3 of the original paper. Basemap © OpenStreetMap contributors.

The exported red polygons can be seen overlaid on the building outlines in the imagery. These test-image examples demonstrate the connection to GIS and are separate from the quantitative evaluation on the validation data in the preceding section.

Here, “GIS-ready” means that the output can be delivered to GIS as vector data with geographic coordinates and checked geometric validity. It does not guarantee survey-grade accuracy or map updates without visual inspection.

11. Current Limitations and Future Development

The current implementation has the following limitations:

Issue	Current limitation or future direction
Dependence on instance segmentation	Buildings missed by the preceding stage cannot be extracted. A more robust preceding model is needed for different regions and sensors.
Holes, courtyards, and multiple exterior rings	The representation assumes one exterior cycle. Explicit support for multiple cycles and holes is needed.
Maximum edge count	Fixed capacity and simplification of ground-truth geometry limit the representation of complex buildings.
Quality-check thresholds	The thresholds are manually designed and need adaptation to different resolutions and building forms.
Precision and recall	Duplicate suppression may also remove valid predictions.
Evaluation scope	In addition to aerial imagery, evaluation on satellite imagery, other regions, and other resolutions is needed.

The paper identifies a learned polygon-quality prediction head, differentiable feedback from geometric construction to edge prediction, and support for multiple cycles and holes as future directions.

The Adapter Neck is also designed as an interface that converts intermediate DINOv3 features into a common downstream representation. This is intended to make the downstream EdgeQuery architecture easier to reuse when adopting future vision foundation models. An actual replacement would still require feature integration, training, and evaluation.

The presentation also introduced integration with Spatial Arrow as future work: generating model inputs at runtime from georeferenced raster and vector data, then connecting them to DINO-EdgeQuery's vector outputs. The evaluation reported here does not demonstrate that integration.

DINO-EdgeQuery aims to turn building recognition results into structured edges and inspectable geometry. By learning edge positions, directions, and successor relations and constructing exterior boundaries through geometric processing, it connects image recognition with practical use in GIS.

Paper and Presentation

• Kobayashi, Y. and Lai, Y. C. (2026): DINO-EdgeQuery: Edge-First Polygon Decoding for Building Footprint Extraction from Satellite Imagery. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, L-4/W1-2026, 119–126. Paper / DOI
• FOSS4G Hiroshima 2026 presentation (Speaker Deck)

The equations reproduced in this article are the relevant portions of Equations (2), (4), (5), (7), and (9) in the paper. The quantitative results are from Tables 2 and 3. The GPU memory measurements, capacity settings, and explanation of the transition from joint optimization to staged training are based on the presentation slides.