A Practical GeoAI Pipeline for Building Footprint Extraction from NAIP Imagery

One common mistake in building-footprint work is to treat segmentation as the whole pipeline.

If the only thing you measure is “the model outputs a mask,” you can miss the parts that decide whether the result is actually useful downstream: how the imagery is tiled, how labels are aligned, how predictions are cleaned, and whether the output is still valid after vectorization. In geospatial workflows, the model is only one component. The dataset preparation and post-processing often matter just as much.

This tutorial-style workflow is a good example of that broader view. It uses high-resolution NAIP aerial imagery and building labels to build an end-to-end GeoAI pipeline for footprint extraction with a U-Net model, then extends the workflow with zero-shot segmentation and a pretrained Mask R-CNN baseline. The point is not just to produce a mask, but to turn aerial pixels into georeferenced building polygons you can actually use.

What the workflow covers

The pipeline starts with a geospatial deep learning environment and a small set of configuration values that control tiling, model training, inference, and optional experiments. From there, it walks through the following stages:

• download NAIP raster imagery and vector building labels
• inspect raster and vector properties
• generate georeferenced image chips and matching masks
• train a U-Net with a ResNet-34 encoder
• run sliding-window inference on unseen imagery
• convert predicted masks into cleaned building polygons
• compute IoU and F1 metrics
• compare against Grounding DINO + SAM
• compare against a pretrained Mask R-CNN instance segmentation model

The workflow also shows how the same steps can be applied to a real area of interest using NAIP imagery from Microsoft Planetary Computer and building labels from Overture Maps.

Why the tiling step is not optional

A lot of beginners want to jump straight from “download imagery” to “train model.” For GeoAI, that usually leads to brittle results.

The source workflow first exports overlapping image chips from the training raster and builds corresponding label masks from vector footprints. That step is important for several reasons:

1. Memory management: NAIP scenes are large, and full-resolution training is usually impractical.
2. Geospatial consistency: the chips and masks must stay aligned in the same coordinate space.
3. Better training samples: overlapping tiles increase the chance that buildings near tile edges are still represented.
4. Balanced supervision: you can inspect how many tiles actually contain features instead of blindly assuming the dataset is usable.

The tutorial uses a 512-pixel tile size with 256-pixel overlap, which is a reasonable starting point for building segmentation on aerial imagery. That overlap also helps reduce edge artifacts during inference later.

Training a semantic segmentation model

The core model in the workflow is a U-Net with a ResNet-34 encoder pretrained on ImageNet. This is a practical choice for a first pass because it gives you a strong backbone without requiring an extremely large dataset.

The training setup includes:

• batch training on image-mask chips
• a validation split
• early stopping
• checkpointing the best model
• tracking the training history for later analysis

That last point is worth keeping: if you do not inspect the curves, you are guessing. The workflow explicitly loads the saved history and checks the epoch with the best validation IoU. It also calls out the two most common failure modes:

• training loss drops while validation loss rises → overfitting
• both stay flat and high → underfitting

That is a small detail, but it is the difference between “the notebook ran” and “the model is actually learning.”

Inference on a full scene

Training on chips is not the same as predicting on a complete raster. For unseen imagery, the workflow uses sliding-window inference with overlap. This is the right pattern when the source image is larger than what your model can accept in one pass.

The output includes:

• a predicted mask raster
• a probability raster
• a side-by-side visualization comparing the original scene and the predicted mask

This matters in practice because full-scene inference often reveals problems that look fine during training. Common examples include broken building edges, merged roofs, or missed small structures. Sliding-window inference plus overlap reduces those issues, though it does not eliminate them entirely.

From mask to building polygons

A mask is useful, but most geospatial applications need vectors.

After inference, the workflow performs several cleanup steps:

1. remove small noisy regions
2. convert the mask to polygons
3. orthogonalize the shapes
4. regularize the boundaries
5. add geometric properties such as area, perimeter, solidity, elongation, and orientation

This sequence is important because raw polygonization tends to produce jagged, over-detailed outlines. Regularization makes the output easier to interpret and more suitable for mapping or downstream GIS use.

There is a tradeoff here: every cleanup step simplifies the geometry a little, which can improve usability but may also smooth away fine architectural details. If your goal is cadastral-grade precision, you would be more conservative. If your goal is approximate footprint extraction at scale, regularization is usually the better default.

Measuring quality the right way

The workflow evaluates predictions against rasterized ground-truth labels using pixel-wise IoU and F1. That is the right first metric set for segmentation, but there is an important interpretation detail:

building-class IoU is the metric that matters most.

Background IoU can look artificially good because the negative class dominates the image. If you only glance at overall accuracy or background performance, you will overestimate the model.

The tutorial also visualizes imagery, predictions, and labels together, which is still one of the fastest ways to catch systematic errors that scalar metrics do not show.

Zero-shot and pretrained alternatives

A strong implementation choice in the tutorial is that it does not pretend U-Net is the only answer.

It also shows two alternatives:

Grounding DINO + SAM

This path uses text prompts like “building,” “house,” or “rooftop” to do zero-shot segmentation without additional task-specific training. That can be useful for fast exploration or low-data situations.

The tradeoff is predictability. Prompt-based segmentation can be surprisingly effective, but it is usually less controllable than a supervised model trained on your domain data.

Pretrained Mask R-CNN

The workflow also uses a pretrained building instance segmentation model. This is helpful when you want individual objects instead of semantic masks.

That difference matters:

• U-Net can merge adjacent roofs into one connected building region
• Mask R-CNN is better aligned with instance-level outputs, but may split or miss objects depending on confidence and thresholding

So the right model depends on the downstream question. If you need a continuous building layer for mapping or area estimation, semantic segmentation plus vectorization may be enough. If you need object counts or per-building instances, Mask R-CNN may be the better fit.

Extending the pipeline to real data

The final practical lesson is that the same workflow can be reused on real-world areas of interest. The source tutorial shows how to pull NAIP imagery from Microsoft Planetary Computer and match it with Overture Maps building labels, then feed that data into the same chip-generation and training process.

That makes the pipeline portable. The key is not the specific sample dataset; it is the structure:

• inspect inputs
• tile them
• train a model
• infer on a larger scene
• convert outputs into GIS-ready vectors
• compare against alternative approaches

Takeaway

If you are building GeoAI tooling, the biggest mistake is focusing only on the model architecture. For building footprint extraction, the real work is the workflow around it: geospatial alignment, tiling strategy, sliding-window inference, polygon cleanup, and metric interpretation.

This NAIP-based pipeline is a solid template because it connects those pieces end to end. It gives you a supervised baseline with U-Net, a prompt-based alternative with Grounding DINO + SAM, and an instance-segmentation comparison with Mask R-CNN. More importantly, it shows how to turn raw aerial imagery into structured building footprints that are ready for analysis.

For builders, that is the real goal: not just masks, but usable geospatial outputs.