Client Profile
Our client is a U.S.-based startup specializing in automated roof measurement for the insurance industry. Their core business involves providing insurers with precise roof dimensions, structural layouts, and damage assessments based on drone imagery. To improve accuracy and reduce manual effort, they needed a custom software solution that could automatically reconstruct roofs in 3D, extract relevant measurements, and generate clean 2D plans suitable for underwriting and claims.
Challenge
1) Over-detailed 3D Mesh from NodeODM
After photo-based reconstruction with NodeODM, the resulting mesh was extremely dense, with thousands of tiny triangles—even for flat roof areas. This over-fragmentation caused performance bottlenecks and made segmentation significantly harder, as trees and leaves had similar polygon density.
2) Loss of Geometric Integrity During Decimation
To simplify the mesh, decimation was applied. However, some algorithms degraded geometric quality—rounding sharp corners and distorting roof planes ("melting" edges into organic shapes). Choosing the right algorithm that preserved structure while reducing complexity required trial, testing, and compromise.
3) Low Accuracy of Heuristic Segmentation
Initial roof detection relied on heuristic filters (e.g., surface orientation and flatness). These methods struggled with real-world variation: roofs were missed entirely, or vegetation and terrain were falsely identified as part of the roof. Wall surfaces were sometimes incorrectly retained.
4) Unreliable Ground Plane Detection
Some models were generated with incorrect orientation—e.g., buildings rotated sideways due to poor camera alignment or metadata issues. The system misidentified vertical surfaces as horizontal, breaking the assumption that the ground plane is the largest horizontal surface.
5) Limitations of Neural Network Alone
To improve precision, a fine-tuned MeshCNN model was added after heuristics. Since the network relies only on geometric features (angles, curvature, connectivity), it helped reduce false positives but sometimes excluded valid roof segments in visually ambiguous cases where geometry alone was not enough.
6) Roof Color Blending with Environment
A major challenge appeared when roof surfaces visually blended with their surroundings — for example, green roofs or moss-covered areas next to trees. Since MeshCNN relies only on geometry, such zones were often left out entirely. Additional color analysis had to be introduced to re-include these segments based on median similarity to validated roof areas.
7) Imprecise Geometry in Final Roof Layout
Even after segmentation, the raw roof layout was often visually "off": lines were skewed, corners slightly misaligned, and shapes deviated from architectural norms. These imperfections, though minor, were problematic in professional insurance outputs and required post-processing corrections.
8) Complexity in 2D Plan Generation
The final requirement was to generate several distinct 2D plan types (e.g., area, length, slope, joint type). This required automatic annotation logic and formatting, adapted to downstream use in insurance documentation—while preserving clarity, alignment, and accuracy.
Solution
3D Model Reconstruction
We used NodeODM to reconstruct a high-resolution 3D mesh from drone photos taken in a circular flight path around each building. The output preserved fine-grained surface detail and spatial accuracy, serving as a base for identifying roof elements like planes, ridges, slopes, and joints, as well as distinguishing them from surrounding objects such as trees and walls.
Mesh Simplification
We simplified the over-detailed 3D mesh using carefully selected decimation algorithms that reduced triangle count while preserving roof geometry. This made the model lighter and easier to process without distorting key architectural features.
Roof Candidate Identification (Heuristics)
We used heuristic methods based on normal vector orientation and surface angles to identify potential roof areas. Large flat surfaces were classified as ground, steep vertical planes as walls, and only moderately sloped surfaces within defined angle ranges were retained as roof candidates. This filtering step significantly reduced irrelevant geometry and focused the pipeline on plausible roof segments before neural network analysis.
Neural Network-Based Refinement
To enhance segmentation accuracy, we used a fine-tuned MeshCNN model to classify mesh segments as roof or non-roof. The network was trained on a curated dataset containing diverse roof types and edge cases, allowing it to correct errors from the heuristic stage, such as misclassifying trees, terrain, or architectural noise.
Color-Based Segment Recovery
To improve detection accuracy, we added a color-based refinement step after neural classification. It analyzed unclassified segments by comparing their color histograms to confirmed roof areas. If a segment’s color closely matched the known roof surfaces, it was reclassified as part of the roof. This helped recover areas missed by the neural net, especially in visually complex or low-contrast environments.
Roof Scheme Assembly and Optimization
Classified roof segments were assembled into a structured 3D diagram representing the building’s geometry. A beautification step aligned edges, corrected near-90° angles, and removed minor distortions, ensuring a clean and architecturally accurate model ready for measurement and 2D plan generation.
2D Plan Generation
Using the cleaned 3D model, we automatically produced a set of 2D roof plans tailored for insurance analysis. Each plan highlighted different structural details — including surface areas, segment lengths, roof slopes, and joint classifications. The outputs were formatted as layered PDFs and CAD-compatible files
Features
1. Automated 3D Roof Modeling from Drone Footage
Upload drone imagery from a circular or grid flight path, and the system reconstructs a high-resolution 3D mesh of the entire building envelope, including complex roof geometry, without any manual labeling or post-processing.
2. Accurate Measurements Without On-Site Visits
Once the 3D model is generated, the system automatically extracts surface area, edge lengths, pitch angles, and structural joints. Results meet documentation standards for underwriting and claims — with no need for field visits or hand measurements.
3. AI-Enhanced Segmentation
After the 3D model is built, an AI module classifies each surface to isolate true roof segments. It handles visual ambiguity like shadows, vegetation overlap, or color blending (e.g., mossy roofs vs. trees), combining neural network predictions with color-based refinement to ensure accurate segmentation.
4. Customizable Roof Plan Reports
Generate 2D plan views directly from the cleaned 3D model. Each report highlights key attributes, such as surface area per segment, edge lengths, slope angles, and structural joints, formatted as layered PDFs or CAD files to match different insurance workflows.
5. Architectural Output Clean-Up
The system automatically snaps angles (e.g. 88° → 90°), straightens edges, and aligns segments to create clean, architecturally accurate diagrams — ideal for technical review and insurance documentation.
6. Multi-Roof and Batch Processing Support
Upload multiple properties at once and generate reports in parallel. Suitable for insurers handling portfolios, regional risk assessments, or post-disaster claims at scale.
Development Process
Drone Photo Acquisition
The input consisted of geotagged images captured by drones flying in circular or lawnmower (grid) patterns around each building. These flights ensured overlapping coverage from multiple angles, providing sufficient parallax for 3D reconstruction.
3D Mesh Reconstruction with NodeODM
We used NodeODM to convert photo sets from drone flyovers into dense 3D surface meshes. The pipeline included feature matching across overlapping images, camera pose estimation, point cloud generation, and surface reconstruction. The output was a detailed mesh with high geometric fidelity, accurately capturing roof contours, slopes, ridges, and surrounding elements.
Output: Extremely dense triangle mesh, especially in high-texture areas (e.g., roof shingles, vegetation), which preserved fine detail but required further simplification for downstream processing.
Mesh Simplification (Decimation)
After reconstruction, the raw mesh contained excessive polygon detail, especially on textured surfaces like shingles and foliage. To reduce computational load and prepare the mesh for segmentation, we implemented a decimation pipeline using Open3D and custom routines.
We tested and benchmarked multiple simplification algorithms — including quadric error, edge collapse, and custom planar clustering — to find the best balance between reduction and structural fidelity. Key criteria included:
- Maintaining planar roof surfaces
- Preserving straight ridges and 90° corners
- Filtering out high-frequency noise (e.g., vegetation)
The resulting mesh had ~10–15× fewer triangles, significantly speeding up processing while retaining all critical architectural geometry.
Initial Roof Candidate Filtering (Heuristics)
We analyzed triangle angles in the 3D mesh to estimate surface orientation. Flat areas (0°–10°) were marked as ground, vertical ones (80°–90°) as walls, and sloped surfaces (10°–60°) were kept as roof candidates. This step removed irrelevant geometry and focused processing on plausible roof zones.
Neural Network-Based Refinement (MeshCNN)
Next, we applied a custom MeshCNN model to classify the pre-filtered mesh into roof and non-roof segments. Trained on labeled 3D meshes, including dormers, green roofs, and surrounding clutter, the network used geometry features like connectivity and curvature to reduce false positives and improve segmentation accuracy.
Color-Based Recovery
To recover valid roof areas mistakenly excluded by the neural network, we introduced a color histogram matching module. It compared unclassified segments to confirmed roof surfaces and re-included those with similar visual profiles. This step was particularly effective for roofs with distinct colors (e.g., red or blue tiles), where color contrast improved detection. For green or mossy roofs blending with vegetation, we relied more on the neural network’s geometric analysis.
3D Roof Scheme Assembly
After finalizing the segmentation, we aggregated the verified roof triangles into structured components, aligning them into a unified, watertight 3D model. This model served as a clean architectural base, with clearly defined ridges, slopes, and junctions — optimized for plan generation and further editing.
Model Optimization (Beautification)
To prepare the model for documentation and inspection, we applied geometric corrections that cleaned up minor distortions from the reconstruction and segmentation steps. This included snapping corners close to 90°, aligning nearly parallel edges, and transforming irregular shapes (e.g., trapezoids) into architecturally accurate rectangles. The result was a more readable and technically precise roof structure.
2D Plan Generation
From the optimized 3D roof model, we generated multiple types of annotated 2D plans tailored to insurance and engineering needs. These included surface area diagrams, edge length measurements, slope and pitch visualizations, and joint type labels. Outputs were exported in both PDF and CAD-compatible formats, based on target layout standards from industry benchmarks.
Impact
- 80% Reduction in Processing Time
Automated mesh optimization and segmentation reduced roof processing from hours to minutes per property.
- 99% Measurement Accuracy Achieved
Final outputs matched field measurements within industry tolerance, meeting insurance documentation standards.
- 60% Drop in Manual QA Effort
Clean segmentation and geometric correction minimized the need for manual editing or post-cleanup before generating reports.
- Scalable Portfolio Analysis
Enabled batch processing of entire property portfolios, supporting regional claim triage and underwriting assessments.
Top comments (0)