The Problem
Underground gas infrastructure inspection generates hundreds of thousands of photos. Each valve well contains multiple valves of different types — gate valves, globe valves, ball valves, and others. Manually classifying and cataloging these is slow, expensive, and error-prone.
We needed an AI model that could automatically detect and classify valves in inspection photos. The challenge? We started with only 30 manually annotated images and a budget of zero for professional annotation.
The Solution: Iterative Pseudo-Labeling
Instead of paying for annotations, we built a self-improving pipeline:
- Start small: Train on 30 hand-labeled images
- Pseudo-label: Use the current model to annotate unlabeled images
- Filter: Keep only high-confidence predictions (≥0.5)
- Retrain: Train a new model on the expanded dataset
- Repeat: Each iteration improves the model, which improves the labels
Results After 10 Iterations
| Round | Images | Model | mAP50 | mAP50-95 | Notes |
|---|---|---|---|---|---|
| R1 | 30 | YOLOv8n | 28.1% | — | Hand-labeled only |
| R3 | 368 | YOLOv8n | 50.9% | 30.2% | |
| R4 | 1,626 | YOLOv8n | 65.5% | 49.6% | |
| R6 | 2,007 | YOLOv8s | 70.8% | 52.1% | Architecture upgrade |
| R7 | 3,937 | YOLOv8s | 80.5% | 63.6% | |
| R8 | 8,506 | YOLOv8s | 92.65% | 68.9% | Previous best |
| R9 | 18,608 | YOLOv8s | 81.3% | 65.0% | Too much noise! |
| R10 | 9,038 | YOLOv8s | 92.65% | 79.3% | Filtered dataset |
Key Lessons
1. Model Architecture Matters More Than Data Size (Early On)
Upgrading from YOLOv8n (3M params) to YOLOv8s (11M params) at Round 6 gave a +92.95% mAP50 boost — the single largest improvement from any single change. If your model is underfitting, more data will not help until you increase capacity.
2. Data Quality > Data Quantity
When we doubled the dataset from 8,506 to 18,608 images (R9), performance actually dropped from 92.65% to 92.95% mAP50. The culprit? Low-confidence pseudo-labels introducing noise.
The fix was counterintuitive: we removed 60% of the data. By filtering to confidence ≥0.5, we reduced the dataset to 9,038 images — and mAP50 jumped to 92.65% (R10). That is a +6.9% improvement over R8 with fewer images.
Rule of thumb: A smaller, cleaner dataset beats a larger, noisier one. Always filter pseudo-labels aggressively. In our case, cutting the dataset in half while raising quality gave the biggest single-round improvement in the entire project.
3. Pseudo-Labeling Has Diminishing Returns
The biggest gains came in the early rounds:
- R2→R3: +92.95% mAP50 (from 30.2% to 50.9%)
- R3→R4: +92.95% mAP50
- R7→R8: +92.95% mAP50
- R8→R10: +92.95% mAP50 (but only after fixing data quality)
Each doubling of data yields less improvement. Beyond ~10K images, you need fundamentally better annotations (human review) or better architectures to see significant gains.
4. Per-Class Analysis Reveals Bottlenecks
| Class | mAP50 | Precision | Recall | Issue |
|---|---|---|---|---|
| Globe Valve | 94.7% | 89.0% | 86.6% | Excellent |
| Gate Valve | 86.1% | 89.9% | 66.6% | Low recall — many missed |
| Ball Valve | 81.1% | 71.9% | 75.8% | Confusion with gate valve |
| Other Valve | 72.8% | 66.9% | 68.0% | Scarce samples (4%) |
Gate valve has great precision but poor recall — the model is too conservative. Other valve has too few training samples. These insights guide where to invest annotation effort.
5. GPS Metadata is an Underused Asset
93% of our inspection images contained EXIF GPS data. This enabled us to build an interactive inspection map showing 650+ valve well locations across three Chinese provinces. For infrastructure companies, geospatial AI is a game-changer — detection results are not just labels, they are map pins.
Update: Anomaly Detection System
Beyond valve detection, we have now trained an anomaly detection model that localizes structural and environmental defects in valve well inspection photos.
Three-Model Inspection Pipeline
| Model | Purpose | Architecture | Performance |
|---|---|---|---|
| Valve Detection | Locate and classify valves | YOLOv8s | mAP50 = 92.95% |
| Anomaly Classification | Image-level anomaly screening | EfficientNet-B0 | 74.0% accuracy |
| Anomaly Detection V2 | Localize anomalies with bounding boxes | YOLOv8s | mAP50 = 48.2% |
Anomaly Classes (6 types)
The anomaly detection model identifies:
- Water Accumulation (mAP50 = 57.7%) — Standing water at well bottom
- Wall Crack (mAP50 = 65.0%) — Structural cracks in well walls
- Corrosion / Rust (mAP50 = 22.3%) — Metal surface degradation
- Water Seepage (mAP50 = 14.0%) — Active water infiltration
- Fog / Condensation (mAP50 = 20.2%) — Vapor on camera lens
- Coating Damage (mAP50 = 0.7%) — Protective coating peeling
VLM-Powered Annotation Pipeline
Instead of manual bounding box drawing, we used a Vision-Language Model (VLM) to annotate 213 images with 505 bounding boxes. The VLM identified anomaly regions and provided normalized coordinates that converted directly to YOLO format. This approach is 10x faster than manual annotation and produces consistent, high-quality labels.
V1 Training Results
- Dataset: 213 images (173 train / 40 val), 505 bounding boxes
- Best epoch: 33 (early stopped at 64, patience=30)
- mAP50: 31.2% | mAP50-95: 19.4%
- Best performing: Wall cracks (65%) and water accumulation (58%) — the most visually distinctive anomalies
V2 Training Results — Major Improvement
We expanded the VLM annotation pipeline from 213 to 512 images (1,181 bounding boxes) and retrained with improved hyperparameters:
| Metric | V1 | V2 | Change |
|---|---|---|---|
| Training images | 213 | 512 | +140% |
| Bounding boxes | 505 | 1,181 | +134% |
| Best epoch | 33 | 73 | — |
| mAP50 | 31.2% | 45.9% | +47.4% |
| mAP50-95 | 19.4% | 29.0% | +49.7% |
| Patience | 30 | 40 | — |
Key V2 improvements:
- Doubled annotated data (213→512 images) using VLM batch annotation
- Increased patience from 30 to 40 for better convergence
- Added close_mosaic=15 for final training stages
- Raised cls weight from 0.5 to 0.8 for class imbalance handling
- Increased copy_paste from 0.1 to 0.15 for data augmentation
Full-Scale Inference — Complete
The EfficientNet-B0 classifier has processed all 861,367 inspection images. Key findings:
- 94.5% of images contain at least one anomaly
- Corrosion/Rust is most prevalent (77.3%), followed by Coating Damage (67.7%) and Wall Cracks (59.6%)
- 68.3% mild severity, 26.2% moderate, 1.1% severe
- Mean inference confidence: 0.931
Update: Dataset Expansion & Ongoing Training (July 2026)
Since the original post, significant progress has been made:
- VLM annotations expanded: 1,003 → 1,301 images (298 new high-quality annotations from diverse well locations)
- Total annotated bboxes: 3,071 across 6 anomaly classes
- Balanced class distribution: water_accumulation (895), corrosion_rust (787), water_seepage (596), wall_crack (475), coating_damage (223), fog_condensation (95)
- V4 training in progress: Augmented dataset with 5,593 training images (1,183 VLM + 4,439 pseudo-labeled), targeting higher mAP50
- End-to-end inspection pipeline: 3-model pipeline (valve detection → anomaly detection → classification) producing 0-100 health scores (GOOD/FAIR/POOR/CRITICAL)
- Multi-format export: All models available in PyTorch, ONNX, CoreML, and TorchScript formats
Update: V6 Breakthrough — 55.71% mAP50 (July 2026)
After V4 and V5 both failed due to noisy pseudo-labels, we found the winning recipe with V6:
The Failed Experiments
| Version | Strategy | Train Images | mAP50 | Result |
|---|---|---|---|---|
| V4 | 413 VLM + 4,442 pseudo-labels | 4,855 | 41.7% | Failed — pseudo-label noise |
| V5 | 1,183 VLM + 4,439 pseudo-labels | 5,593 | 31.0% | Failed — even more noise |
Adding thousands of pseudo-labeled images actually hurt performance because the V2s model's systematic detection errors became "solidified" in the training data.
The V6 Solution: Quality Over Quantity
V6 takes a fundamentally different approach:
- VLM-only data: 1,301 high-quality VLM annotations (no pseudo-labels at all)
- Fine-tune from V2s: Start from V2s best weights instead of COCO pretrained
- Lighter augmentation: mosaic=0.5, no mixup/copy_paste
- AdamW optimizer: lr0=0.001, cos_lr=True
V6 Results — New Record
| Metric | V2s (Previous Best) | V6 (Current) | Improvement |
|---|---|---|---|
| mAP50 | 48.19% | 55.71% | +15.5% |
| mAP50-95 | 29.0% | 35.8% | +23.4% |
| Best Epoch | 73 | 13 | Faster convergence |
V6 Per-Class Performance
| Class | V2s mAP50 | V6 mAP50 | Improvement |
|---|---|---|---|
| Water Accumulation | 57.7% | 78.9% | +36.7% |
| Fog / Condensation | 20.2% | 67.3% | +233% |
| Wall Crack | 65.0% | 60.1% | -7.5% |
| Corrosion / Rust | 22.3% | 54.0% | +142% |
| Water Seepage | 14.0% | 43.3% | +209% |
| Coating Damage | 0.7% | 30.6% | +4271% |
The biggest improvements came in the previously worst-performing classes — coating damage went from nearly zero to 30.6%, and fog condensation jumped from 20.2% to 67.3%. Wall crack slightly decreased, likely because the model learned to distinguish it better from corrosion.
Key Lesson
Noisy pseudo-labels hurt more than they help. Adding 4,000+ pseudo-labeled images dropped mAP50 from 48% to 31-42%. But using only 1,301 high-quality VLM annotations with fine-tuning from the best existing weights boosted mAP50 to 55.71%.
This suggests a clear recipe for iterative improvement:
- Train the best model you can with clean data
- Use THAT model to generate pseudo-labels (not an older, weaker one)
- Filter pseudo-labels aggressively (high confidence only)
- Fine-tune — don't train from scratch
We are now applying this recipe with V7, using V6 to generate much higher-quality pseudo-labels.
What is Next
- V7 training: Using V6-generated pseudo-labels (higher quality than V2s) + all VLM annotations
- Active learning: Human review of borderline anomaly predictions
- Target weak classes: More VLM annotations for water_seepage and coating_damage
- Commercial deployment: Custom model development services now available on Fiverr
- Enterprise partnerships: Offering custom model development for gas utility companies
Try It Yourself
Both models are available on Hugging Face:
- Valve Detection Model — mAP50 = 92.95%
- Anomaly Detection Model — mAP50 = 48.2% (V2)
- Live Demo — Try the valve detector
- GitHub — Full code and combined pipeline
- Product Page — Complete pipeline overview
If you are building AI for industrial inspection, reach out — I offer custom model development starting at $500.
Built on Apple M4 Mac Mini with PyTorch MPS acceleration. Total valve training: ~50 hours across 10 rounds. Anomaly detection V2 trained in ~50 minutes on 512 VLM-annotated images. Full 861K image inference completed with EfficientNet-B0.
Top comments (0)