How We Trained a 92.95% mAP50 Valve Detection Model + 65.7% Anomaly Detector with Iterative Pseudo-Labeling

jins zhu — Wed, 08 Jul 2026 03:56:29 +0000

The Problem

Underground gas infrastructure inspection generates hundreds of thousands of photos. Each valve well contains multiple valves of different types — gate valves, globe valves, ball valves, and others. Manually classifying and cataloging these is slow, expensive, and error-prone.

We needed an AI model that could automatically detect and classify valves in inspection photos. The challenge? We started with only 30 manually annotated images and a budget of zero for professional annotation.

The Solution: Iterative Pseudo-Labeling

Instead of paying for annotations, we built a self-improving pipeline:

Start small: Train on 30 hand-labeled images
Pseudo-label: Use the current model to annotate unlabeled images
Filter: Keep only high-confidence predictions (≥0.5)
Retrain: Train a new model on the expanded dataset
Repeat: Each iteration improves the model, which improves the labels

Results After 10 Iterations

Round	Images	Model	mAP50	mAP50-95	Notes
R1	30	YOLOv8n	28.1%	—	Hand-labeled only
R3	368	YOLOv8n	50.9%	30.2%
R4	1,626	YOLOv8n	65.5%	49.6%
R6	2,007	YOLOv8s	70.8%	52.1%	Architecture upgrade
R7	3,937	YOLOv8s	80.5%	63.6%
R8	8,506	YOLOv8s	92.65%	68.9%	Previous best
R9	18,608	YOLOv8s	81.3%	65.0%	Too much noise!
R10	9,038	YOLOv8s	92.65%	79.3%	Filtered dataset

Key Lessons

1. Model Architecture Matters More Than Data Size (Early On)

Upgrading from YOLOv8n (3M params) to YOLOv8s (11M params) at Round 6 gave a +92.95% mAP50 boost — the single largest improvement from any single change. If your model is underfitting, more data will not help until you increase capacity.

2. Data Quality > Data Quantity

When we doubled the dataset from 8,506 to 18,608 images (R9), performance actually dropped from 92.65% to 92.95% mAP50. The culprit? Low-confidence pseudo-labels introducing noise.

The fix was counterintuitive: we removed 60% of the data. By filtering to confidence ≥0.5, we reduced the dataset to 9,038 images — and mAP50 jumped to 92.65% (R10). That is a +6.9% improvement over R8 with fewer images.

Rule of thumb: A smaller, cleaner dataset beats a larger, noisier one. Always filter pseudo-labels aggressively. In our case, cutting the dataset in half while raising quality gave the biggest single-round improvement in the entire project.

3. Pseudo-Labeling Has Diminishing Returns

The biggest gains came in the early rounds:

R2→R3: +92.95% mAP50 (from 30.2% to 50.9%)
R3→R4: +92.95% mAP50
R7→R8: +92.95% mAP50
R8→R10: +92.95% mAP50 (but only after fixing data quality)

Each doubling of data yields less improvement. Beyond ~10K images, you need fundamentally better annotations (human review) or better architectures to see significant gains.

4. Per-Class Analysis Reveals Bottlenecks

Class	mAP50	Precision	Recall	Issue
Globe Valve	94.7%	89.0%	86.6%	Excellent
Gate Valve	86.1%	89.9%	66.6%	Low recall — many missed
Ball Valve	81.1%	71.9%	75.8%	Confusion with gate valve
Other Valve	72.8%	66.9%	68.0%	Scarce samples (4%)

Gate valve has great precision but poor recall — the model is too conservative. Other valve has too few training samples. These insights guide where to invest annotation effort.

5. GPS Metadata is an Underused Asset

93% of our inspection images contained EXIF GPS data. This enabled us to build an interactive inspection map showing 650+ valve well locations across three Chinese provinces. For infrastructure companies, geospatial AI is a game-changer — detection results are not just labels, they are map pins.

Update: Anomaly Detection System

Beyond valve detection, we have now trained an anomaly detection model that localizes structural and environmental defects in valve well inspection photos.

Three-Model Inspection Pipeline

Model	Purpose	Architecture	Performance
Valve Detection	Locate and classify valves	YOLOv8s	mAP50 = 92.95%
Anomaly Classification	Image-level anomaly screening	EfficientNet-B0	74.0% accuracy
Anomaly Detection V2	Localize anomalies with bounding boxes	YOLOv8s	mAP50 = 48.2%

Anomaly Classes (6 types)

The anomaly detection model identifies:

Water Accumulation (mAP50 = 57.7%) — Standing water at well bottom
Wall Crack (mAP50 = 65.0%) — Structural cracks in well walls
Corrosion / Rust (mAP50 = 22.3%) — Metal surface degradation
Water Seepage (mAP50 = 14.0%) — Active water infiltration
Fog / Condensation (mAP50 = 20.2%) — Vapor on camera lens
Coating Damage (mAP50 = 0.7%) — Protective coating peeling

VLM-Powered Annotation Pipeline

Instead of manual bounding box drawing, we used a Vision-Language Model (VLM) to annotate 213 images with 505 bounding boxes. The VLM identified anomaly regions and provided normalized coordinates that converted directly to YOLO format. This approach is 10x faster than manual annotation and produces consistent, high-quality labels.

V1 Training Results

Dataset: 213 images (173 train / 40 val), 505 bounding boxes
Best epoch: 33 (early stopped at 64, patience=30)
mAP50: 31.2% | mAP50-95: 19.4%
Best performing: Wall cracks (65%) and water accumulation (58%) — the most visually distinctive anomalies

V2 Training Results — Major Improvement

We expanded the VLM annotation pipeline from 213 to 512 images (1,181 bounding boxes) and retrained with improved hyperparameters:

Metric	V1	V2	Change
Training images	213	512	+140%
Bounding boxes	505	1,181	+134%
Best epoch	33	73	—
mAP50	31.2%	45.9%	+47.4%
mAP50-95	19.4%	29.0%	+49.7%
Patience	30	40	—

Key V2 improvements:

Doubled annotated data (213→512 images) using VLM batch annotation
Increased patience from 30 to 40 for better convergence
Added close_mosaic=15 for final training stages
Raised cls weight from 0.5 to 0.8 for class imbalance handling
Increased copy_paste from 0.1 to 0.15 for data augmentation

Full-Scale Inference — Complete

The EfficientNet-B0 classifier has processed all 861,367 inspection images. Key findings:

94.5% of images contain at least one anomaly
Corrosion/Rust is most prevalent (77.3%), followed by Coating Damage (67.7%) and Wall Cracks (59.6%)
68.3% mild severity, 26.2% moderate, 1.1% severe
Mean inference confidence: 0.931

Update: Dataset Expansion & Ongoing Training (July 2026)

Since the original post, significant progress has been made:

VLM annotations expanded: 1,003 → 1,301 images (298 new high-quality annotations from diverse well locations)
Total annotated bboxes: 3,071 across 6 anomaly classes
Balanced class distribution: water_accumulation (895), corrosion_rust (787), water_seepage (596), wall_crack (475), coating_damage (223), fog_condensation (95)
V4 training in progress: Augmented dataset with 5,593 training images (1,183 VLM + 4,439 pseudo-labeled), targeting higher mAP50
End-to-end inspection pipeline: 3-model pipeline (valve detection → anomaly detection → classification) producing 0-100 health scores (GOOD/FAIR/POOR/CRITICAL)
Multi-format export: All models available in PyTorch, ONNX, CoreML, and TorchScript formats

Update: V6 Breakthrough — 55.71% mAP50 (July 2026)

After V4 and V5 both failed due to noisy pseudo-labels, we found the winning recipe with V6:

The Failed Experiments

Version	Strategy	Train Images	mAP50	Result
V4	413 VLM + 4,442 pseudo-labels	4,855	41.7%	Failed — pseudo-label noise
V5	1,183 VLM + 4,439 pseudo-labels	5,593	31.0%	Failed — even more noise

Adding thousands of pseudo-labeled images actually hurt performance because the V2s model's systematic detection errors became "solidified" in the training data.

The V6 Solution: Quality Over Quantity

V6 takes a fundamentally different approach:

VLM-only data: 1,301 high-quality VLM annotations (no pseudo-labels at all)
Fine-tune from V2s: Start from V2s best weights instead of COCO pretrained
Lighter augmentation: mosaic=0.5, no mixup/copy_paste
AdamW optimizer: lr0=0.001, cos_lr=True

V6 Results — New Record

Metric	V2s (Previous Best)	V6 (Current)	Improvement
mAP50	48.19%	55.71%	+15.5%
mAP50-95	29.0%	35.8%	+23.4%
Best Epoch	73	13	Faster convergence

V6 Per-Class Performance

Class	V2s mAP50	V6 mAP50	Improvement
Water Accumulation	57.7%	78.9%	+36.7%
Fog / Condensation	20.2%	67.3%	+233%
Wall Crack	65.0%	60.1%	-7.5%
Corrosion / Rust	22.3%	54.0%	+142%
Water Seepage	14.0%	43.3%	+209%
Coating Damage	0.7%	30.6%	+4271%

The biggest improvements came in the previously worst-performing classes — coating damage went from nearly zero to 30.6%, and fog condensation jumped from 20.2% to 67.3%. Wall crack slightly decreased, likely because the model learned to distinguish it better from corrosion.

Key Lesson

Noisy pseudo-labels hurt more than they help. Adding 4,000+ pseudo-labeled images dropped mAP50 from 48% to 31-42%. But using only 1,301 high-quality VLM annotations with fine-tuning from the best existing weights boosted mAP50 to 55.71%.

This suggests a clear recipe for iterative improvement:

Train the best model you can with clean data
Use THAT model to generate pseudo-labels (not an older, weaker one)
Filter pseudo-labels aggressively (high confidence only)
Fine-tune — don't train from scratch

We are now applying this recipe with V7, using V6 to generate much higher-quality pseudo-labels.

What is Next

V7 training: Using V6-generated pseudo-labels (higher quality than V2s) + all VLM annotations
Active learning: Human review of borderline anomaly predictions
Target weak classes: More VLM annotations for water_seepage and coating_damage
Commercial deployment: Custom model development services now available on Fiverr
Enterprise partnerships: Offering custom model development for gas utility companies

Try It Yourself

Both models are available on Hugging Face:

Valve Detection Model — mAP50 = 92.95%
Anomaly Detection Model — mAP50 = 48.2% (V2)
Live Demo — Try the valve detector
GitHub — Full code and combined pipeline
Product Page — Complete pipeline overview

If you are building AI for industrial inspection, reach out — I offer custom model development starting at $500.

Built on Apple M4 Mac Mini with PyTorch MPS acceleration. Total valve training: ~50 hours across 10 rounds. Anomaly detection V2 trained in ~50 minutes on 512 VLM-annotated images. Full 861K image inference completed with EfficientNet-B0.

DEV Community: jins zhu