AI Detection of Structural Defects from Photos

AI diagnosis of structural defects — cracks in concrete, rebar corrosion, masonry delamination — is critical for building safety. Traditional manual inspection is subjective: different inspectors classify the same crack differently. A neural network provides reproducible assessment with quantitative metrics. For example, crack width is measured to 0.1 mm accuracy, and corrosion area with less than 5% error. Our team specializes in such solutions: over 5 years of experience, over 50 deployments for construction expertise. We guarantee severity classification accuracy of 94%.

Task: Classification and Segmentation of Defects

Structural defects require pixel-level accuracy, not just bounding boxes — we care about crack length, width, orientation. This is a computer vision task, specifically semantic segmentation (see Wikipedia). Comparison: a neural network works 20 times faster than a manual inspector, with severity classification accuracy reaching 94%.

import torch import numpy as np import cv2 import segmentation_models_pytorch as smp from PIL import Image from torchvision import transforms from dataclasses import dataclass from typing import Optional @dataclass class DefectAnalysis: defect_type: str severity: str # 'hairline', 'minor', 'moderate', 'severe', 'critical' area_px: int area_ratio: float max_width_px: Optional[float] max_length_px: Optional[float] orientation: Optional[float] # degrees from vertical bounding_box: list class StructuralDefectDetector: def __init__(self, model_path: str): """ UNet++ with EfficientNet-B5 encoder. Fine-tuned on Concrete Crack Images Dataset (40k images) + own dataset with corrosion and delamination. """ self.model = smp.UnetPlusPlus( encoder_name='efficientnet-b5', encoder_weights=None, in_channels=3, classes=5, # bg, crack, corrosion, spalling, delamination activation=None ) checkpoint = torch.load(model_path, map_location='cpu') self.model.load_state_dict(checkpoint['model']) self.model.eval() self.class_names = { 0: 'background', 1: 'crack', 2: 'corrosion', 3: 'spalling', 4: 'delamination' } self.transform = transforms.Compose([ transforms.Resize((512, 512)), transforms.ToTensor(), transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) ]) @torch.no_grad() def analyze(self, image: np.ndarray, gsd_mm_per_pixel: Optional[float] = None) -> list[DefectAnalysis]: """ gsd_mm_per_pixel: scale (from drone or laser metadata). Allows returning sizes in mm instead of pixels. """ h, w = image.shape[:2] pil_img = Image.fromarray(cv2.cvtColor(image, cv2.COLOR_BGR2RGB)) tensor = self.transform(pil_img).unsqueeze(0) logits = self.model(tensor) # (1, 5, 512, 512) mask = logits.argmax(dim=1)[0].numpy() # (512, 512) # Scale mask back to original size mask_full = cv2.resize( mask.astype(np.uint8), (w, h), interpolation=cv2.INTER_NEAREST ) defects = [] for cls_id in range(1, 5): cls_mask = (mask_full == cls_id).astype(np.uint8) if cls_mask.sum() < 100: # noise filter continue contours, _ = cv2.findContours(cls_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) for cnt in contours: area = int(cv2.contourArea(cnt)) if area < 50: continue x, y, cw, ch = cv2.boundingRect(cnt) area_ratio = area / (w * h) # For cracks — skeletonization for length/width max_width = None max_length = None orientation = None if cls_id == 1: # crack max_width, max_length, orientation = self._analyze_crack( cls_mask[y:y+ch, x:x+cw] ) defects.append(DefectAnalysis( defect_type=self.class_names[cls_id], severity=self._classify_severity(cls_id, area_ratio, max_width, gsd_mm_per_pixel), area_px=area, area_ratio=area_ratio, max_width_px=max_width, max_length_px=max_length, orientation=orientation, bounding_box=[x, y, x+cw, y+ch] )) return defects def _analyze_crack(self, crack_roi: np.ndarray) -> tuple: """Skeletonization of crack for width and length measurement""" from skimage.morphology import skeletonize skeleton = skeletonize(crack_roi > 0) length = float(skeleton.sum()) # skeleton pixels ≈ length # Width via distance transform dist = cv2.distanceTransform(crack_roi, cv2.DIST_L2, 5) max_width = float(dist.max() * 2) if dist.max() > 0 else 0 # Orientation via PCA pts = np.column_stack(np.where(skeleton)) if len(pts) > 10: mean = pts.mean(axis=0) centered = pts - mean _, _, vt = np.linalg.svd(centered) angle = np.degrees(np.arctan2(vt[0, 0], vt[0, 1])) else: angle = 0.0 return max_width, length, angle def _classify_severity(self, cls_id: int, area_ratio: float, width_px: Optional[float], gsd: Optional[float]) -> str: if cls_id == 1: # crack severity by width (mm) width_mm = (width_px * gsd) if (width_px and gsd) else None if width_mm: if width_mm < 0.2: return 'hairline' if width_mm < 0.5: return 'minor' if width_mm < 1.5: return 'moderate' if width_mm < 5.0: return 'severe' return 'critical' # For others — by area if area_ratio < 0.005: return 'minor' if area_ratio < 0.02: return 'moderate' if area_ratio < 0.05: return 'severe' return 'critical' 

Why Segmentation is More Accurate than Bounding Boxes for Cracks?

Cracks occupy a small area, often less than 1% of pixels. Bounding boxes capture much background, distorting metrics. Segmentation provides a mask for each pixel, allowing accurate measurement of length, width, and orientation. For hairline cracks (width < 0.2 mm), a bounding box can show 10 times the actual area. Segmentation solves this.

How Does the Neural Network Determine Defect Type?

The model is trained on 40,000 images from open datasets and our own collection, labeled by construction experts. It uses the UNet++ architecture with an EfficientNet-B5 encoder — it achieves the best IoU for small objects. It detects 4 types of defects: cracks, corrosion, spalling, and delamination.

Defect Assessment Standards

Case Study: Inspection of 120 Overpass Supports

From our practice: technical condition assessment of an overpass using drone photography.

• 120 supports, 3500 photos with GSD 0.5–1.5 mm/pixel
• Processing: 2.5 hours on RTX 3090
• Found: 847 cracks (23 critical, width > 1 mm), 156 corrosion areas
• Manual verification of 5% random results: 94% severity classification accuracy
• Budget savings for expertise amounted to 200,000 RUB per object
• For another object with 50 supports, savings reached 180,000 RUB
• System payback period — less than one year due to reduced costs for on-site inspections and re-inspections

Implementation Process: Step by Step

1. Object analysis and requirements gathering — determine defect types, shooting conditions, required accuracy.
2. Data collection and labeling — at least 2000 images for your object. Labeling is done by construction engineers.
3. Model training — use UNet++ with augmentation, fine-tuning for your domain.
4. Integration — REST API on FastAPI, support for PNG/JPG, EXIF metadata.
5. Testing and validation — check on independent test set, compare with manual expertise.
6. Documentation and handover — API description, shooting requirements, training of your specialists.

Checklist: What Matters for Accurate Diagnosis

• Quality shooting with GSD 0.5–1.5 mm/pixel
• Uniform lighting, no glare
• Metadata (focal length, shooting height) for pixel-to-mm conversion
• At least 200 images of your object for fine-tuning (if needed)

What Is Included in Our Work

• Data collection and labeling (minimum 2000 images for your object).
• Training segmentation model with augmentation and scale (GSD) recognition.
• Integration: REST API on FastAPI, support for PNG/JPG, EXIF metadata.
• Documentation: API description, shooting requirements, integration examples.
• Warranty: 6 months free model refinement if shooting conditions change.

Estimated Timelines

Order a pilot inspection of one object — we will demonstrate accuracy on your data. Contact us to evaluate your project. Get a consultation from an engineer on automating building inspection with AI.