AI Defect Detection via Mobile App Camera

Standard machine vision systems (Cognex, Keyence) are effective but expensive and rigidly tied to the line. A mobile inspector based on AI is a flexible alternative: cheaper, more mobile, no equipment rearrangement required. However, new engineering challenges arise: unstable lighting, variable distance to the object, vibration from handheld capture. We solve them with on-device AI tailored to your process. With over 5 years of experience in industrial computer vision and 30+ implementations on production lines, savings of up to 40% compared to off-the-shelf systems are not uncommon. Contact us for a consultation — we'll help you choose the optimal solution.

AI Defect Detection via Mobile App Camera: Technical Challenges

Every millisecond counts, and internet in the workshop is a luxury. On-device inference eliminates latency and network dependency. Model size is limited by device RAM, speed by throughput requirements.

// iOS: industrial defect detection via CoreML class DefectDetectionEngine { private let model: VNCoreMLModel private var confidenceThreshold: Float = 0.5 // adjustable on the test stand private var iouThreshold: Float = 0.45 // Dedicated queue for stable FPS private let inferenceQueue = DispatchQueue( label: "defect.inference", qos: .userInteractive ) func analyze(sampleBuffer: CMSampleBuffer) async throws -> [DefectDetection] { guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { throw DefectError.invalidFrame } return try await withCheckedThrowingContinuation { continuation in inferenceQueue.async { let request = VNCoreMLRequest(model: self.model) { req, error in if let error = error { continuation.resume(throwing: error) return } let detections = (req.results as? [VNRecognizedObjectObservation])? .filter { $0.confidence >= self.confidenceThreshold } .map { obs in DefectDetection( type: DefectType(rawValue: obs.labels.first?.identifier ?? "") ?? .unknown, confidence: obs.confidence, boundingBox: obs.boundingBox, // normalized [0,1] severity: self.classifySeverity(obs) ) } ?? [] continuation.resume(returning: detections) } request.imageCropAndScaleOption = .scaleFill let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer) try? handler.perform([request]) } } } } 

Performance. YOLOv8n in CoreML on iPhone 14: ~15 ms per inference (66 FPS potential). YOLOv8s: ~25 ms. For a line requiring >30 frames/sec with handheld capture — choose the n-variant.

Image Stabilization for Handheld Scanning — AI Defect Detection

Hand vibration is the enemy of small defects. Several techniques: frame buffering with sharpness selection (Laplacian), automatic exposure, optical stabilization if available.

// Frame buffering + selecting the sharpest frame class StabilizedFrameSelector { private var frameBuffer: RingBuffer<CMSampleBuffer> = RingBuffer(capacity: 8) private var sharpnessScores: [Float] = [] func addFrame(_ buffer: CMSampleBuffer) { let sharpness = computeLaplacianVariance(buffer) frameBuffer.push(buffer) sharpnessScores.append(sharpness) } // For analysis — take the frame with peak sharpness from the last N frames var bestFrame: CMSampleBuffer? { guard let maxIdx = sharpnessScores.indices.max(by: { sharpnessScores[$0] < sharpnessScores[$1] }) else { return nil } return frameBuffer[maxIdx] } } 

Also: AVCaptureDevice.activeVideoMinFrameDuration + exposureMode = .continuousAutoExposure + stabilization via videoStabilizationMode = .cinematic.

How to Fine-Tune the Model on Real Data?

Ready-made datasets for specific production are unavailable. You need your own labeling. The process:

1. Capture 200–500 samples on the production line (normal + defective)
2. Label in Label Studio or CVAT (bounding boxes + defect classes)
3. Augmentation: brightness ±30%, rotation ±15°, horizontal flip, Gaussian noise — simulating real shooting conditions
4. Train YOLOv8s/m (depending on speed requirements)
5. Convert to CoreML (.mlpackage) or TFLite
6. Iterative fine-tuning on production errors — every 2–4 weeks

# Fine-tuning on new production data from ultralytics import YOLO model = YOLO("defect_detection_v2.pt") # previous version as base results = model.train( data="production_defects.yaml", epochs=50, imgsz=640, batch=16, lr0=0.001, # lower LR for fine-tuning freeze=10, # freeze first 10 layers of backbone augment=True, hsv_h=0.015, hsv_s=0.7, degrees=10.0, translate=0.1, scale=0.5, mosaic=1.0 ) 

Every 2–4 weeks we fine-tune the model on new production data, increasing accuracy up to 98%. Contact us for an audit of your data — we'll estimate the required labeling volume.

How Do We Integrate the Solution with Your Systems?

The mobile inspector must record results in the MES/ERP system. We use an offline-first approach: first local storage in Room/CoreData, then synchronization via REST/GraphQL when a network connection appears. This guarantees no inspection is lost.

// Android: sending inspection result data class InspectionResult( val productId: String, val batchId: String, val inspectorId: String, val timestamp: Instant, val detections: List<DefectDetection>, val verdict: InspectionVerdict, // PASS, FAIL, REVIEW val imageUrl: String, // saved photo with annotations val deviceId: String ) suspend fun submitInspection(result: InspectionResult) { // First — local queue (production may be without Wi-Fi) localQueue.enqueue(result) // Sync when network appears syncManager.triggerSync() } 

Offline-first is critically important: workshop Wi-Fi is unstable, loss of an inspection result is unacceptable.

Challenge: Specifics of Industrial Quality Control

Defects in production vary drastically by industry. Below are examples of the most common cases.

For each industry — its own model. A universal defect model doesn't work: what is a defect on a PCB may be normal on metal.

What Does Our Work Include?

All components are configured for your production processes.

Timeline Estimates

MVP with a baseline model (200–300 labeled samples), on-device inference, local inspection history — 3–4 weeks. Full system with a fine-tuned model for a specific production process, image stabilization, offline-first sync with MES/ERP, defect statistics dashboard, and support for iOS + Android — 2–3 months. Exact cost is calculated individually after an audit of your production.

Payback for such a solution is less than 6 months due to reduced scrap and faster inspection. Contact us for a free evaluation of your project. Order a pilot implementation today.

According to Apple's documentation, on-device inference ensures privacy and speed.