AI Car Recognition (Make & Model) for Mobile Apps

AI Car Recognition (Make & Model) for Mobile Apps

Recognizing a car's make and model from a photo is a well-studied problem. Models trained on Stanford Cars Dataset (196 classes) or CompCars achieve 90%+ accuracy on clean side shots. The main production challenges are angles, partial visibility (front only or rear only), nighttime conditions, and niche market vehicles.

Our team has 5+ years of experience in mobile AI solutions and has delivered over 50 Computer Vision projects for automotive. We've tackled these issues in real projects for insurance and dealer apps. In this article, we'll cover how to build a robust car recognition system that works under challenging conditions, and compare implementation options—from ready-made APIs to custom CoreML/TFLite models. We'll describe specific technical solutions including multi-angle capture and a hybrid VIN+Visual approach. If you need an estimate for a similar project, contact us for a free consultation.

Ready APIs and Their Limitations

For most projects: a custom CoreML/TFLite model based on EfficientNetV2, fine-tuned on a combined dataset (Stanford Cars + VMMRdb). Model size — 25–40 MB, Top-1 accuracy on popular models — 88–93%. A custom model yields 10-15% higher accuracy than ready APIs, especially on rare cars. Inference time on iPhone 13 — under 50 ms.

How to Choose the Approach for Car Recognition?

The choice between ready API and custom model depends on your requirements. If you only need to recognize popular models (top 100-200) and accuracy isn't critical, CarAPI will suffice. For insurance or dealer apps where every detail matters, a custom model with multi-angle capture is the only reliable option. We recommend starting with an API prototype to evaluate accuracy on real data, then migrating to a custom solution.

iOS Implementation with CoreML

class CarRecognitionService { private lazy var model: VNCoreMLModel = { let config = MLModelConfiguration() config.computeUnits = .cpuAndNeuralEngine let mlModel = try! CarClassifierV3(configuration: config).model return try! VNCoreMLModel(for: mlModel) }() func recognize(image: UIImage) async throws -> [CarPrediction] { guard let cgImage = image.cgImage else { throw CarError.invalidImage } return try await withCheckedThrowingContinuation { continuation in let request = VNCoreMLRequest(model: model) { request, error in if let error = error { continuation.resume(throwing: error) return } let results = (request.results as? [VNClassificationObservation]) ?? [] let predictions = results .filter { $0.confidence > 0.05 } .prefix(5) .map { CarPrediction( makeModel: $0.identifier, // "Toyota Camry" confidence: $0.confidence )} continuation.resume(returning: Array(predictions)) } // Normalizing image orientation is critical — otherwise accuracy drops request.imageCropAndScaleOption = .centerCrop let handler = VNImageRequestHandler(cgImage: cgImage, orientation: image.cgImageOrientation) try? handler.perform([request]) } } } 

The parameter imageCropAndScaleOption = .centerCrop is a non-obvious detail. By default, CoreML scales images differently than the model expected during training, causing a 5–8% accuracy loss.

Why Multi-Angle Capture Improves Accuracy?

For high-accuracy tasks (insurance apps, car dealers), one shot is not enough. We request three angles:

enum CarPhotoAngle: CaseIterable { case frontThreeQuarter // 3/4 front — optimal for make/model case rear // for rear (additional verification) case side // side — for body style and generation var instruction: String { switch self { case .frontThreeQuarter: return "Photograph the car from front-side (45°)" case .rear: return "Photograph from the rear" case .side: return "Photograph strictly from the side" } } } // Aggregating results from three shots — weighted voting func aggregatePredictions(_ predictions: [[CarPrediction]]) -> CarPrediction { let weights: [Double] = [0.5, 0.3, 0.2] // frontThreeQuarter more important // ... weighted voting by makeModel } 

Year and Generation Identification

Visually determining the year is harder than make/model: facelifts alter appearance minimally. Two approaches:

• Generation classifier (separate head in multi-task model)
• Hybrid: VIN via OCR (if visible) + visual generation classification

VIN approach is more accurate: if OCR reads the VIN from the license plate or windshield, all data (make, model, year, trim) is decoded without AI via NHTSA API or paid VIN decoders. VIN OCR recognition time is about 200 ms.

Work Stages (Turnkey)

1. Requirements analysis and dataset collection (if rare models needed)
2. Model training and validation (EfficientNetV2, CoreML/TFLite)
3. Recognition module integration with app UI
4. Testing on real photos under different conditions
5. Post-release support (fine-tuning, API updates)

What's Included in the Deliverable

Our experience reduces labeling costs by 30% through transfer learning and selective key image sampling.

Timeline Estimates

Integration of a ready CoreML model with result display UI — 3–5 days. Full system with multi-angle capture, hybrid VIN+Visual approach, car feature database, and iOS + Android — 1–2 weeks.

To get an estimate for your project, fill out the form or write to us — we'll reply within a day. Get a free consultation.