AI-Powered Document Verification via Mobile Camera
Document verification via camera isn't just OCR. It's a pipeline of multiple stages: detecting document in frame, assessing image quality, extracting data via OCR, structuring fields, validating accuracy, and detecting forgery signs. Each stage can fail—and here lies all the complexity.
Why Raw OCR Doesn't Work
Attempting to read a passport via Vision framework (iOS) or ML Kit (Android) and parse the result with regex—the most common first approach. It gives 70–80% accuracy in lab conditions and 40–60% on real users: glare, shooting angle, creased pages, document wear, non-standard fonts—everything breaks simple OCR.
The correct pipeline adds three layers over OCR:
Image preprocessing. Perspective correction (document shot at an angle), contrast enhancement, glare removal. On iOS—CIFilter + CIPerspectiveCorrection. On Android—OpenCV via JNI or CameraX with custom ImageAnalysis.
Specialized document OCR. Not general OCR, but models trained on documents: Microsoft Azure Document Intelligence, Google Document AI, Amazon Textract. They return not just text, but structured fields—surname, given_names, date_of_birth, document_number—already properly bound to document zones.
Machine Readable Zone (MRZ) parsing. Passports contain MRZ—two ICAO 9303 standard lines at the bottom. Most reliable data source: standardized font, clear structure, built-in checksum. Libraries: mrz-java, passport-reader for iOS, or custom parser implementation.
Azure Document Intelligence Integration
// iOS — Swift
import AzureAIDocumentIntelligence
class DocumentVerificationService {
private let client: DocumentIntelligenceClient
func analyzePassport(imageData: Data) async throws -> PassportData {
let request = AnalyzeDocumentRequest(
urlSource: nil,
base64Source: imageData.base64EncodedString()
)
let operation = try await client.beginAnalyzeDocument(
"prebuilt-idDocument",
analyzeRequest: request
)
let result = try await operation.waitForResult()
guard let document = result.documents?.first else {
throw DocumentError.noDocumentDetected
}
return PassportData(
firstName: document.fields?["FirstName"]?.valueString,
lastName: document.fields?["LastName"]?.valueString,
documentNumber: document.fields?["DocumentNumber"]?.valueString,
dateOfBirth: document.fields?["DateOfBirth"]?.valueDate,
expiryDate: document.fields?["ExpirationDate"]?.valueDate,
nationality: document.fields?["CountryRegion"]?.valueCountryRegion,
mrz: document.fields?["MachineReadableZone"]?.valueString,
confidence: document.confidence ?? 0
)
}
}
Confidence score—critical parameter. When confidence < 0.8, request retake with user guidance (better lighting, hold straight, don't cover edges).
Real-Time Camera Guidance
Users shouldn't make multiple blind attempts. Real-time feedback while shooting via Vision framework on iOS:
// Detects document bounds in realtime while camera is active
func detectDocumentInFrame(_ pixelBuffer: CVPixelBuffer) {
let request = VNDetectRectanglesRequest { [weak self] request, error in
guard let observation = request.results?.first as? VNRectangleObservation else {
self?.cameraGuidance = .noDocumentFound // "Point camera at document"
return
}
let area = observation.boundingBox.width * observation.boundingBox.height
if area < 0.4 {
self?.cameraGuidance = .tooFar // "Move closer"
} else if area > 0.9 {
self?.cameraGuidance = .tooClose // "Move camera away"
} else {
self?.cameraGuidance = .ready // Auto-capture
}
}
request.minimumAspectRatio = 0.5
request.maximumAspectRatio = 1.0
request.minimumConfidence = 0.7
try? VNImageRequestHandler(cvPixelBuffer: pixelBuffer).perform([request])
}
Auto-capture at perfect positioning eliminates button-pressing—reduces poor photos.
Forgery Detection
Basic anti-spoofing for mobile verification includes:
Physical document vs document photo check. If user photographs a printed or screen-displayed document—models detect screen pixel patterns (moire effect) or uneven paper texture. Microsoft Azure and Onfido have built-in detectors.
Liveness check. For document + selfie binding, liveness check is mandatory: random head movements, blinking. AWS Rekognition and FaceTec provide SDKs.
MRZ vs visual fields cross-check. Date of birth in MRZ and visual zone should match—simple but effective check.
Development Process
Analyze documents to support → select OCR provider (Azure / Google Document AI / Regula) → implement real-time camera guidance → integrate document analysis API → MRZ parsing and field cross-validation → anti-spoofing → compliance checks (data storage regulations).
Timeframe Estimates
MVP with Azure Document Intelligence and basic guidance—2–3 weeks. Complete system with liveness check, anti-spoofing, multiple document types—4–6 weeks.