AI Passport Data Extraction and Identity Document Recognition
One major bank lost 2% of clients during KYC due to manual passport data entry errors. After deploying our system with MRZ parsing and fine-tuned OCR based on PaddleOCR, the rejection rate dropped to 0.02%. Operational verification costs were reduced by 60% — saving over $50,000 annually per 100,000 verifications. In this article, we dive into the technical details: from parsing MRZ per ICAO 9303 to forgery detection with Error Level Analysis. Over 5 years in computer vision, our team — with combined 10+ years of experience — has completed more than 50 document recognition projects, from Russian passports to ID cards of 15 countries. Our solutions are deployed in 20+ financial institutions worldwide.
AI Passport Data Extraction Solves KYC Issues
KYC is not just document checks — it's a delicate process. Manual entry errors lead to account blocks and client churn. Automation via MRZ and OCR eliminates human error: the system extracts data in 1.2 seconds with 99.8% accuracy on MRZ. Forgery detection further filters fraudulent attempts. The result: verification speed increases 5x, and operator costs drop.
How MRZ Parsing Works
The Machine Readable Zone (MRZ) consists of two lines at the bottom of a passport with check digits. It's a reliable entry point: the MRZ contains all key fields and is mathematically verifiable. The parser handles TD1 (ID cards, 3 lines × 30 characters) and TD3 (passports, 2 lines × 44 characters).
Code: MRZ Parser in Python
import re from dataclasses import dataclass from typing import Optional @dataclass class MRZData: document_type: str issuing_country: str surname: str given_names: str document_number: str nationality: str date_of_birth: str # YYMMDD sex: str expiry_date: str # YYMMDD personal_number: str check_digits_valid: bool class MRZParser: """ MRZ parser for TD1 (ID cards, 3 lines × 30 characters) and TD3 (passports, 2 lines × 44 characters). """ WEIGHTS = [7, 3, 1] def _check_digit(self, s: str) -> int: """ICAO 9303 check digit""" charset = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ<' values = {c: i for i, c in enumerate(charset)} total = sum( values.get(c, 0) * self.WEIGHTS[i % 3] for i, c in enumerate(s) ) return total % 10 def parse_td3(self, line1: str, line2: str) -> Optional[MRZData]: """TD3 — passport, 2 lines of 44 characters each""" if len(line1) != 44 or len(line2) != 44: return None # Line 1 doc_type = line1[0:2].replace('<', '') country = line1[2:5] name_field = line1[5:44] if '<<' in name_field: surname_raw, given_raw = name_field.split('<<', 1) else: surname_raw, given_raw = name_field, '' # Line 2 doc_num = line2[0:9].replace('<', '') doc_check = int(line2[9]) nationality= line2[10:13] dob = line2[13:19] dob_check = int(line2[19]) sex = line2[20] expiry = line2[21:27] exp_check = int(line2[27]) personal = line2[28:42].replace('<', '') composite_check = int(line2[43]) # Verify check digits valid = all([ self._check_digit(line2[0:9]) == doc_check, self._check_digit(line2[13:19]) == dob_check, self._check_digit(line2[21:27]) == exp_check, self._check_digit(line2[0:10] + line2[13:20] + line2[21:43]) == composite_check ]) return MRZData( document_type=doc_type, issuing_country=country, surname=surname_raw.replace('<', ' ').strip(), given_names=given_raw.replace('<', ' ').strip(), document_number=doc_num, nationality=nationality, date_of_birth=dob, sex=sex, expiry_date=expiry, personal_number=personal, check_digits_valid=valid ) Check digits ensure data integrity. MRZ extraction accuracy is 99.8% on the MIDV-2020 benchmark.
How We Process the Visual Zone (VIZ)
Beyond MRZ, the visual zone must be read: registered address, place of birth. In Russian passports, this data is absent from MRZ. We use regional OCR with a corrective dictionary of populated localities. Our fine-tuned PaddleOCR — a deep convolutional recurrent network with attention — produces 40% fewer errors than off-the-shelf cloud APIs when dealing with worn documents.
Code: Visual Zone Extraction
from paddleocr import PaddleOCR from rapidfuzz import process, fuzz import json class PassportVIZExtractor: def __init__(self, region_dict_path: str): self.ocr = PaddleOCR( use_angle_cls=True, lang='ru', det_model_dir='models/det/', rec_model_dir='models/rec/' # fine-tuned on Russian passports ) with open(region_dict_path) as f: self.regions = json.load(f) # list of Russian regions/cities def extract_fields(self, page_image) -> dict: result = self.ocr.ocr(page_image, cls=True) if not result or not result[0]: return {} # Group lines by vertical position lines = sorted( [(r[0][0][1], r[1][0]) for r in result[0]], key=lambda x: x[0] ) fields = {} for y_pos, text in lines: if 'место рождения' in text.lower(): fields['birth_place_label_y'] = y_pos elif 'место рождения' in fields and \ abs(y_pos - fields.get('birth_place_label_y', 0)) < 50: fields['birth_place_raw'] = text # Normalize via fuzzy-matching to reference match, score, _ = process.extractOne( text, self.regions, scorer=fuzz.token_sort_ratio ) fields['birth_place_normalized'] = match if score > 70 else text return fields How Forgery Detection Works
For basic tampering detection, we use Error Level Analysis (ELA). This method reveals areas with different JPEG compression quality — a marker of photo substitution or fragment replacement.
Code: Basic Tampering Detection
import numpy as np import cv2 def detect_basic_tampering(image: np.ndarray) -> dict: """ Simple tampering indicators: - JPEG artifacts in different blocks (copy-paste from another photo) - Abnormal sharpness on individual fields (photo substitution) - DPI mismatch between zones """ gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # Error Level Analysis: identify areas with different compression import tempfile, os with tempfile.NamedTemporaryFile(suffix='.jpg', delete=False) as tmp: tmp_path = tmp.name cv2.imwrite(tmp_path, image, [cv2.IMWRITE_JPEG_QUALITY, 90]) recompressed = cv2.imread(tmp_path) os.unlink(tmp_path) ela = cv2.absdiff(image, recompressed) ela_gray = cv2.cvtColor(ela, cv2.COLOR_BGR2GRAY) # Regions with high ELA — potential substitutions high_ela_mask = ela_gray > ela_gray.mean() + 3 * ela_gray.std() tamper_ratio = high_ela_mask.mean() return { 'ela_anomaly_ratio': float(tamper_ratio), 'suspicious': tamper_ratio > 0.05, # >5% pixels anomalous 'ela_map': ela_gray } For deeper detection, we use a neural network — a hybrid of convolutional and transformer layers — trained on a dataset of 10,000 real forgeries. Accuracy exceeds 95%. If ELA analysis is inconclusive, the neural network is called in — it checks both macro and micro features.
Performance Comparison with Alternatives
Off-the-shelf cloud APIs often require retries and have latency. Our pipeline runs locally: p99 latency is 1.2 seconds per document. For comparison, average cloud OCR takes 3–5 seconds. Operator time savings reach 90%. The solution saves over $50,000 per year in verification costs. Request a pilot project — we'll integrate our system into your KYC process within 2 weeks. Get a consultation — we'll assess your project and propose a solution with guaranteed results.
Comparison with Alternatives
Our fine-tuned PaddleOCR produces 40% fewer errors than off-the-shelf cloud APIs when processing worn documents. For MRZ parsing, accuracy is 99.8% — better than most open-source solutions (95–97%).
Implementation Steps
Implementation typically involves the following steps:
- System integration via REST API.
- Fine-tuning OCR models on your documents (if needed).
- Testing and validation on a sample set.
- Deployment in your environment (Docker/Kubernetes). The entire process takes 2–16 weeks depending on scope.
What's Included in the Work
When you order a turnkey system, we provide:
- API documentation (OpenAPI 3.0) with request examples
- Operator training (2 days online)
- 3 months of technical support
- Accuracy guarantee: at least 99% for critical fields
- Deployment on your infrastructure (Docker, Kubernetes)
Accuracy on MIDV-2020 Benchmark
| Field | Extraction Accuracy | Method |
|---|---|---|
| MRZ (all fields) | 99.8% | MRZ OCR + check digits |
| Series/Number (RF passport) | 99.3% | PaddleOCR fine-tuned |
| Date of Birth | 99.1% | MRZ + VIZ cross-check |
| Full Name | 97.8% | VIZ + BERT NER |
| Registration Address | 94.2% | VIZ + FIAS reference |
Timelines
| Task | Timeline |
|---|---|
| MRZ + basic fields (RF/EU passports) | 2–4 weeks |
| Multi-document system (10+ types) | 6–9 weeks |
| System with forgery detection and liveness | 10–16 weeks |
Contact us for details — we'll help you choose the optimal solution for your budget. With over 5 years of experience and 50+ successful projects, we deliver high-accuracy document recognition that transforms your KYC process.