OCR training: failures on rare fonts and languages
Standard recognition models are trained on mass corpora: typographic typefaces, clean scans, one language per page.
As soon as a rare font enters the stream — narrow form-based, historical, with non-standard ligatures — character confidence drops, and similar symbols begin to collapse into one.
A single letter substitution in an account number or surname breaks everything built higher in the process.
Poor scan quality amplifies the effect: low contrast, copier noise, skewed lines, an area filled by a seal. Preprocessing aligns such pages but does not restore lost pixels — the model simply does not see some characters and reconstructs them by meaning.
A separate category is handwritten text in documents: signatures, handwritten edits, hand-filled form fields.
The spread of slant, thickness, and continuity of strokes is almost absent from training samples, so recognition either skips the fragment or produces a plausible but incorrect value.
Mixed languages in one document complicate it even more: a header in one language, details in another, some lines in transliteration. Recognition with a fixed dictionary is forced to choose one alphabet and rewrites the rest into it.
Field extraction quality drops more than per-character accuracy: a single substitution in a date is enough for the document to go to manual review.
We break down how this is addressed: where custom labeling for a specific font and language is needed, where synthetic samples of forms and handwritten insertions are needed, and where post-processing setup and field validation rules are sufficient.
What model adaptation gives: measurable effects
The base recognition model is trained on an averaged set — typical forms, printed font, one language. On your corpus, it stumbles over what simply was not in the training data: narrow office font, handwritten notes over printed text, bilingual forms where details are mixed.
Field extraction accuracy
Fine-tuning on labeled samples of your documents raises accuracy for specific fields — account number, date, amount in words, department code.
We calculate the metric not for the document as a whole, but for each field separately: where a character is confused with a similar one (0/O, 1/l, 3/Z), and where the block boundary is lost and the value moves into the adjacent cell.
Reduction of manual correction
The operator checks only fields where model confidence is below a set threshold — usually 5–15% of the document instead of all one hundred percent.
The system marks questionable fragments on the form itself, and manual correction becomes selective control: time per document drops severalfold, not by percentages.
Stability on new form types
The main effect is that the model does not fall apart when a counterparty sends a new-template form.
Fine-tuning with distortions of source files (rotation, scanner noise, broken lines) makes recognition resistant to form changes, and two or three dozen new examples cover an unfamiliar document type without rebuilding the entire processing chain.
As a result, document entry automation stops requiring constant manual support. This also works on open engines like Tesseract: what decides is not the model itself, but the quality of corpus labeling and the post-processing logic for the structure of your fields.
Approaches to OCR training: comparison of options
The base model confidently reads uniform printed forms, but falls apart where the font changes, a handwritten insertion appears, or two languages coexist in one field.
The refinement method is chosen based on document composition: how many different forms are in the stream and how critical an error is in a specific cell.
The options are not mutually exclusive — they are compared by preparation effort and by the class of errors each one closes.
| Option | When it fits | Limitations |
|---|---|---|
| Fine-tuning on custom labels | Narrow font, non-standard layout, uniform document stream | Requires labeled examples — hundreds per class; relabeling when the form changes |
| Post-processing of recognition results | Errors are predictable: confusion of similar characters, extra spaces, line merging | Fixes the consequence, not the cause; complex distortions still go to manual review |
| Dictionaries and field rules | There are reference books of codes, names, addresses and strict field formats | Breaks on values outside the reference book; rules must be maintained |
| Hybrid scheme | Heterogeneous stream: printed forms, scans, handwritten notes | Longer setup and more components to test together |
In practice, a hybrid usually wins: fine-tuning pulls up character recognition, while dictionaries and rules bring the field to a valid value — the document number is checked against a checksum.
Pure post-processing is appropriate where the stream is homogeneous and errors are of the same type.
To choose a scheme, we analyze your samples: calculate the share of fields read incorrectly and break errors down by type. This shows where rules are sufficient and where custom labeling is needed.
How the work proceeds: stages from audit to handover
We start not with model selection, but with an analysis of what actually arrives at the input. Documents are rarely homogeneous: printed forms sit next to handwritten insertions, scans differ in resolution and skew, and a rare typeface appears in isolated copies. This picture determines both the labeling scheme and which metrics are worth calculating at all.
-
Audit of the document corpus. We break samples down by types, languages, typefaces, and media quality, and calculate the share of pages in each group. This shows that a rare class is a fraction of a percent and will require synthetic data or additional examples.
-
Labeling scheme. We fix the granularity (lines, words, blocks), annotation format, and rules for disputed places: hyphenation, stamps, crossed-out text. We agree on this before the start.
-
Labeling preparation. We label some pages twice and compare — discrepancies show where the instruction is ambiguous. We supplement rare fonts and languages with synthetic data with realistic distortions.
-
Model fine-tuning. We select distortions and the order of example submission using the validation part of the corpus. We do not touch the test set — otherwise the metric stops meaning anything.
-
Validation on a holdout set. We calculate error on pages that participated neither in training nor in tuning, and break the result down by subgroups: print, handwriting, rare typeface. The average figure for the corpus hides failure on a rare class.
-
Handover of the model to operation. We provide weights, configurations, the retraining procedure, and monitoring metrics, and connect the model to the accounting system. We set the confidence threshold below which a fragment goes to an operator for review.
Handover package: model, labeling, documentation
The handover package determines whether you can develop the solution on your own. A model without labeling, configurations, and a description of weak points is a black box: a change of form or font turns into a new project. Therefore we hand over not only the weights, but the entire reproduction loop.
The package is assembled during the work, not before delivery — at acceptance you have already seen every artifact in action.
- Documentation set for the model: a card with the scope of applicability, metrics by document types (scans, photos, faxes), a list of known error classes, and applicability boundaries.
- Labeling and labeling versioning: source images, annotations, symbol dictionary, and a rule linking “labeling revision — model version,” so metrics are compared on the same set.
- Training and recognition configurations: preprocessing parameters, distortions, confidence thresholds — the run is reproducible on your side.
- Integration scheme with the accounting system: call points, exchange formats, behavior when the service is unavailable, reprocessing without duplicates.
- Retraining procedure: triggers (accuracy drop on the control set, new form type), frequency, who labels, and within what timeframes.
- Monitoring loop: share of recognitions with low confidence, manual review queue, automatic export of disputed cases back into labeling.
- Environment and launch: containers, dependencies, instructions for deployment and training on your hardware.
- Knowledge transfer: sessions with your team and a decision log recording why these architectures and thresholds were chosen.
Case: archive of contracts in three languages
The customer brought an archive of contracts covering ten years — about 400 thousand pages in three languages, more than half scans from printed forms. The forms are typeset in a non-standard typeface: narrow serifs, ligatures, compressed numeric fields.
Off-the-shelf recognition fell apart on such pages — similar glyphs were confused, key details drifted into free text.
Base Tesseract without fine-tuning produced 11–16% character errors specifically in fields with number and amount.
For a legally significant document, this means manual verification of every page: the share of pages that an operator corrected by hand stayed at 30–35%, and the flow of scans grew faster than the review staff.
What was changed
We assembled a training set from a labeled corpus and synthetic data: form typefaces were overlaid on clean backgrounds, noise, skew, seals, and signatures over text were added — what was scarce in the original scans.
The model was fine-tuned separately by language, but with a shared layer for numeric details, so identical fields in different languages would not diverge in interpretation.
On top we placed post-processing: dictionaries, checksums and date formats, filtering out obviously impossible values.
What changed
On a control set of 20 thousand pages, character errors fell from 13% to 1.2–1.8% depending on language. The share of pages going to manual correction decreased from 33% to 4%, and extraction accuracy for key details — number, date, party, amount — reached 97–99%. The full archive pass took weeks instead of months of manual entry.
A separate result is reproducibility. Adding a fourth language took days: only the sample set and dictionary changed, not the entire recognition scheme.
New typefaces are connected as separate example sets and do not break quality on old forms — OCR training turns from a one-off project into an adjustable process.
What to do if a model is already running in your environment
First, sort the errors into categories — there is no need to change the engine immediately.
Assemble a reference set of real documents and calculate the share of correct fields separately by fonts, languages, and scan quality: often the model confuses not text but a table area, or does not correct skew — then preprocessing helps, not fine-tuning.
If errors persist specifically on special fonts and languages, there are two paths: extend the already deployed recognition model or install a new one.
Fine-tuning preserves current settings, but requires custom labeling and runs into architecture: classic engines like Tesseract learn well on one typeface and degrade on mixed pages with tables and insertions in another language.
Compatibility with the current environment is checked before work starts: response format and block coordinates, encodings, reading order, behavior during failures and empty responses, reference books for validation.
Then we put an adapter layer that brings responses from any engine to a single form — so replacing the recognition engine does not turn into rewriting integrations.
A staged transition without downtime relies on shadow mode: the new model runs through the same stream in parallel, discrepancies with the reference are analyzed, and its share grows from a few percent to the full volume. Rollback is one switch; the old environment remains operational.
In practice this is rarely “either-or”: we fine-tune on your fonts, add dictionary validation against reference books, and install a second model for problematic types.
Describe what is already deployed and where exactly it makes mistakes — we will come back with an estimate and a plan.
Technical doubts: labeling, languages, privacy
During fine-tuning of optical recognition, the same questions come up: how many pages to label, what to do with personal data in scans, how a new language is connected, and what happens to the model after handover. Below are short answers as we resolve this in projects.
How much data needs to be labeled?
The volume depends on the distance between the target font and the base model: handwriting, a typographic form, and standard printed text require different amounts.
We do not label “by eye” — we take the first batch, measure character and word accuracy on a holdout set, and increase the volume until the gain plateaus. Labeling stops where it stops yielding quality.
What about personal data in documents?
Training can be conducted without taking scans outside the customer's environment: labeling and fine-tuning run on a dedicated server inside your infrastructure, and only model parameters are transferred outside.
If we need the pages, they are anonymized — fields with names, numbers, and addresses are cut out, and the source files are deleted after acceptance.
How is a new language added?
A new writing system means an extended alphabet, a post-processing dictionary, and a separate labeling set.
Feature transfer from an already trained model covers most of the work, so adding a language to an existing model and starting from scratch take different times — we name the timeframe after testing on your samples.
What happens to the model after launch?
The model is not a one-off artifact: we monitor the share of confident recognitions and the share of pages sent to manual review. Questionable pages automatically enter the queue — operator corrections become labeling for the next fine-tuning. Model versions are saved, so rollback to the previous one remains available.
Let's discuss the task: what to send for evaluation
To come back with an estimate rather than a range “from and to,” we need your documents in the form in which they arrive for processing.
20–30 samples per type are enough — this is sufficient to see the spread of quality and calculate accuracy on real data, not on a demonstration set.
- document samples in original quality, necessarily with problematic cases: faded print, skew, seals and handwritten notes over text;
- list of document types and fields: what we read line by line, what we extract as a separate detail;
- languages and alphabets, including mixed lines and transliteration within one document;
- non-standard fonts — narrow, pre-reform, handwritten, dot-matrix: what the current solution stumbles on;
- accuracy requirements: where an error is unacceptable, where it is acceptable, and what share of discrepancies is considered workable;
- operating conditions: closed environment or external network, page flow per day, who handles disputed cases;
- current process: what you use for recognition now, how you verify the result, and where the finished document goes.
Based on this data, we describe a plan: on which sample we train, where validation rules are sufficient, where manual labeling is needed — and within what timeframes the first working result on your documents will appear. Describe the task and send samples — we will come back with an estimate, stages, and timeframes.
