Why table recognition breaks on scans and PDFs
Table recognition failure rarely begins with a character recognition error — the task gets substituted earlier. The model returns lines of text, whereas a table is a structure where a value can only be read together with its coordinate and column header.
As soon as text is extracted without geometry, complex table layouts fall apart before the validation stage.
Merged cells are the first point of failure: the meaning of a value is spread across two or three header levels, while the value itself sits in a single cell. With flat parsing, the link is lost, and the row lands in the wrong column.
Multiline headers and empty separator cells meaning “same as above” finish the job — without restoring the header hierarchy, the data looks plausible but is wrong.
Scans and PDFs add a second group of causes.
A PDF with a text layer provides coordinates, but it is often a document rebuilt after printing, with lost grid lines; a scanned file is only an image, where a rotation of a couple of degrees, perspective, and noise shift the boundaries relative to the original.
Without line alignment and robust grid detection, a cell “drifts,” values merge, or they land in the neighboring column.
A separate cause is page breaks. A table split across two pages arrives as a second half without a header, and the repeated heading ends up in the data as an ordinary row.
Data extraction quality does not reach the threshold at which the export can be sent to the accounting system, and the operator starts transferring values by hand. We examine how each of these failures is addressed.
Measurable effect: accuracy, speed, processing volume
We measure the effect of data entry automation at three points, not “for the project as a whole.” The first is the share of documents that enter the accounting system without operator correction: at the start, this is usually 60–70%, because scan skew, merged cells, and multiline headers break parsing.
The second is the time for a batch of a thousand pages: two to three shifts of manual entry versus minutes of machine parsing. The third is the share of rows that the model flags as questionable and sends for review.
Extraction accuracy grows unevenly, and this is visible in advance. Numeric columns aligned to the right are read almost without errors, while composite headers spanning two or three rows and empty separators inside the table require a separate rule.
Therefore, in the report, accuracy is broken down by table and field types: it is clear where autofill can be enabled immediately, and where sum control or reconciliation with reference data is needed.
An entire class of manual entry errors disappears: digit shift, column swapping during transfer, a missed row at a page boundary, a duplicated line item.
Machine parsing applies one rule to the entire batch, and reconciliation of the sum of row values with the total row cuts off distortions before they are recorded in accounting.
If the control does not reconcile, the document is not “completed by eye” but sent to an exception queue with the problem area marked.
Operator workload is reduced by cutting manual operations, not by speeding up typing. People stop transferring numbers and handle exceptions — as a rule, this is 5–15% of pages: crumpled scans, handwritten edits, tables without a header.
We select the stack for the document type: printed forms are parsed by an open-source engine (Tesseract) with structure post-processing, while handwritten forms are handled by a multimodal model, so document processing speed grows without a proportional increase in team size.
Document processing options: templates, rules, learning
The processing mode is chosen not by the “complexity” of the document, but by the stability of its form. Template extraction relies on the coordinates of blocks on the page, parsing rules rely on text structure, and learning from examples relies on statistics across many samples.
A wrong choice manifests in the same way: the share of tables requiring manual correction grows along with the volume of the flow.
| Option | When it fits | Limitations |
|---|---|---|
| Template extraction | The form is fixed: blocks and columns do not shift from delivery to delivery | Breaks when the form version changes; each new type needs a separate setup |
| Parsing rules | The structure is predictable: stable headers, separators, repeating rows | Sensitivity to typos and “similar but different” documents; rules proliferate |
| Learning from examples | Heterogeneous scans, multiple suppliers, floating layout | Requires a labeled sample; quality depends on coverage of rare forms |
A mixed document flow is handled by a hybrid scheme: the document is first identified by form type, then goes into its own mode — template, rules, or model.
Cells below the confidence threshold do not go into the export but enter a manual verification queue, and the operator sees a highlighted fragment of the source image, not an empty field.
We test the mode choice on the customer's historical data: we run a corpus of documents and calculate the share of tables parsed without correction, as well as the volume of rules that will need to be maintained at the next form change.
If the rules grow faster than accuracy, we change the mode — this measurement is done before production launch, not after.
Stages: document parsing, setup, validation, handover
Documents rarely arrive uniform: skewed scans, stitched sheets, headers repeated across pages, handwritten marks over printed text. On such a set, the recognizer makes uneven errors — on one form type it runs cleanly, on another it confuses column boundaries.
Therefore, we build the implementation stages with a checkpoint at each step: it is clear what has already been fixed and what has not.
- Sample audit — we collect real documents, including inconvenient ones: poor scans, multipage tables, templates from different counterparties.
- Document parsing by class — each type has its own field schema and rules for merged cells and repeating headers.
- Reference sample — we manually label 100–200 pages and split them into training and holdout parts; quality is measured by cells, not by page.
- Extraction assembly — table region detection, structure recognition, bringing values to a unified format.
- Quality validation — run on the holdout part, report with the share of correct cells and analysis of typical errors for each document class.
- Tuning based on results — we refine the rules and post-processing, then repeat the run on the same reference set so the comparison is fair.
- Handover to operation — launch scheme, exception handling, manual review queue for disputed pages, documentation, and training for your team.
What stays with you: configurations, schemas, regulations
The package is handed over not “upon delivery,” but as a working support tool: with it, your team resolves routine situations on its own, without contacting us. Below is exactly what you receive together with working table recognition.
- Integration schemas — where documents come from (email, network folder, accounting system), where the result goes, where the failure points are, and how to switch to a backup route without losing files.
- Access rules — roles and permissions: who can see source scans, who changes confidence thresholds and the field dictionary, and where the operation log is kept.
- Field dictionary and labeling rules — how columns are mapped to accounting system fields, what to do with merged cells, multiline headers, and rotated scans.
- Support procedure — frequency of quality checks, procedure for updating configuration, rollout of changes, and rollback to the previous set of settings.
- Handling disputed cases — operator instructions: what the manual review queue looks like, what to check first, and how to mark an error so it goes into retraining.
- Solution documentation — configuration description, quality metrics for each form type, list of known limitations and workarounds.
- Reference sets and checks — document samples and run scenarios that show, after any change, whether accuracy on complex tables has dropped.
Such a package allows support to be handed over to an internal team and returning to us only for further development: new document types, changes to the export schema, or accuracy improvements.
Case study: invoice and act flow before and after changes
A wholesale company receives about eight thousand invoices and acts per month from four hundred counterparties. Some arrive as electronic PDFs with a text layer, some as scans and photos of varying quality: faint impressions, a rotated sheet, a table spanning two pages.
Line items must be entered row by row into the 1C accounting environment — finance and warehouse reconcile rows, not the document total.
Before: manual entry in two shifts
Two operators processed the flow manually: opened the file, typed in the rows, checked the total. The monthly batch was closed in three working days, the error rate in numeric columns reached 4–6%, and errors were found during reconciliation, reopening the documents.
After: automatic parsing and quality control
Now the flow is divided by source type: text files are parsed directly, scans go through table recognition with restoration of cells, headers, and total rows.
Each line item is normalized — code, unit, rate, amount — and checked with control sums: the sum of rows is reconciled with the table total, and a discrepancy above the threshold goes to the operator. 84% of rows enter accounting without corrections, and about 12% of documents require corrections.
Refinements along the way
After the first weeks, we added templates for non-standard acts, parsing of multipage tables with header carryover, and processing of rotated scans. Each change was tested on the reference sample so as not to lose the achieved accuracy.
Result: the monthly batch is processed in one day by an operator and a reviewer, instead of three days by two specialists. Sum discrepancies are cut off before accounting, rather than surfacing during reconciliation.
How table recognition differs from ordinary text
Ordinary text recognition is a one-dimensional task: preserve the order of characters in a line. It is enough to correctly segment the page into characters and words, and quality is measured by the share of correctly recognized characters.
A table is two-dimensional: a value is determined by the intersection of a row and a column, and if the grid is lost, the fragment “Total 1,250” ceases to be a number with a clear meaning.
It is not the font that gets in the way, but the geometry. Merged cells, multilevel headers, an empty cell as a meaningful feature, tables without lines — all of this must be resolved into a “row — column — value” model.
A combination works: first we detect lines and block boundaries (basic algorithms for this kind of parsing exist in open-source libraries such as Camelot), then we assemble an addressable grid, and only then recognize the contents of each cell separately — taking into account that a column of numbers and a column of dates require different rules.
The key is cell relationships. A header title is linked to all values in the column, and a multilevel header produces a composite key that must be joined into a single field name.
If relationships are not restored, the text is recognized perfectly, but totals and slices in the report are calculated incorrectly, and rows land in the wrong places when loaded into the accounting system.
This leads to different validation requirements. Character-by-character accuracy ceases to be an acceptance criterion: it can be high while the columns have already shifted.
We validate structurally — consistency of the number of rows and columns, matching sums by rows and by columns, completeness of keys, absence of duplicates, and stability of the result on a repeat run over a control sample of documents.
Therefore, acceptance is built not on “text from an image,” but on the suitability of the export: a set of records with valid keys and verified relationships that can be loaded without manual correction.
How many rows shifted and how many records failed validation — these are the quality metrics for table recognition.
Technical doubts we resolve at the start of the project
Engineering doubts rarely concern the model itself — they are about input data, result acceptance, and responsibility boundaries. Therefore, before the start, we address scan quality, responsibility for errors, pilot scope, and expansion of coverage.
Our scans are poor — will recognition work at all?
First, we calculate the metric on your sample: the share of cells extracted correctly across a couple hundred pages.
Noise, skew, and a skewed grid are treated by alignment and line detection; handwritten insertions and merged characters cause a drop — we mark such fields for manual review.
If accuracy is below the threshold, we say plainly what preprocessing will achieve and what will require rescanning.
Who is responsible for an error in an extracted value?
In the acceptance scheme, a confidence threshold is set for each field: above it, the value goes into the accounting system on its own; below it, it goes to the operator review queue. An error does not silently go into accounting; it stops at control.
We are responsible for the pipeline and metric stability, you are responsible for the rule: which fields are critical and what to do on a mismatch.
What does the pilot stage look like?
We take one document type and one process area and run the entire path from upload to recording in your system. The output is a working flow, a report on the accuracy of each field, and a list of places where rule adjustments are needed. You make the scaling decision based on the numbers.
How difficult is it to expand document coverage later?
The pipeline is assembled from stages: page parsing, table detection, structure restoration, column-to-schema mapping. A new document type usually means a new template and normalization rules at the last stages, not a rewrite of the core.
Therefore, the second and third form types are connected noticeably faster than the first, and support load does not grow linearly with the number of forms.
Let's discuss the task and come back with an estimate and work plan
To come back with an estimate and work plan, we do not need the entire document archive — a slice of the real flow is enough.
The more precisely you describe which tables arrive and in what form, the fewer assumptions will remain in the plan and the faster the pilot launch will reach your pages, not demo pages.
Discussing the task takes one conversation: we review the flow, note which templates can be covered by configuration and where a separate model is needed, and agree on acceptance metrics. The minimum data set for this is as follows:
- Document types: invoices, delivery notes, statements, industry reports — what is scanned, what arrives as a digital file, what is photographed.
- Flow volume: how many pages per month and its profile — even or with peaks during reporting periods.
- Table structure: fixed or arbitrary templates, merged cells, multilevel headers, multiline line items.
- Scan quality: resolution, skew, noise, handwritten edits — this is the main factor in final accuracy.
- Where to send the result: Excel, accounting system, database, or API exchange — format and reconciliation rules.
- People's roles: who handles disputed rows and what share of manual correction is acceptable to you.
- Environment and access: cloud or internal infrastructure, information security requirements, availability of a test environment.
- Samples: a few dozen pages covering all encountered template variants.
From this set, a pilot plan is assembled: the sample size for validation, target metrics by rows and cells, stages, and working terms — we fix them before the start so that the result is accepted against pre-agreed numbers.
Describe the task in free form and attach samples — we will come back with a timeline estimate and the procedure for preparing for the start.
