Why formula recognition breaks on real documents
Formula recognition breaks not at the character level but at the structure level. A formula is a two-dimensional layout: the position of a glyph relative to the baseline determines whether it is an index, an exponent, or a multiplier.
Models that read a line left to right lose this geometry and return correct markup that describes a different expression.
Overlapping indices are the first class of recognition errors: nested powers and double subscripts collapse into a single level.
Multi-tier fractions and matrices add a second axis, where row and column boundaries are determined by gaps, and variable-height brackets make it hard to restore the markup.
Handwritten formulas make the picture worse: handwriting does not provide stable proportions, and edits over printed text are read as part of the expression.
Scan quality acts as a multiplier for all these failures. Page skew, uneven lighting, compression artifacts, and thin strokes turn an index into noise, so the binarization threshold has to be selected page by page — otherwise some characters blend into the background. Even solutions that are strong on printed expressions degrade on such files.
A separate category is the context around the expression. Continuous numbering within a paragraph, references like “see (14),” and a formula split across lines or columns require not character recognition but parsing of the document structure.
That is why we measure the share of fully matched expressions, not accuracy by individual characters: the latter metric masks a shifted index and a lost bracket.
Measurable effect: accuracy, speed, share of manual edits
We calculate recognition accuracy not by the “number of guessed characters” but by the formula as a whole: we normalize the markup and compare it with the reference prepared by a methodologist.
On a control sample of typical educational materials, single fractions and powers are recognized almost without errors, while the entire remaining error is concentrated in multi-line systems, matrices with indices, and fractions with nested brackets — that is where the share of manual edits appears, visible in the report.
We work through the local recognizer Pix2Tex: it runs on our infrastructure, pages do not go to external services, and at the output we get a confidence estimate for each fragment.
A formula below the threshold does not enter the finished document automatically — it goes into a queue for review with questionable places highlighted.
This closes the most expensive class of failures: a silently incorrect formula that reaches the editor and lives in the layout until typesetting.
The speed of processing an archive is determined not by recognition but by verification.
Batch mode processes thousands of pages in hours, and the bottleneck shifts to review: engineers annotate complex cases themselves, while simple ones pass by the confidence rule without human involvement.
The output includes the share of manual edits as a percentage of the number of formulas, the distribution of errors by type, and a set of the customer’s complex formulas, on which we accept the result and track regression after any changes.
Formula parsing options: comparison of approaches and modes
The choice of formula recognition mode is determined by three things: the nature of the source materials (printed typesetting, handwriting, low-quality scans), confidentiality requirements, and the acceptable share of errors at the output.
Formulas cannot be “corrected by meaning” — a lost exponent index or fraction boundary changes the result, so the mode is selected for the task, not “the most accurate in general.” We separately fix the output format: LaTeX for insertion into editors and MathML for the web — what the pipeline does next depends on it.
| Option | When it fits | Limitations |
|---|---|---|
| Ready-made external service | One-off parsing of small volumes, simple printed typesetting | Pages go to a third party; quality cannot be tuned to your corpus |
| Local engine | Confidential documents and a stream of pages within your perimeter | On complex notation, accuracy is lower without tuning; you need your own launch and model updates |
| Hybrid scheme | High-volume stream where an error is costly: a draft pass plus human review | An editing interface and regulations are needed — part of the work remains with the editor |
| Fine-tuning on your own corpus | Narrow notation, your own designations, many similar documents | A labeled corpus and time are required; the gain is noticeable from several thousand formulas |
We do not oppose the hybrid scheme and fine-tuning: labeled edits from the review interface become the corpus for the next iteration, and the share of formulas that passed without edits grows from run to run.
Before putting the stream into operation, we run your document set and verify the rendering of LaTeX and MathML in the editor where the formulas will be used — this way, output incompatibility is discovered during testing, not in the middle of an export.
Stages of implementing formula recognition into a document pipeline
Formula recognition breaks not on the model but on the gap between a demo example and a real corpus. A single stream includes noisy and skewed scans, vector PDFs where glyphs are laid out by coordinates, and photos from a whiteboard.
That is why the work follows a fixed sequence: first we measure the corpus, then we build the parsing scheme, then we tune and hand it over.
- Document corpus audit. We calculate the share of pages with formulas, source types, fonts, and notation languages. We record what occurs: fractions, matrices, indices, multi-line derivations. The output is a map: where parsing works immediately and where preprocessing is needed.
- Reference sample. Several hundred manually labeled pages, balanced by formula types. The metric is the share of derivations recognized entirely and with correct structure: character-level accuracy hides a lost index or nesting.
- Parsing scheme. We define the order: detecting formula regions, cropping with margin, normalizing scale and contrast, recognition, and reassembling into the document markup. We choose the engine based on its result on the sample, not on a description.
- Parsing tuning. For your corpus: region detection threshold, command dictionary, post-processing — restoring brackets and service commands, resolving conflicts with the page’s text layer.
- Validation on the reference sample. We run the entire pipeline and break errors down by class: merged indices, lost characters, incorrect nesting. A separate blind set of pages that did not participate in tuning separates real accuracy from overfitting.
- Solution handover. We provide the code, configuration, dictionaries, and operating regulations: how to connect new formula types, what to check when accuracy drops, where to view logs, and how to update the sample.
What remains with the customer: artifacts and regulations
We hand over the solution not as a service running on our side but as a kit that your team maintains without us. Each material is checked during acceptance: if the process cannot be reproduced from the documentation, the handover is not closed.
- Recognition configurations: page preprocessing parameters, confidence thresholds, and the rule for sending a fragment to manual review.
- Parsing schemes: how an image turns into a formula structure — block boundaries, fractions, indices, matrices, multi-line expressions.
- Notation dictionary: mapping of characters to the notation adopted by you, so the result matches how the formula is written in your materials.
- Quality protocol: a set of reference pages with markup and the accuracy metric that the solution must return to after every edit.
- Instructions for editors: what to do with uncertain fragments, how to mark edits, and which errors to consider systemic and which as isolated.
- Change regulations: a new formula type goes through the same quality protocol rather than modifying the configuration locally.
- Handover procedure: access, environment, request log, and the responsible engineer during the support period.
Before closing acceptance, the kit is run on your side: your specialist brings up the environment and repeats the reference run. A match with our result is the proof that the solution has indeed been handed over.
Case study: a scientific publisher and a stream of handwritten articles
An academic publisher accepted manuscripts as scans and photographs: author edits, marginal notes, two-line handwritten formulas.
The editor checked each expression against the image and rekeyed it — an article with 40–60 formulas took almost a full workday, and the proofreading queue stretched for weeks.
What we built
A formula recognition pipeline: detecting regions with expressions on the page, recognition, conversion to LaTeX, then structural validation — bracket balance, fraction nesting, index pairing, and checking characters against the dictionary and the source raster. Only fragments where the validator did not match the image remain for manual proofreading.
At the start, we took the Mathpix service as a quality benchmark and against it selected a model for handwritten scans: for printed pages, one branch was enough; for drafts, fine-tuning on the editorial team’s own markup was needed.
Metrics before and after
| Indicator | Before | After |
|---|---|---|
| Time to proofread expressions in an article | about 6 hours | 40–50 minutes |
| Share of formulas with manual edits | 100% | 12–15% |
| Errors surfacing after typesetting | every fifth formula | isolated cases |
We completely eliminated the classes of errors that previously were caught only at proofreading: swapped superscripts and subscripts, lost fraction bars and brackets, and substitution of similar characters.
Validation cuts them off before the expression enters the layout — the editor spends time on meaning, not on checking characters.
How does formula parsing differ from ordinary text recognition?
A linear recognizer assumes that text is linear: characters go left to right, and the relationship between them is determined only by order.
A formula is not structured that way — the numerator sits above the denominator, an index goes up or down from the base, a radical covers an expression with a bar, and a matrix defines a grid.
The same characters in different geometry are read as different expressions, so text recognition returns a string of characters but not the meaning.
Hence three layers of work on top of ordinary recognition. The first is detection and classification of characters: thin strokes, serifs, and italic style interfere just as much as handwritten input.
The second is markup of the expression structure: relative positions and sizes determine fraction boundaries, root degree, summation limits, and bracket nesting.
A shifted fraction boundary changes the meaning entirely, so the structure is restored and checked separately from the characters.
The third layer is the character dictionary and notation grammar. Recognized characters are matched against known operators, Greek letters, and functions, variables are linked to their indices, and the assembled structure is checked for syntax.
This removes classic ambiguities: “l” and one, the letter x and the multiplication sign, zero and uppercase O. The dictionary limits the hypothesis space, and validation cuts off branches that are formally assembled but mathematically meaningless.
Context picks up the remaining ambiguity: neighboring characters and the overall shape of the expression suggest whether we have a limit or an index. Models that see the expression as a whole rather than sliding along the line make noticeably fewer errors in such places.
At the output, ordinary recognition gives text that still has to be reread and retyped, while formula parsing gives an expression tree ready for rendering and computation.
Technical doubts: quality, validation, implementation
Printed typesetting and handwritten derivations are two different modes, and they cannot be covered by a single pipeline.
For printed formulas, a ready-made parsing service such as Mathpix is suitable, but it sends the page to an external server — for a closed perimeter this is unacceptable.
In a closed perimeter, we deploy the model on your infrastructure: documents do not leave the perimeter, and the processing log remains with you.
We fine-tune handwritten input on your samples: several hundred labeled pages provide gains exactly where the base model is confused — similar indices, lowercase and uppercase, and the derivative sign.
Manual proofreading does not disappear but becomes targeted: the editor does not look at everything but at fragments below the confidence threshold, which the system highlights itself.
Can we do without external services?
Yes. The model is deployed on an isolated server without network access, and markup and fine-tuning take place within the perimeter.
Is continuous manual proofreading needed?
No. Confidence is calculated for each fragment, and the editor checks only disputed places — usually fractions of a percent of the volume. We leave the continuous pass for final verification.
What should be done when the model makes mistakes on new formulas?
Editor edits automatically become markup and enter the set for model fine-tuning. After several cycles, the error class is closed, and the share of manual edits drops.
Let's discuss the task: what to clarify before starting work
The more precise the inputs, the fewer assumptions in the estimate and the faster we return with a work plan. The points listed below determine the formula recognition architecture — if data is missing for any of them, we will record the assumption directly in the estimate.
- Source types: scans of printed publications, photographs of handwritten notes, PDFs with a text layer, screenshots. Preprocessing depends on this — alignment, noise removal, binarization — as does model selection.
- Corpus volume: how many pages, what share of them contain formulas, how much has already been manually labeled. This determines the labor intensity of data preparation and the training mode.
- Target metrics: character-level accuracy and the share of formulas recognized entirely without edits. A reference set of several hundred pages allows quality to be measured numerically rather than by eye.
- Perimeter requirements: the customer’s closed server, cloud, or hybrid scheme; whether offline mode is needed, what can be transferred outside, and what to write to access logs.
- Result format: LaTeX, MathML, or custom markup; whether formula coordinates on the page and a link to the source image are needed.
- What happens to the recognized result next: manual review by an operator, automatic syntax validation, or formula search. This sets the quality threshold below which the result is not accepted.
- Handover format: a deployed perimeter, trained model, training scripts, labeled reference set, documentation, and support instructions.
Describe the task within these boundaries — we will return with an estimate, work plan, and architecture options.
