Why AI document generation breaks on real contracts
AI document generation works confidently on a homogeneous flow and falls apart when a second counterparty with its own structure lands in the folder.
Clauses are renamed, numbering is off, appendices are moved into separate files — the rule that looked for "Subject of the contract" by heading no longer finds anything.
Contract automation begins not with a template, but with the disparity of formats, and that is exactly what eats up the first month of work.
A separate line of failures is scans without a text layer. Documents arrive as photos and PDFs after printing and signing, and extracting details from scans is critical to every character: recognition confuses similar digits, loses spaces in an account number, splits INN and BIK.
An error in one character means payment goes to the wrong place and the package returns for another round of approval.
Templates live their own lives. A lawyer edits the wording for a specific deal, the edit remains in the file, and a month later there are five almost identical versions in use, and it is unknown which one is the reference.
Manual editing of templates becomes constant, because every new condition has to be written in anew and re-checked by eye.
The cumulative effect is missed approval deadlines. The document goes along the chain, at each step someone manually double-checks the details, versions diverge between departments, and responsibility for the error is blurred.
We will analyze how this is closed: a data extraction scheme, recognition quality control, and template version management.
What changes after implementation: deadlines, errors, volume
The time to prepare a package is reduced not by speeding up typing, but by eliminating handoffs between people.
Document generation comes from the request data: values are substituted directly, the model fills the fields that cannot be derived by a formula, and the lawyer receives a finished package for review.
A package that used to take several hours to assemble and sat waiting for an available specialist is now prepared in minutes.
The class of data transfer errors disappears along with manual copying: typos in details, mixed-up amounts and dates, mismatched appendix numbering no longer survive to the final file.
Each value is taken from the source once, and completeness and arithmetic checks trigger before sending, not at signing. Errors from the source itself remain — they are cut off by validation at the input.
The volume of processing grows without hiring staff: standard contracts go through the template with almost no human involvement, and a specialist gets involved where there are deviations from the rules. The queue is regulated by routing rules, so peak load does not turn into an emergency.
Document version control becomes exportable rather than reconstructed from memory: you can see which template revision was applied, who changed a field and when, and how the previous version differed.
The share of manual operations falls — reconciliation, forwarding, and searching for the current file stop being separate work, and an audit according to ISO 9001 is done from the log, not by reconstructing events.
Operating modes: template, language model, hybrid
Choosing a mode is a choice between predictability and flexibility, and on a live document flow it is almost never just one.
Standard deals are cheaper to assemble by substitution into a rigid contract template, free generation by a model helps on non-standard sections, and the variable part — amounts, deadlines, details — is safer to keep under verification. Below is how these three modes differ in practice.
| Option | When it fits | Limitations |
|---|---|---|
| Rigid contract template | Mass homogeneous documents with a stable structure and a fixed set of fields | Wording does not adapt: any deviation from standard terms goes to the lawyer manually |
| Free generation by a model | Drafts, letters, explanatory notes, descriptive sections for a specific deal | The wording changes from run to run; without external verification, details and references to appendices are not guaranteed |
| Hybrid scheme | Contracts with a variable part: the framework and details are fixed, the descriptive part is assembled by the model | Requires described field validation rules and a test set for each template revision |
In the hybrid approach to generation, the framework, section numbering, and details come from the template and reference books, and the model fills only those blocks where wording for the situation is needed.
Deterministic output is ensured not by the model, but by field validation rules: requiredness, allowed format, reconciliation with the accounting system, and referential integrity of appendices.
If a rule is not met, the document does not go for signature, but returns to the assembly step with an indication of the specific field.
Work order: from parsing the array to delivery
The order is fixed, because each subsequent stage relies on a verifiable artifact of the previous one. This way, at any step it is clear where exactly the error is — in the parsing rules, in the model, or in the source data — and not "something is wrong at the output".
- Document flow audit. We look at which types of contracts and accompanying papers actually circulate in the company, in what formats, how many of them are scans and tables. The output is a map of types with shares and a list of fields that need to be extracted.
- Designing the processing scheme. We break down each type along the route: where parsing is done by rules and dictionaries, where the model is connected, at what point the document goes to manual review. We fix what is considered successful parsing.
- Setting up field extraction. We create counterparty reference books, detail templates, and rules for normalizing dates, amounts, and numbering. For each field we set a confidence threshold: below it, the document does not proceed further without a human.
- Assembling generation templates. Standard documents are assembled from verified blocks with substitution of fields from the card, and invalid combinations of conditions are cut off before export.
- Validation on historical examples. We run the array of past contracts and compare the result with what a human once filled in. We analyze discrepancies and close them with rules or adjust thresholds.
- Handover to operation. We provide processing regulations, a description of the verification queue, configured quality monitoring, and employee training. After that, support proceeds by metrics: the share parsed without edits, typical errors, and the volume of the manual queue.
Delivery package: templates, rules, regulations
The delivery package is not a folder with templates, but a working system that can be handed over to another team without verbal explanations.
Each artifact has a version, an owner, and a change procedure, so new deal types are added by editing settings, not by rewriting the logic.
- Contract template library — divided by deal types, fields are marked as mandatory, conditional, and optional. The template version is fixed in the issued document, so you can always see which revision it was assembled from.
- Field extraction and validation rules — describes what is considered a value, which formats of dates, amounts, and details are allowed, and what the result is reconciled with. The trigger threshold and verification strictness are configured without changing code.
- Document flow scheme — the route from loading a draft to issuance: data extraction, verification, approval, signature. The scheme marks the points where a human gets involved and what data they see.
- Exception handling regulations — what to do when a field is not recognized or conditions conflict: queue, responsible person, response time, decision recording. A resolved case becomes a new rule, not a one-off edit.
- Employee instructions — roles, typical scenarios, actions when there is a discrepancy with the counterparty registry. Written so that a new person can start independent work without support.
- Decision log and metrics — the share of documents that passed without manual edits, cycle time, top reasons for deviations. They show where the bottleneck is and whether automation should be expanded.
- Package update procedure — who approves a new template or rule revision, how to test the change on historical documents, and how to roll back to the previous version.
Let's discuss your task — we will come back with an estimate and a plan.
Case study: contract preparation before and after implementation
Before implementation, the contract package was assembled manually: texts lived in file copies, and each edit was rolled out across them separately.
A package of 60 documents took 3-4 working days, and at the end of the quarter, when twice as many requests came in, the queue stretched to a week.
What measurements showed before
- Returns for rework: about a third of templates went for a second round — discrepancies in details and diverging wording.
- Peak load: the department closed the end of the month with overtime, up to 40% of time went to reconciling copies, not legal work.
- There was no single point of edits: changing a clause required manual synchronization across the entire document set.
What was changed and what it gave
Templates were moved to a common library: versions, conditional blocks for the deal type, and mandatory fields became unified.
The draft is assembled by generation on OpenAI and then passes verification by structure and details — the model fills variables, rules cut off what does not pass.
Assembling the package takes 3-4 hours instead of 3-4 days. Returns for rework fell from a third to 8%: discrepancies in templates no longer arise — an edit is made in one place and inherited by all. The department gets through peak load without overtime, the queue grows linearly.
Throughput grew approximately threefold with the same staff, and the freed-up time went to approving non-standard terms.
How does template assembly differ from model work?
A template is deterministic assembly: values are substituted into pre-marked fields, the structure and wording remain unchanged. The result is predictable to the character, and quality control comes down to checking the input data, not the finished text.
The model works differently: AI document generation builds wording for the context and pulls meaning from unstructured places.
This is the only way to cover documents that do not reduce to fixed fields, but flexibility has a price — a human checks the result, and the volume of edits depends on the document type.
| Mode | When it fits | Limitations |
|---|---|---|
| Template assembly | Standard contracts, acts, and invoices with a stable structure | Does not cover deviations from the form — a non-standard case is edited by a human |
| Model generation | Letters, claims, descriptions, free sections | Requires human verification, quality variance is higher |
| Hybrid | Framework and mandatory blocks — from the template, variable ones — from the model | More complex rule marking and configuration |
The cost of quality control is the main selection criterion. Where the wording must match literally (details, dates, subject of the contract), template assembly is cheaper: data is checked, not text.
Where language and adaptation to the situation matter, human verification is inevitable — its volume is planned in advance, selectively or by confidence thresholds.
The rule is simple: if a document is described by a set of fields, it is a template; if the meaning rests on text, it is a model; if the structure is stable but the content changes, it is a hybrid.
Common doubts: accuracy, verification, and document volume
Objections to such projects are almost always technical: how the system will behave on poor scans, whether the wording will 'float', who sees drafts, and who is responsible for issuance.
It is precisely by these boundaries that it is determined where AI document generation is fully appropriate and where a manual step is mandatory.
What happens with poor-quality scans?
Recognition proceeds in two passes: first text, then structure — details, tables, marks.
If confidence for a field is below the threshold, the document does not go to generation, but enters the manual verification queue: the operator sees the highlighted fragment directly on the image. A recognition error does not turn into an error in the contract.
Will the wording not start to 'float' from document to document?
The text is assembled not by free retelling by the model, but according to an approved template with a fixed structure and a dictionary of wordings.
The template is versioned, the version is fixed in the document card — for any deal you can see which revision the text was assembled from.
Who sees drafts and how is access delineated?
Drafts remain inside the customer's perimeter, only finished files go outside by an explicit action. Access is built on roles: the author sees their documents, the approver — only those sent to them, the administrator — the action log. Every opening and edit is recorded.
Who is responsible for the issued document?
Issuance is always up to a human: the system prepares the draft and substitutes data, while signing and sending are up to the employee with the required role. The log records who changed what before issuance, so resolving a disputed situation takes minutes, not days.
Let's discuss the task: what is needed to estimate timelines
To calculate timelines, a task description is not enough — the quality of AI document generation is influenced not by the models themselves, but by templates, substitution rules, and where data is allowed to be processed.
Send the materials from the list below, we will review them and come back with an estimate and a work plan: stages, set of checks, and points where your side's participation will be needed.
- Document samples — 10-15 real files for each type: with wording variants, appendices, footnotes, and manual edits by managers. They show where the template holds shape and where text generation is needed.
- Template map — which documents are issued, which fields are substituted automatically and which are filled by a human, and what happens under non-standard deal conditions.
- Data sources — where field values come from (accounting system, registry, email), in what form, and who is responsible for their currency.
- Perimeter requirements — where documents are allowed to be processed, whether an isolated perimeter is needed, and what information must not leave your infrastructure.
- Roles and access — who initiates generation, who approves, who signs, and what rights each role needs.
- Volume and flow — how many documents per month and what the peak looks like, in order to estimate load and speed headroom.
- Acceptance criteria — on which set of documents we consider the result correct, who performs the check, and what share of edits after generation is acceptable.
If some materials are not at hand — describe them in your own words, we will clarify the rest in conversation. After the review, we will come back with a work plan and an estimate by stages.
