Why AI EHR Falls Apart on Real Clinical Data
On a test set, the system behaves predictably: fields are filled, reference data match, and the model confidently labels diagnoses. On a live stream, this breaks in the first 24 hours.
AI EHR implementation runs up against not the choice of model but the quality of clinical data coming from a dozen sources in different formats.
The laboratory returns a structured response via its own channel, while the conclusion of an instrumental study arrives as a scan with skewed lines and a stamp.
OCR of conclusions on such scans confuses dosages, units of measurement, and dates—and then this goes into the chart as fact. The physician's free text adds its own: abbreviations, local department templates, formulations without a reference date.
On top of that come manual entry errors at the front desk: insurance policy, date of birth, duplicate patients. An error does not remain local—it breaks the linkage of episodes, the medical history is assembled incompletely, and clinical check rules trigger on the wrong data.
The bottleneck in processing charts is most often not in the model but at the interfaces: a broken integration with the laboratory, a stalled message queue, a mismatch in service reference data.
Until the interface is closed, clinical data quality cannot be raised by any fine-tuned model—the physician again has to verify everything manually. We analyze how this is closed.
What AI EHR Delivers in the Clinic's Daily Practice
AI EHR is based on speech recognition and structuring what is said according to a given appointment template: complaints, history, examination, orders. The physician speaks, and the system lays the text out across fields rather than writing it as a continuous stream.
We do not choose the storage format by eye: records are stored in the HL7 FHIR model—an international standard for medical data exchange—so the chart interfaces with the laboratory, pharmacy, and external systems without manual transfer.
We measure effects before and after. The share of charts closed without manual typing at high-volume appointments reaches 70–80%, and preparation of an appointment record is reduced from 6–8 minutes to 1–2.
Chart returns for revision fall severalfold—most often they were caused by missed mandatory fields, not the substance of the record.
A separate class of input errors disappears: typos in dosages, discrepancies between the order and the chart text, lost lines when copying a previous appointment.
Integrity is maintained by checks on save: the dose is verified against weight, age, and concomitant orders, and incompatible items are flagged. The completeness of the clinical chart no longer depends on how tired the physician was by the end of the shift.
The physician's workload shifts: they review and confirm rather than type. The final word remains with the human, and any edit is recorded in the log with author and time.
Next we break down what such a system consists of and how to integrate it into the clinic's current environment.
Deployment Options: Cloud, Clinic Environment, Hybrid
The choice of deployment scheme is dictated not by infrastructure convenience but by clinic policy: where personal data processing is permitted and which outbound channels are open at all.
We handle such approvals together with the customer's IT service, because both the architecture and the operating regulations depend on the answer.
| Option | When it fits | Limitations |
|---|---|---|
| Cloud | A clinic network with high and uneven load, external compute is needed | Medical data outside the environment: a processing agreement, encryption in transit and at rest, and separate access control are mandatory |
| Clinic environment | Strict requirements for personal data storage, isolated network | Requires own compute capacity for the model; we install updates in an agreed window, and external integrations only through an approved gateway |
| Hybrid | Text processing inside the clinic, while laboratory exchange and heavy computations go outside | Two environments: reference data synchronization, role separation, and end-to-end request logging are required |
In the hybrid scheme, the boundary runs along a specific field, not the entire system: everything containing patient identifiers stays in the environment, a de-identified task goes outside, and the response returns linked to the chart already on the clinic side.
Exchange with the laboratory is built the same way—by results and orders, without exporting the full history.
What is equally important in all three options: an immutable access log for charts, encryption keys remain with the customer, and if connectivity is lost the system switches to read mode and does not lose what has already been entered. We run these checks before handover and record them in the regulations.
How We Implement the System: Stages of Engineering Work
Implementation starts not with the model but with data: until it is clear how current charts are structured and who fills them out, automation produces a convincing demo and garbage in real work.
That is why we follow a fixed sequence where each stage relies on the result of the previous one and can be stopped without losing what has already been done.
- Medical data audit — a de-identified sample of charts: field completeness, share of free text, variation in wording across physicians and departments.
- Data schema — we agree on the record structure, reference data, and normalization rules, as well as what the system fills in itself and what remains with the physician.
- Integration schema — exchange points with the medical information system and laboratory, flow direction, and behavior when an external service is unavailable.
- Prototype on the archive — we run recognition and entity extraction on historical records to see errors before the clinic rather than inside it.
- Pilot in a department — the system runs "in the shadow": physicians keep charts as usual, and its suggestions are compared with the final records.
- Recognition quality check — precision and recall on a labeled sample, with critical error classes separately: dosages, allergies, diagnoses.
- Handover to operation — regulations, roles and access rights, action logging, and instructions for physicians and support service.
We fix acceptance criteria and the set of labeled examples before the pilot starts—otherwise the dispute about quality comes down to subjective assessments and the project descends into endless revision.
What We Hand Over to the Clinic: Configurations, Schemas, Regulations
The project is considered delivered not when the model confidently answers a demo query, but when the clinic operates the system without us.
Therefore acceptance includes a package of artifacts that the on-duty engineer and department head use to deploy, verify, and maintain the solution themselves.
- Deployment configurations — declarative manifests for test and production environments: connection parameters to the chart storage, load limits, and retry policy when an external service fails. The environment comes up with one command, and divergence between environments is eliminated.
- Data flow diagram — the path of a record from the physician's workstation through the integration layer to the model and back, with explicit marking of where personal data is de-identified, what is stored in the audit log, and which fields do not go outside.
- Data access matrix — a "role / field set / permitted action" table for the physician, nurse, front desk, administrator, and service engineer. Prohibitions on bulk export and the procedure for issuing temporary access are recorded separately.
- Implementation documentation — step-by-step deployment procedure, reference data migration, and control checks after each step: how to verify that the reference data matched and old records are read correctly.
- Instructions for staff — one- to two-page scenarios: how to accept a suggestion, how to reject it, how to report an error. Separate versions for physicians and for the support service, without technical details.
- Maintenance regulations — which quality and latency metrics are monitored daily, at what thresholds alerts trigger, who responds and within what time frame, and how a model update is rolled back.
- Set of acceptance scenarios — a fixed list of checks on real de-identified charts, by which the clinic itself confirms operability after each update.
Case Study: How We Reduced Manual Entry in the Admissions Department
The clinic's admissions department operated in its usual mode: the physician dictated or typed the chart after examination, the front desk entered data again, and some information settled in text notes.
Preparing one chart took 11–13 minutes, mandatory fields were filled in 68% of charts, and only 62% of them were closed on the day of admission.
What We Restructured in the Data Flow
We deployed speech recognition with a medical vocabulary and binding to the chart structure: what is said during the appointment is laid out across fields—complaints, history, examination, orders.
The medication reference suggests dosage wording, and a contradiction check prevents signing the chart if an order conflicts with allergies. The draft is ready by the end of the examination, and the physician only needs to confirm and correct.
Exchange with the medical system goes through HL7 FHIR—data goes into the same fields as before, without manual transfer.
Pilot Results
The pilot ran for six weeks across two shifts of the admissions department and covered about 1,400 admissions. The source of measurements is the medical system logs and time tracking on 120 admissions, verified with the senior physicians on shift.
Chart preparation time decreased from 11–13 minutes to 3–4, mandatory field completion increased from 68% to 97%, and the share of charts closed on the day of admission—from 62% to 94%. Disputed orders are cut off by the check before signing, not at dispensing.
The figures were taken on a live shift, not in a test environment: the flow of admissions was not stopped, and physicians worked as usual. Next we extend automatic completion to the inpatient department and discharge summaries—the same fields and the same checks.
How Implementation from Scratch Differs from Enhancing an Existing System
The fork is decided not by clinic size but by the state of the core record-keeping system.
If records are structured, there is a unified patient identifier and working reference data, enhancing the existing system is faster and more predictable: the AI layer is added on top without breaking the appointment logic familiar to physicians.
If, however, record-keeping rests on text files, scans, and manual journals, implementation from scratch will be more honest—otherwise source analysis will stretch across the entire project.
Enhancement starts with a storage audit: we look at which fields are actually filled in during an appointment, where patient duplicates are, and how export works.
Then the environment connects to the existing database in read mode and gradually takes over entity extraction from records, code suggestions, and history summaries.
Migration without downtime is built on shadow comparison: the new logic computes in parallel, we analyze discrepancies on live charts, and the switch happens department by department, not across the entire clinic at once.
Implementation from scratch is justified when sources are fragmented and there is no agreed terminology: then the data model, reference data, and coding rules are designed before the first record.
This removes the main risks of system replacement—loss of history linkage, duplicates, and a gap between admission, laboratory, and billing.
We think through rollback in advance: the old environment remains available for reading until the new one confirms the completeness of transfer during reconciliation.
We assemble the transition plan in stages: source audit, prototype on a sample of charts, pilot in one department, completeness reconciliation, training, phased rollout.
The criterion for exiting the pilot is not an overall impression but the share of correctly coded records and the time to close an admission. Describe how your record-keeping is structured today—we will come back with a transition plan and estimate.
Technical Concerns Before Kickoff: Point-by-Point Answers
Before kickoff, technical teams ask the same set of questions: where medical data resides, who is responsible for a model error, how to separate access, and how long validation on real records will take. We answer point by point.
Where are charts stored and who accesses them?
Data does not leave the customer's environment or a dedicated private segment. Every access to a chart is written to an immutable log; transmission and storage are encrypted; and de-identified samples with preserved medical history linkage are used for training.
Who is responsible for a model error?
The model formulates a suggestion; the physician makes the decision. Each output is accompanied by a reference to the source records and a confidence estimate: below the threshold, the system does not insert a value but highlights the field.
Physician edits are logged and go into fine-tuning—responsibility remains in the clinical process rather than being shifted to the algorithm.
How is access separated between employees?
Roles are separated according to the principle of least privilege: a physician sees their own patients, a resident—the assigned department, an administrator—only metadata. Bulk export requires separate confirmation, and all actions are audited.
How long does the pilot last and by what metrics is it evaluated?
We keep the pilot within one department or specialty: 6–10 weeks, with pre-agreed metrics—share of fields filled automatically, time to prepare an appointment, completeness of extracted entities. The results show where accuracy is sufficient for daily work and where human control is needed.
What to do with heterogeneous records?
Scans, free text, and different export formats pass through a normalization layer: recognition, conversion to a unified schema, and marking ambiguous cases for manual review.
This way the patient history is assembled into a coherent picture rather than remaining a set of attachments.
Let's Discuss the Task and Come Back with an Estimate and Work Plan
To move to an estimate, we need one substantive conversation: which processes are already digitized, which specialists work with charts daily, and which operations take the most time.
A description of a typical appointment and a de-identified sample of records is enough—that is sufficient to understand the scope and not build a plan on assumptions.
In response, we send a package that allows a decision to be made rather than intentions to be discussed:
- Order of stages: process and data analysis → prototype on your sample of charts → pilot on a limited group → expansion to other departments.
- Pilot composition: one department and 8–15 users, a fixed list of scenarios (record draft, field structuring, history search, discharge preparation), and a clear set of exceptions that the system does not take on.
- Control points: after data analysis, after the prototype, and midway through the pilot—with metrics for each scenario, not an interface demonstration.
- Acceptance criteria: share of fields filled without physician edits, time to prepare an appointment record, number of accesses to the patient history per visit.
- Handover format: description of integration with your chart management system, role and access schema, regulations for specialist review of model outputs, and instructions for staff.
- Responsible parties on both sides: who confirms the scenarios, who grants access to the data sample, who accepts the result at each point.
The plan is assembled for your environment, not a standard template: if an external system does not provide data in the required form, this is identified at the first stage and changes the scope of work, not the delivery timeline.
Describe the task and current process—we will come back with an estimate, order of stages, and pilot plan.
