AI in healthcare: where a pilot project breaks
An AI pilot project in a clinic most often stalls not on the model, but on the data.
The study archive was collected over years: images from different machines, exports from old systems, some are scans of films without original quality, descriptions are free-form and do not match in structure between departments. Until the dataset is brought to a uniform form, no algorithm produces a reproducible result on it.
The second point of failure is labeling. When labels are assigned "by eye" from the report text, without a common protocol and without monitoring agreement between physicians, the model learns from contradictions.
One specialist considers a finding significant, another considers it borderline, rare classes are almost unrepresented, frequent ones dominate. The quality of medical data sets the ceiling: no matter how many layers you add later, you cannot rise above it.
The third scenario is a model trained on someone else's data. On a public dataset, the metrics look convincing, but the clinic has a different fleet of machines, a different population, and a different proportion of normal cases.
When transferred, accuracy drops, and the growth in false positives directly increases the physician's workload.
And the fourth is the lack of validation on your own flow. Without reviewing disputed cases and without measuring the rate of misses and false alarms on a local sample, diagnostic errors go into the clinic unnoticed.
We break down how this is closed step by step: from archive inventory and a labeling protocol to physician oversight on every model update.
AI in healthcare: measurable effect on patient flow
The introduction of AI in healthcare changes not so much diagnostics itself as the throughput of the department.
An image from the modality enters the triage queue, the model marks suspicious areas and raises the priority of studies where pathology is more likely, and the result goes into the same DICOM stream that the physician's workstation already reads.
The effect of AI implementation is visible in four metrics that we capture before and after launch: study review time, the miss rate in triage, the volume of repeat orders, and the workload of the reporting physician.
Published clinical summaries Lancet Digital Health on screening programs give a comparable order of magnitude — review time is reduced by 20–40%, and the miss rate falls severalfold relative to manual queue distribution.
Diagnostic accuracy grows not "on average across the model", but in subgroups.
We calculate sensitivity separately by machine type, image quality, age group, and comorbidities: if there is a failure in a narrow subgroup, the trigger threshold is raised specifically for it, not for the entire queue.
The workload on physicians is redistributed, not eliminated. Typical cases without findings are closed faster, and the freed-up time goes to complex studies and discrepancy review.
The flip side is that excessive sensitivity generates unnecessary follow-up examinations and an increase in repeat orders, so we calibrate the threshold based on the cost of a false miss and a false alarm together with the clinical team.
A model without built-in control degrades when the machine or imaging protocol changes.
Therefore, the delivery includes monitoring of feature distribution and regular re-validation on fresh data — the effect lasts for years, rather than appearing only in the first month after launch.
Deployment formats: cloud, clinic environment, hybrid
The deployment scheme is chosen not by development convenience, but by where patient data physically resides and which access channels to it are permissible.
The three options are not "better and worse", but different sets of trade-offs between speed of change, perimeter control, and maintenance cost.
Cloud deployment provides quick access to computing power for training and peak load, but requires either de-identifying data before transfer or placing it in an environment with confirmed compliance with medical data protection requirements.
An internal environment removes the issue of external transfer, but is limited by the clinic's resources and makes each model update a separate coordinated procedure.
A hybrid scheme separates the flows: source studies remain inside, while de-identified samples go to external computing power.
| Scheme | When it fits | Limitations |
|---|---|---|
| Cloud deployment | Fast start, variable load, de-identified datasets | External transfer requires de-identification and an agreement with the provider; dependence on the communication channel |
| Internal environment | Data cannot leave the perimeter, strict protection requirements | Capacity is limited by the clinic's equipment; model updates are slower, and a dedicated standby environment is needed |
| Hybrid scheme | Heavy training outside, sensitive data inside | Two support environments; a synchronization regulation and control at the junction are needed |
In practice, the choice is fixed in a document: which categories of data are stored and processed where, who has access, how rights are demarcated, and how the access log is maintained.
This same description becomes the basis for audits — and it does not change when the model inside the system is replaced.
Implementation stages: from data audit to operation
In a clinical environment, a solution is validated not only by metrics, but also by whether it fits into real patient intake.
Therefore, the stages proceed sequentially: each next one relies on an artifact of the previous one — the audit report, the labeling protocol, the agreed integration scheme.
A skipped stage comes back as a discrepancy on the physician sample or an integration failure already in pilot operation.
- Data audit. We review the archive: volume, share of labeled cases, gaps, heterogeneity of sources, and quality of de-identification. The output is a report showing whether the data is sufficient for the claimed quality and what will need additional labeling.
- Clinical problem statement. Together with physicians, we formulate the target outcome and metric: which error we consider costly, and at what sensitivity it is acceptable. We record who makes a decision based on the output and how.
- Solution and integration scheme. We determine where the model runs — in a secure environment or on a dedicated node, how it receives images, and how it returns the result to the physician's interface. The scheme is agreed upon before the first line of code.
- Sample preparation and tuning. We split the data into training, validation, and test sets without patient overlap — otherwise the metrics are inflated. We train, select thresholds on validation, and freeze them before the test.
- Validation on the physician sample. We run it on a blind sample labeled independently, and review every discrepancy case. We look at metrics broken down by subgroups: machine, clinic, patient profile.
- Transfer to operation. We hand over the regulation: data drift monitoring, degradation control, model update procedure, staff instructions, and incident review procedure.
What we hand over: models, labeling, regulations, documentation
A solution that cannot be handed over to another team and validated without us is not an implementation, but dependence on the contractor.
Therefore, we close every project with a package of artifacts: configurations, data flow diagrams, access rules, staff instructions, and a support regulation. Specifically, what remains with the clinic:
- Model configurations and artifacts. Versions, launch parameters, reproducible environments — training and inference are brought up on a new environment without our involvement.
- Data flow diagrams. Where studies come from, at which step identifiers are removed, where data is stored, and how it enters training and quality review.
- Data labeling. Labeling instructions, rules for disputed cases, the share of examples that passed a second review, and the agreement metric between annotators.
- Access rules and roles. Who works with de-identified datasets, who sees source data, who has the right to launch retraining and change thresholds.
- AI solution documentation. Purpose, inputs and outputs, error classes, confidence levels, and known limitations by patient subgroups.
- Staff instructions. How a physician accepts a model suggestion, what to do at low confidence, and where to report a discrepancy with their own conclusion.
- Monitoring regulation. Which quality metrics and sample shift metrics are tracked, at what frequency, who responds, and at what threshold stops use.
We verify the handover at acceptance without formalities: an external engineer uses our documents to bring up the environment on test data and explains why the model behaves this way and not otherwise. If that works, there is no dependence on us.
Case study: reviewing study flow in a diagnostic center
A diagnostic center received several hundred studies per day: images and reports were piled into a common archive, and the queue for review lived in a duty officer's spreadsheet.
The physician manually selected a series, compared it with a previous examination, and typed the conclusion in an editor. Urgent and routine cases went in a single flow — a suspected pathology sometimes waited longer than a follow-up image.
What was changed in the pipeline
A processing layer was placed in front of the physician: it receives studies in DICOM format, brings series to a uniform form, filters out technical defects, and sorts the queue by priority.
The model marks areas of interest and returns them together with the source series — this is a hint for the eye, not a diagnosis. The physician signs the conclusion; the model's decision does not go into the patient record.
Priority is calculated from a combination of features: referral, age, repeat status, and model findings. Urgent cases automatically surface to the top, while routine ones go into background review. If the model is unavailable, the queue degrades to manual sorting rather than stopping.
Operational metrics: time from image to signed conclusion — from 2-3 days to 4-6 hours on the urgent flow; 94% of studies go through automatic prioritization; misses on repeat images, where pathology is visible only over time, decreased by about threefold.
Manual sorting almost disappeared from the duty officer's work — the freed-up time went to complex cases.
How does a medical model differ from a general-purpose one?
A general-purpose model is trained on a general corpus of texts and images: it handles language well, but does not know the department's local terminology, units of measurement, or the structure of discharge summaries.
In a clinical flow, this produces predictable failures — an abbreviation like "CRP" is parsed as a random set of letters, reference intervals are substituted from general logic, and the answer sounds confident without support from fact. The error is noticed not by the user, but by the physician — already at the review stage.
A medical model differs not in size, but in that during training it sees labeled clinical data: diagnoses, prescriptions, laboratory results, examination protocols.
Training on clinical data ties it to real reference intervals and code directories, so answers come in the clinic's terms, not in general formulations.
Fine-tuning on a local corpus is not retraining from scratch: representations are adapted on "text — diagnosis code" pairs, and the abbreviation dictionary is aligned to the department.
Before that, records are depersonalized and split by patient so that one person's data does not end up simultaneously in the training and control samples. Structured exchange, for example via HL7 FHIR, removes part of manual labeling — the fields are already labeled by meaning.
The boundary of applicability is concrete: the model is reliable where there is data for fine-tuning and degrades on rare diagnoses and new practices.
Therefore, the output includes an explicit refusal "insufficient data" instead of a plausible guess, and quality is controlled by patient subgroups and reassessed when clinical guidelines change.
Labeling, physician oversight, and boundaries of applicability
Data arrives heterogeneous: images from a CT scanner, free-form conclusions, exports from laboratory systems, signals from devices.
We bring them to a single schema, define a class dictionary and labeling protocol, and send disputed cases to arbitration by two independent specialists. We check the quality of the dataset by agreement between annotators, not by the volume of exported data.
In the clinical environment, the model ranks and highlights, while the final decision remains with the physician. We secure responsibility for the output with a regulation and a log: we save what the model proposed, which specialist confirmed the result, and on what basis.
This leads to interface requirements — visible confidence, a link to source data, and a separate "disagree" action.
Who is responsible for an erroneous output?
The model is a decision support tool, the physician confirms the output, and this is recorded in the log together with the model version.
We review disputed cases together with the customer's clinical team at acceptance so that the regulation matches the department's real practice.
What happens when equipment or imaging protocol changes?
We validate the model on new data before putting it into operation, and if metrics drop, we leave it in shadow mode. We store weight versions separately — rollback to the previous one takes minutes.
Let's discuss the task and come back with an estimate and work plan
To come back with an estimate and a plan, rather than general words, at the first conversation we need a minimum: how the process is structured now, what data it runs on, and who on the customer's side will confirm the correctness of the result.
A de-identified export for one to two months and access to a test environment say more than an hour of descriptions — they show the class distribution, the miss rate, and the real variability of records.
The conversation takes about an hour: we review the clinical or operational process, the points where time is lost, and the constraints — personal data requirements, mandatory specialist review, and integration with an existing system.
We immediately record what we consider success in measurable terms: the share of automatically processed cases, completeness and accuracy on the control sample, and conclusion preparation time.
Then we come back within a few business days with a package:
- project estimate: stages, team effort, dependencies on data and external systems;
- solution scheme — where data is processed, what goes to manual review, and how rollback is arranged;
- work plan by stages with points where you can stop without losing the result;
- pilot implementation plan for one department or a narrow scenario, with metrics captured before launch;
- team composition and roles on the customer's side, including an expert physician for labeling and acceptance;
- acceptance criteria and the procedure for bringing the solution into production.
Pilot implementation is the first step, not a separate product: its result determines whether we expand the system or change the approach. Describe the task — we will come back with an estimate and a stage plan.
