Automatic transcription of medical dictations with WER 2-4%

Automatic transcription of medical dictations: where it breaks

Automatic transcription of medical dictations breaks not on the recognition model, but on the input data. A doctor dictates between appointments — continuously, without pauses at word boundaries, swallowing endings and saying numbers on the fly.

The model receives a stream of sound without anchor points and fills in phrases by probabilities, and in medicine such completion is more consequential than an empty line.

Noise in the appointment recording is the second reason.

An open door, conversation in the office, rustling paper, dictation over the operation of equipment: the useful signal and interference arrive at the same level, and the accuracy of medical speech transcription falls not smoothly, but collapses.

This also includes different formats of incoming recordings — voice recorder, headset, telephone channel, attachments from several systems: the same phrase from different sources is recognized differently.

The third is professional vocabulary. Drug names, dosages, body sides, abbreviations, and Latin terms are in the doctor’s dictionary but are rare in training sets.

An unintelligible doctor’s dictation turns into a similar-sounding word, and data loss during transcription is not visible to the eye: the text looks coherent and plausible.

The consequence of all three failures is the same — manual proofreading of dictations. While every line has to be checked against the sound, time is not saved: there is a transcription, but there is no trust in it. Next we analyze how this is closed — from audio preparation to checking against terms and rules.

Implementation effect: time to a finished document and quality control

While a dictation goes into a common pool for transcription, the finished document appears by the next morning: the appointment has ended, and the text is still waiting for a typist.

After moving dictations to automatic transcription, the draft is ready by the end of the appointment, and document preparation time is reduced from hours to minutes — signing and uploading to the record happen the same day.

Quality is visible by the share of manual editing. On a fine-tuned medical corpus, the error rate stays in the 2–4% range — a level comparable to Whisper — and errors are concentrated in rare terms, surnames, and dosages.

The editor checks not the entire text but the highlighted segments: the share of manual editing drops from full rewriting to selective proofreading.

Completeness of the appointment record no longer depends on whether the doctor managed to dictate everything before the end of the shift. The draft is formed immediately, audio is stored next to the text, and missed fragments are added from it.

Access control to records is built on roles and an access log: you can see who and when opened a specific document; export outside the perimeter is closed.

The main effect is a reduction in the class of failures. A whole group disappears: lost dictations, unreadable transcriptions, documents stuck in the typing queue. What remains is a targeted correction of a term for a few seconds instead of a retyping cycle.

Connection options: local perimeter, cloud, mixed mode

The choice of scheme is determined not by convenience but by where the recording is allowed to be stored and processed, and what load peaks need to be sustained.

We work with three options: a fully local perimeter, cloud transcription, and a mixed scheme in which audio stays inside and only a depersonalized stream goes out for recognition. Below are the applicability conditions and limitations that should be planned for in advance.

In practice, the scheme can be changed without reworking the integration: the dictation intake point and the output format remain the same; only the processing route changes.

If it is unclear which option will withstand your data isolation and load requirements, describe the task — we will come back with an assessment and a plan.

How implementation proceeds: from parsing recordings to handover

The order of steps here is not a formality: until real dictations are analyzed, any model tuning is work blind. First we collect a corpus of recordings and understand what we will have to deal with, and only then do we tune recognition and test it on data it has not seen.

1. Audit of appointment recordings. We export dictations from different departments and different doctors: we assess sound quality, speech rate, the share of Latin terms, abbreviations, and drug names. The output is a map of problem areas, not general assumptions about quality.
2. Data flow scheme. We determine how audio enters processing, at which step it is depersonalized, how the finished text returns to the medical information system, and who has access to it.
3. Terminology dictionary. We collect an agreed list of diagnoses, drugs, studies, and internal clinic abbreviations with unified spelling — it becomes the basis for recognition.
4. Tuning for the clinic’s vocabulary. We adapt the model on the collected corpus and dictionary, separately working through numbers, dosages, and units of measurement, where an error is most consequential.
5. Testing on a control sample of recordings. Fragments that did not participate in tuning are run through the solution and the word-level error rate is calculated; disputed places are checked manually so as to see real accuracy, not an average from a report.
6. Handover of the solution to the customer. We deploy it in the clinic’s perimeter, hand over documentation and the dictionary replenishment procedure, and train employees — after that, glossary support is handled by the clinic itself.

What remains with the clinic: dictionaries, rules, regulations

The clinic receives not only a working service but also documents by which it can develop the system on its own — without us. Everything is formatted as working files in understandable formats, not as a project report.

• Dictionary of medical terms — the department’s specialized vocabulary: drugs, abbreviations, names of techniques, doctors’ abbreviations. The file is editable: a new name is added with one line, and the next dictation is already recognized with it.
• Transcription editing rules — what counts as a recognition error and what is a speaker’s slip; how to mark a disputed fragment and when to add a correction to the dictionary and when to leave it in the appointment text.
• Procedure for working with drafts — how long a recording remains unconfirmed, who proofreads it, what happens to drafts without confirmation, and how the fact of a doctor’s review is recorded.
• Recording processing scheme — the path of audio from dictation to medical history, indicating storage locations and points where rollback to the original recording is possible.
• Instructions for staff — a short memo for doctors and administrators: how to record a dictation, where to view status, what to do in case of a failure.
• Responsibility matrix and change log — who on the clinic side approves dictionary edits, whom to contact in case of an incident, and on which control dictations quality is checked after each change.

This set is enough to change the dictionary and regulations independently, and to involve us only in development: expansion to new departments or connection of other types of recordings.

Case study: how the error rate and proofreading time were reduced

A clinic with a flow of several hundred dictations per day: doctors’ reports, imaging descriptions, appointment notes. At the start, quality was measured on a sample of 400 recordings — the word-level error rate was 12–15%, with drugs, dosages, and abbreviations recognized worst.

Proofreading one report took 4–5 minutes, and processing covered about half of the recordings; the rest were transcribed manually.

What changed

We collected a corpus of dictations by department profiles and a dictionary of drugs, Latin names, and abbreviations accepted in the clinic.

Recognition was built on the open Whisper model with domain adaptation, and after it we placed a post-processing layer: normalization of dosages, units, and dates, bringing the text to the medical system’s templates. Dictations go through a priority queue — urgent reports are processed first.

What the measurements showed

On the flow, the error rate stays in the 2–4% range: high-volume profiles give the lower bound, rare ones the upper. Proofreading was reduced to 1–1.5 minutes per report, that is, three times less time per document, and coverage grew to all recordings, including night shifts.

We separately established a control loop: once a week we analyze a sample of fresh dictations; errors go into the dictionary and into fine-tuning. Without such a loop, quality drops on new drugs and changes in diction — with it, it holds on the flow.

How is live speech recognition different from archive transcription?

These are two different modes of working with the same sound, and the difference is not in the recognition model but in when the text must be ready. In real-time processing, the result is needed within seconds after the doctor finishes the phrase.

In deferred archive transcription, throughput is more important: the text can be delivered by morning, but the entire accumulated volume must be processed overnight.

In streaming mode, the recording is cut into short windows, the intermediate result is refined as context accumulates, and the terminology reference helps the model with rare names.

What is critical is something else — resilience to connection interruptions: a lost fragment cannot be restored after the fact, so the stream is duplicated to disk, and in case of failure the dictation goes to the archive for a repeat run.

Deferred transcription is organized as a queue: recordings go in order of arrival, but urgent dictations have higher priority — a dictation from the admission department overtakes yesterday’s array.

The scheduler distributes the load, and updating the terminology reference triggers a repeat run only where questionable sections remained.

Both scenarios work in one system, so the 2–4% recognition error rate holds in any mode: what differs is not quality but the time until the text is ready.

Therefore the mode is chosen not by accuracy but by where the result is needed — at the patient’s exam table or in the archive by the start of the shift.

Technical questions: dialect, noise, specialization

Before the start, the customer usually brings a list of engineering doubts: “what if the doctor speaks with an accent,” “what if it is noisy in the office,” “will the model handle our narrow terminology.” Below are our substantive answers, without promises of “universal recognition of everything.” Each risk is translated into a measurable hypothesis and tested on your recordings.

There is constant hum in the office — is that critical?

Background noise level matters, but in different ways: steady hum (ventilation, equipment) is removed by preprocessing, one-off spikes are cut off by signal level. We look at the recordings and decide: refine the model or change the microphone scheme.

We have a narrow department with its own terminology — will the model cope?

A department’s narrow specialization is covered by a separate dictionary and fine-tuning on the corpus of exactly that profile. The general model does not break — growing domains do not interfere with each other.

How are homonyms in drug names resolved?

Homonyms are separated by the reference dictionary and dosage context — form and regimen give the model enough signal. Disputed recognitions are marked with a hint so that the doctor confirms them with one movement.

How will we verify quality before any costs and timelines?

Validation on your own recordings: we take your set of dictations and calculate the error by section, dictation type, and speaker. The output is a report showing which scenario is already ready and which requires refinement.

Let’s discuss the task and come back with an assessment and plan

So that the assessment of scope and timelines is based on facts rather than assumptions, a few artifacts are enough as input.

The closer the samples are to real dictation, the more accurate the forecast for error rate, processing speed, and labor effort for dictionary maintenance.

What to send to define the transcription task:

• 30-60 minutes of recordings from the office — with the microphones and headsets that doctors actually use, including noise and different specialties;
• reference transcriptions for 10 minutes of audio: we will compare the markup with them and show how we calculate errors on words and on medical terms;
• volume estimate: number of dictations per day, average duration, peak periods, share of edits after proofreading;
• output requirements: text structure by report sections, department dictionary, format of transfer to your records system;
• restrictions on access and storage: where recordings are allowed to be processed, who gets access to audio and texts, when they are deleted;
• acceptance criteria: target error rate, text return speed, system behavior with poor sound quality.

In response, we will return a staged work plan — preparation of the recording corpus, configuration of the terminology dictionary, control proofreading by a doctor, handover of the system to operation.

It will include the project team composition with roles, timelines for each stage, and the procedure for accepting the result on your side. Describe the task — we will come back with an assessment and a plan.