Why call transcription breaks on real recordings
Open models like Whisper produce almost verbatim text on clean studio recordings and create a false impression that the task is solved.
On a telephony stream, the same conversation transcription turns into mush: fragments of phrases, wrong names, utterances attributed to the wrong participant.
The first reason is recording quality. A narrowband telephone channel, echo, packet loss, and recompression cut the high frequencies by which consonants are distinguished.
Noise suppression on the PBX side finishes off quiet speech — and the model reconstructs the word by meaning rather than by sound.
The second is overlapping voices and speaker changes. The customer and the operator interrupt each other, speaker separation does not switch in time and glues two phrases into one. On short “uh-huh,” “yes-yes,” numbers, and addresses, recognition errors accumulate fastest.
The third is industry slang. The model writes tariff names, SKUs, abbreviations, and contract numbers by phonetics, not by a reference dictionary.
Without a domain dictionary and post-processing, such text cannot be loaded into the system of record — it will have to be proofread manually.
The fourth is the archive itself. If the recording is mixed to mono or made with a voice recorder near the phone, no model can restore what was lost.
That is why we start with an audit of the recordings: whether channels were recorded per side of the conversation, what happens during compression, and where speech disappears entirely. Next — how this is addressed at each level.
What changes after implementation: effects in operational metrics
The first metric that shifts is the share of conversations the operator no longer listens to manually.
Speech turns into text on the fly, and a typical request is resolved from the transcript: the operator reads the needed fragment instead of scrolling through the entire recording. Recordings that previously were never gotten around to stop being dead weight.
Reviewing the accumulated archive stops being a separate project. Search runs over utterances — by order number, service name, complaint wording — and a sample for the required period is assembled in minutes instead of days of listening.
The index is built across all lines at once, so an incident is analyzed by the customer’s actual words, not by a retelling.
The completeness of the ticket card grows through entity extraction from the transcript: address, order ID, promised deadline, reason for refusal. Fields are filled automatically and remain verifiable — each value is tied to a fragment of the conversation.
The share of tickets with empty required fields drops severalfold, and a disputed case is analyzed by a quote, not by memory.
Quality control changes form: the reviewer works not with what they managed to listen to, but with a cross-section across all lines. Systemic issues are found — scripts that operators bypass and topics customers return about repeatedly.
Review coverage is measured by the share of conversations, not by the number of hours listened, so sampling stops hiding problems.
Call transcription: batch, streaming, hybrid mode
The processing mode is chosen not by archive volume, but by what is needed at the moment of the conversation: deliver text to the operator immediately or analyze the recording afterward.
Batch processing exports recordings from telephony on a schedule and transcribes them in a batch; streaming sends audio for recognition during the conversation; hybrid mode enables both branches on different lines.
The difference is visible in output latency. Streaming transcription delivers utterances within seconds after they end — this is needed where operator prompts or script control work in real time.
Batch mode is responsible for the entire recording: it survives nightly peaks and connection drops and reprocesses failed files, but the result arrives with a delay of minutes and hours.
| Mode | When It Fits | Limitations |
|---|---|---|
| Batch processing | Archive, reporting, shift quality review | Text unavailable during the call, output delay from minutes |
| Streaming transcription | Operator prompts, script control, alert scenarios | Requires a stable channel and backup: if the connection drops, a segment of the conversation is lost |
| Hybrid mode | Contact center where some lines operate in real time | Harder to maintain: two processing branches and end-to-end reconciliation of results |
The choice comes down to two values: how many conversations happen simultaneously and how soon after the call the text must appear in the CRM.
We consolidate the results of both branches into one storage with labels for channels, speakers, and utterance times — with open recognition models like Whisper, audio and text remain on your side.
If some lines require real time and some only reporting, hybrid mode removes the need to maintain two systems.
How we implement: audit, scheme, setup, validation, handover
Transcription starts not with model selection, but with analysis of what is already recorded. On a real set of calls, you can see the share of speech overlaps, contact center noise, dialects, slang, and channel quality — this data defines the processing scheme. Below is the sequence of steps from the first listening session to handover into operation.
- Recording audit. We take a sample across different lines, operators, and ticket types, and calculate the share of speech, overlaps, and unintelligible fragments. The result is a report with an estimate of achievable accuracy and a list of problematic groups.
- Processing scheme. We decide where recognition should sit: streaming during the conversation or after it ends. We fix speaker separation, the output format with timestamps, retention periods, and the link between text and the recording in the system of record.
- Dictionary setup. We extend the base lexicon with product names, job titles, abbreviations, and typical errors from the audit. For bilingual lines, we configure language switching within a phrase, and we keep dictionary versions for rollback.
- Connection to the source. We pull recordings from the system, configure the queue and retries on failures, and delimit access: the call text is available only to those who have the right to the recording itself.
- Quality acceptance. We run a sample that was not used in setup and compare it with manual transcription by the share of correct words and key entities — numbers, addresses, amounts. The threshold is agreed in advance and recorded in the protocol.
- Handover into operation. We provide regulations, the scheme, dictionaries, support instructions, and a dashboard with daily quality; for the first week, we accompany the system and analyze deviations.
Delivery package: configurations, schemes, and regulations
The delivery package is not a folder of files, but a working system that your team runs on its own after handover. Configurations are edited without us, schemes explain what flows where, and regulations define behavior during a failure.
We provide every artifact in a format you can maintain with your own resources — without proprietary editors and notes that are “clear only to the author.”
- System configuration — telephony connection parameters, processing queue, recognition confidence thresholds, retention periods for recordings and transcripts. Values are kept separate from code: a rule can be changed without a rebuild and release.
- Data flow diagram — the path of a call from recording to finished transcript: where the audio is stored, at which step recognition starts, what checks the text passes through, and where the result goes.
- Glossary of terms — product names, departments, job titles, abbreviations, and names that are misheard without the dictionary. It is updated on the fly, without retraining.
- System documentation — service composition, integration points with the customer records system and storage, export formats, environment variables, and update procedure.
- Operations regulation — what to do when telephony or the recognition service is unavailable, how to restart the queue, how to check processing latency and export completeness, and whom to notify about a failure.
- Quality control methodology — a reference sample of calls, the procedure for measuring the error rate, rules for revising dictionaries, and criteria under which recognition is considered degraded.
We accept the package together: your engineers bring the system up on a test environment using our documentation. If a step is not described or is described in a way that requires calling the author, that is our oversight, and we fix it.
Contact center case: transcription accuracy before and after
An inbound support contact center — about four thousand calls per day, sixty operators on the line.
Transcription ran as nightly batch processing on a general model: the word error rate held at 28–34%, so the recording was only good enough to “understand what the conversation was about in general.”
A separate pain point was numeric and terminology entities. Contract numbers, error codes, and tariff names were output by the model as a random set of characters, and everything requiring accuracy was entered by the operator into the system of record by hand. Quality control was sampled: 10–12 recordings listened to per day, less than 0.3% of conversations.
What we did
We assembled a separate recognition pipeline: streaming transcription on Whisper with fine-tuning on the contact center’s own archive of recordings, an entity dictionary for the product — tariffs, codes, names — and normalization of numbers and dates before feeding them to the model.
We added speaker separation: utterances are tied to a specific participant rather than merged into one stream.
What changed in numbers
Word error rate — 6–9% on a clean channel and 12–14% on noisy mobile recordings. Manual proofreading shrank from every conversation to sampled review, roughly every twentieth one.
A whole class of errors disappeared: numbers, codes, and service names are no longer lost, and disputed charges are analyzed by a quote from the call rather than by the operator’s memory.
Continuous quality control grew from 0.3% to 100% of conversations — without expanding the department’s staff.
What to do if the recording archive is stored in an old format
An inherited archive of telephone recordings is almost never homogeneous. Some files are compressed with narrowband telephony codecs (G.729, GSM, adaptive multi-rate), some are already transcoded copies, and some are streams without a correct header with a .wav extension.
The extension and metadata cannot be trusted: the format is determined by signature and the result of trial decoding.
The key rule during transcoding is to allow exactly one loss over the entire history of the recording. The compressed stream is decoded once into a lossless master, and all subsequent processing runs from the master.
If you start call transcription from the source and save intermediate results in a compressed format, artifacts accumulate: prediction noise, ringing, eaten consonants — the model begins to confuse words that sound similar.
Upsampling does not bring back information that is not present in a narrowband recording. It is more useful to feed it to a model trained on a telephone channel than to inflate it to wideband: the accuracy is the same, while the volume and processing time are higher.
Path defects are fixed separately — level overload, automatic gain control operation, dropouts — otherwise they cause false triggers on utterances.
Archive migration is built as an idempotent process. First, cataloging: for each file, the codec, duration, number of channels, and integrity are recorded. Then checkpoints by recording — an interrupted pass does not force a restart and does not create duplicates.
Originals are kept until reconciliation with the switch log and records systems: this reveals conversations mistakenly linked to another subscriber.
Technical doubts we resolve before work starts
Before the start, the customer’s technical side usually checks not the idea of transcription itself, but how it joins with the infrastructure: where the recordings physically reside, what leaves the perimeter, how the system behaves on their vocabulary, and what happens if quality turns out lower than expected. Below are answers to what most often comes up during approval.
Where is the system hosted, and do recordings leave our perimeter?
We deploy in the customer’s environment on their infrastructure or in a dedicated environment in a chosen region — audio is not sent to external recognition services.
Access to recordings and texts is delimited by roles, requests to storage are logged, and fragments such as card numbers and passport data are cut out before the transcript is saved.
How does the system handle our vocabulary — product names, SKUs, names?
Base recognition works from day one, but we improve domain vocabulary separately: we build a dictionary of terms and abbreviations, add it to the model, and validate it on a labeled sample of real calls.
Usually two or three iterations are enough for SKUs and names to stop being recognized syllable by syllable.
What is considered pilot success, and how do you measure it?
We calculate the error rate in words and in key entities — order numbers, amounts, dates — on a holdout sample that was not used during setup.
We also separately look at how many calls need additional manual labeling: if their share drops severalfold relative to manual transcription, the task is solved.
What happens if the quality does not satisfy us?
We launch in shadow mode: the system transcribes the stream in parallel with the current process, results are compared, but nothing is replaced. We switch over in traffic fractions — from one channel to the rest, with the ability to return to manual processing at any moment.
What do you need from us to start, and how long will it take?
We need access to call recordings from a past period, an understanding of how they are stored, and one contact on the telephony side. Using this data, we set up the dictionary and metrics, then run a shadow pass and gradually enable it. Let’s discuss your task — we will come back with an estimate and a stage-by-stage plan.
Let’s discuss the task and come back with an estimate and plan
To come back with a project estimate and implementation plan, a lengthy brief is not needed — one review session with your engineer or department head is enough.
We are interested in where call recordings currently reside, how telephony is set up, and where the finished text should go: into the system of record, storage, or a separate quality control interface. That is enough to see bottlenecks before work begins.
In return, you receive a document you can use to make a decision without additional clarification:
- flow diagram: from recording in telephony to text in your system, with storage locations for audio and transcripts;
- quality requirements: target share of correctly recognized words, speaker separation accuracy, dictionary of terms, names, and people’s names;
- implementation plan by stages: a pilot on one department, quality measurement on real calls, expansion to the remaining lines;
- list of data and access needed from your side and the responsible person who accepts the result of each stage;
- personal data policy: where audio and text are stored, retention periods, deletion on request;
- project estimate: labor effort by stage and support composition after launch;
- acceptance criteria — what exactly we consider a working solution.
Next step — describe the task: how many lines and calls per month, what telephony you use, and what you currently do with transcripts. We will come back with an estimate and implementation plan.