Speech synthesis: where quality breaks and what the client notices
Synthesis quality is judged not by timbre but by prosody: where pauses are placed, which word receives phrasal stress, how tone changes by the end of a phrase.
A model trained on even narrator reading maintains the same intonation, and the listener stops distinguishing semantic blocks. In a voice menu or a long instruction, this results in loss of meaning by ear: the person turns off the audio and returns to the text.
Another class of failures is phonetics at junctions and text preparation. Endings and prepositions are swallowed at fast pace, names and abbreviations are read letter by letter, numbers take the wrong case.
Formally the phrase is pronounced in full, but it is unintelligible by ear, so we check not only acoustics but also normalization before feeding into the speech synthesis model.
A separate risk area is voice customization. Cloning from a short sample reproduces timbre but does not adopt the speaker's prosody: the voice drifts over long paragraphs, the emotional range narrows, and on terminology it sounds alien. Therefore we measure naturalness of sound on the client's real texts, not on demo phrases.
In synchronous scenarios, desynchronization with subtitles is added: timings are calculated from phoneme boundaries, and with incorrect markup the caption appears before the line or lags behind it, while in dubbing the match to articulation breaks.
Each of these defects is visible in metrics and reproducible on a test set — we analyze how this is closed at the level of data, prosody settings, and rejection of duplicate takes.
What a custom voice gives: measurable effect for the product
A custom voice ceases to be a matter of taste when it is measured. We compare synthesis on one's own voice profile with a baseline speaker along three axes: share of intelligible phrases, time to voice an array of texts, and number of repeat requests to the recording. This way the effect becomes verifiable before release, rather than assessed "by ear".
Intelligibility and brand recognition
We calculate intelligibility using the subjective assessment methodology Mean Opinion Score: assessors plus automatic comparison with a reference transcript yield the share of phrases understood without asking for repetition.
On names, abbreviations, and numbers, a custom voice keeps it noticeably higher than a universal speaker. Brand recognition is a consequence: the listener recognizes the company's voice in the first seconds, even before the name is heard.
Voiceover speed and repeat requests
An array of several thousand short lines is voiced in a single run instead of multi-day studio recording, and editing the text does not require a new take.
Repeat requests to the recording drop: the meaning is grasped on the first listen, the share of clarification requests in support decreases manyfold, and the same voice works in advertising, an app, and an auto attendant without discrepancies.
Approaches to speech synthesis: comparison of options by task
The choice of speech synthesis method is determined by the task, not by fashion for models: how many phrases are in the dictionary, how unique the voice must be, whether the speaker is available, and how much time is allotted before launch.
Ready-made voices cover typical scenarios, while customization is needed where the voice is part of the brand or a carrier of industry terminology.
Before training, we fix three parameters: requirements for pronunciation of names and terms, permissible synthesis latency in dialogue, and legal cleanliness of the speaker's recording.
How speech synthesis methods differ
| Option | When it fits | Limitations |
|---|---|---|
| Ready-made voice | Typical scenarios: interface voiceover, auto attendant, draft tracks | Uniqueness is minimal, terms are corrected with a pronunciation dictionary |
| Fine-tuning a model on your own recordings | A recognizable voice and stable industry vocabulary are needed | Requires studio recording of the speaker: single microphone, even tone, marked-up text |
| Voice cloning from a sample | Quick start when there is a short high-quality speaker fragment | Sensitive to noise and volume drops in the sample, less control over intonation |
| Hybrid scheme | Several voices and roles in one product | Two update loops and separate quality acceptance |
What we choose in practice
The hybrid scheme wins where the product speaks in different voices: main content on a fine-tuned speaker voice, short service lines on a ready-made one, so as not to spend model resources on repetitive phrases.
We accept quality by blind listening: intelligibility, naturalness of intonation, errors in terms and names. We plan the speaker recording as a separate early stage — this is the longest part of preparation, and it should not be moved to the end of the project.
Work order: from reviewing recordings to acceptance of the result
Speech cannot be fine-tuned "by eye": the result depends on what is in the source recordings. Therefore the work order begins with inventory — if a third of the archive is duplicate takes and the noise level varies, this is found out before training, not at acceptance.
Audit, scheme, corpus
- Material audit: count hours of clean recordings per speaker, find duplicates, retakes, fragments with music and conference noise. The output is a table: how much material is suitable and where the markup diverges from the audio.
- Solution scheme: fix the target voice, domain words and abbreviations that the system must pronounce correctly, and the set of phrases for acceptance.
- Corpus preparation: a single recording format, level alignment, cleaning of pauses and clicks, synchronization of text with audio, a pronunciation dictionary for numbers and proper names.
Training and acceptance
- Model training: first baseline, then adaptation to the target voice. We run each version on control examples — on a small corpus it is easy to get overfitting and "floating" sound.
- Check on control examples: the set is agreed in advance and does not change — intelligibility, intonation, numbers, Latin script, rare terms. Assessment is double: machine metric plus blind listening.
- Acceptance of the result: metrics on the held-out part of the corpus, export of the model, dictionary, and synthesis configuration, documentation for reproducing the result. The previous version remains in place — rollback takes minutes, not retraining.
What we hand over: voice model and documentation
We hand over not only the sound but also everything that controls this sound going forward. The handover package is assembled so that the client's team can change pronunciation, texts, and delivery style with their own hands, without coming back to us for every edit.
Voice model and pronunciation dictionary
- Trained voice model with a card: timbre range, tempo, pause placement, permissible reading styles.
- Pronunciation dictionary: stresses, homographs, abbreviations, toponyms, product and job titles — edits are made without retraining.
- Synthesis profiles for scenarios: short prompts, long texts, emotional delivery for advertising inserts.
Regulations, tests, and handover
- Update regulations: by what signs we launch fine-tuning, who approves, how we return to the previous version of the voice model.
- Sets of test phrases with reference recordings — a mandatory run before every change to the model or dictionary.
- Integration schemes: streaming synthesis, batch generation, reuse of ready-made fragments.
- Instructions for the client's team: connection, typical errors, what to do when intelligibility drops.
- Acceptance checklist: how to verify that the new model and dictionary have actually been applied in the production loop.
Every item is closed with a demonstration on the client's real phrases: if speech synthesis sounds different from what was expected, the correction is made before signing the handover.
Implementation example: voice assistant and growth in answer accuracy
A voice assistant handled incoming requests on tariffs, connections, and appointments at service centers — up to 1,500 conversations per day.
The bottleneck turned out to be not speech recognition but synthesis: the share of incorrectly voiced names reached 6%, and almost as many lines the client asked to repeat.
Starting point
Recognition worked normally, so the cause was sought for a long time. Stresses broke in compound names, abbreviations were read letter by letter, and toponyms sounded according to the model's general rules, not the way they are pronounced in the region.
Preparing a separate voice for the new scenario took up to three weeks: almost all the time went to manual markup and pronunciation edits.
What implementation gave
We assembled a pronunciation lexicon: transcriptions, stressed syllables, rules for reading abbreviations and numerals across the entire nomenclature.
The voice was fine-tuned on the speaker's recordings while preserving timbre, and we run updates through an acceptance set of 1,500 live phrases — discrepancies go into the regression set.
We calculate quality not as an average score on the MOS scale, but as the share of failures on specific names.
| Indicator | Before | After |
|---|---|---|
| Share of incorrectly voiced names | 6% | 0.4% |
| Clarification requests per conversation | 3.4 | 0.9 |
| Preparation of a new voice | 3 weeks | 2 days |
The procedure transfers to any scenario: synthesis is checked on the client's live nomenclature, not on neutral examples. The lexicon and regression set remain part of the release — the new voice does not bring back errors that are already closed.
How is voice cloning different from speaker recording?
These are two different paths to the same result. Speaker recording gives a reference set: each phrase is pronounced by a live person, and synthesis assembles speech from recorded fragments.
Voice cloning builds a timbre model from samples and generates any phrases, including those the speaker never pronounced.
The difference in the volume of materials is orders of magnitude. Speaker recording requires a studio session of several hours of clean text with dense phonetic coverage, otherwise synthesis stumbles on rare combinations.
Cloning needs only a few minutes of clean speech — that is enough for models like XTTS without fine-tuning, or up to an hour of recordings if the timbre is adjusted for the project separately.
The limitations of the approaches are different in nature. Cloning inherits defects from the samples: noise, cut-off frequencies, and different microphones between fragments are transferred by the model into synthesis.
Speaker recording is limited by the person — their schedule, fatigue, and written consent to use the voice.
The choice rests on the scenario: for updatable content and dozens of voices, cloning wins; for reference quality, recording.
Common doubts before starting: timelines, rights, support
Before starting, clients ask the same set of questions — and these are the right questions. Below are short answers for each: timelines, model rights, recording privacy, accent, and support.
How long does voice adaptation take?
For a ready-made model, timbre is configured in days: half an hour of clean recording is enough. Your own voice from scratch — several weeks, and most of it goes to corpus markup, not training.
Who owns the rights to the model?
We transfer the model, its weights, and the data preparation code by acceptance act — you can fine-tune it with your own team or another contractor. The speaker recordings remain with you.
Where do texts and audio go?
In your environment, synthesis runs on your equipment: audio and text do not leave the perimeter, external services are not used. Request logs are anonymized.
What about accent and dialect?
The speaker's accent is transferred along with the timbre — this is a consequence of training on their recordings. If a neutral accent is needed, we add reference speech to the corpus.
What happens after delivery?
We hand over documentation and train your team. After that — a support procedure: quality control in production, fine-tuning when the subject matter changes, analysis of deviations.
Let's discuss the task: send examples of recordings and requirements
Speech synthesis is evaluated by materials, not by description: timbre, tempo, and intelligibility on complex words determine both the choice of model and the timeline. Therefore the next step is simple — send what you already have on hand.
What is useful to attach to the request:
- examples of recordings — 3–5 fragments of 1–3 minutes per voice, with different intonation: narration, question, announcement;
- voice requirements — timbre, tempo, gender, age, accent, permissible deviations; separately note whether style transfer to a new voice is needed;
- volume of voiceover — how many minutes or hours are synthesized now and how the corpus will grow: one-off lines, a stream of announcements, or a constantly updated archive;
- scenarios and channels — phone line, app, speech interface, videos; whether real-time responses and dialogues are needed;
- pronunciation — names, abbreviations, terms, numbers, and units of measurement that the voice must read correctly; a replacement dictionary, if one already exists;
- constraints — response time, peak load, operation without external services, rules for processing and storing data.
Based on this data, we assemble a work plan: stages, duration of each, acceptance scope, and what will be required from your side. If some materials are not ready yet — describe the task in words, we will clarify the details and suggest where to start.