Automatic subtitle translation: where manual assembly breaks
A typical failure begins even before translation. The input consists of an assembled master, draft subtitles from the editing program, and a list of editor's corrections in email; each source lives its own life.
By the time video localization into 20 languages is needed, there is no single source of timings: in one file a cue starts earlier, in another later, and some lines are not marked by speaker at all.
Then scale finishes off the process. Translation into 20 languages means 20 independent sets of timings: the German line is longer than the source, Arabic and Hebrew require right-to-left writing, Chinese text is segmented by meaning differently.
Desynchronization of speech and text ceases to be a random error and becomes systemic — the cue either lags behind the voice or remains on screen after the frame changes.
Manual subtitle assembly finally breaks at this point. Proofreading relies on native speakers, but the bottleneck is not translation itself, but consolidation: controlling line length, line breaks, merging adjacent cues, unified markup when exporting to SRT and WebVTT.
One missed line break in a rare-language version comes back as an edit in all the others, because the master track has long since diverged across copies.
The cost of such a process is weeks of calendar time per release and divergence of versions between screening platforms.
That is why below we break down how automatic subtitle translation is handled by a single pipeline: one master timing track, machine checking of reading speed and line length, and targeted native-speaker involvement where the wording is genuinely debatable.
Measurable effect: language coverage, release speed, quality
Counting the effect is worth doing not by the number of languages in a presentation, but by how many tracks come out of one run and in what window.
One pass — the entire language set
The source track goes through the chain “recognition → segmentation → translation,” and the output is the entire language set at once. We use a Whisper-compatible model for recognition, and a language model with a project-specific style profile for translation.
A new language is a line in the configuration and a repeat run, not a separate project: coverage grows without rebuilding the process.
The series is released in one window
Previously a ten-episode series went out in waves: languages were translated one after another, and the last episode waited for the first.
Now the entire set comes out in one window, and the release time of the series is determined by the duration of the longest episode, not the number of languages in it. Localization speed no longer depends on how many markets you open.
Unified glossary: terms and timing
The glossary keeps product names, proper names, and terms identical across all languages and series: an edit to one term is applied to the entire set, not to each track separately.
Timings are inherited from the source track, so cue desynchronization and lines that outgrow the cue are caught by checks before release. A whole class of comments disappears: different names in different series and shifted subtitles.
Existing work is reused
Translation memory, style profiles, and approved terminology are preserved between projects.
The next series starts with accepted wording, not from scratch, so the share of edits after proofreading falls from series to series, and a new language is connected with an already prepared glossary.
Translation and post-editing modes: what to choose
Translation mode is not “quality versus speed,” but a choice of where the human sits in the chain. The same clip can be run through machine translation and delivered in minutes, or placed in a stream with native-speaker proofreading and produce a broadcast version.
The difference shows not in an individual phrase, but in the class of errors: drifting names, broken humor, different variants of the same term in adjacent episodes.
Workflow options
| Option | When it fits | Limitations |
|---|---|---|
| Machine translation without editing | One-off clips, draft review, languages where the audience tolerates rough edges | Terms and proper names “drift,” idioms break, there is no unified vocabulary between episodes |
| Trigger-based post-editing | Regular releases, limited time budget | Only low-confidence segments, long cues, and numbers are edited — the rest remains machine-translated |
| Hybrid workflow | Mixed content: interviews, training courses, technical talks | Requires a pre-assembled glossary and translation memory; the first pass needs more manual work |
| Native-speaker proofreading workflow | Public releases, legally significant wording, brand localization | The longest chain in time; requires a native speaker with a profile in the subject, not “any language” |
| Batch processing | Archive of dozens of clips, seasonal catalog update | Cannot be edited on the fly; if the glossary changes between batches, terms diverge |
The mode is chosen based on the cost of error and volume: where subtitles are seen by a mass audience, proofreading pays off, while in a training archive trigger-based post-editing is enough.
Below we show how we set up a hybrid workflow and by which metrics we consider a translation ready for publication.
Implementation order: from material audit to handover
The order of work is structured so that translation does not depend on how well the model performed on a particular series. Each stage ends with an artifact that can be checked: a list of problematic files, a workflow diagram, a glossary, a report on the test batch.
This removes the classic issue — a translation fails acceptance not because of text quality, but because of discrepancies in timecodes and terminology between seasons.
- Material audit. We collect all sources: languages, track formats, timecode discrepancies, embedded styles, duplicates, and missing cues. The output is a list of files with defect descriptions and a decision on what we clean up and what we translate as is.
- Workflow diagram. We fix where subtitles are taken from, how material is cut into segments, by what attribute cues are merged, and in what form the result is returned to storage or player according to individual calculationGlossary and rules. Terms, names, product and job titles, case, form of address, restrictions on line length and minimum display duration. Separately — a ban on literal transfer of terms if a language has an established equivalent.
- Test batch. We take a fragment that contains all types of speech: dialogue, voice-over, subtitles, speech with terms. We look not at the average score, but at the share of segments that a native speaker would not edit.
- Checks. Automatic — timecode overlap, reading pace, lost cues. On top — selective native-speaker proofreading against a pre-agreed list of comments.
- Acceptance of the result. Agreed criteria: share of segments without edits, glossary stability, absence of time shifts. Deviation from the criteria is a reason to refine the rules, not to rewrite manually.
- Handover. We provide regulations, per-language glossaries, diagrams, and instructions on how to add a new language or season without our involvement.
Glossaries, markup, and proofreading regulations
After translation is launched, the client is left not with scripts but with working documentation: with it, a new language is connected in days rather than from scratch, and any editor understands what counts as an error. We hand over the set in open formats — edits are made without developer involvement.
- Project glossary. Subject-matter terms, proper names, and product names with approved translation and a list of prohibited variants. For each language — its own column and notes “do not inflect,” “do not translate.”
- Terminology rules. Who proposes a new term, who approves it, within what timeframe, what to do in case of conflict with an already approved variant, and how to mark temporary exceptions.
- Subtitle markup schemes. Permissible line length and number of lines per block, line-breaking rules, punctuation, speaker labels, sound events. Separate profiles for languages written right-to-left and without spaces.
- Proofreading regulations. Check levels: automatic glossary verification, editor proofreading, final client review. For each — a checklist and criteria for sending a series back for revision.
- Edit regulations. How a comment turns into a rule: an editor's edit goes into the glossary or translation memory, and the next release comes out with it accounted for without manual work.
- Workflow documentation. Pipeline diagram from source files to finished tracks: formats, control points, behavior during timing desynchronization, instructions for handing over to a new team.
Case study: a series of releases in 12 languages without manual assembly
Starting point
The client — the streaming platform StreamCast — released a season in 12 languages sequentially: the English version went first, and the others caught up over weeks.
Each language version was assembled by hand — segmentation, binding to timecodes, terminology verification between episodes. In a test batch of 900 cues, about 40% required manual semantic editing, and more than half of the discrepancies were in character names and device names.
What we measured on the test batch
We ran the same batch through automatic subtitle translation with glossary support: the model sees neighboring cues and a predefined list of terms rather than translating a segment in isolation.
The share of segments that the editor edited semantically dropped to 6–8%, and terminology inconsistency between episodes disappeared — the glossary became a single point of truth.
We separately checked for regression: timecodes, line segmentation, and cue length limits after the run remained in place.
What changed in the release cycle
The release series now proceeds in one pass: a draft translation into 12 languages is ready in the same window as the source version, and editors work only with flagged segments.
An edit made in one language version is carried over to the others — discrepancies between versions do not accumulate. The reduction in manual edits delivered the main thing: time to publication of a language version no longer depends on the number of languages.
How to check synchronization and terminology before publication
Pre-publication checking is not proofreading the entire array by hand, but a set of automatic checks on the subtitle file before release. Each block receives attributes: line length in characters, number of lines, reading speed, display duration, and gaps with neighboring blocks. Everything that goes beyond thresholds is flagged — the rest is published without edits.
Synchronization checking relies on comparison with the source track: block boundaries should not drift relative to speech, and offset tends to accumulate toward the end of the recording — so the beginning and finale are controlled separately.
Block overlaps, too-short display, and gaps that cause the subtitle to go dark before the phrase is spoken are caught separately.
Terminology control is built on a single project dictionary: we check that the same term is translated identically throughout the file, not only in an individual cue.
It also includes prohibited formulations — things that must not be translated literally or named on air — and consistency in addressing the viewer.
Line readability is checked by formal attributes: line length, number of lines per block, line break at a syntactic boundary rather than in the middle of a phrase. Names, numbers, and compound terms are not split between lines — otherwise the viewer loses meaning at the junction.
What remains for a human is not the entire array, but only flagged fragments: disputed terminology, dense speech, places where synchronization drifts.
After the run, a report with labels and reasons is compiled, so edits are made in a targeted way, and the share of segments with manual intervention drops from the majority to single-digit percentages.
Technical concerns: voice, synchronization, terms
Diarization and speech overlap are the first things automatic subtitles stumble over.
We split the track by voiceprints (diarization) and mark areas where speakers overlap: such a fragment is not “collapsed” into one cue, but either remains in the source track or is marked as ambiguous for manual review.
In interviews with two speakers, this gives speaker-labeling accuracy at a level where the viewer does not stop understanding who said what.
What happens if several people speak at the same time?
The recognition engine on Whisper and similar models is trained on a single voice, so during overlap it tends to merge cues or lose part of the words.
We detect overlap by energy and spectrum, mark the problematic interval, and separately run it with source separation — the output is either a correct cue or a clear note for the editor, but not “mush” in the subtitles.
How will the translator understand names, brands, and narrow terms?
Before the run, we assemble a glossary: names, company and product names, industry abbreviations are entered into the dictionary with fixed translation and casing.
The model receives this list as a constraint, not as a wish, so “Smith” does not turn into “blacksmith,” and the product name remains recognizable in each of the 20+ language versions.
Can you set a custom subtitle style?
Yes, the style is set by a preset: line length, characters per second, line breaks by semantic groups, position relative to safe areas of the frame.
One preset is applied to all languages at once, so versions do not diverge in layout, and the vertical format for short clips gets its own grid without manual reconfiguration.
What about edits after publication?
Subtitles are stored as a separate track from the video, so editing a term or timing does not require reassembly and re-uploading the video.
The editor changes a line, the change goes to the required language versions, and the previous variant remains in history — you can return to it if the edit proved unsuccessful.
Let's discuss the task: what is needed for an estimate
To come back with a project estimate and work plan, we only need initial information — describing internal processes and opening access at this stage is not necessary. The more specific the source data, the fewer assumptions in the plan and edits after the start.
- Volume of materials — how many hours of recording or files need to be translated and how often new ones appear: a one-off series or a continuous stream.
- Language pairs — from which languages and into which, which directions are priority, and whether rare pairs via a pivot language are needed.
- Source format — SRT, VTT, or text transcript: this determines whether preliminary track assembly and timing synchronization are needed.
- Terminology — glossaries, names, product names, and industry terms that cannot be translated literally.
- Reading requirements — character limit per line and display time, line-breaking rules, whether a cue needs to fit into one or two lines.
- Reference examples — two or three fragments translated manually: we use them to configure checks and assess discrepancies.
- Operating mode — a one-off catalog or a regular stream, monthly volume, and delivery deadlines.
Once we receive this data, we will come back with a scope estimate and plan: we will show the stages, quality control points, team composition, and acceptance procedure. Describe the task in a message — we will clarify the details and propose the order of work.
