Train Custom Translation Models for Your Industry
A legal department localizing contracts into 10+ languages spent 40 hours per week post-editing machine translations. Generic models confused terms in 15% of cases: "consideration" became "рассмотрение" instead of "встречное удовлетворение". We fine-tuned MarianMT on 50K parallel legal sentences — BLEU jumped from 28 to 46, post-editing time dropped to 10 hours. This reduced post-editing costs by 40%, saving the client $12,000 annually. ROI on the fine-tuning investment (under $5,000) was achieved in 5 months. Below is how we replicate such results.
Why Fine-Tuning a Machine Translation Model Is Critical for Business?
Generic models achieve BLEU 25–35 on technical texts. After custom fine-tuning on domain corpora, we raise BLEU to 40–50 and COMET by 0.1–0.15. This cuts post-editing volume by 30–50% and eliminates critical semantic errors. Without customization, you lose up to 20% translation accuracy on complex domains. Our fine-tuned models outperform baselines by a factor of two in BLEU on specialized domains.
Comparison of Base Architectures
| Model | Languages | Size | Resources for Fine-tuning | When to Use |
|---|---|---|---|---|
| MarianMT (Helsinki-NLP) | 1000+ pairs | 150–300M params | 1 GPU, 10–50K sentences | Quick fine-tuning for one language pair |
| NLLB-200 (Meta) | 200 languages | 1.3B–3.3B params | 4–8 GPUs, 100K+ sentences | Multilingual scenarios, rare languages |
| SeamlessM4T (Meta) | 100 languages (text+speech) | 2.3B params | 8+ GPUs, 200K+ sentences | Integrating STT and translation in one pipeline |
MarianMT trains 3 times faster than NLLB with comparable quality on single-pair tasks.
Step-by-Step Fine-Tuning Guide
Our pipeline consists of five stages:
Step 1: Data Analytics
We collect and analyze your corpora, define metrics, and select the best architecture based on language pairs and budget. This takes 2–5 days.
Step 2: Data Preparation
Corpora are cleaned, deduplicated, tokenized, and aligned. We use OPUS, EMEA, JRC-Acquis, and your own data. Minimum parallel sentences: 10K for MarianMT, 100K+ for NLLB. Duration: 3–10 days.
Step 3: Training and Optimization
We fine-tune using libraries like Transformers and LoRA for efficiency. Hyperparameter tuning includes learning rate (1e-5–5e-5), dropout (0.1), and early stopping. For large models, we apply 4-bit quantization. Example training code:
from transformers import MarianMTModel, MarianTokenizer, Seq2SeqTrainingArguments, Seq2SeqTrainer
import sacrebleu
model_name = "Helsinki-NLP/opus-mt-ru-en"
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)
def preprocess(examples):
inputs = tokenizer(examples["ru"], max_length=512, truncation=True, padding=True)
targets = tokenizer(text_target=examples["en"], max_length=512, truncation=True, padding=True)
inputs["labels"] = targets["input_ids"]
return inputs
training_args = Seq2SeqTrainingArguments(
output_dir="./marian_legal",
predict_with_generate=True,
per_device_train_batch_size=8,
num_train_epochs=5,
learning_rate=5e-5,
fp16=True,
generation_max_length=512,
)Training takes 1–5 days.
Step 4: Evaluation and Iterations
We evaluate using BLEU and COMET. Typical improvement: +3–8 BLEU, +0.05–0.1 COMET. We also manually test 200–500 sentences to catch artifacts. If needed, we retrain with adjusted parameters. Duration: 2–5 days.
# BLEU
bleu = sacrebleu.corpus_bleu(hypotheses, [references])
print(f"BLEU: {bleu.score:.2f}")
# COMET
from comet import download_model, load_from_checkpoint
model_path = download_model("Unbabel/wmt22-comet-da")
comet_model = load_from_checkpoint(model_path)
scores = comet_model.predict(data, batch_size=8, gpus=1)Step 5: Deployment
We export the model to ONNX or TorchScript, containerize it with Docker, and provide a REST API. Monitoring and latency optimization ensure p99 < 500ms. Duration: 1–2 days.
Typical Mistakes in Fine-Tuning and How to Avoid Them
- Overfitting on small corpus: use dropout 0.1, early stopping, data augmentation (back-translation).
- Domain shift: add 10–20% general data (e.g., OPUS).
- Quality loss on general domain: multitask learning — train simultaneously on domain and general corpus.
- Hallucinations: enable forced decoding with length penalty, apply beam search with length penalty.
What's Included in the Work
- Preparation and cleaning of parallel corpora (including ETL pipeline)
- Selection and configuration of base model (MarianMT/NLLB/SeamlessM4T)
- Training with hyperparameter tuning (LR, batch size, dropout, number of beams)
- Quality evaluation (BLEU, COMET, manual validation on 200–500 sentences)
- Export of model to ONNX/TorchScript with latency p99 optimization
- Documentation: model card, metrics report, deployment instructions
- Support for 30 days after delivery
Process and Timelines
| Stage | What We Do | Timeline | Cost Range |
|---|---|---|---|
| Analytics | Data collection, metric definition, architecture selection | 2–5 days | $1,000–$2,000 |
| Data Preparation | Cleaning, tokenization, alignment | 3–10 days | $2,000–$5,000 |
| Training | Experiments with hyperparams, LoRA, quantization | 1–5 days | $2,000–$8,000 |
| Evaluation and Iterations | Testing on hold-out, fixing artifacts | 2–5 days | $1,000–$3,000 |
| Deployment | Docker, REST API, monitoring | 1–2 days | $500–$1,500 |
Total estimated timeline: from 10 working days for MarianMT to 30 days for NLLB. Total cost typically ranges from $6,500 to $20,000 depending on complexity.
Why Trust Us?
We have 5+ years of experience in NLP production projects. We have delivered 15+ machine translation customizations for legal, medical, and technical domains. We guarantee transparent reporting at every stage: you receive a model card, metrics, and code. Contact us — we'll evaluate your project and choose the optimal architecture. Schedule a consultation on fine-tuning today.