Fine-tuning an embedding model: when search does not understand the domain
Semantic search over documents is simple: a query and a text fragment are converted into vectors, similarity is calculated by cosine measure, and results are assembled from the nearest neighbors. As long as the base is homogeneous and the vocabulary is common, this is enough.
But if an internal term gets into the query — a department abbreviation, a node designation, a part number from a supplier catalog — the model drifts into the nearest general-language area, returning formally similar but irrelevant paragraphs.
The reason is that the base model was trained on general texts, where these words either do not exist or appear in a different context.
The vector of such a term turns out to be an almost random point in space, and the distance to the needed document does not differ from the distance to a hundred unrelated ones.
On domain queries, lexical search by word occurrence often outperforms semantic search — a clear sign that the representation of terms does not match your corpus.
Growth in the number of documents makes this worse rather than fixing it. The denser the base, the more neighbors an uncertain vector has, and ranking starts to be determined by noise: the needed fragment moves to the end of the first hundred.
By uploading new regulations and specifications without working on domain vocabulary in search, you only increase the volume of candidates among which the model makes mistakes.
The symptom is recognized by three signs: employees stop formulating queries in words and search by numbers, result quality drops as the base is filled, and on domain formulations search fails to find what is needed even when the document is live in the index.
This is exactly the case where fine-tuning an embedding model on "query — relevant fragment" pairs from your subject area restores top hits. We explain how this is closed.
What changes in search quality after domain adaptation
After fine-tuning on an internal corpus, it is not the "feeling" of search that changes, but measurable values. Basic matching searches by exact formulations, so the query "apply for unpaid leave" works only if the document contains the same words.
Adaptation shifts the vector space toward the language of the knowledge base: the share of relevant results in the top three on the reference set grows from 40–55% to 85–90%, and the average position of the first needed document is roughly halved.
The effect is most visible on synonyms, transliteration, and internal abbreviations.
"GO", "KP", "SF", department abbreviations, and names of internal systems are almost indistinguishable from typos for the general model, while for the adapted one they are stable semantic anchors.
It was precisely on such queries that result quality dropped most noticeably, and employees rewrote them manually two or three times.
Reduction in manual queries is the most honest indicator. We calculate the share of sessions where the user reformulated the query or went to search for a file manually: after adaptation it drops severalfold, and the share of sessions resolved on the first query grows. This is read from logs without surveys and does not depend on subjective assessments.
We check the effect on the same set of reference queries as before fine-tuning. We compare the share of top-three hits, the average position of the first relevant document, and the share of queries with no relevant answer at all.
If the metrics do not improve, the result is not accepted: the base model remains, and the corpus goes for additional labeling.
Ways to adapt search: comparison of approaches
Approaches to search adaptation differ in what exactly we change: rules and labeling, the structure of the vector index, or the representation model itself.
Simple options are implemented faster and more predictably, but hit a quality ceiling; fine-tuning gives the maximum on complex vocabulary — at the cost of a domain corpus and an evaluation loop.
| Option | When it fits | Limitations |
|---|---|---|
| Dictionaries, synonyms, and ranking rules | The domain is narrow, and results need to be fixed pointwise and quickly | Does not generalize: new query formulations remain outside the rules |
| Vector index tuning: fragment size, similarity metric, field filters | Documents are homogeneous, queries are short | Improves hitting the right fragment, but does not close semantic gaps of the model |
| Hybrid search: lexical and vector with weight recalculation | Queries contain part numbers, abbreviations, exact terms | Requires weight selection and separate validation for each query type |
| Reranking candidates with a separate model | The first dozens of results are relevant, but the order is wrong | Adds response latency, requires labeling of "query — document" pairs |
| Fine-tuning representations on a domain corpus | The vocabulary and meanings of the subject area are far from general data | Requires a corpus, an evaluation loop, and control of degradation on general queries |
We go from bottom to top: first rules and vector index tuning, then hybrid search, and only after measuring the ceiling — fine-tuning.
The same set of queries is run at each step, so in the comparison of search approaches it is visible which layer gave the gain and which can be rolled back without loss of quality.
How model fine-tuning on domain data proceeds
Fine-tuning begins not with starting training, but with inventorying what the model already does poorly. Otherwise you get a system that answers better on prepared examples and makes the same mistakes on live queries.
-
Query audit. We export search requests, group them by scenario, and mark where results diverge from expectations: internal slang, abbreviations, part numbers. The output is a list of problem groups with frequency.
-
Labeling domain pairs. We collect "query — relevant document" pairs and add hard negatives: related in topic but not suitable. Labeling is done by subject-matter experts, and disputable cases are resolved by a separate rule.
-
Training scheme. We fix the base model, vector dimensionality, share of negative examples, and frozen layers. Hyperparameters are selected on the validation part; we do not touch the test part until the final stage.
-
Validation on a holdout set. We run reference queries and look at the share of cases where the correct document hit the top five, as well as category bias.
-
Search regression tests. The set of reference queries stays in the repository, so each subsequent model version goes through the same checks and degradation is visible before rollout.
-
Deployment and handover. The model goes into the index under versioning, rollback to the previous build is configured, and we hand over retraining scripts, labeling, and the update procedure.
After handover, the customer's team fine-tunes the model on new categories independently — we join when the search scheme changes.
What we deliver at the output: configurations, schemes, procedures
A working search pipeline differs from a successful launch in that it can be handed over to another engineer.
Therefore, along with the fine-tuned model, you receive the supporting framework: configurations, schemes, and documents by which the system can be reproduced, extended, and safely updated without our involvement.
- Search configuration in the repository — parameters for splitting documents into fragments, field weights, hybrid ranking rules, and filters. Not settings in someone else's panel, but files under version control.
- Pipeline scheme — from corpus loading to answer output: where training happens, where reindexing happens, which nodes are dependent, and what happens if each fails.
- Model card — on which corpus it was trained, what metrics on the holdout set, where the applicability boundaries are, and which query types it handles worse.
- Model update procedure — triggers (quality drop, new document array, nomenclature change), step-by-step retraining order, who approves rollout, and how to roll back to the previous checkpoint.
- Set of evaluation queries with reference answers — to catch regression automatically after each reindexing, rather than from user complaints.
- Access rules — who has the right to change configuration, who can launch retraining, where vectors are stored, and how they are deleted together with the source document.
- Model documentation and operations manual — launch, diagnostics of typical failures, and escalation procedure to our team when requirements change.
Keys and passwords are not handed over as separate files: access is set up for your accounts, and we revoke ours after acceptance.
Case study: search accuracy on an internal knowledge base before and after
A model fine-tuning case: the internal knowledge base of a service company — regulations, contract templates, history of project decisions. Employees' vocabulary diverged from general language: abbreviations, names of internal systems, local shortenings.
On such queries, the base model returned the needed document in the top 3 in less than half of cases, and people went back to manually browsing folders.
What was measured before work
We collected real search queries with relevance labels from relevant departments and recorded three metrics: share of top-3 hits, recall at top-10, and average position of the correct document.
Separately, we held out an independent sample — 400 queries that participated neither in fine-tuning nor in parameter selection.
What was changed and what happened
Fine-tuning of the embedding model was performed on "query — relevant fragment" pairs labeled by domain experts. To the vector index on OpenSearch, we added keyword search, reranking, and an access rights filter.
Top-3 hits grew from 47% to 86%, recall at top-10 — from 71% to 94%, and the average position of the correct document shifted from sixth to second.
The gain was checked on the holdout set: the discrepancy with training metrics is within 2 percentage points. Rollback is provided: model weights and index version are switched with a single setting, without rebuilding or downtime.
What to do if search already runs on a ready-made solution
A ready-made search solution usually relies on an index and a text representation model selected for the general case.
They do not know industry vocabulary: internal abbreviations, part numbers, node names, and jargon are decomposed incorrectly in vectors, and results lose to simple word matching.
Fine-tuning an embedding model closes exactly this layer — ranking logic, interface, and familiar scenarios stay in place.
The enhancement proceeds without stopping the service. The new model version computes vectors into a separate index, the running one continues to answer queries, and switching is done only after quality is confirmed by measurements on a labeled query set, not by eye.
A full index rebuild is also not required: recalculation goes in batches, with priority for frequent documents and those that more often appear at the top of results.
The rest is recalculated in the background, so the delay in the appearance of new materials is measured in minutes, not hours.
Search migration is built as a reversible operation: two index versions live simultaneously, the share of queries is switched gradually, and rollback is one line in the configuration. Previous vectors are retained until the new model completes a full cycle on your data.
Next we analyze your collection: where exactly the ready-made solution makes mistakes and what volume of recalculation is needed to fix it.
Technical doubts: how much data is needed and how to verify
Before launching embedding model fine-tuning, the same questions come up: how much labeling is needed, whether the model degrades on general queries, how to measure the effect if there is no reference set, and how often to update the result. Below are short engineering answers to each of these doubts.
How much data is needed for fine-tuning?
Usually a few thousand "query — relevant document" pairs are enough: site search logs, support requests, internal regulations. Quality is more important than volume — contradictory pairs harm more than an added thousand examples help.
On a narrow domain, we start with a smaller set and collect more data based on the errors of the first iteration.
Will the model get worse on general queries?
There is such a risk, so domain pairs are mixed with general ones in a selected proportion, and some layers remain frozen.
We measure the effect not only on the domain, but also on a separate set of general queries: if the share of correct hits there drops, training is stopped early.
How to verify quality without a reference set?
The set is assembled quickly: a few hundred queries with known correct answers already provide a stable estimate.
Additionally, we look at indirect signals — cluster shift, the share of queries where the model stopped finding what it previously found, and comparison of the old and new versions on the same stream in shadow mode.
How often should the model be fine-tuned?
Not by calendar, but by data shift: when a noticeable share of new nomenclature or terminology appears in queries, and the metric on the control set drops. In practice, this is an update once every one to two quarters, not continuous retraining.
Let's discuss the task: where to start and what we need from you
To come back with an estimate and work plan, we do not need access to your production and do not require a pre-written technical specification.
It is enough to show how search is currently structured and which queries it handles poorly — this is sufficient to understand the scope of embedding model adaptation for your domain and subject-area language.
What is worth preparing for the first conversation:
- Examples of real queries — 20-50 formulations where results are irrelevant, and next to them what the user meant.
- The current search scheme: whether a vector index is used, which fields ranking is built on, and whether there are filters.
- The corpus of texts on which search runs: descriptions, knowledge base articles, request history — in any readable format.
- Quality statistics: share of queries with no result, transitions to the card, repeat support requests after unsuccessful search.
- Constraints: where the data physically resides, whether processing outside your environment is allowed, and what requirements exist for retention periods and access separation.
- Guidelines on timelines and priorities: which part of search hurts most and what counts as success for you.
Then we take on the work: analyze the corpus, assemble labeling of "query — relevant answer" pairs, fine-tune the model on the domain, and compare it with the current one on a holdout set.
As a result, you receive a work plan with stages, checkpoints, and quality metrics by which the decision to start the project is made.
