Semantic search: why word-based matching does not find the answer
Full-text matching compares characters, not meaning: it splits the query and documents into words, reduces them to stems and looks for intersections. As long as an employee enters the same terms as the document author, everything works.
But as soon as they formulate a thought in their own words, the results are empty, even though the needed file is in the database.
The first gap — synonyms and word forms.
Stemming removes some discrepancies but does not help when the query says "return goods" and the policy says "refusal of purchase." There is no word overlap — so for search the document is irrelevant, although for a person it is the same process.
The second gap — query formulation.
Employees ask questions in natural language, while documents are written in bureaucratic style: "who is responsible for acceptance" versus "responsibility for carrying out acceptance." Different constructions, different parts of speech, zero word overlap — and empty results on the fifth or sixth query force people to search manually through folders.
Semantic search works differently: it converts the query and document fragments into vector representations and compares them by proximity of meaning, not by letters. Synonyms, word forms and different formulations of the same thought cease to be a barrier. Below we discuss how this is arranged and what it gives on documents of different volumes.
What semantic search provides: effect in measurable metrics
The first measurement we take before and after implementation is the share of queries where the needed document appears in the top results.
On a corpus of several thousand pages, keyword search gives it within 35–45%: the query wording does not match the wording in the text, and synonyms and abbreviations are not found.
After moving to semantic search with a hybrid scheme — lexical retrieval plus vector representation search — the metric rises to 75–90%. We work on OpenSearch: both modes live in the same index, and a second system is not needed.
The second metric is time to the first useful document. Previously an employee spent 10–20 minutes going through irrelevant links or went to ask a colleague; now semantic results close the question in a minute or two.
The consequence is visible in repeat requests: questions like "where is the policy" reach knowledge holders far less often, and this is already a change in workload, not a self-assessment of convenience.
Separately, we count the failure class "the document exists but is not found" — when the text is in the system, but word search does not surface it because of different wording or section title.
In the reference sample, such queries failed in about half of cases and were closed by a person; after implementation they are found by search.
We keep output quality under control: there is a reference set of queries with labeled correct documents, we use it to calculate metrics before and after implementation, and we separately review rare formulations.
Comparison of approaches: vector index, hybrid and rules
The choice of mode determines not "search quality in general" but which misses you will get on your corpus. A vector index finds semantic matches during paraphrasing but loses exact identifiers — contract numbers, SKUs, rare abbreviations. Rule-based search is predictable and explainable but requires the query to match the schema.
Hybrid mode removes this contradiction: it combines the lexical signal and embedding proximity, while metadata filters cut off the excess before ranking. Which suits which corpus is in the table.
| Option | When it fits | Limitations |
|---|---|---|
| Vector index | Heterogeneous corpus, free-form queries, resilience to synonyms matters | Requires rebuilding when the embedding model changes, poorly catches exact codes and rare terms |
| Hybrid mode | Mixed queries: meaning plus exact identifier, several document types in one index | Need to coordinate the contribution of signals and the cutoff threshold, higher response time |
| Rule-based search | Narrow domain, requirement for explainability and repeatability, regulated queries | Fragile to paraphrasing, every new document field means manual rule editing |
In practice, the mode is chosen by the composition of queries. If the corpus contains both prose and exact identifiers — start with hybrid, and leave rules for mandatory conditions: access, term, document type.
The limitations of each approach are known in advance, and the choice of mode becomes a data-driven decision, not a matter of taste.
How we implement: work stages from audit to handover
We arrange the stages so that search quality is measured at every step, not at the end of the project. Each stage ends with a verifiable artifact: a corpus report, a chunking scheme, a set of test queries.
- Document corpus audit. We count volume, formats, share of scans and tables, find duplicates and outdated revisions. The output is a list of sources and a rule for what goes into the index and what is cut off as noise.
- Chunking and indexing scheme. We define fragment boundaries — section, paragraph, table — so that a piece of text retains meaning without its neighbors. Next to the fragment we store the clause number, date, document type and access rights.
- Query labeling. We collect real employee formulations and label them by topics and expected documents. This gives a set of checks against which search variants are then compared.
- Output mode configuration. We look for a balance between exact word matching and semantic search, and set filters by rights and freshness. We separately test short queries and queries with a clause number.
- Quality measurement. We calculate the share of queries where the needed document appeared in the top five results and analyze failures manually. We agree on the acceptance threshold before work begins.
- Pilot operation and handover. We launch on a limited group, see what people search for and where they drop off, and adjust chunking and output based on facts. We hand over configurations, the indexing scheme and the reindexing policy, and train administrators.
Output artifacts: schemes, dictionaries and update policies
Semantic search remains a working tool only when it can be changed without the system's authors.
Therefore we hand over not a "black box" service but a set of artifacts that your support team can use to update the corpus, expand dictionaries and investigate search incidents on its own.
- Indexing scheme — which document fields go into the index, into which fragments it is split, with what overlap, and what metadata (section, version, date, rights) is stored next to the text.
- Domain term and synonym dictionary — internal abbreviations, product names, professional jargon; it is expanded without rebuilding the index or changing code.
- Preprocessing rules — markup normalization, parsing of tables and attachments, a separate recognition layer for scans: without it, part of the corpus simply is not found.
- Section access policies — which document collections are visible to whom; filtering happens at the search stage, not after it, otherwise a restricted document leaks through the output.
- Reindexing policy — what is updated incrementally when a document changes, what requires a full pass, in what window it is launched, and how to verify that the new index version is not worse than the previous one.
- Reference query set and quality metrics — control questions with expected documents; with them you can distinguish degradation from a one-off user complaint.
- Operational documentation — service map, failure points, procedure for rolling back to the previous index version, typical failures and actions for them.
We hand over the package together with a walkthrough on a running system: we show how to add a new document section and how to roll back if output quality drops after an update. After that, support is handled by your team or ours — your choice.
Case: time to find an answer in the document base decreased
Implementation case: a manufacturing company with a corpus of technical and regulatory documentation — tens of thousands of pages of policies, equipment passports, operating instructions and internal standards.
Search worked by file names and word matching, so the result depended on whether the specialist guessed the wording.
Before implementation, the answer was sought by manual browsing: they opened 8-12 documents, checked revisions, and clarified with colleagues.
Average time to find an answer was 30-40 minutes, and about every fifth query ended with the conclusion "it is not in the documents," although the needed clause existed in a different wording.
The load on subject-matter specialists grew: up to a third of working time went to searching instead of their main work.
What was changed
Documents were parsed by structure — sections, clauses, tables, appendices — and split into semantic fragments while preserving a link to the source.
Semantic search was assembled as a hybrid scheme on Elasticsearch: vector representations are responsible for meaning, the exact index for numbers, SKUs and abbreviations. The output includes a text fragment and a link to the original page.
Metrics before and after
The share of queries with a found answer grew from 78% to 96%, and average time to find decreased from half an hour to two or three minutes.
The load on subject-matter specialists fell by about half: questions that used to be closed by manual browsing are now resolved independently.
In parallel, we expanded index updating — a new revision of a document enters search on the same day, and outdated versions do not surface.
How a vector index differs from word matching
An inverted index answers exact matching: the query is split into tokens, and a document appears in the output only if it contains the same tokens.
This is fast and predictable, but it works only when the wording is known in advance: "termination policy" will not find text about the "procedure for ending a contract." Stemming and synonym dictionaries cover some cases, but not paraphrasing, translation or typos.
A vector representation converts the document and query into a single numeric space, where semantic matching is calculated as vector proximity, not as string matching.
The query "how to process a return" finds paragraphs about the "procedure for canceling an order," although they have no words in common. Relevance here is determined by the distance between vectors — output is ranked by meaning, not by term frequency.
Exact matching is not canceled, however. SKUs, contract numbers, full names, department codes and abbreviations can be blurred by a vector model into semantically similar but wrong values, and an error here is the most costly.
Such fields remain on string search with mandatory exact occurrence — the same applies to legal wording and negations.
In a working system, both mechanisms are assembled into combined output: string search provides exact hits and filters, vector search provides semantic neighbors, and rank merging and weights are tuned for the specific corpus.
Some queries are routed explicitly: a numbered one goes to the exact index, a natural-language question goes to the vector one. We set up the scheme on standard components, for example Qdrant for vectors paired with an inverted index for strings.
Weight tuning is a separate effort with labeling of real queries: we look at where hybrid loses to pure word matching and adjust rank normalization so that relevance grows not on tests but on the live flow of requests.
Common technical concerns before starting work
Concerns before starting usually relate not to the idea of semantic search itself, but to how it will behave on a real corpus: heterogeneous files, scans, access rights and constant edits. Below are answers to what engineers and department heads most often clarify before work begins.
What will happen to "dirty" documents — tables, headers/footers, broken lines?
Before indexing, the text is normalized: repeated headers and footers are removed, broken lines are joined, table structure is restored, and the document is split into fragments with overlap and binding to a section.
Then noise does not enter the vector representation and is not mixed into the output during ranking.
What if the documents are scans without a text layer?
Recognition is connected: the page is converted to text, and each fragment receives a confidence score. Fragments with low confidence are marked and sent for manual review — so search does not start confidently answering based on garbage recognition. We measure the share of correctly recognized pages on a pilot sample of the customer's documents.
How to delimit access so that an employee does not see too much?
Rights are inherited from the source: file storage or document management system. The filter by documents available to the user is applied before ranking, not after — otherwise the top of the output may be empty, even though relevant materials exist. The log stores the query and the visibility scope in which it was executed.
Can search be deployed in an internal environment?
Yes. The solution is deployed in the customer's closed environment: the model, index and storage work locally, without contacting external services. Updates are also an internal procedure.
What happens when the knowledge base is updated?
Reindexing is incremental: if a document changes, only it is recalculated, and outdated versions are marked and drop out of the output.
Changing the representation model requires a full recalculation — this is a planned procedure with a maintenance window, and we give advance notice about it.
How to prove that search quality has actually improved?
We fix a set of real questions with reference fragments and calculate the share of hits in the top 5 and answer completeness. The same set is run through keyword search — comparison on the same data shows exactly where semantics wins and where exact matching is sufficient.
Let's discuss the task: how we will come back with an estimate and plan
So that the semantic search estimate is based on facts, not guesses, it is useful to collect several inputs for the first conversation.
The more precise the source data, the fewer assumptions in the stage plan, and the sooner it is visible exactly where matching quality will drop on your corpus.
- Query examples — 15–20 formulations from your users, including typos, synonyms and narrow domain terms.
- Corpus composition and volume — document types, formats (PDF, DOCX, scans, tables), approximate volume, languages, whether there is search history or manual labeling.
- Access requirements — where we deploy the environment, whether isolation without external calls is needed, who issues accounts and approves network rules.
- Current matching mechanisms — how search works now: full-text index, field filters, manual selections, what irritates users.
- Expected metrics — share of queries with a relevant first result, time to answer, number of clarifying re-queries.
Based on the discussion, we send an estimate of work and a stage plan: corpus preparation, model selection, search configuration, validation on your queries, delivery to the environment and documentation handover. Describe the task — we will come back with an estimate and plan.
