Why RAG Qdrant fails on narrow queries
The scheme looks like this: documents are split into chunks, each one is turned into a vector and goes into the Qdrant vector database. A query is vectorized in the same way, the database returns the nearest chunks, and the generative model answers based on them.
This works on broad questions — which is why document search passes acceptance, but it breaks down in production.
A narrow query is an SKU, a regulation number, a last name, a negation, a condition using “except”.
The embedding averages meaning down to the topic, so chunks “about this in general” make it into the top, while the needed paragraph with the exact wording ends up outside the first results.
The database returns irrelevant results; the stricter the similarity threshold, the more often nothing at all makes it into the context.
The second source of failures is chunking. A fact is torn across a chunk boundary: the condition in one, the exception in the next. Search brings back half, the model fills in the rest from its weights and answers just as confidently as with complete data. This is model hallucination: the context is formally there, it is just incomplete.
The third layer is the absence of result verification: similarity scoring is not calibrated, candidates are not reranked, and the model does not receive the signal “there is no answer in the documents” and takes the answer from memory. Metadata, meanwhile, does not always work as a filter.
Next we break down how this is closed: hybrid search, metadata filters, candidate reranking, and an abstention mode instead of a confident guess.
What changes after implementing RAG Qdrant
Search quality stops depending on the words used to ask the question.
The Qdrant vector database stores document chunks as vectors and returns nearest neighbors with metadata filters — the share of correct hits in the top three results is determined by the meaning of the question, not by wording matches.
Rephrase the question, change a synonym, make it shorter — the set of sources remains the same, and the variance between reformulations is measured and kept within a narrow corridor.
Whole classes of failures disappear, rather than “become less frequent.” An answer without a source, a link to a nonexistent chunk, a stitching together of statements from different documents — every sentence in the output is tied to a specific chunk, and this link is visible in the interface.
If the best found chunk is below the relevance threshold, the system honestly reports that there is no data in the database, instead of a plausible fabrication.
Output stability is tested on a set of real questions with pre-labeled reference sources.
We calculate three numbers: the share of reference hits in the top three positions, the share of answers with a correct document citation, and the share of correct abstentions where there is no answer in the database.
Before implementation, this is a conversation along the lines of “it seems to have become more accurate”; after, it is metrics by which regression is visible in the next index update.
A document was added or replaced — only it is indexed, without rebuilding the entire database. Access rights filters are applied at the search stage, so the same index serves different roles and does not return chunks to a person that they are not entitled to. This removes the most common fear of vector search in a corporate environment.
Search organization methods: comparison of approaches
What actually makes it into the model context is determined not by the vector database, but by the search mode and how documents are split into chunks.
We choose between pure vector search, hybrid search, selection with metadata filters, and a scheme with reranking — there is no universal option, each has its own cost in latency and retrieval completeness.
| Option | When it fits | Limitations |
|---|---|---|
| Vector search | Queries are formulated by meaning, terminology is fluid | Does not catch SKUs, codes, and rare terms — they have no close neighbors in the embedding space |
| Hybrid search | The query has both meaning and exact strings: abbreviations, identifiers | Two result sets need to be merged and weights calibrated; the index grows, search is more expensive |
| Metadata filters | Data is labeled by document, role, period, version | With incomplete labeling, relevant chunks are filtered out along with noise |
| Chunking | Long regulations where the answer is spread across sections | Small chunks lose context, large ones blur the vector |
| Reranking | The first selection yields dozens of candidates, order matters | Adds query latency and a separate ranking model to the pipeline |
In Qdrant, these modes work on a single index, so they can be combined without a second storage.
In practice, the filter narrows the search space, the hybrid pulls out exact terms, reranking fixes the order of the top results, and we select chunking to match the document structure and test it on the client's real questions.
Implementation stages: from audit to handover
Implementation begins not with the model, but with the document corpus: until it is clear what is in it and who can see what, any search tuning will be guesswork. That is why the work proceeds in stages, and each ends with an artifact you can verify before moving to the next.
- Data audit. We analyze the corpus: formats and sources, duplicates and outdated versions, the share of scans without a text layer, access separation by department. The output is a list of what is subject to indexing and record-level access rules.
- Collection schema. We decide how many collections are needed and why, what metadata accompanies each chunk, how a document is divided into parts, which embedding model is used, and which fields are filtered. The schema is fixed before the first load — redoing it on a live index is more expensive.
- Search setup. We assemble hybrid retrieval: vector search by meaning plus exact matching by terms, numbers, and abbreviations, with reranking and a cutoff threshold on top, so the system honestly says “there is no answer in the documents” instead of a plausible fabrication.
- Build and population. We create the indexing pipeline: initial corpus load, then incremental updates as documents are edited, with separate version storage and the ability to reindex one collection without affecting the others.
- Quality check. We prepare a set of questions with reference answers and sources, measure the share of hits in the right document, completeness and accuracy of citations, robustness to paraphrasing, and behavior on out-of-corpus questions. We analyze errors by cause: chunking, metadata, search, or query wording.
- Solution handover. We hand over the schema and configurations, corpus update and reindexing procedures, support instructions, train your team, and stay in touch during the first edits.
This order delivers a cumulative effect: by launch, you have not only working document search, but also a clear procedure for maintaining it.
Documentation and deliverables
A working system without documentation quickly turns into a black box: the first change of sources or switch of embedding model runs into trial and error.
That is why the solution is handed over together with a set of artifacts stored in the same repository as the code and updated along with it.
- Collection schemas in Qdrant — collection name, vector dimensionality, similarity metric, composition of service fields, and indexes on filterable attributes. They show what can narrow the selection and what to change as the corpus grows.
- Document chunking rules — chunk size, overlap, mapping of metadata to source and section. Without them, a new knowledge base block breaks retrieval quality.
- Update procedure — rebuild frequency, procedure for loading new documents, removal of outdated versions, and rollback to the previous index snapshot.
- Access policies — roles, separate read and write keys, visibility restrictions by department, data access log.
- Solution documentation — component diagram, data flows, list of failure points and the action for each: what to restart, what to reindex, where to look at logs.
- A set of validation questions with reference answers — run after every model change or search setting change, so degradation is visible before user complaints.
- Team instructions — how to bring up the environment, add a source, read search quality metrics, and where to look when answers are empty.
- Change log and responsibility boundaries — what we maintain and what remains on the client's side.
This set reduces incident triage to minutes and removes dependence on specific people: a new engineer joins the project through documents, not phone calls.
Case study: how we reduced the share of incorrect answers
Measurements were taken on a corpus of regulatory documentation: several hundred documents, revisions introduced at different times, some clauses canceling each other.
Before implementation, an engineer searched for the answer manually — opened regulations, checked clause numbers and effective dates.
One question took 12–15 minutes, and the share of incorrect answers reached a third: the person either took a canceled revision or could not find the needed clause and answered from memory.
| Metric | Before | After retuning |
|---|---|---|
| Time to find an answer | 12–15 minutes | 20–30 seconds |
| Share of incorrect answers | 34% | 6% |
| Source citations | not recorded | in every answer, with clause number |
What we changed
We split documents not by pages, but by structural units — clause, section, appendix — so that a whole chunk, rather than a fragment of a table, made it into the output.
Search was made hybrid: semantic similarity via Qdrant plus exact matching by clause numbers and terms, with candidate reranking and a filter by the current revision on top.
We separately set a confidence threshold: if there are no close chunks, the system answers “there is no answer in the corpus” instead of a plausible fabrication.
We tested on a control set of questions with reference answers — the same questions were run before and after the changes, so the figures are comparable. The main effect is not speed: every answer can now be verified with one click on a link to a specific document clause.
How hybrid search differs from vector search
Semantic search compares the query vector with chunk vectors and finds those close in meaning — it will return a paragraph about equipment service life even if the query is phrased as “how long does the device work.” The weak spot is rare tokens: SKUs, abbreviations, standard names.
The vector averages them down to general meaning and returns “similar in topic” rather than an exact hit.
Exact matching works by term occurrences: if the query contains a part code or standard designation, the lexical index finds exactly those chunks where this string occurs.
The flip side is paraphrasing: the user writes “accelerated delivery,” the documents contain “express logistics,” and there is no match.
Hybrid search runs both channels and merges the two candidate lists, while ranking weights determine how much the lexical part affects the final order.
The weights are not chosen by eye: they are tuned on a set of real queries and checked by the share of relevant answers in the top results.
The choice of mode is dictated by the composition of queries, not by the fashion for hybrid.
We look at request logs and calculate the share of queries with codes, abbreviations, and exact names: if it is small, semantic search gives the same result with fewer computations and is easier to debug. If queries are mixed, hybrid is enabled with separate weights for different question types.
In Qdrant, semantic and lexical vectors are stored in one collection, so hybrid does not require a second database or index synchronization: the mode is switched by a query parameter, and weights change without reindexing the corpus.
Technical concerns and frequently asked questions
Document search rarely raises questions at launch — they appear later, when the corpus starts changing and data must be kept inside the perimeter. Below are the things asked about most often.
How is an index migrated from another database?
By rebuilding: documents are read from the source, split into chunks using the previous logic, and loaded into Qdrant in batches with collection count verification.
The share of queries where the new output diverges from the old one is analyzed before switching traffic, so rollback to the previous database remains possible.
The corpus is updated weekly — will the output drift?
Only edits on top without reindexing the relationships between chunks are harmful: search starts returning outdated pieces. We version collections — the new one is populated in the background, the old one answers queries, and the switch happens when ready.
Can everything be deployed in an isolated environment?
Yes: models, the vector database, and the search service live in an isolated network without external calls, and updates go through an internal artifact registry. Access is separated by roles — only chunks permitted for a specific user are returned.
What happens with a multiple increase in document volume?
Thresholds are set in advance: collection size, number of segments, compaction and outdated-version removal rules.
As it grows, replicas are added and the collection is distributed across nodes, while quality is checked with a set of reference questions with expected sources — if the metric drops, the release is not rolled out.
Let's discuss the task and come back with an estimate and plan
To produce a working plan after analyzing the task, rather than general wording, we need a minimum of input from you: where the documents are stored and in what form, who uses them, which answers the system must give accurately, and where an honest “no data” is better than a guess.
We discuss access separately — which sources are read in full, which only by sections, and whether there is data that should not enter search.
Then we come back with a package you can use to make decisions, not with a presentation of an idea:
- Problem statement — list of scenarios, validation questions with expected answers, and evaluation criteria: the answer is correct, incomplete, or dangerous.
- Work plan — stages, their order, what we check at each, and which result determines the transition to the next.
- Solution composition — document parsing, vector index on Qdrant, answer generation, access rights layer; what we take ready-made and what we write for you.
- Data update scheme — how documents enter the index, how they are reindexed after edits, and what happens when an employee's rights are revoked.
- Acceptance procedure — a set of control questions, the share of correct answers, and testing of system behavior on topics not in the database.
- Project start — team composition on our side, points of contact, what is required from your specialists and when.
- Boundaries of the first version — what is deliberately not included in it and why.
Describe the task in free form — sources, users, and what is currently getting in the way. We will come back with a timeline estimate, work plan, and acceptance procedure, so you can see the result before the start.
