Why RAG indexing of corporate knowledge bases produces junk answers
RAG indexing of a corporate knowledge base looks like uploading documents into a vector store, but the model builds its answer not from a whole page, but from several retrieved fragments.
If a fragment is assembled incorrectly, the model will not stay silent — it will neatly complete the answer and deliver it with the same confidence as a correct one.
The first source of failures is heterogeneous markup. Pages in Confluence, Notion, and SharePoint consist of nested blocks, callouts, and task lists; a flat parser strips away the hierarchy, and a section heading gets separated from its body.
Steps from a neighboring instruction end up in the output, and the sequence of actions loses its meaning.
The second is versions and copies. One instruction exists in three places and in two revisions; after chunking, they are almost indistinguishable by vector, and documentation search pulls any of them.
Broken tables add their own issues: parameters come through without column headers, and attachments — diagrams and policies in files — do not enter the index at all.
Together, this produces irrelevant answers while search formally works: the model receives adjacent paragraphs instead of the needed item and fills gaps with a guess. Output quality drops not because of the model, but because of what it was fed.
Next, we break down how this is addressed: structural markup parsing, chunking by semantic blocks instead of fixed length, deduplication of revisions, and separate passes for tables and attachments.
What changes after a managed rebuild of knowledge base search
The first thing that changes is that output stops depending on whether the employee guessed the right wording. Hybrid search combines exact keyword matching and vector similarity, while reranking lifts fragments that actually contain the answer to the top.
We index Confluence, Notion, and SharePoint spaces in a single RAG pipeline: the sources stay in place, and only the search layer changes.
Output completeness and grounding in the source
Before the rebuild, pages were indexed whole: a long document was blurred into a single vector, and the needed paragraph did not make it into the top.
After chunking into semantic blocks with context overlap, the share of queries where the correct fragment appeared in the top five grew from 61% to 93%.
Answers without grounding in a source decreased by about a factor of four — if there is no relevant block, the system says so rather than completing the wording.
Response speed and knowledge base freshness
Output completeness is useless if the answer takes half a minute: people go back to manual folder search. We keep response speed predictable under peak load — the query queue is served in single-digit seconds because retrieval and reranking are cached by normalized query.
Index updates are event-driven: a page edit enters search within minutes, not within a day, and broken links to renamed sections no longer surface.
The result is measured not by "it became more convenient," but by the share of answers with a confirmed source, response time under load, and how quickly the knowledge base catches up with changes. Next — how these metrics look on a specific base before and after.
Full, incremental, and hybrid rebuilds: what to choose
The index update mode is part of the service commitments, not an implementation detail: it determines how quickly a new answer appears in search and how much load is sent to the source systems.
The choice is determined by two measurements — the share of documents changing per day and the acceptable lag between an edit and the index. Without them, the service either answers from outdated versions or constantly rereads entire repositories.
We compare three modes by data volume per cycle, sensitivity to edit frequency, and availability requirements during build.
| Mode | When it fits | Limitations |
|---|---|---|
| Full rebuild | Rare edits in batches, a maintenance window is available | Reads the entire corpus: on large volumes, the cycle takes hours |
| Incremental build | A steady, targeted stream of edits | Changes and deletions must be tracked; when markup changes, discrepancies accumulate |
| Hybrid index | Mixed sources: some change often, others are stable | Two update pipelines, completeness reconciliation and cleanup of outdated fragments are required |
How to choose a mode for the change stream
We look at the edit profile, not the base size. A steady stream of a fraction of a percent of pages per day is handled by incremental build; rare mass changes to structure or markup are a reason to periodically run a full rebuild and maintain a hybrid index.
We switch without downtime: the new version is built alongside, enabled after checks on validation questions, and rollback goes to the previous version.
RAG indexing stages: from source audit to acceptance
The workflow is the same for any set of sources. First we record what actually lives in the bases and how often it changes, and only then do we build the index — otherwise some documents do not enter search, and some exist as outdated copies.
- Source audit. We traverse Confluence, Notion, and SharePoint spaces and sections: volume, edit frequency, owners, duplicates. Restricted sections are either excluded or tagged with an access level so that permissions are preserved in the output as well.
- Chunking scheme. We split by semantic blocks — headings, tables, instruction steps — not by a fixed window. A single fragment size for a policy and correspondence produces broken meaning at the boundaries.
- Document metadata. We attach source, section, owner, modification date, and access level to each fragment. Without them, filters by space and document revocation from the index do not work.
- Update pipeline. We configure incremental reindexing on change and deletion events, and a scheduled full pass — as insurance against missed events.
- Quality check. We assemble a validation set of questions with reference answers and see whether the needed fragment appears in the top positions. Misses are analyzed separately: most often they are duplicates or an outdated version.
- Handover to operation. We formalize the reindexing policy, the person responsible for knowledge base quality, and the procedure for connecting new sources. Acceptance runs on your data, not on a demo set.
Text chunking schemes, access policies, and update policies
Indexing is not a one-time launch, but a set of artifacts that remain in your environment and then live a life of their own. We hand them over together with the working build: for each item, your team has a file or policy, not verbal agreements.
- Source map. Which spaces, sections, and bases enter the index, who owns each, what is deliberately excluded and by whose decision.
- Permission inheritance rules. Access rights from the source system are carried into the index along the same hierarchy: an employee receives in the answer only those fragments that are open to them in the wiki or repository itself.
- Build configurations. Chunking schemes by document type — policies, tickets, tables, correspondence — with fragment size and overlap for each type.
- Update policy. Reindexing frequency by source, handling of deleted and moved documents, behavior when access rights change.
- Quality metrics. Validation question set, share of answers with a link to a supporting fragment, share of honest "I don't know" refusals, iteration dynamics.
- Team instructions. How to connect a new source, how to revoke access, what to do if an answer diverges from the source document.
Support without us
Everything listed is working configurations and textual policies, not closed-source code: a new source or a changed access policy is edited by your engineers in a single iteration.
We get involved when the document corpus itself changes — then we revisit chunking schemes and quality metric thresholds.
Case study: how we reduced the share of irrelevant answers in support
Before implementation, the knowledge base lived in three systems: policies in Confluence, product notes in Notion, and email templates in SharePoint.
Search worked by keywords, so for the question "how to process a return," an operator received a dozen pages and decided for themselves which one was current.
Answers went to customers without grounding in a document, and discrepancies surfaced only at the repeat-contact stage.
We consolidated the sources into one index, split documents into fragments by heading hierarchy, and preserved metadata: section owner, last edit date, status "active/archive." We added a confidence threshold to the answer pipeline — if there is not enough grounding in the documents, the system does not make things up but passes the question to an operator along with the retrieved fragments.
Metric dynamics
| Metric | Before | After |
|---|---|---|
| Answers without grounding in a document | 38% | 6% |
| Share of irrelevant answers | 31% | 8% |
| Answer preparation time | 7 minutes | 45 seconds |
| Requests requiring repeat review | 44% | 13% |
| Answers with a source link | no | 100% |
Team feedback
Operators stopped cross-checking three systems manually: a link to the source document eliminates disputes about where the wording came from.
The support manager notes something else: new employees reach independent answers noticeably faster because the system shows where the standard is located.
Edits in the base became targeted. Logs show which fragment produced an incorrect answer — a specific section is corrected, not the entire policy, and after reindexing the fix immediately enters the answers.
What to do if the knowledge base has already been indexed by a third-party service?
First we determine what remains from the third-party service: chunking parameters, metadata composition, access rights accounting.
If this is present in the export, we transfer the settings and rebuild what is missing; if only ready-made vectors without structure were provided, it is cleaner to reindex the sources directly. An index without metadata almost always produces worse output.
Two losses are immediately visible. Permissions: documents were indexed under a service account — inheritance is lost, and pages unavailable to the requester end up in the answer.
Duplicates: one instruction in different Confluence spaces, in an attachment, and in a personal copy produces three similar fragments and pushes out the rest. We address this with deduplication by content hash and link canonicalization.
Soft migration without downtime: the new index is built alongside the old one, populated with a snapshot of the sources, and caught up on changes. Requests still go to the old one, while the same sample goes to the new one in shadow mode — we compare quality on a live stream.
Switching is done by flag when the metrics match; rollback uses the same flag, and we keep the old index in reserve for several weeks.
We compare approaches across three scenarios: transferring settings is fast but inherits old chunking errors; parallel launch is more resource-intensive but tests quality on real questions; full reindexing is the most predictable and requires a window for the first pass.
Acceptance is reference questions with known answers: the share of hits of the needed fragment in the top three and zero answers from documents outside the user's permissions.
Technical concerns: access rights, markup, loss of context
Concerns at the evaluation stage are usually the same: who will see a fragment, what to do with tables and attachments, and whether meaning will fall apart during chunking.
All of this is addressed at the indexing pipeline level: each fragment has an owner and metadata, and search filters candidates by them before the text reaches the model.
How are access rights to fragments inherited?
Rights are read from the source together with the content — page, space, groups, personal exceptions — and stored in the index next to the fragment.
Search checks them for a specific user before retrieval: an unavailable piece does not enter the candidate set, and the model does not see it. Permission changes are picked up during the next synchronization.
Will tables and attachments turn into mush?
Tables are parsed structurally: the row and column header are preserved, otherwise the value loses meaning. Attachments — documents, presentations, PDFs — go through a separate pipeline and are linked to the parent page.
Each fragment retains a link to the source section, and the answer can be verified in the original source.
How do you avoid losing context during chunking?
Fragments are split along semantic boundaries with overlap, and section headings and the path through the page tree are written into metadata. The model receives not an isolated piece, but a fragment together with its hierarchy. This reduces the share of answers where a phrase torn from context is interpreted incorrectly.
What about outdated pages and contradictions?
Versions are deduplicated by content; in a conflict, priority is given to the fresher fragment confirmed by the owner. Archived spaces and drafts are excluded from the index by an explicit rule.
Documents in different languages?
Multilingual sources are indexed in a single vector space: the question and the document are brought to a common representation, so relevant content is found regardless of the page language. The answer is formulated in the language of the question.
Let's discuss the task: where we start and what we need from you
To come back with an estimate rather than generalities, we need a minimum of inputs: where the knowledge lives, how quickly it changes, and who is responsible for access.
The first task review is one conversation with an engineer, in which we go over the source structure and agree on a work plan. Below is what to prepare for this meeting.
- Source audit. A list of Confluence spaces, Notion bases, SharePoint libraries, file repositories, and internal portals — with a note on what is actually used in work and what remains an archive and will not enter the index.
- Edit frequency. How quickly pages become outdated, whether there is a publication policy, who monitors freshness, and whether everything needs reindexing or only changed content is enough.
- Access and exports. A service account with read permission, a page export in an open format, or a one-time snapshot of the base. If the environment is closed, we discuss building inside your network.
- Data restrictions. Sections that cannot be indexed: HR records, contracts, personal data. We build exclusions into collection rules at the start, not afterward.
- Expected scenarios. 5–10 real questions the assistant should answer — they show which document types are critical and which can be left alone.
- Acceptance owner. One person on your side who confirms answer completeness and resolves disputed cases during setup.
After the review, we send a work plan: index composition, update mode, a scheme for checking answers on a test set, and an estimate of timelines by stage. Describe the task — we will come back with an estimate and a plan.
