Why an AI chatbot in Teams does not find answers in documents
The bot is installed in Teams, responds quickly and confidently — but when asked about the vacation policy, it gives general reasoning.
The reason is almost never the model: search across internal documents is either not connected to real sources, or it is assembled from a single network drive, while knowledge lives in the wiki, ticketing system, and email. A model without relevant context does not admit it does not know and fills in a plausible answer.
The second layer is access rights.
A bot that accesses storage under a service account sees either everything or nothing: in the first case, an employee receives a fragment of a confidential contract; in the second — "the documents contain no information." The working option is a query on behalf of the user with group permissions inherited at the fragment level, not at the level of entire files: otherwise a shared document with one confidential chapter drops out of retrieval entirely.
The third layer is retrieval quality. Splitting text into fixed blocks breaks tables and headings, so vector search returns "similar in topic" rather than an answer to the question.
A hybrid scheme — exact terms and abbreviations together with vectors — and reranking candidates move the needed fragment to the top of the context. Without a source link, the answer is useless: an engineer cannot verify which revision of the document the figure came from.
Separately, there is index freshness: without reindexing, the bot quotes a revoked order for months.
Corporate search most often fails to find answers not because of the model, but because of the gap between the index, permissions, and retrieval — we break down how to connect them into one environment.
What changes after implementation: speed and accuracy of answers
Before implementation, an employee looks for an answer to a work question for an average of 8–12 minutes: going through file storage, scrolling through correspondence, asking in a general channel.
Some of these questions go to support again — a noticeable share of requests are on typical topics, and the first line answers the same thing in different words.
The assistant lives in Microsoft Teams, so search happens in the same window where the person already works: there is no need to switch context and reformulate the question.
The answer is assembled not "from memory" of the model, but from fragments of your documents: the search index is rebuilt on schedule, and each answer contains a link to the source file and section where the confirmation is located.
This produces two measurable effects. Time to first answer drops to 20–40 seconds, including the time to formulate the question.
Accuracy stops being a matter of faith: we count the share of answers with a confirming source and the share of cases where the assistant honestly hands the question to a human instead of guessing.
We take metrics on a prepared set of control questions and on the real flow of requests, so that growth does not turn out to be a novelty effect.
Offloading the support service is a consequence, not a slogan. Repeat requests on topics covered by the knowledge base leave the first-line queue, and the service handles nonstandard cases that require investigation.
In measurements after launch, the share of such requests decreases by 40–60%, and support response speed grows because the queue stops consisting of identical questions.
Search options across internal data: comparison of approaches
The search scheme determines what the bot returns to an employee's question in Teams, and it is worth choosing before development, not after the first complaints about answer quality.
Keyword search is precise on identifiers and names, but breaks on paraphrasing; vector search across documents understands the meaning of the question, but skips SKUs and contract numbers.
Hybrid search usually wins in comparison with single schemes: it combines both signals and adds source-grounded search, where the answer must be accompanied by a link to a fragment. Below are four real options with their applicability boundaries.
| Option | When it fits | Limitations |
|---|---|---|
| Keyword search | A base with stable terminology, search by numbers and names | Does not find synonyms and paraphrasing; quality depends on manual rules |
| Vector search across documents | Heterogeneous texts, questions formulated in one's own words | Loses exact identifiers and rare abbreviations; requires reindexing when the base is updated |
| Source-grounded answers | A verifiable link to a fragment is needed, not a ready-made conclusion | Requires a labeled corpus and access control, otherwise it answers confidently but off target |
| Hybrid scheme | Mixed queries: meaning plus exact identifiers | Ranking setup is more complex; without regular quality evaluation, the share of irrelevant fragments grows |
Off-the-shelf connectors have their own limitations: they pull documents from the source as is, without fragment-level access labeling, so permissions have to be additionally configured.
Separately, we check the scheme's behavior when the base updates — this determines how long it takes for a new document to become discoverable.
Stages of work: from source audit to environment handover
Work proceeds in iterations: the result of each stage is an artifact that the next stage relies on. Below is the sequence for implementing corporate search, from knowledge inventory to handover of the environment to your team.
- Knowledge source audit. We collect a list of systems — file storage, wiki, email archives, request databases — with volume and update frequency; separately note duplicates and dead sections. The result is a source map with owners and connection priority.
- Search design. We determine which document types are chunked, how metadata is stored (department, date, type), and how semantic and exact search are combined. This determines whether the bot finds the needed clause of a policy or just a similar paragraph.
- Access control. Rights are inherited from source systems: in the answer, an employee sees only what is directly open to them. We test on mixed roles — manager, external contractor, new employee.
- Bot deployment design and configuration. We decide where the indexing service and model live, how the index updates, and what happens when a source is unavailable. The bot itself is embedded in Microsoft Teams as a regular application.
- Testing on control questions. Several dozen questions with reference answers: we record the share of correct source hits and questions where there is no answer. We analyze errors by causes — no document, outdated, lost during splitting.
- Acceptance of the search solution. Metrics are agreed in advance: source hit accuracy, response time, no leaks of confidential data. Separately, we check honest "I don't know" instead of a fabricated answer.
- Handover of the environment. We hand over the deployment design, index update procedure, administrator instructions, and development plan. The team supports the environment itself; we remain available to step in.
What we hand over: configurations, diagrams, and procedures
The handover package is not a folder of files "for viewing," but a set of artifacts your team can use to support the solution without us.
Each document is tied to a system node: what is configured, which parameter values are selected and why, and what happens on deviation.
- Configuration descriptions — index dictionary, embedding model, chunk size and overlap, how many sources are returned in the answer, relevance threshold. With rationale: why these values were chosen and how to change them.
- Data flow diagram — from sources (file storage, wiki, email) through parsing and indexing to the answer in chat. With notes where data is encrypted and what is deleted after processing.
- Data access policies — bot rights are tied to the employee account: the answer will not include what is closed in the source system. Role and exception matrix attached.
- Search implementation documentation — procedure for connecting new sources, format requirements, checklist of checks after addition.
- Employee guide — how to formulate a query, how to evaluate an answer, where to report an error; short one-page version for distribution.
- Knowledge base update procedure — frequency of full and partial reindexing, owner of each source, actions on desync.
- Rollback plan and change log — which artifact versions are compatible and how to roll the index back to a previous state.
Acceptance is based on control questions: does the system deploy in your environment, is the answer reproducible for the same query, is access control observed. Anything that fails the check — we refine before handover.
Case study: knowledge base search for a service team
What it was before
The service team answered employees' internal questions manually: requests came via email, a general chat, and direct messages to on-duty engineers.
The knowledge base existed in two places and diverged — equipment maintenance instructions, escalation policies, and customer response templates were updated without synchronization.
An employee with the question "which policy is currently in effect" waited 25-40 minutes for an answer if they got into the queue, and received a link to an outdated version of the document in about one in five cases.
What we changed
We built a corporate chatbot in Microsoft Teams and connected it to the knowledge base through semantic search: the system finds relevant fragments of policies and generates an answer with a mandatory source link.
If search confidence is low, the bot does not speculate but immediately offers escalation to an on-duty specialist with the question context already attached.
The index is rebuilt on a document change event, not on schedule, so yesterday's policy update is visible to an employee today, not in a week. Only active versions remain in the output — drafts and archived revisions do not enter the index.
What was the result
The share of questions closed without specialist involvement grew from 34% to 71%: on-duty engineers are no longer the first line and handle only real deviations. Time to first correct answer decreased from tens of minutes of waiting in the queue to a few seconds.
A class of errors "answer based on an outdated policy" also disappeared — every bot statement is accompanied by a link to the current revision, and a disputed answer can be verified in one click.
What to do if the chatbot is already deployed but answers inaccurately?
Inaccurate answers are almost never related to the model itself — it answers exactly to the extent that the search index under it is complete, fresh, and correctly chunked.
We run diagnostics across four points: source freshness, chunking documents into fragments, access control, and verification of the answer against the primary source. If even one is not covered, the bot answers inaccurately even to simple questions.
Most often, outdated sources are to blame: the index was built during implementation, documents have been edited since, and reindexing runs once a day or not at all — the bot confidently cites a revoked revision of the policy.
Data actualization for search should be event-driven: a document change triggers an update of its fragments in the index within minutes, not within a day.
The second cause is lack of chunking: a document of dozens of pages enters the index whole, and its vector averages several topics at once, so a question about approval deadlines gets an answer about the signing procedure.
Splitting documents into fragments by semantic sections, with overlap at boundaries and metadata (version, date, department), restores the hit into the needed paragraph.
We filter access rights before search, not after retrieval: otherwise part of the answers is assembled from closed documents or, with coarse cutoff, the bot says "not found" where an answer exists.
And final — verification of the answer against the primary source: without a link to a specific fragment, we do not show the answer. We measure improvement in answer quality on a set of reference questions: the share of answers with a confirmed source before and after edits.
Common technical concerns before starting work
Concerns before starting relate not to the model but to search behavior in a corporate environment: what it will return for a specific question, to whom it will show the found document, and what will happen to the base in six months.
How accurate will search be across our documents?
Accuracy is determined not by the model but by corpus preparation: chunking, overlap, metadata, ranking. The answer comes with a link to the source fragment — the error is visible immediately.
How does search account for access rights?
Access labels from the source are written to each index fragment. The query is executed on behalf of the user, so an employee sees exactly the set allowed to them in the source system.
What to do with heterogeneous sources?
Systems are connected by separate connectors: knowledge bases, file storage, correspondence, tickets. Everything is brought to a unified structure, but the link to the original is preserved — the answer can be checked against the primary source.
Where are data and index stored?
In your environment: on your infrastructure or in a private dedicated segment. The index, model, and query logs do not leave the perimeter; only explicitly allowed data goes outside.
Who keeps the knowledge base up to date?
Reindexing runs on schedule and on event — when a document is changed or deleted. Outdated fragments are cleaned out, and a coverage report shows sources that have not been updated for a long time.
Let's discuss the task: we'll come back with an estimate and work plan
To come back with an estimate and work plan, not follow-up questions, it helps to gather several inputs. A formal document is not needed — enough to understand where knowledge lies, who needs it, and what constraints exist for the environment.
These answers determine the architecture of the search chatbot and the composition of the stages. What is most often worth preparing for the conversation:
- A list of knowledge sources that should enter search: policies, project documentation, knowledge base, tickets, file storage — and the owner of each.
- Access model: how rights are currently separated so the answer is assembled only from documents the employee is entitled to see and does not pull in others' data.
- Twenty to thirty real employee questions with expected answers — on this sample we check output quality and configure fragment selection.
- Corpus volume and change frequency: how quickly documents become outdated and how often materials need reindexing.
- Requirements for the environment: where documents and vectors may be stored, whether external services are prohibited, who grants access to Teams and storage.
- Who on the customer side accepts the result and in what windows the pilot user group can be launched.
Separately, it is useful to agree in advance how we will understand that search works: the share of answers with a link to a confirming source, a reduction in the flow of similar support requests, and the time to first correct answer for a new employee.
If some points are not yet formulated — this does not block the start. We'll discuss search implementation based on what exists and come back with an estimate, work plan, and boundaries of the first stage.
Even a short request for chatbot implementation — describe users, sources, and constraints — is sufficient for us to calculate timelines.
