Improve Support Efficiency with AI Agent Assist for Knowledge Base Search
Support agents spend up to 30% of their time searching for the right answer in the knowledge base. Every extra second increases Average Handle Time (AHT) and lowers Customer Satisfaction (CSAT). Traditional keyword search delivers irrelevant results: synonyms and context are ignored, forcing agents to scroll through dozens of articles. Studies indicate that deploying an AI assistant in support reduces AHT by 25–40%.
Our AI-powered support system analyzes conversations in real time and retrieves relevant articles, instructions, and response templates. We have deployed such solutions for a dozen companies handling 500 to 10,000 daily tickets, achieving an average AHT reduction of 30–40%. A critical requirement is p99 latency below 1 second, achieved by caching embeddings, precomputing indices, and using GPU inference (T4 or A10G). For a team of 50 agents handling 200 tickets daily, this translates to annual labor savings of over $150,000.
How Answer Suggestion Works
Every new customer message triggers a chain of actions:
- Intent detection — identify the topic (e.g., "return product" or "password reset").
- Parallel search across three channels:
- Vector search on knowledge base embeddings (using
text-embedding-3-smallwith dimension 1536 and HNSW indexing for fast approximate nearest neighbor); - BM25 search on FAQs and templates (sparse retrieval);
- Search through resolved tickets with the same intent.
- Vector search on knowledge base embeddings (using
- Re-ranking — an ensemble of a lightweight CatBoost model and a document-freshness rule, plus a cross-encoder for precision.
- Brief summary generation (optional, via LLM for complex cases).
Why Vector Search Outperforms BM25 Alone?
BM25 works well with exact keywords but fails on synonyms and context. Vector search (cosine similarity on embeddings, a form of dense retrieval) finds semantically close documents even if wording differs. For example, a query "how to change my password" retrieves the article "account reset".
In practice, the best results come from a hybrid approach: combining BM25 and vector search with weights that we tune via cross-validation on your historical data. Hybrid search provides 85% recall@5, which is 25% better than BM25 alone.
Data Sources Used
- Knowledge base: Confluence, Notion, internal Wiki — articles and instructions.
- Ticket archive: resolved tickets tagged "successful" — practical solutions.
- Response templates: ready-made phrases for typical cases (up to 80% of tickets).
- Product documentation: technical specs, API docs.
All sources are connected via REST APIs or direct integration using ETL pipelines (Apache Airflow). The overall pipeline is a Retrieval-Augmented Generation (RAG) architecture.
Search Method Comparison
| Method | Latency | Recall@5 | Semantic Flexibility |
|---|---|---|---|
| BM25 (sparse) | <50 ms | ~60% | Low |
| Vector (dense) | <100 ms | ~75% | High |
| Hybrid | <150 ms | ~85% | High |
Impact on Support Metrics
| Metric | Before | After |
|---|---|---|
| AHT | 8–12 min | 5–8 min |
| Acceptance rate | 30–40% | 55–70% |
| CSAT | 3.8–4.2 | 4.3–4.7 |
Operator Panel Interface
A side panel shows 3–5 most relevant articles with brief summaries. One click inserts the full content into the reply box, editable. For templates, an "Insert as-is" button is available.
Adoption metrics we track:
- Suggestion acceptance rate — target >55%.
- Modification rate — how often the agent edits the suggestion.
- AHT before and after — reduction of 30–40%.
- CSAT — answer quality does not suffer; often improves.
Learning from Accepted and Rejected Suggestions
Every acceptance or rejection is a signal to retrain the ranker. We use ranking fine-tuning to adapt to your data. We run A/B tests with different algorithms on a subset of agents. Over 3–6 months, the acceptance rate grows from 40% to 60–70% as the system learns from your data.
Example Integration with Zendesk
The system connects via the Zendesk App Framework API. On every new ticket, a webhook sends the message content. The response returns a JSON array of suggestions. The UI widget renders on the agent sidebar.What's Included in the Work
- Audit of your current knowledge base: evaluate quality and completeness, remove duplicates.
- Build data pipeline: extraction, cleaning, vectorization (using
sentence-transformersor OpenAI embeddings). - Select and tune the ranker: hybrid BM25 + embeddings, trained on your history.
- Integration with CRM/Helpdesk: Zendesk, Freshdesk, Bitrix24, or your system via API.
- UI component for the agent panel (web widget or extension).
- A/B testing and calibration.
- Documentation and training for the support team.
Based on 50+ AI solution deployments, the project from audit to first hypothesis takes 4–8 weeks. Full rollout with learning takes up to 3 months. With 5+ years of experience in AI support solutions, we have deployed over 50 projects. Typical investment ranges from $20,000 to $40,000 for a mid-size company, with ROI achieved within 3-6 months.
Contact us to discuss your knowledge base and get a preliminary project assessment. Request an audit of your current knowledge base — we guarantee an acceptance rate above 50% after the first two weeks of use. Try a demo version to see the effect on your own data.