How to Build an Enterprise AI Assistant in Microsoft Teams with RAG

Employees spend up to 30% of their working time searching for information in SharePoint and communicating in Teams. For a company of 1000 people, this amounts to tens of thousands of unproductive hours per year — substantial losses for the business. The ticket system is overloaded with repetitive HR, IT, and document management questions. An AI chatbot embedded in Microsoft Teams solves both problems: it answers queries directly in the chat using the corporate knowledge base. We develop such bots turnkey — from architecture design to tenant-wide deployment.

At the core is a hybrid RAG architecture: an LLM (GPT-4, LLaMA 3, or Mistral) generates responses, and a vector database (Qdrant, ChromaDB, pgvector) stores 1536-dimensional document embeddings. This provides up-to-date answers without retraining the model, with hallucination control through chunk-based search. The 128K token context window handles entire documents, and chain-of-thought prompting improves complex query quality.

What problems does the Teams bot solve?

Ticket overload: 40% of helpdesk queries are routine (password reset, ticket status, policies). The bot responds instantly, offloading first-line support by 60%. The savings on helpdesk are substantial for medium-sized businesses, potentially saving over $100,000 annually for a 1000-employee company.

• Slow document search: An employee spends on average 12 minutes finding the right file in SharePoint. The RAG bot finds an answer in 2-3 seconds using semantic search on embeddings.
• No single entry point: Information is scattered across Outlook, Teams, OneDrive, and internal portals. The bot aggregates data via Graph API and Adaptive Cards, displaying results as interactive cards.

Why RAG over fine-tuning?

Fine-tuning a model on corporate knowledge requires labeled data, computational resources, and risks overfitting. RAG supplements the prompt with relevant documents at query time — no retraining, guaranteed relevance. In practice, RAG architecture shows 60% higher accuracy (F1 metric) on frequently updated documents compared to fine-tuning. RAG on Wikipedia Additionally, RAG allows changing data sources quickly without restarting the bot, whereas fine-tuning requires retraining.

Architecture of a Teams bot

The Teams bot is built on Azure Bot Framework. The platform supports automatic scaling and deep Teams integration. Key components:

• Bot Framework SDK (Python/C#/Node.js) — bot logic
• Azure Bot Service — registration and routing
• Bot Framework Connector — Teams integration

from botbuilder.core import ActivityHandler, TurnContext from botbuilder.schema import ChannelAccount, Activity class MyBot(ActivityHandler): async def on_message_activity(self, turn_context: TurnContext): user_input = turn_context.activity.text response = await ai_handler.process( user_input, user_id=turn_context.activity.from_property.id ) await turn_context.send_activity(Activity(type="message", text=response)) async def on_members_added_activity( self, members_added: list[ChannelAccount], turn_context: TurnContext ): for member in members_added: if member.id != turn_context.activity.recipient.id: await turn_context.send_activity("Hello! How can I help?") 

Adaptive Cards

Teams Adaptive Cards are JSON descriptions of UI components. They are far richer than text messages:

{ "type": "AdaptiveCard", "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "version": "1.5", "body": [ {"type": "TextBlock", "text": "Analysis Result", "weight": "bolder"}, {"type": "TextBlock", "text": "{{analysis_text}}", "wrap": true} ], "actions": [ {"type": "Action.Submit", "title": "Accept", "data": {"action": "accept"}}, {"type": "Action.OpenUrl", "title": "Details", "url": "{{details_url}}"} ] } 

Integration with Microsoft 365

The Teams bot can access Graph API with the user context:

• SharePoint: search documents, retrieve files
• Outlook Calendar: create meetings, check availability
• Azure AD: org structure, find colleagues, groups
• OneDrive: access user files

Deployment approach comparison

Vector database comparison

Qdrant is up to 2x faster than ChromaDB and 3x faster than pgvector, making it ideal for high-throughput enterprise scenarios.