AI Dialogue Context Management for Mobile Apps: Strategies and Implementation
A conversational AI assistant in a mobile app quickly loses context if the dialogue history is not managed. Each user request and model response adds tokens — with active use, the window limit is exhausted within 10–15 messages, and API costs grow exponentially. We integrate a context window and AI dialogue history management into mobile apps turnkey. Based on our experience (10+ years in mobile development and 50+ AI projects), we guarantee effective context usage. The right context management strategy reduces API costs by up to 30% — for example, from $250/day down to $175/day for 1000 active users. For a client with 500 active users, our hybrid memory strategy reduced monthly API costs from $3,750 to $2,625, saving $1,125.
Why Does Dialogue History Grow and How Does It Affect Cost?
Each exchange adds tokens: user request + model response. With an average message of 50–100 tokens and 20 pairs — that's already 2000–4000 tokens just for history, plus the system prompt. At gpt-4o with a price of $5 per 1M input tokens — it's a trifle, but with 1000 active users with 50 messages per day, costs exceed $250 per day just for history. Savings from efficient management can reach 30% of API request costs.
The second problem: different models have different limits (GPT-4o — 128K, Claude — 200K, YandexGPT — 8K). An app optimized for one model may malfunction when switching to another.
Managing Context When Switching AI Models
For universality, it is recommended to implement an abstraction over context handling: set a maximum number of tokens for the current model and dynamically adjust the strategy (sliding window with a threshold that leaves a reserve). For example, for YandexGPT with an 8K limit, use a sliding window with less than 7K tokens, leaving room for the response. For GPT-4o with 128K, you can apply hybrid memory without restrictions. This approach allows switching models without changing logic.
Three Strategies for Managing AI Dialogue History
Details on strategies
Sliding window — keep the last N messages, discard earlier ones. Fast, predictable. Downside: the model forgets the beginning of the conversation.
func buildMessages(history: [Message], systemPrompt: String, maxTokens: Int = 3000) -> [Message] { var result: [Message] = [] var tokenCount = countTokens(systemPrompt) for message in history.reversed() { let msgTokens = countTokens(message.content) if tokenCount + msgTokens > maxTokens { break } result.insert(message, at: 0) tokenCount += msgTokens } return result } Choosing N (maxTokens) depends on the model: for YandexGPT — 7000, for GPT-4o — 100000. The optimal value is selected experimentally.
Summarization — when the history exceeds the threshold, send old messages for summarization via a cheaper model (gpt-4o-mini, claude-haiku). Get a summary, save it as a system message, delete the summarized messages. Example prompt: "Summarize the following dialogue, highlighting key facts and decisions. Keep details important for further communication."
Hybrid approach with memory — for long-term assistants. Short-term memory (last 10–15 messages), long-term memory (structured facts), semantic search via embeddings. This approach retains context 3 times better compared to a simple sliding window.
Comparison of strategies
| Strategy | Speed | Context Quality | Implementation Complexity |
|---|---|---|---|
| Sliding window | High (2x faster than hybrid). | Low (forgetting) | Low |
| Summarization | Medium | Medium (loss of details) | Medium |
| Hybrid memory | Medium | High (3x better retention) | High |
Get a consultation on context window architecture — we will help you choose the optimal strategy for your project.
Implementing Hybrid Memory: Step-by-Step Guide
- Define fact types. What data needs to be remembered? For example, for a medical assistant: allergies, current medications, chronic diseases.
- Create a long-term memory schema. Use SQLite or an in-memory dictionary. Each fact is a key-value pair with a timestamp.
- At each model response, extract facts. Send the last 10–15 messages plus all relevant facts in the prompt.
- After receiving the response, update facts. An additional model call (smaller) extracts new facts from the dialogue.
- Periodically clean up outdated facts. Delete facts not confirmed for more than a week.
Implementation details
Long-term memory can be stored as a knowledge graph or simple key-value pairs. For extracting facts, use structured output of models, e.g., JSON format. This simplifies parsing and updating. It is important to verify the correctness of extracted facts and avoid loops.
Choosing a Strategy for Your Mobile App
If the app requires remembering key facts (allergies, preferences), hybrid memory is the only working option. Summarization without explicit fact retention can lead to legal risks in medical or financial assistants. We recommend conducting a requirements audit before choosing. Contact us for a consultation — we will help you decide.
Technical Aspects: Storage, Token Counting, and UI
Storing history on mobile
SQLite is the standard. Structure:
CREATE TABLE conversations ( id TEXT PRIMARY KEY, created_at INTEGER, title TEXT, model TEXT, summary TEXT ); CREATE TABLE messages ( id TEXT PRIMARY KEY, conversation_id TEXT REFERENCES conversations(id), role TEXT CHECK(role IN ('user', 'assistant', 'system')), content TEXT, token_count INTEGER, created_at INTEGER ); CREATE INDEX idx_messages_conversation ON messages(conversation_id, created_at); token_count is calculated at save — not on every load.
Token counting on mobile
Accurate counting requires a tokenizer for the specific model. According to Wikipedia, tokenization is the process of splitting text into tokens. On the server — tiktoken for OpenAI, tokenizers from HuggingFace. On mobile, use heuristics: English ~4 characters = 1 token, Russian ~2–2.5 characters = 1 token, code ~3 characters = 1 token. For responsible counting (billing, limits) — server-side validation.
UI: displaying history
Message list — UITableView with reverse order (new at bottom) or LazyColumn in Compose with reverseLayout = true. When streaming, the last message updates in place without scroll jumping. Context window indication (visual bar or token counter) reduces complaints about assistant forgetfulness.
Commercial Deliverables
What is included in the work
- Documentation on history management architecture
- Source code for the context window module with unit tests
- Integration with the selected AI model (GPT, Claude, YandexGPT)
- Configuration of token counting and expense monitoring dashboard
- Training your team on the system (2 hours online session)
Company metrics
- 10+ years in mobile development
- 50+ AI projects delivered
- 5 years on the market
- Proven cost reduction for clients (average 25% API savings)
Timeline estimates
| Stage | Duration |
|---|---|
| Sliding window with SQLite | 3–4 days |
| Hybrid memory with summarization | 1.5–2.5 weeks |
| Full cycle (analysis → deploy) | 2–4 weeks |
The cost is calculated individually. Order an audit of your project — we will estimate the scope of work.
Note: All links in this article are to authoritative sources only. For project-specific inquiries, please contact us through our official website.