AI Summarization of Long Texts: Fitting Everything into the Context Window
Summarizing a long text immediately hits the context window limit. GPT-4o accepts 128K tokens (roughly 100K words), Claude 3 — 200K. At first glance, that's enough, but legal contracts of 200 pages, technical reports, or books often exceed the limit. Even if the text fits, a long context increases cost and response latency. For example, a typical supply contract may contain 150 pages of fine print — a direct request would cost about $0.03 to process, but the result may be incomplete due to the "lost in the middle" effect. Research on Lost in the Middle shows that models recall less information from the middle of a document. Therefore, for large documents, decomposition strategies are needed.
We use three main strategies: direct summarization, Map-Reduce, and Refine. Each suits a different size and quality requirement. Below are the details.
Strategies for Different Text Lengths
| Strategy | Token volume | Time | Quality | Complexity |
|---|---|---|---|---|
| Direct | up to 80K | Low | High | Low |
| Map-Reduce | up to 500K+ | Low (parallel) | Medium | Medium |
| Refine | any | High (sequential) | Very high | High |
Direct summarization works for texts up to 50–80K tokens. We send the entire text in one request and ask for a summary. Simple, cheap to implement. The limitation is token cost and latency (the model processes a large context more slowly).
Map-Reduce is for texts that don't fit into the context. We split into chunks → summarize each chunk → summarize the summaries. Map-Reduce is 3 times faster than Refine for large documents because it processes chunks in parallel.
Map-Reduce Implementation Example
async def map_reduce_summarize(text: str, chunk_size: int = 4000) -> str: chunks = split_text(text, chunk_size) chunk_summaries = await asyncio.gather(*[ summarize_chunk(chunk) for chunk in chunks ]) combined = "\n\n".join(chunk_summaries) if count_tokens(combined) > chunk_size: return await map_reduce_summarize(combined, chunk_size) return await summarize_final(combined) asyncio.gather — parallel API requests for all chunks simultaneously. For 10 chunks, the time is nearly the same as for one.
Refine — summarize the first chunk, then refine the summary with each subsequent chunk. The final summary is enriched sequentially. Quality is higher than Map-Reduce for connected narrative texts, but slower — requests are sequential.
How to Manage Tokens and Avoid Errors?
The main mistake is not counting tokens before sending. tiktoken (Python) or gpt-tokenizer (JS) give accurate counts:
import tiktoken enc = tiktoken.encoding_for_model("gpt-4o") token_count = len(enc.encode(text)) if token_count < 100_000: return await direct_summarize(text) elif token_count < 500_000: return await map_reduce_summarize(text, chunk_size=8000) else: return await map_reduce_summarize(text, chunk_size=4000) Different summary types require different prompts:
- Executive summary (for executives): 3–5 sentences, only key decisions and figures
- Detailed retelling: structured list with subtitles
- Key points list: bullets without narrative
- Answer to a question: "what is this document and what needs to be done?"
On mobile (using Swift or Kotlin), we offer the user to choose the summary type before launch.
How to Avoid Duplication in Summarization?
With Map-Reduce, the final summary may repeat similar points from different chunks. Duplication is eliminated by explicitly stating in the prompt: "Combine similar points, do not repeat the same idea twice." For legal and financial documents, we use structured output in JSON format with fixed fields (parties, obligations, deadlines, key_figures). This is more reliable than free text.
How to Configure Summarization for Your Document
- Determine the model's maximum context size (e.g., 128K for GPT-4o).
- Split the text into chunks of 4–8K tokens, respecting paragraph boundaries.
- Choose a strategy: direct for short texts, Map-Reduce for medium, Refine for connected narratives.
- Configure prompts for each summary type (executive, detailed, etc.).
- Implement caching by document hash and progress updates via SSE.
Summarization Progress on Mobile
Summarizing a 100-page document takes 15–60 seconds. Without a progress indicator, the UX suffers. The server sends events via SSE:
event: progress data: {"step": "chunking", "total_chunks": 12, "completed": 0} event: progress data: {"step": "summarizing", "total_chunks": 12, "completed": 4} event: result data: {"summary": "...", "word_count": 450} On the mobile client, a progress bar with a step description and animated text "Processing pages 1–25...".
Caching
Summarizing a document costs money. Cache the result by content hash + summary type. Redis with a TTL of 7–30 days is standard. If the document changes, invalidate the cache by document_id.
What Our Work Includes
- Task analysis and strategy selection (direct, Map-Reduce, Refine)
- Server pipeline development with model integration
- Caching and streaming setup
- Mobile UI with summary type selection and progress bar
- Testing on real customer documents
- Detailed documentation, system access, team training, and 2 weeks of post-launch support
Phases and Timelines
| Phase | Duration |
|---|---|
| Analysis and design | 2–3 days |
| Pipeline implementation (Map-Reduce + streaming) | 1–2 weeks |
| Mobile UI and testing | 1–2 weeks |
| Full launch with caching and training | 3–5 weeks |
With over 5 years of experience and 30+ AI projects delivered, we guarantee reliable summarization. This solution saves companies up to $10,000 annually in manual summarization costs. Our certified AI expertise and secure infrastructure ensure your data remains confidential. We'll assess your project in one day — contact us for a consultation. Get a demo in 2–3 days.