When you search for "листья" with Snowball stemming in Elasticsearch, it only returns "листь" — missing "лист", "листовой". This is a typical issue with Russian morphological analysis: rule-based stemming clips endings, losing up to 40% of relevant documents. We use lemmatization based on Hunspell dictionaries to guarantee accurate search across all word forms of Russian and English. Over 5 years, we have completed 30+ projects configuring search for online stores, news portals, and corporate portals. Our engineers hold Elastic Certified Engineer certifications and have deep understanding of linguistic algorithms.
Morphological analysis is not just suffix stripping — it's dictionary-based parsing. According to Elasticsearch documentation, lemmatization provides 30–50% more relevant results than stemming. Below we break down which approaches work in practice and how we implement them.
Why Stemming Falls Short of Lemmatization?
Stemming (Snowball, Porter) clips endings by rules. Fast — 1–2 ms per token, but inaccurate. For example, "бегать" and "бег" produce different stems, even though they are semantically related. For Russian, this is critical: word forms can differ drastically (бежать, бежал, бегущий). Lemmatization uses dictionaries (Hunspell, Mystem) and reduces words to their base form. It is slower during indexing (3–10x), but search accuracy improves by 30–50%. In our projects, Hunspell lemmatization is 1.5 times more accurate than Snowball.
| Parameter | Stemming (Snowball) | Lemmatization (Hunspell) |
|---|---|---|
| Indexing speed | 1–2 ms/token | 5–20 ms/token |
| Search accuracy for Russian | ~60% | ~90% |
| Dictionary dependency | no | requires 50–500 MB dictionary |
| Word form support | limited | full lemmatization |
How to Configure Hunspell for Russian and English?
Installing dictionaries takes one business day. We obtain dictionaries from the LibreOffice repository.
mkdir -p /etc/elasticsearch/hunspell/ru_RU cd /etc/elasticsearch/hunspell/ru_RU wget https://cgit.freedesktop.org/libreoffice/dictionaries/plain/ru_RU/ru_RU.dic wget https://cgit.freedesktop.org/libreoffice/dictionaries/plain/ru_RU/ru_RU.aff # Similarly for en_US After adding dictionaries, restart Elasticsearch and create an index with the analyzer.
PUT /articles { "settings": { "analysis": { "filter": { "ru_hunspell": { "type": "hunspell", "locale": "ru_RU", "dedup": true }, "en_hunspell": { "type": "hunspell", "locale": "en_US", "dedup": true } }, "analyzer": { "ru_en_morphology": { "tokenizer": "standard", "filter": ["lowercase", "ru_hunspell", "en_hunspell", "unique"] } } } }, "mappings": { "properties": { "content": { "type": "text", "analyzer": "ru_en_morphology", "search_analyzer": "ru_en_morphology" } } } } Quality check:
POST /articles/_analyze { "analyzer": "ru_en_morphology", "text": "Разработчики создали приложение для управления задачами" } # Expected tokens: разработчик, создать, приложение, управление, задача Complications arise with bilingual content. We use multi-field with different analyzers for Russian and English, then multi-match with boosting. This increases relevance for mixed queries.
How to Improve Search on Mixed Content?
For sites with Russian and English content, we apply multi-field: one field with ru_hunspell, another with en_hunspell. Search via multi_match with a coefficient of 1.5 for the primary language. This boosts accuracy for mixed queries by 25%. For example, the query "управление tasks" finds documents with both languages.
| Method | Accuracy for ru | Accuracy for en | Response time |
|---|---|---|---|
| Standard only | 55% | 70% | <30 ms |
| Hunspell ru/en multi-field | 88% | 85% | <50 ms |
Case Study: Furniture Online Store
Our client, a furniture online store, faced an issue: searching for "стул" did not return "стулья", "стульчик". After implementing Hunspell, search accuracy improved from 62% to 89%. Additionally, we configured multi-field for the catalog with Italian names in English. As a result, search conversion increased by 12%, and revenue grew by 15%. The setup took 3 business days.
Turnkey Setup Process
- Data analysis: estimate volume, language composition, query types.
- Dictionary selection: Hunspell for ru/en, plus custom user dictionaries if needed.
- Index configuration: configure analyzers, test on a sample.
- Reindexing: create a new index with morphology, migrate data.
- Optimization: tune
refresh_interval,number_of_replicas,forcemerge. - Acceptance testing: compare search results before/after, adjust stop words.
- Documentation and handover: index schema, maintenance procedures.
What Is Included
- Preparation and installation of Hunspell dictionaries for Russian and English.
- Analyzer configuration tailored to content specifics (stop words, deduplication).
- Index schema with multi-field for bilingual search.
- Reindexing scripts and performance optimization (bulk, forcemerge).
- Configuration documentation and instructions for adding new dictionaries.
- Use of licensed Hunspell dictionaries.
- Analyzer operation guarantee for 30 days after delivery.
Example configuration for bilingual index
PUT /articles_bilingual { "settings": { "analysis": { "filter": { "ru_hunspell": { "type": "hunspell", "locale": "ru_RU", "dedup": true }, "en_hunspell": { "type": "hunspell", "locale": "en_US", "dedup": true } }, "analyzer": { "ru_analyzer": { "tokenizer": "standard", "filter": ["lowercase", "ru_hunspell", "unique"] }, "en_analyzer": { "tokenizer": "standard", "filter": ["lowercase", "en_hunspell", "unique"] } } } }, "mappings": { "properties": { "title": { "type": "text", "fields": { "russian": { "type": "text", "analyzer": "ru_analyzer" }, "english": { "type": "text", "analyzer": "en_analyzer" } } } } } } Timelines
Estimated time: 2 to 5 business days depending on data volume and configuration complexity. Pricing is calculated individually. Average budget savings on search refinements after Hunspell implementation is 30% compared to alternative solutions.
We guarantee that search accuracy for word forms will increase by at least 30% compared to standard Snowball stemming.
If you want to improve search on your project — contact us for a consultation. We will evaluate your task for free. Order morphological search setup in Elasticsearch. Get a free consultation.