OpenSearch configuration: where search breaks after migration
The first thing that breaks when migrating a search cluster is not the search itself, but trust in it: results start returning yesterday's products, filters find the wrong things, and some documents disappear from results.
The cause is almost always the same — index desynchronization: data reindexing happens separately from settings reindexing, and the cluster lives with half the rules for a while.
If this isn't noticed right away, the discrepancy makes it into production and surfaces on live users.
The second typical failure is a drop in search response speed after the move.
On the old cluster, shards were laid out for the real query profile, but on the new one they were distributed "by default": heavy aggregations start competing with point queries, and response time grows severalfold on the same hardware.
Separately, mismatched field mappings surface: where there used to be keyword for exact matching, there turns out to be text with word splitting — and sorting by price or SKU stops working correctly.
The third layer is lost analyzers and normalization chains. Russian morphology, synonyms, typo handling, and stop words live in the cluster configuration, not in the data, and during migration from Elasticsearch they are easy to leave behind.
On the surface, search works, but "sneakers" stops finding "sneaker," and hyphenated brands drop out of filters. It's at this stage that the client most often learns that what was migrated was the data, not the search logic.
We start with an inventory: we capture the actual mappings, analyzers, and index templates, compare them against the target configuration, and only then plan the reindexing. Next comes a controlled cutover with validation on shadow traffic. Below we break down how each of these failures is closed at the OpenSearch configuration stage.
What a properly configured search cluster delivers
Response time stops depending on data volume
Proper sharding and replica configuration means search returns results in tens of milliseconds even when the catalog grows several times over.
We select the number of shards based on the actual volume and query profile, not "by default": incorrect splitting is the most common reason a cluster starts to slow down out of nowhere.
We separately configure caching for filters and aggregations so that repeated queries are not recalculated from scratch.
Reindexing proceeds predictably and without downtime
Changing the mapping or document structure should not turn into a nightmare with search downtime. We configure aliases and switch traffic to the new index atomically — the old one remains available until the new one is ready.
Parallel loading via bulk operations with a tuned batch size keeps the load on the cluster steady rather than spiky.
The cluster survives node failure without data loss
Replicas and shard allocation settings ensure fault tolerance: if a node goes down, search continues to work, and data is restored automatically. We verify this deliberately — we shut down a node in the test environment and watch how the cluster redistributes the load. Index integrity is confirmed by comparing document counts before and after the failure.
Index integrity is under control
We set up cluster health checks and alerts for "yellow" and "red" status so that degradation does not go unnoticed. Regular index snapshots provide a rollback point rather than hoping for the best. As a result, search runs reliably, and the team learns about problems before users do.
Deployment Options: Cloud, On-Premises, Hybrid
The choice of platform determines not only request latency, but also who is responsible for a shard failure at three in the morning. Below are three modes of hosting a search cluster and their real constraints on load, maintenance, and total cost of ownership.
| Option | When it fits | Constraints |
|---|---|---|
| Managed search in the cloud | No in-house operations team, predictable load, need a fast start | Less control over the version and JVM heap parameters, rising bill as the number of nodes and index volume grow, vendor lock-in |
| Own cluster on servers (on-premise) | Data must not leave the perimeter, stable load profile, engineers are available | CapEx for hardware and redundancy, you handle updates, backups, monitoring, and disk replacement yourself |
| Hybrid scheme | Hot data and queries in the cloud, archive and sensitive indices on your own perimeter | More complex networking and snapshot synchronization, requires a unified request routing layer and shared monitoring |
What really changes the picture
Managed search takes the routine off the team's hands: security updates, replication, snapshots to object storage — all on the provider's side. You pay for this with less freedom in fine-tuning sharding and a ceiling on cluster size.
On-premise gives full control over index layout, heap size, and lifecycle policies, but requires on-call duty and spare hardware for peak load. Hybrid covers both scenarios if replication between perimeters is set up correctly.
We deploy a cluster for a specific load profile and document what goes to the provider and what stays with your team.
Stages of OpenSearch implementation and configuration
The work proceeds in sequential stages: each subsequent one relies on the artifacts of the previous, so skipping any step surfaces under load or in production. Below is the order in which we run a project from audit to handing the cluster over to your team.
- Search cluster audit. We capture the current state: which indices and aliases exist, how mappings are structured, where unused fields accumulate, how shards and replicas are distributed across nodes, which queries drive the main load. The output is a map of bottlenecks and a list of what we migrate as-is and what we redesign.
- Index schema design. We lay out data across indices and aliases so that reindexing and settings changes don't require stopping search. We define templates, index lifecycle, and retention policy — old data rolls off on schedule, hot data stays fast.
- Analyzer configuration. We build tokenization and filter chains for the language and nature of the queries: word forms, synonyms, typos, morphology. We test on real search phrases from logs, not on synthetic ones — otherwise the analyzer works beautifully in a test and falls apart on live traffic.
- Data and query migration. We move indices and rewrite incompatible queries and aggregations, verifying output line by line. We record discrepancies before the switch, not after.
- Load testing. We run a real traffic profile with peaks: we look at response time at percentiles, behavior when a node fails, and shard recovery speed. We select the number of shards and replicas based on actual results, not on a formula from the documentation.
- Deployment and monitoring. We configure the cluster in production, collect metrics on latency, failures, and resource consumption, and set up alerts for degradation.
- Handover to operations. We hand over regulations, schemas, and instructions for typical operations — your team then runs the cluster on its own.
Describe your task — we'll come back with an estimate and a plan based on these stages.
Access policies, roles, and permission separation
In OpenSearch, permission separation is built on three levels: cluster permissions, index permissions, and document permissions. A role is a named set of permissions that is assigned to a user or a service account.
Cluster permissions are responsible for managing the cluster itself: snapshots, settings, monitoring. Index permissions are responsible for reading, writing, deleting, and managing specific indices or templates.
This separation makes it possible to grant a service write access only to its own index and prevent it from changing the state of the cluster.
Index permission templates are defined by masks: logs-app-* for the application, logs-search-* for the search service. A mask covers an entire group of indices at once, including future ones — there is no need to rewrite the role every time a new index appears.
For multi-tenant scenarios, permissions are narrowed down to documents through a query filter: a user sees only their own records, even though physically everything is stored in a single index.
Action auditing is enabled at the cluster level and writes access events to a separate index. The log records: who, when, which index was accessed, and with what result.
This provides a basis for incident investigation and confirmation of compliance — you can see not only the fact of a denial, but also attempts to bypass permissions.
Service roles are configured separately: for snapshots, for monitoring, for cross-cluster replication. They are issued not to people, but to service accounts with a limited lifetime.
Rotation of such accounts happens on a schedule, not manually — otherwise credentials get copied between teams and stop restricting anything.
Practice shows: without an explicit role model, a cluster quickly turns into "everyone sees everything." We start with an inventory of services, for each one we record the minimum required set of permissions, and only after that do we enable auditing — otherwise the log will fill up with noise from excessive permissions.
What the customer receives after cluster setup
After cluster setup, the customer receives not "working search" but a set of artifacts by which the system can be operated, extended, and handed over to another team without us. Below is what exactly remains in the repository and in the documentation.
- Cluster configuration — node and role settings files, sharding and replication parameters, policies for distributing shards across availability zones. Deployment is reproduced from these files, not "from the engineer's memory."
- Index schema — a description of indexes, their templates, and relationships: which fields are analyzed, which are stored as is, where synonyms and text parsing rules are applied. The schema records why search finds what is needed, rather than simply "that's how it's configured."
- Lifecycle policies — rules for index and snapshot rotation: how much data we keep in the hot tier, when we move it to the cold tier, how often we take backups and how long we retain them. The registry stops growing uncontrollably, and rollback to a previous state remains possible.
- Maintenance regulations — the procedure for planned work: updating nodes without downtime, checking cluster health, responding to disk overflow and growing query latency. With it, the on-call engineer understands what to do without calling a developer.
- Monitoring and alerts — a set of cluster health metrics and the thresholds at which a signal is triggered: search latency, query queue, shard state, free space. A problem is visible before a user notices it.
- Search documentation — an index map, a field dictionary, examples of typical queries, and an analysis of common errors. A new engineer gets up to speed on the project from it, not from chat messages.
- Migration procedure — a step-by-step plan for migrating from the previous search engine: the switchover order, data completeness checks, the point of return. Migration proceeds without stopping the service for users.
This set is the result of the setup: the cluster can be developed, scaled, and maintained by your own team. Describe your task — we'll come back with an estimate and a work plan.
How does configuring OpenSearch differ from Elasticsearch?
Formally, OpenSearch is a fork of Elasticsearch, and the basic search and indexing API is compatible: _search and _bulk queries, mappings, and analyzers carry over without rewriting.
But compatibility does not mean identity, and the differences surface precisely at the configuration level, not in the application code.
The key difference is the licensing model and the set of plugins.
In OpenSearch, security, alerting, anomaly detection, SQL, and dashboards are included in the distribution and configured out of the box, whereas in Elasticsearch some of these capabilities live in a separate paid subscription.
For configuration, this means: roles, index permissions, and access policies are described in the same files, but they are enabled by default, and they need to be deliberately restricted rather than purchased as an add-on.
The second is cluster management. Sharding, replication, and lifecycle policy settings are defined with similar syntax, but parameter names and the behavior of some plugins diverged after the projects split.
Directly transferring elasticsearch.yml and index templates "as is" produces silent discrepancies: some options are ignored, while others change segment behavior and merge speed.
During migration, we transfer not the files but the meaning of the settings: we verify mappings and analyzers, rebuild index templates, check how the search plugins work, and only then switch the traffic. That way, migrating settings does not turn into a surprise in production.
Case study: search migration without service downtime
The project is a marketplace with a catalog of several million items: search and facets relied on Elasticsearch, and a multi-node cluster ran under constant load.
Peak indexing coincided with morning traffic, and during those hours search began returning delayed responses, while some requests timed out.
A separate pain point was mapping updates: reindexing the catalog took hours, and throughout that time two indexes had to be kept running and the alias switched manually.
What we did
We deployed the target cluster in parallel with the production one and configured replication so that both held the same dataset.
Indexing was routed through a queue: first we write to the current cluster, then replay the same events into the new one — divergence is caught by comparing document checksums.
Traffic was switched not all at once, but in fractions: first a small percentage of read requests, then half, then the entire stream. At each step we compared search metrics — the share of slow requests, response latency, and cluster load.
What changed
The switchover took minutes and happened without downtime: users did not notice the engine change. The share of requests exceeding the target response time dropped by roughly threefold, and peak cluster load stopped hitting the ceiling — part of the reads moved to replicas.
Recovery time after a node failure shrank from tens of minutes to single digits: replicas come up automatically, with no manual intervention required.
Rollback was planned in advance — the old cluster remained operational until the end of the observation period, and returning to it was possible with the same fractional traffic switch.
Let's discuss your search setup task
Search setup is a task where the cost of a mistake is immediately visible: an incorrect mapping or an unsuccessful sharding scheme shows up not on a test index, but on production traffic, when reindexing takes hours.
That is why we start not with configuration, but with an analysis of your scenario: what data and in what volume is indexed, which queries are speed-critical, whether morphology and synonym support is needed, and how search results are delivered today.
To make the conversation concrete, send us the inputs — our engineers will use them to put together a work plan and an estimate for the setup:
- the current search stack and branch version (Elasticsearch or OpenSearch), index volume, and daily data growth;
- a list of typical queries with examples and latency requirements;
- the cluster layout: number of nodes, roles, resource headroom, presence of replicas;
- how data loading is organized — sources, update frequency, whether incremental reindexing is needed;
- ranking requirements: field boosts, synonyms, stop words, autocomplete;
- constraints on access, backups, and maintenance windows;
- what you have already tried and where you got stuck, if migration or setup is stalling.
Based on this data, we will come back with a plan: stages, the order of traffic switching, rollback points, and the scope of work for migration from Elasticsearch and deployment.
An engineer's consultation at this step is not a general conversation, but an analysis of your index scheme and search delivery bottlenecks. Describe the task in whatever format is convenient — we will come back with an estimate and a work plan.