Where manual cluster tuning breaks under peak load
A self-hosted cluster is almost always tuned for the load present at launch: the connection pool, memory limits, the number of application worker processes.
As long as the number of concurrent connections grows linearly with traffic, everything looks operational — but every connection has its own memory and its own scheduler time, and at some point they run out.
The first to fail is not the database but the application: it gets a connection refused and goes into retries, which finish off the cluster for good.
Peak load on the database is rarely uniform: a promotion, a mass mailing, or a seasonal spike causes growth within seconds.
A query bottleneck does not manifest as an outage but as degradation as traffic grows — the queue grows, response time creeps up, a heavy export holds locks and slows everyone else down. At that point, diagnostics are done from logs, when the service is already down.
A separate story is node failure. In a manual installation, failover is done by an engineer: see the alert, confirm, promote the replica, switch traffic.
The downtime window here equals the on-call engineer's reaction time, and if the replica was lagging, after failover the data diverges and has to be reconciled manually.
We'll look at how this is addressed with managed databases Selectel — and why some of the failures listed above simply do not occur there.
Selectel managed databases: what changes in practice
Managed databases Selectel are a change in the operating model, not moving the same server under someone else's administration. The provider's database maintenance covers the routine: minor updates, free space monitoring, parameter review, replication checks. The team stops keeping the schedule of these tasks in their heads.
The first measurable effect is the share of manual operations. Volume expansion, adding a read replica, updating a minor version cease to be separate tasks with window coordination.
Automatic backups run on a schedule, and their result is verified with built-in tools rather than manually.
The second is reduced downtime. Point-in-time recovery to before the failure eliminates parsing the chain of increments: the database is restored to a point minutes before the error, not to the last nightly copy.
Failover to the standby node when the primary fails happens automatically, and the application reconnects using the same name.
Separately, a class of operational failures goes away: a full disk, a missed update, a diverged replica, a backup that has never been restored. Until the moment of recovery, such failures do not look like incidents.
When traffic spikes, behavior becomes predictable: reads are distributed across replicas, connection limits protect against overload, and resource limits prevent a single heavy query from bringing down the instance.
What to check before migrating: backup retention period, target recovery point, failover scheme for node failure, and team access to metrics and logs.
Deployment options: dedicated cluster, shared pool, replica
The configuration is chosen not by database size but by load profile and availability requirements. Transactional writes, analytical queries over large tables, and peak read surges hit different resources, so databases of the same volume live in different setups.
In Selectel, managed databases are deployed in several variants, and the boundaries between them run along isolation, replica lag, and behavior during node failure.
| Option | When it fits | What it constrains |
|---|---|---|
| Dedicated database cluster | Production workload with transactions, isolation and predictable response required | Resources are dedicated to one service and are not shared with neighbors even when idle |
| Shared resource pool | Internal services, staging environments, queues, reporting with uneven load | Neighbors in the pool affect peaks, upper latency bound is not guaranteed |
| Read replica | Reports, exports, search, and heavy queries on a large database | Data arrives with lag, writes are not sent to the replica |
| High-availability configuration | A service for which database unavailability equals product unavailability | Standby resources are occupied continuously, and the application must survive failover |
Migrating an already running database does not require stopping the service: the target configuration is brought up alongside, streaming replication is enabled, they wait until the lag is closed, and the application is switched to the new address.
The previous cluster remains available for rollback until the new setup has completed a full cycle under load.
Implementation stages: from load audit to handover to operations
We start not with configuration but with measurements: we capture the hourly load profile from the production database, the top queries by total time, the read/write ratio, and connection pool behavior at peak.
These numbers determine the instance class, the number of replicas, and the list of queries to optimize before migration — otherwise the migration will simply move the bottleneck to new hardware.
- Database load audit. We collect the top queries, locks, spikes, table and index growth, and headroom for disk and connections. The output is the target cluster parameters and a list of what we fix before migration.
- High-availability scheme. We define the topology: primary node and a replica in another availability zone, automatic failover, backup retention periods, and point-in-time recovery.
- Cluster deployment. We bring up a managed Selectel Managed Databases cluster and configure roles, network access, encryption, and the backup schedule — for the measured profile, not defaults.
- Zero-downtime migration. We migrate data with logical replication: the subscription catches up to the primary database, we keep lag under control, then switch the application — downtime fits within seconds and affects only writes.
- Failover testing. We stage a drill failure: shut down a node and check whether recovery fits within the target time, whether integrity is preserved, and whether the application continues running. We repeat until the result becomes reproducible.
- Observability. We configure alerts for replica lag, connection exhaustion, disk space, and query time growth, with explicitly fixed response thresholds.
- Handover to operations. We hand over cluster documentation and train the team: how to trigger failover, restore from backup, and act during degradation. For the first while, we troubleshoot incidents together with your engineers.
What we hand over: configurations, diagrams, and procedures
We hand over not access to the control panel but a set of artifacts that let your team maintain managed databases Selectel — without contacting us for every change. Below is exactly what is included in the handover and in what form it is documented.
- Cluster configuration — node composition, their placement across availability zones, parameters changed from defaults, and the reason for each deviation.
- Replication scheme — the “primary node and replicas” topology, synchronization mode, expected replica lag, and cluster behavior when a node is lost.
- Database access policies — roles and their permissions, the list of networks from which connections are allowed, channel encryption requirements, and credential rotation procedure.
- Backup policy — snapshot frequency, retention depth, where backups are exported, and how often we test restoration in a separate environment.
- Incident response procedure — step-by-step scenarios: node unavailability, load degradation, erroneous data change, indicating who makes the decision and in what sequence.
- Monitoring and alerts — which metrics we collect, at what thresholds an alert fires, whom it is addressed to, and what is considered a false positive.
- Maintenance documentation — where configurations and diagrams are stored, how to roll out a change, checklists for planned work and acceptance after it.
Case study: how we reduced recovery time after node failure
Before: node failure was recovered manually
The production environment of an online store ran on three server nodes, but failover was done manually.
The on-call engineer learned about the outage from a notification, promoted the replica, edited the connection string in the application configuration, and restarted services — on average 40–60 minutes of database unavailability, during which orders were not placed.
What we changed and what the result was
We migrated the databases to managed databases Selectel with automatic failover. The leader node is selected by specified priorities, health checks run every few seconds, and the application connects to a permanent endpoint and no longer knows the addresses of individual nodes.
Only planned operations were left manual — scaling, parameter changes — and each follows a procedure with a preliminary replica check.
Recovery time after cluster node failure dropped to 60–90 seconds, and the share of manual operations in recovery fell from almost a full set of steps to individual confirmations.
Database availability over the quarter grew from 99.3% to 99.95%, and there were no outages due to a failover error.
Before going live, we ran drill failures for a month and a half: we shut down a node during working hours and checked that the application reconnects on its own and transaction counters are not lost.
We are applying the same order — first a failure rehearsal, then abandoning manual steps — to the remaining environments.
What to do if the database is already deployed on your own servers?
Yes, we migrate a deployed database without stopping the service — but not with a single full snapshot in a nightly window, but in stages.
We enable logical replication between your installation and a managed Selectel Managed Databases cluster, leaving the application writing to its usual database. Changes stream to the cloud from the transaction log, so the load on the production server remains the same.
We check compatibility before failover, not during. We look at the kernel branch and available set of extensions, encoding and collation rules, session time zone, transaction isolation level, behavior of auto-increment fields, triggers, views, and role permissions.
We separately review places where the application relies on superuser operations or file system access — in a managed cluster these are covered by built-in tools, and this is the only class of changes that must be made in the code in advance.
The transition happens in stages. First, the replica catches up and runs in read-only mode: we run analytics on it and compare plans for heavy queries.
Then we catch up the lag, stop writes on the source database for just a few seconds, compare row counts and checksums for key tables — and switch the application to the new connection addresses. The write downtime window is measured in seconds, not hours.
We plan rollback in advance: the old installation remains untouched until the end of observation, and if the reconciliation diverges, traffic returns to it with a single configuration change.
After failover, we monitor replication lag, slow queries, and locks to catch what did not show up in the test run.
If some data is no longer needed — historical partitions, archive tables — it does not have to be migrated to the new location at all. This reduces replication volume and catch-up time, while integrity is preserved by migrating related tables in one step.
Describe your current installation — we will come back with a migration plan and an estimate of the failover window.
Common technical concerns: access, backups, downtime
Engineering concerns around managed databases are always the same: whether access to the server remains, how backup integrity is confirmed, what happens to applications during failover, and whether it is later visible who changed settings. Below are short answers that usually resolve these questions before work begins.
A managed database is not “a server you were given access to” but a set of network boundaries, policies, and observability. We configure them for your setup and hand them over along with documentation so that operations do not depend on a specific engineer.
Is direct access to the database server required?
No. Connections come only from your private network: no public address is issued, access is limited by allowlists and accounts with role separation. Administrative operations are performed through the panel and described procedures; logging in to the host is not required.
How do you verify that a backup will actually restore?
Backups are taken on a schedule with a specified retention period. We test restoration on a separate instance in an isolated environment: we deploy it, compare the structure and checksums.
The recovery point is selected by time, rollback goes to a new instance — the production database is not overwritten.
What happens during failover?
The role moves to the replica automatically, applications briefly lose connections and reconnect. We configure retries and timeouts in the drivers, and check replication state in advance — otherwise the downtime window stretches to minutes instead of seconds.
Is it recorded who changed what in the database?
Yes, action logging is enabled: logins, administrative operations, and changes to parameters and structure are saved. We export logs to your environment, so the question “who changed the setting” is answered with facts, not correspondence.
Let's discuss your task: what to send for an estimate and work plan
To come back with an estimate for migrating to a managed database, rather than generalities, we need a minimum of inputs. You will not need to prepare anything special — what your team already has will be enough.
- load profile: peak queries per second, read/write ratio, daily and weekly fluctuations;
- current scheme: where data lives, volume, number of databases and heaviest queries, server version;
- availability requirements: acceptable downtime, recovery point, whether a standby node in another zone is needed;
- access and constraints: network rules, encryption, personal data and logging regulations;
- environments: how many there are, how they are separated, what is considered the source of truth during discrepancies;
- maintenance windows: when failovers are allowed and whether there are periods when nothing can be changed.
This is enough for a database implementation plan: placement scheme, migration procedure with a rehearsal on a copy, rollback points, and a set of checks after failover.
If the target platform is managed databases Selectel, we will separately specify which parameters we take out of the box and what we configure manually.
Let's discuss your database task — we will come back with an estimate, work plan, and division of responsibilities.