Managed Kubernetes in Selectel: where infrastructure breaks
Managed Kubernetes in Selectel removes control plane maintenance from the team: its updates and API availability are the provider's concern. But cluster failure points do not disappear—they shift to the workload level: node groups, environments, and the recovery scenario.
Manual node scaling is the first such place. While the on-call engineer calculates capacity reserve, the peak arrives faster than a machine is added: pods stay pending, readiness probes fail, and some requests time out. Nodes provisioned with reserve sit idle for months and still require updates.
Environment drift accumulates unnoticed. A fix was made manually in the test cluster, bypassing configuration—it never made it to production; Terraform state diverged from actual, and the next run shows dozens of unplanned changes. Debugging turns into comparing environments instead of finding the root cause of the failure.
Recovery after a failure without rehearsal is a lottery. The cluster state copy was not tested, StorageClass with a volume deletion policy wipes data along with PVC, and the order of bringing up dependent services was not recorded. Then downtime is measured not in hours of planned work but in days of experiments.
We explain how this is addressed: declarative node groups, a single source of configuration for all environments, and recovery that is run in advance, before an incident.
Effect of a dedicated cluster: what changes in operations
The effect of implementing a managed cluster is visible not in presentations but in four metrics the team captures before and after: environment deployment time, service recovery time, share of manual operations, and the failure class that stops reproducing. Below—what exactly changes and due to what.
- Environment deployment — from days to tens of minutes. Infrastructure description in code gives identical environments for development, testing, and production: a new site comes up with the same configuration already verified, and discrepancies between stands cease to be a separate task.
- Service recovery — by runbook, not memory. Nodes are recreated automatically, load is redistributed among the remaining ones, and storage state is described by a storage class. Service downtime is measured in minutes and does not depend on which engineer is on call.
- Share of manual operations drops. Version updates, scaling, certificate issuance, and traffic routing are performed declaratively: a change in the infrastructure description is applied to the whole cluster, without logging into individual machines.
- The failure class "fixed manually and forgot" goes away. Configuration stops living only in administrators' heads, any deviation is visible on the next apply of the description and is either rolled back or consciously added to it.
We deploy the cluster on Selectel: the control plane and node fault tolerance are on the provider's side, while we are responsible for infrastructure description, routing, and storage policies. Describe the task—we'll return with an estimate and migration plan.
Managed cluster operating modes: comparing options
Cluster mode is chosen along two axes—cost of error and cost of isolation. While there is one product and the team is small, a shared contour speeds everything up: one update point, one set of permissions, fewer entities under control.
As soon as test data, payment scenarios, and a second team appear, the same cluster becomes a source of mutual interference—a failed release or resource shortage on one service affects the others.
Below are four modes that teams actually choose between in managed Kubernetes.
| Option | When suitable | Limitation |
|---|---|---|
| One shared cluster | One product, small team, early stage | No environment separation: test and prod share nodes and control plane, an error is immediately visible to users |
| Cluster per environment | Multiple teams, test data requirements | Number of control entities grows linearly: more updates, certificates, and monitoring points |
| Namespaces inside the cluster | Moderate isolation needed without growing the number of clusters | Shared node pool: neighboring workload at peak affects latency and evicts pods |
| Dedicated node pool | Payments, personal data, sensitive workloads | Some nodes idle, scheduler configured manually—node labels and admission rules |
What remains shared in any mode
The limitation of a managed solution is that the control plane belongs to the provider: it sets versions and update order itself, and a separate control plane per task is not provided. Therefore workload isolation is built on node pools, namespaces, and quotas.
The working approach is to start with one cluster and separate environments when a second team appears or there is a requirement to keep test data separate from production.
How implementation proceeds: stages of engineering work
Work proceeds sequentially: first we fix the current state of the environment, then design and assemble the cluster, and only then test it under real load.
This order leaves no room for edits "along the way"—each decision is either confirmed by measurement or recorded in the design.
Order of work
- Infrastructure audit. Inventory of services, dependencies, and manual operations; output—dependency map and list of constraints that cannot be violated during migration.
- Cluster design. We design node pools for different task classes, update policies, access separation, and namespaces. We agree before creating the first resources—changes in the design are cheaper than rebuilding a working environment.
- Environment assembly. We configure the traffic entry point, certificates, storage classes for different data profiles, and volume backups.
- Service migration. We describe configuration as code: state is stored in the repository, the environment is reproduced from scratch, and secrets are moved out of manifests into a secure vault.
- Load testing. We run a real load profile: peak requests, node and individual component failure, autoscaling behavior, and the moment quotas are exhausted.
- Acceptance and handover of runbooks. We verify target metrics for recovery time and successful request share, and hand over documentation: update, rollback, adding a service, incident actions.
What remains with the customer: configurations, diagrams, runbooks
We hand over the managed Kubernetes cluster on Selectel not as panel access but as a set of artifacts in your repository: an engineer from your team brings up the environment from scratch and does not ask clarifying questions. Below—what exactly remains in your hands after handover.
- Infrastructure as code. Modules for the cluster, node groups, network, and balancer: any change goes through review and plan rather than being edited manually in the console—the environment state is reproducible.
- Infrastructure diagram. Network topology, private and public segments, storage connection points, traffic path from the external balancer to services inside the cluster.
- Access policies. Roles and permission bindings, separate accounts for people and automation, service accounts for build pipelines, restrictions on system namespaces.
- Storage placement rules. Storage classes for different workload types, volume sizes, snapshot policies, and data lifecycle—so a volume survives node recreation.
- Update runbook. Order and windows for updating the cluster version, node groups, and add-ons, mandatory test environment check, and a rollback plan to the previous state.
- Recovery instructions. Backups of configurations and volumes, step-by-step cluster bring-up on a clean site, responsible persons, and checkpoints at each step.
- Operational documentation. Service and owner map, monitoring and alerting scheme, analysis of typical failures, and on-call engineer procedure.
Any item can be left with us for support, but the acceptance criterion is different: the team deploys a copy of the environment, performs an update, and recovers after a failure without our participation.
Case: migrating services to a cluster and metrics before and after
What was before migration
A service of six applications and two queues lived on manually configured virtual machines: release rollout took about forty minutes and required an on-call engineer, while rollback went through snapshot restore.
Node failure stretched for hours, and half of incidents were caused by setting discrepancies between stands. We built the migration to a managed Selectel cluster specifically around these two points.
What changed
Infrastructure was moved to a declarative description: Terraform creates the cluster, node groups, and network, Ingress lives as a separate object, and StorageClass with a retention policy is tied to database volumes.
Applications are built into images with immutable tags, schema migrations run as a separate job before rollout, and readiness probes do not let traffic to an unready pod. Live traffic was switched through the balancer, so the migration went without downtime.
Metrics before and after
Rollout was reduced from forty minutes to six, release frequency grew from one every two weeks to several per week with the same team size. Pod recovery takes minutes: the node is drained from the pool, load is redistributed, and state is brought up from the volume.
Failures of the class "stand settings diverged" no longer repeat—the environment is assembled from one description.
Why infrastructure as code is better than manual setup
With manual cluster setup, actions are not recorded anywhere: node pool parameters, volume limits, and routing rule edits live only in the console and the engineer's memory.
Formally the environment works, but it can no longer be reproduced from scratch—and this is the main support risk.
Infrastructure as code moves these actions into declarative files: the description of the cluster, node groups, networks, routing, and storage classes lies in the repository next to application manifests.
Environment reproducibility stops being a verbal agreement: the development stand and the load-testing stand are deployed from one branch and differ only in variable values.
Configuration versioning adds change history at the commit level: you can see who expanded a node pool or changed a storage class and why. Edits pass review before apply, rather than being analyzed after the fact via logs.
Rollback of changes becomes an ordinary operation: we return the previous version of the description and apply it again—the cluster is brought to its past state without restoring from memory.
With manual setup, the same rollback is hours in the console under pressure and a high chance of missing a step.
Without infrastructure description, time costs for support grow with every change: a new engineer takes longer to get up to speed, incidents are investigated more slowly.
In managed Kubernetes, the provider maintains only the control plane, while nodes, network, and storage remain with the team—these we keep in code, describing them via Terraform.
Technical doubts: access, storage, updates
Doubts about a managed cluster usually come down to four questions: who can change what, where database and log data lives, how updates proceed, and what remains after a failure. Below—short answers for each.
How to separate rights between teams and contractors?
We issue roles at the namespace level, not the whole cluster: a developer sees their services, the build pipeline gets a separate account only for rollout. Secrets are pulled by a controller and are not stored in manifests. All changes go to the audit log.
Which storage class to choose for databases and for logs?
For databases—a class with guaranteed performance and a volume retention policy: data survives pod recreation. For logs and metrics, a network disk with delayed attach is enough. Separation by classes removes the case when analytics fills database disks.
Do updates proceed without downtime?
Nodes are updated pool by pool one at a time: workload is evicted, disruption budgets prevent all service replicas from being terminated at once. The provider updates the control plane separately, so node versions lag by a minor version—this is normal mode.
What about backup and recovery?
The provider takes cluster snapshots, but an individual application cannot be restored from them. We add namespace and volume backup with a schedule and retention of multiple generations. Once a quarter we deploy a copy on the test environment: an untested copy equals a missing one.
Let's discuss the task: where work begins
To estimate timelines, a conversation and a few inputs are enough for us: how many environments and workloads need to be migrated now, whether there are availability and maintenance window requirements, who owns the cloud provider account, and what security policies are already in place.
This is enough to understand the scope of work and the order of steps without taking the entire cluster apart.
We return with a work plan that includes:
- breakdown by stages: preparing access and image registry, infrastructure as code, configuring ingress traffic and storage classes, migrating workloads;
- timeline estimate for each stage and an overall duration guide;
- list of what we need from you: access, list of services, downtime requirements;
- rollback scheme and procedure for verifying the result at each step;
- responsible persons on both sides and points where we wait for your decision;
- option for a minimal first step—validate the approach on one environment without touching the others.
The first step is always the same: we fix the current state, describe the cluster as code, and bring up a test environment—after it, real timelines and risks are visible, not estimated ones. Describe the task in free form: we'll return with a work plan, timelines, and a start proposal.