Where tenant isolation breaks in a shared cluster
Isolation in a shared cluster is more declarative than real.
A namespace separates object names and rights to them, but not network, disks, or databases: until network policies are enabled, pods of one client freely access services of another, and service accounts by default receive more rights than a service needs.
The most expensive point of failure is data. A shared database with a tenant attribute in rows rests on one condition: every query filters by that attribute.
One unclosed report, one shared cache without a tenant prefix, or one mounted volume from a neighboring namespace — and the client sees someone else's records. The leak is uncovered not immediately, but during incident review.
Resource conflict looks different. Without quotas and limits, a batch export or reindexing eats disk operations and network on the node; neighboring services begin to return timeout errors, and the scheduler under pressure evicts pods not of the one causing the noise. The culprit keeps working, others suffer.
Environment boundaries also break. A test environment in the same cluster with a shared load balancer, queues, and secret storage means a migration run on staging hits the production database. Next, we break down how this is closed — across network, access, and data levels.
What changes after environment separation: effect in metrics
Isolation is built on three levels: a separate namespace with CPU and memory quotas, network policies with default deny, and a dedicated schema or database instance per tenant.
Kubernetes provides ready primitives for this, so boundaries are described declaratively and reproduced without manual edits.
The first consequence is visible at project start: an environment for a new client deploys from a template along with quotas, network rules, and database migrations.
Client onboarding time decreases from days of manual cloning to tens of minutes, and it no longer grows with the number of tenants.
Second — recovery time. A failure is localized in one namespace, so an image rollback or database backup restore affects only the impacted environment. Recovery is measured in minutes and does not depend on how many clients live nearby.
Third — the share of incidents affecting neighbors. Quotas cut consumption spikes, and policies do not pass traffic between tenants: a "noisy neighbor" stops taking down the shared node.
At the same time, a whole class of failures goes away — schema version conflict during migration and access to another's table through a shared connection.
Fourth — release predictability. Rollout goes tenant by tenant: an error is visible on one environment, others stay on the previous version, and the release schedule no longer depends on someone else's.
Tenant isolation options: shared namespaces or dedicated environments
Tenant isolation is not a binary choice between "everything in one" and "separate infrastructure per tenant".
Between these extremes there are workable levels, and the difference between them is what exactly is separated: objects, network, database, or the entire environment. Below are three options realistically chosen at project start.
The easiest level — namespaces in Kubernetes: they separate objects but not network or database, so they must be supplemented with access policies and separate accounts. Such a scheme supports dozens of small tenants until formal separation requirements appear.
| Option | When suitable | Limitations |
|---|---|---|
| Shared namespaces | dozens of small tenants in one environment, formal separation requirements | resource limits and access policies are configured manually, an error in rules opens access to others' data, and an active neighbor consumes resources |
| Hybrid scheme | some tenants require enhanced isolation, some do not | two sets of templates and regulations: support is more expensive, audit is separate |
| Dedicated environment | regulated data, large customers, separate network and database requirements | higher operating cost: updates and backups are replicated for each environment |
In practice, the hybrid scheme often turns out to be a working state rather than a compromise: the core lives in a shared environment under strict policies, sensitive tenants are moved to separate groups of namespaces with their own limits.
Transition between levels is designed in advance — then moving a tenant to a dedicated environment becomes a planned operation, not an infrastructure rewrite.
Implementation procedure: from audit to handover to operations
The work order is structured so isolation appears on a live system without stopping the product. First we record the current state, then design the target scheme and test it on staging — production workloads are migrated only after that. Each stage ends with an artifact that can be checked and passed on.
- Configuration audit. Capture the actual state: which namespaces exist, who is connected to them, where network policies overlap, and how services access databases. Output — a list of discrepancies between documentation and reality.
- Dependency map. Mark connections between services, shared storages, and external systems. It shows what cannot be separated directly and where an intermediate access layer is needed.
- Isolation scheme design. Define tenant boundaries on three levels: namespaces for compute, network policies for traffic, separate schemas and accounts for data.
- Pilot staging. Assemble a copy of the environment on a limited set of services and run negative scenarios: access from another namespace, a shared volume, a reused secret.
- Migration plan. Split migration into waves by service connectivity, for each define order, rollback point, and maintenance window. A wave does not start until the previous one passes validation.
- Acceptance and handover. Run the agreed set of checks, record the result, and hand over the scheme to the on-call shift along with regulations and analysis of typical incidents.
After handover, not only working isolation remains but also documentation: dependency map, access policies, procedure for onboarding a new tenant. The on-call shift knows what to do when boundaries are crossed, and changes go through the same process as the first launch.
What remains with the team: configurations, schemes, and regulations
We hand over not "panel access" but a working kit with which your team can onboard a new tenant, handle an incident, and return the system to a known state.
Every item is verified on a live environment together with your engineers — during acceptance it is clear whether the document works or was written for a report.
- Deployment templates — parameterized staging descriptions: per-tenant variables, quotas, dedicated environment, shared services. A new client is brought up by changing parameters, without logic edits.
- Separation scheme description — boundary map: where namespaces, network policies, and separate databases run, which resources are intentionally shared and why this does not create overlaps.
- Access policies — roles and rights per zone, procedure for granting and revoking, secret storage and rotation, rule "access by request with justification".
- On-call instructions — typical scenarios: how to verify tenant data does not overlap, what to check when someone reports seeing others' data, whom to escalate to and within what time.
- Rollback procedure — return points, contents to roll back (data schema, policies, configurations), sequence of steps, and post-recovery validation.
- Change regulation — how edits in the shared part are approved, which checks are mandatory before rollout, and who is responsible for regression.
Everything lives in the repository next to the code, is versioned, and changes by the same change as the configuration. Otherwise, in six months the instruction diverges from the actual state — and the on-call engineer makes decisions based on an outdated scheme.
How it looked in practice: separating environments with shared databases
The customer — a platform with several business lines — kept all environments in one PostgreSQL cluster: shared database, shared connection pool, and one account with access to both production data and test environments.
A long migration of one line blocked tables that others read, and an error in the test environment reached real data. The fix cost hours of downtime for the entire cluster.
Separation started with access, not data copying: each line got its own schemas and a role with rights only to them, applications got separate connection strings instead of a shared pool. Migration followed the plan — first non-critical environments, then reads, then writes.
The schema at each step remained backward compatible, so old and new code worked in parallel, without stopping services.
What metrics showed
Cross-access to another's database — from hundreds per day to zero. Recovery time after a deployment error decreased from several hours to ten minutes: only one line's environment rolls back. Table locks between teams disappeared — migrations no longer wait for others' releases.
A new environment stopped being a separate project: previously a copy of the cluster was provisioned for it, now a database in minutes. Backup volume did not grow, and recovery of one environment does not require deploying everything.
How does network restriction differ from database access control?
Network rules answer the question of who can connect to whom: they allow or block traffic between tenant environments and their services. But within an allowed route they do not see whose data is in the request — the rule passes any request that reaches the database.
Data-level rights work with content. Roles and permissions restrict which tables and operations are available to a service, and row filtering by tenant ID prevents returning others' records even with a correct connection.
A role is issued not to an abstract user but to a specific service in a specific environment — then an error in one handler does not turn into a full database export.
| Protection level | What it closes | Where it does not help |
|---|---|---|
| Network perimeter | Connections between tenant environments | Does not distinguish requests inside an allowed channel |
| Roles and permissions | Which operations are available to a service | A service with correct rights accesses the wrong data |
| Database rights | Row and schema filtering by tenant | Does not work with a shared account |
| Encryption keys | Reading others' data from a copy | Does not replace permission checks |
One level almost always leaks: the network will not stop a query without a tenant filter, and database rights will not help if the service is connected to a shared account.
Therefore protection levels are assembled into a combination of measures — dedicated environment, own roles, data-level filtering, separate encryption keys. Every attempt to cross the boundary is checked by a test: a request from another environment must fail, not silently pass.
Typical technical doubts before start
Before start, doubts arise not about isolation itself but about compatibility with what already works: shared database, granted permissions, growing number of environments, and the cost of moving back. Below are answers to the objections that most often slow down the decision.
We have one database for everyone — can data really be separated?
We separate not head-on: shared cluster, separate schema and role per tenant, access only through a service account tied to the tenant ID. Tenants with strict requirements are moved to a dedicated node first. The data access layer already knows whose request it is — edits are minimal.
Permissions are already granted broadly — will everything break?
First inventory: who actually accesses what. Then narrow roles and collect requests, and disable old ones when no traffic remains on them. Overlaps are closed by network policy — even with an excessive role, a call between namespaces does not pass.
Environments are growing — won't management become more complex?
On the contrary: namespaces, network policies, schemas, and roles are described declaratively and rolled out from one template. A new environment is a parameter, not manual assembly, so growth in the number of environments stops being a source of discrepancies.
What if we have to return to the previous scheme?
The reverse path is designed in advance: isolation does not delete data but changes access boundaries. Return is removing policies and merging schemas while preserving identifiers, and we test it on a copy before start. The storage format does not change during migration — only who and how accesses it.
Let's discuss your task and come back with a plan
The accuracy of a tenant isolation estimate depends on initial data: from a "two-word" description you get a range where the upper bound is twice as high as the lower one.
If you send the inputs from the list below, we will come back with an estimate of scope and a stage plan, not a random range.
- map of the current environment: which services are deployed where and how environments are separated — development, staging, production;
- how isolation is arranged now: shared database and shared network, separate namespaces per tenant, or hybrid scheme;
- data composition by tenant and its sensitivity: personal, payment, regulatory — this determines the strictness of boundaries;
- load profile and growth: how many tenants now, how many expected, where peaks are;
- constraints on changes: client integrations, release cycle, allowed maintenance window;
- access matrix: who from your team and external contractors remains in the environment and with what rights;
- incident history: were there cases of tenant interference or leaks through a shared layer.
On the call we clarify the task statement and trust boundaries between tenants, regulator requirements, and acceptance criteria — what is interpreted ambiguously in correspondence.
In response we provide a work scheme: order of stages, content of each, checkpoints, and the way to record the result. Then — start of cooperation: agree on the first-priority scope and roles in the team, begin with environment assessment.