Network Security Policies: why traffic between services is open
By default, the network inside the cluster is flat: pods see each other directly, and the route between nodes does not require confirmation.
Until Network Security Policies are defined, any service can reach a database, message broker, or internal API — not because it was designed that way, but because nothing prohibits it.
The cost of this picture becomes visible at the first failure: a configuration error or compromise in one service opens access to neighboring services, and through the same open traffic between services — to external systems and storage. We break down how this is closed.
Workload segmentation is built on three questions: who initiates the connection, where to, and on which port. Rules are described with pod and namespace label selectors, so traffic isolation in the cluster is not tied to addresses and does not break when pods are recreated.
The "deny everything except what is allowed" mode is enabled first, and exceptions are added point by point — based on actual connections, not assumptions.
The order of work is the reverse of the usual: first we capture a map of flows from network logs and metrics, mark required connections and debug routes left after launch.
Then we describe network security policies in Calico or Cilium, including egress rules and DNS resolution, and test them in a test environment.
The output is a set of rules in the repository next to the manifests: a change goes through validation and rollout just like code, not manually in a panel.
Traffic isolation between environments: failure classes and response time
Segmentation starts with default-deny: in each namespace, ingress and egress traffic is closed, then only the required service-to-service pairs and ports are opened.
Rules describe not "who is allowed" but "who is not allowed otherwise" — connections are visible from manifests, not reconstructed from logs.
The approach is supported by common network add-ons, including Calico, and can be transferred between them almost without rewriting rules.
Which failure classes stop spreading: a credential leak in a test namespace no longer gives access to the production database, and an infected pod does not scan its neighbors — lateral movement is cut off at the first deny.
A configuration error that causes a service to reach into someone else's storage is intercepted by the network, rather than discovered through corrupted data.
Incident triage time drops because the policy is an executable diagram of connections.
The on-call engineer sees allowed flows and eliminates half the hypotheses immediately, while a triggered deny is written as a separate event — the "who talked to whom" chain does not need to be pieced together.
Downtime becomes predictable: the blast radius is limited to one segment, and rollback is returning the previous version of the rules, not restoring the entire network. New rules are first applied in monitoring mode and only then block traffic.
Network Security Policies: control modes and limitations
By default, connections inside the cluster are open: any workload can reach any other. Network policies turn this into a "only what is listed is allowed" model — rules describe which services can be reached and on which ports, and rely on labels, not addresses.
They are enforced by the cluster's network component: if the chosen implementation does not support the required set of fields, the policy remains a description without consequences, and a false sense of security costs more than no rules at all.
Four levels of isolation
Control layers solve different tasks and are combined in practice — the choice depends on what exactly needs to be closed.
| Option | When it fits | Limitations |
|---|---|---|
| Built-in cluster rules | Homogeneous cluster, service-to-service connections are predictable | Do not manage traffic outside the cluster; within an allowed group, they do not distinguish who exactly initiated the connection |
| Node-level rules | Need to close connections between nodes and outbound access, part of the workload lives outside the cluster | Defined separately from service descriptions and become out of sync during migrations; debugging is more expensive |
| External perimeter isolation | The entire cluster is in a closed segment, multiple environments under common control | Inside the segment, connections remain open; there is no control over individual services |
| Mutual authentication | It is important not only "is it allowed" but also "who exactly" is connecting | Requires issuance and rotation of credentials; higher maintenance requirements |
The implementation order is usually this: enable built-in policies in monitoring mode, collect real connections from logs, switch monitoring to blocking, and only then add node rules and mutual authentication where protection against lateral movement is needed.
Implementation procedure: from traffic inventory to handover
We build the work order so that no production service loses connectivity. First we capture a map of actual flows, then describe allowed connections, and only after that enable blocking — in a test environment, then in waves across namespaces, with a rollback plan at every step.
- Traffic inventory. We collect network flow logs, DNS queries, and service mesh metrics for a period that covers both scheduled jobs and nightly batch processes. The output is a service → service → port matrix.
- Allowed connection diagram. We keep confirmed connections and coordinate disputed ones with service owners. The baseline is denying everything not explicitly described.
- Configuration in a test environment. We move the policies to a cluster copy and enable them in logging mode: rules are evaluated, traffic is not cut. This way we accumulate hits on a real traffic profile without risk.
- Policy validation. We run legitimate chains and end-to-end paths for monitoring, log collectors, and the image registry: what is allowed passes, and anything extra is recorded in the report.
- Phased enablement of rules. We switch namespaces to blocking mode one by one, starting with non-critical environments. Before each step — monitor availability metrics.
- Blocked connection triage. Using logs, we separate malicious traffic from undocumented but necessary connections: either a precise exception or a fix to the service itself.
- Handover to the team. We hand over the policies in a version control system, the change management process, the connection diagram, and on-call instructions. Rollback is switching to the previous set of rules.
Manifests, flow diagrams, and change management
A policy without documentation lives only until the first change of owner: a new engineer sees a set of rules and does not understand which connections are actually needed and which remain from the previous architecture.
Therefore, along with the working environment, we hand over a package that allows changes to be made consciously without opening anything extra.
- Allowed flow registry — who communicates with whom, on which port, and which service is responsible for the connection. It is referenced during failure triage and when decommissioning a service.
- Ready policy configurations — manifests with clear names, split by namespace, with notes on which rules are mandatory and which are temporary. They are rolled out as is.
- Trust map — a map of segments and traffic directions: what we consider the external perimeter, what is the internal zone, where communication between environments and development environments is allowed.
- Change management process — the change procedure: a request with justification, validation in a test environment, rollout window, and the person responsible for approval. Without it, rules quickly turn into a dumping ground of exceptions.
- Block log and triage procedure — where to look when a service fails, how to distinguish a false positive from a real connection attempt, and within what time the rule is removed.
- Rollback instructions — how to return the previous set of rules with one operation if dependencies break after a change.
- Acceptance scenarios — a list of connections checked after every change so that regressions are caught before production.
The package is kept in the repository together with the manifests, so the documentation does not diverge from the actual state of the cluster: changing a policy and changing its description are done as one coordinated action.
Case study: cluster segmentation without service downtime
Before: flat network
The cluster served production, pre-production, and a test environment in one network — any pod could reach any service on any port.
Almost all connections were allowed: the environment itself restricted nothing, and incident triage began with manually going through logs on all nodes and took at least four hours. The test environment could see production databases.
What was changed
First, traffic was captured over several release cycles and a service → service → port dependency graph was built — it became the source of rules.
Policies were described in Calico: first in logging mode, when connections are only recorded but not blocked, then default deny and point exceptions were enabled.
Each environment received its own set of rules, so pre-production and the test environment stopped routing to production.
Result
The share of allowed connections dropped from 100% to 11% — the rest is closed by explicit denies. Incident triage accelerated several times over: instead of going through nodes, it is enough to look at the log of rejected connections, 30–40 minutes instead of half a working day.
The rules were enabled without stopping the workload: the on-call engineer monitored the failure flow, rollback to the previous profile took minutes, and no downtime window was required.
How do network policies differ from access roles?
Access roles and network policies close different questions, and confusing them leaves gaps in protection. Roles delimit permissions: which account can read, create, and modify objects in the management system.
Network-level policies answer a different question — which connections between workloads are possible at all.
The access model is built around a subject and an action: permission is granted for an operation on an object. The network is described differently — by a flow: source, destination, port. Therefore, a role will not stop an already running pod that reaches into someone else's database.
The difference is visible in an example. A service account is allowed to read secrets in its own namespace — the role permits this, and that is normal.
But this does not mean the same service should reach the billing database: the policy prohibits the connection, and if the pod is compromised, the attacker will not get into someone else's environment.
By default, in a cluster without policies, traffic between namespaces is open. Therefore, implementation starts with default deny and explicit permissions by pod selectors: only the required service → service pairs and outbound DNS access.
Calico and Cilium apply such rules at the network level and drop the connection before it reaches the application.
The comparison of mechanisms comes down to the area of responsibility: roles are responsible for managing the cluster and its objects, policies — for connections between workloads.
Auditing is also different: permission changes are visible in the management system log, rejected connections — in node counters. The access model works only when both layers are configured.
Technical concerns: compatibility, rollbacks, support
Network security policies change how traffic flows inside the cluster, so teams almost always have the same concerns. We answer the ones most often asked before starting.
How can we be sure the policies will not cut off working traffic?
We capture a picture of current flows: which services and namespaces communicate, on which ports, who goes outside. We enable policies in monitoring mode — they only log what would be dropped.
After reviewing discrepancies, we add permissions and only then switch the rules to enforcement mode.
What will happen to rules that already exist in the cluster?
We inventory everything described by manifests and consolidate it into a single set with priority validation: in case of overlap, the order of application determines the result. Controllers calculate this order differently, so we do not mix two in one cluster — only one remains.
How do we roll back changes if access is lost?
The rules live declaratively in the repository, so rollback is returning the previous version and applying it, without manual edits.
We keep the principle: "deny everything" is not applied as the first step — permissions for service traffic remain in place until the switch.
Who supports the policies after handover?
We hand over the repository with rules, the service flow diagram, and diagnostic instructions: which logs show exactly what was dropped. For a while, we stay in touch and edit the rules together with your team until it reaches an independent mode.
Getting started: what we need from you and timelines
The starting procedure is the same for one cluster and for a fleet of dozens: first input data, then work estimation, then an agreed implementation plan with timelines.
Without an environment diagram and a list of services, the estimate becomes a range, and Network Security Policies have to be redesigned along the way — so we put data collection in the first step and record the result in writing.
- Environment diagram: production, pre-production, and development clusters, their connections to each other and to external networks.
- List of services: who communicates with whom, which external addresses and ports are needed, where traffic flows in both directions.
- Entry points: load balancers, gateways, service publication, DNS and image registry access.
- Credentials: a dedicated role with permission to read policies and audit mode enabled in the test environment.
- Target requirements: what exactly we are restricting — connections between namespaces, internet access, database access.
- Constraints: windows in which changes in production are allowed, and services that cannot be stopped.
- Responsible person on your side: an engineer who confirms connections and accepts the result.
Then we return with a staged estimate and plan: a pilot in the test environment in audit mode, when policies only log traffic and block nothing, reconciliation of the log with actual connections, switching to blocking mode one namespace at a time, then production.
From discussing the task to the pilot takes days, to full migration — weeks; the exact timeline depends on the number of services and connections between them.