Why Linux firewall configuration fails in production
Filtering rules accumulate over the years: chains are edited for a specific incident, no one removes old entries, and gradually Linux firewall configuration turns into archaeology.
An engineer who opens the configuration sees hundreds of lines but cannot answer which one closes which service and what will happen if it is deleted.
This leads to two persistent consequences. On the perimeter, open ports remain "just in case" — for debugging, an old staging environment, an integration that has long been unnecessary; each such port outlives a team change and falls out of view.
And there is the opposite problem: blocking legitimate traffic after editing one rule — you close access for one subnet, and along with it monitoring, log export, or backups that used the same range go down.
The root cause is rules without an owner. An entry has no binding to a service, comment, or revision date, so people are afraid to touch it and leave it as is.
Chaos in filtering rules grows not because engineers are careless, but because a change is not tested on a staging environment and is not rolled back in a minute when something goes wrong.
Even switching to a declarative description in nftables does not solve the problem by itself: if rules are not tied to real traffic, services, and owners, the order will remain the same.
We break down how this is addressed — from port inventory to changes that are verified before production.
What a unified set of traffic filtering rules provides
Filtering rules scattered across hosts and edited manually via the console survive until the first outage: no one remembers why a port is open or who will close it.
A unified set is a single policy description that is applied identically on all servers and stored alongside the configuration of other services.
Managed traffic filtering turns network access into a versioned set: changes undergo verification rather than being made "live" on a production host.
The key effect is reducing the downtime window during changes.
The rule set is assembled entirely and applied in one transaction: nftables swaps the table atomically, so connections are not broken mid-edit, and an unsuccessful variant is rolled back by timer if the service has not confirmed availability.
The window goes from minutes of manual work to seconds, and some changes occur without breaking already established connections.
A whole class of failures disappears from the incident list: rules forgotten after a service is removed from a host; duplicates and overlaps where the last match overrides the needed one; ports opened "for debugging" and left forever.
Transparency of network access is achieved because the map of "who talks to whom and on which port" is assembled from the rules and logs themselves, rather than reconstructed from memory during an outage.
Service recovery time no longer depends on which engineer is on shift: the filtering state is reproduced on a new host from the same set in minutes, and returning to a previous revision is a rollback of the set, not a search for an error in a long list of lines. The default-deny policy guarantees that a forgotten rule closes access rather than opening it.
Packet-based, zone-based, and cloud filtering modes
The choice of filtering tool determines where rules live, who changes them, and whether they survive a system update. When configuring a Linux firewall, one usually chooses between four modes — from manual chains to external security groups.
| Option | When it fits | Limitations |
|---|---|---|
| iptables | Ready-made scripts, point rules on a single host | Tables are read linearly: as the number of rules grows, latency grows, and the syntax does not evolve |
| nftables | Complex rule sets, address sets are needed, and common logic for IPv4 and IPv6 | Old configurations have to be rewritten, and some management panels do not yet work with it |
| firewalld and the zone-based filtering model | Hosts with a changing profile: office, VPN, public interface — switching without rebuilding chains | The zone abstraction hides low-level capabilities; exception rules are harder to debug |
| Cloud security groups | Machines in the cloud where filtering is set at the network level | They do not close traffic inside the host and between containers, depend on the platform — a local firewall is still needed |
In practice, modes are combined: external security groups cut off excess before the machine, while the local firewall closes the rest inside. We start the analysis with a map of services and traffic directions.
Implementation procedure: from inventory to regulations
Rules are not written "by eye": first you need to understand what actually travels over the network, otherwise a closed port is either a hole or a failed service.
We work from inventory to regulations so that after handover your engineers can make changes without our involvement.
- Network rule audit — capture current iptables, nftables, and firewalld zone chains, compare them with actually running services. Find duplicates and "any-any" permissions left over from removed services.
- Network flow diagram — record who communicates with whom: source, destination, port, protocol, meaning of the flow. We coordinate the diagram with service owners, not only with administrators.
- Rule configuration — default deny plus explicit permissions grouped by purpose, with a comment for each rule.
- Verification on a staging environment — a copy of the topology, real traffic, and negative scenarios: exactly what will go down if excess is closed.
- Controlled rule migration — apply in stages, with a rollback script and a window when return takes minutes rather than hours.
- Change regulations — the procedure for approving an edit, recording it in a version control system, and periodically reviewing outdated permissions.
After handover, what remains is not a set of commands but a reproducible process: an edit is verified on the staging environment and does not turn into a surprise at three in the morning. The regulations describe who makes changes and how — this removes dependence on a specific person.
What we hand over: diagrams, policies, regulations
We deliver not a working firewall but a set of artifacts your team can use to change the configuration without us.
Everything that goes into the production environment is first recorded in the repository and description — otherwise the next edit turns into archaeology through the history of commands on the server.
- Configurations of the rule set — in the repository, not only on the host: versioned as code, with a note of which snapshot is currently applied and how it differs from the previous one.
- Network access matrix: source, destination, port, protocol, who initiates the connection and why — for each allow line; default denies are described in a separate block.
- Traffic restriction policies: rate limits and concurrent connection limits by service, separate conditions for service subnets and admin access, behavior during spikes.
- Change rollback procedure: how to return the previous set, in what time, and what to check after return, including the scenario of losing access to the server.
- Infrastructure documentation: traffic flow diagram, zones and chains, integration points with the load balancer and reverse proxy.
- Change management regulations: who approves a new rule, where it is verified before the production environment, and how it is recorded in the change log.
- Logging and acceptance: which events are written and where, plus a checklist of checks that your team runs itself and confirms the reproducibility of the configuration.
In this way, the configuration no longer depends on a single engineer: it can be read, checked, and changed according to the document, not from memory.
Case study: how the perimeter was closed and a class of failures was removed
Before: rules without an owner
The perimeter grew chaotically: about four hundred rules accumulated by different teams, a third of them without an owner.
No one could explain why a port was open or what would break if it were closed, so it was left as is "just in case." Processing order was not followed: allow rules were placed above deny rules, and part of the traffic was blocked where it was not expected. Access audit stretched over days, and recovery after a failure was done manually from memory.
What we changed
We consolidated the rules into a single registry, matched each one to a service and owner, and moved the logic into declarative nftables sets — the configuration became identical across all environments and versioned.
Everything not confirmed by an owner was closed by default; access is granted by template for a specific service, rather than by manual editing.
Metrics before and after
- Services accessible from outside: 46 → 11, four times fewer entry points
- Rules with a confirmed owner: 31% → 100%
- Mutual blocks: 9 cases per quarter → 0
- Configuration recovery after a failure: ~45 minutes manually → 6 minutes by deployment from the registry
- Access check before release: 2 days → 20 minutes
The main result is not in the number of closed ports but in predictability: every rule has a justification, a change is rolled back with one operation, and a new service receives access from a template. This is what removed the class of failures associated with mutual blocks.
How does host-level filtering differ from network perimeter filtering?
Host-level filtering answers the question "who can access this service." Rules in nftables, iptables, or firewalld describe which ports listen externally, which addresses are allowed to management interfaces, and how established connections are handled.
This is the last line of defense: if the ingress perimeter let something extra through, the server itself is the one that refuses.
Network perimeter protection works at a different level — it decides which subnets can see each other at all and which services are published externally.
A single host with an open port should not become an entry point into the entire network, so the perimeter describes routing and address translation, not just the list of allowed ports.
Segmentation of internal services adds a third level: the database, queue, and internal API live in different segments, and access between them is opened selectively, by the "source — destination — port" bundle.
Then a compromise of one host does not put the others at risk, and the boundary of attack propagation is known in advance.
The conflict between filtering levels arises not in the rules themselves but in their inconsistency.
A host with a "deny everything except explicitly allowed" policy stops accepting traffic when the perimeter changes, and a segment opened "for debugging" remains open forever.
That is why we describe policy top-down — from perimeter to host — and check that each level denies by default, and that permissions match real traffic, not initial assumptions.
Questions about compatibility and rule rollback
The Linux kernel allows different filtering tools to coexist: while one rule set is running, the second is assembled and tested in parallel.
Therefore, switching between iptables, nftables, and firewalld does not require a downtime window — established connections and routes are not broken.
We follow one scheme for any edit: the rule is tested on a copy of the configuration, applied in the running set, and the previous state is saved as a separate snapshot. Below are typical concerns that arise before such an edit.
Can iptables and nftables be kept on the same host?
Yes. The tools work with different sets inside the kernel and do not interfere with each other as long as a single connection is not described twice. The old set remains active until the new one is verified — only then do we remove duplicates.
What happens if a new rule cuts off access to the server?
A snapshot of the current set is saved before application, and the edit is made from a separate session. Rollback returns the previous state in seconds and does not require rebooting the host.
Do I need to stop the service during edits?
No. Rules are added and removed in the running set, and established connections continue to live. Stopping is needed only when changing the filtering mechanism itself — and even then, after testing the new set.
How can I make sure a rule is safe before applying it?
We run the configuration in test mode: we look at which packets it allows and blocks, and compare it with the access matrix. We remove excessive blocks before the rule reaches a production host.
Let's discuss the task: we'll analyze the scheme and come back with a plan
So that the scope estimate is based on facts rather than guesses, it is useful for us to see your current picture: the active rule sets, the zone and interface diagram, the list of services with inbound traffic, and what is already hindering operations — blocks of legitimate connections or open ports that no one remembers.
In response, we return not "we'll figure it out on site" but a concrete set:
- analysis of current chains and tables: what is actually allowed, what is blocked, and where rules duplicate each other;
- a map of services, ports, and traffic sources — with a note on which of them are confirmed by system owners;
- scope estimate: how many rules we rewrite, what we transfer unchanged, and where new zones are needed;
- implementation plan with the order of rule migration — from peripheral hosts to critical ones, so as not to lose access to the server;
- snapshot and rollback scheme: rule sets are saved before changes, and return to the previous state takes minutes;
- order of checks after migration: which connections we confirm manually, which paths we keep under observation;
- a responsible engineer on call and the point where logs of triggered rules are collected.
We propose discussing the filtering task concretely — based on your diagram, not a template.
The engineering team is available: we will analyze disputed areas, show how the same tasks were solved in similar environments, and come back with a scope estimate and implementation plan.