Configuring Linux routing: where network connectivity breaks
Connectivity between subnets breaks not at the moment a link fails, but more quietly: a service responds from one segment and stays silent from another, some sessions open, others hang until timeout.
Packet loss at the boundary looks like intermittent degradation — monitoring shows normal load, while application checks catch individual handshake errors.
That is exactly why such failures take so long to find: the network formally works, but Linux routing setup lives as a set of rules scattered across nodes.
Most often the cause is overlapping address spaces. When environments merge or a new site is connected, identical ranges begin to exist in two places, and a route with a longer prefix diverts traffic into the wrong segment.
The host is convinced the destination is local and responds itself, bypassing the gateway — the connection establishes in one direction and breaks in the reverse.
The second line of problems is asymmetric traffic. A request leaves through one router, while the response returns through another, and an intermediate filter that tracks connection states discards it as unexpected.
From the outside, this reads as packet loss at the boundary: the connection comes up intermittently, and diagnostics from different nodes give different results.
A separate class is manual entries in routing tables. A rule added for a one-off task outlives the task itself, and after a node migration it remains with the previous gateway address and quietly breaks the path.
Such entries diverge between servers, and the failure does not appear immediately but during the next schema change. Let us break down how this is closed.
What managed network connectivity provides: measurable effects
Managed connectivity means the traffic path is known in advance, rather than discovered during an incident. The five effects below are measured on a test bench before handover to operations, so “it got better” is confirmed by numbers, not feelings.
Link recovery time — seconds instead of hours
A link failure is handled by a pre-tested scenario: the route moves to backup on its own, without manual edits or an on-call engineer at the console. Service restoration fits within tens of seconds versus the time usually spent on diagnostics.
Share of dropped connections — outages stop being background noise
Symmetric return routes and correct filtering remove the losses that looked like “the site opens, then it doesn’t.” The share of terminated sessions falls to a level noticeable only in counters, not in complaints.
Eliminated failure class — path asymmetry and shadow subnets
Two-way route verification eliminates asymmetry where a request leaves through one link but the response does not return. The failure class “connection exists, no data” stops reproducing at all.
Speed of network changes — a new segment in one run
A new subnet, address space, or second carrier is described declaratively and applied without downtime. The change goes through the same set of checks as the current configuration, so edits are no longer postponed until a maintenance window.
Predictability of network behavior — a single source of truth
Addresses, subnets, and routes live in a single registry (NetBox), and working configurations are assembled from it. The discrepancy between documentation and actual state does not accumulate, and any network can be deployed repeatably.
Routing options: static, dynamic, hybrid scheme
The routing mode is chosen by two parameters: the size of the network topology and the frequency of its changes. While there are a dozen servers and subnets are fixed, static rules can be read in a minute, require no daemon, and are restored from configuration.
As soon as backup links appear and addressing changes more often than once a quarter, manual tables begin to diverge between nodes.
Below are three schemes that are actually chosen between. Without evaluative “better/worse”: each has a topology where it is appropriate and its own cost.
| Option | When it fits | Limitations of the approach |
|---|---|---|
| Static rules | Flat network, dozens of nodes, routes rarely change | Does not survive link failure without external monitoring; as the number of segments grows, tables are duplicated on every server and diverge during manual edits |
| Automatic path construction | Multiple subnets, backup links, frequent addressing changes | Requires a routing daemon and unified advertisement rules; an error in filters takes routes outside the environment; the network state must be readable |
| Hybrid scheme | Core and gateways on dynamic routing, peripheral servers and service networks on static | The boundary requires explicit route redistribution; the path is assembled from two sources, debugging is more complex |
We fix the boundary between zones in writing: which prefixes are advertised, where the default route comes from, what happens when the primary gateway fails. This removes the main risk of a hybrid — the assumption that “it will sort itself out.”
How implementation proceeds: from schema review to handover
The work proceeds sequentially: first we capture the actual network state, then design the target one, and only then change settings on production machines.
At every transition there is a rollback point — if traffic behavior diverges from the design, we return to the previous step rather than searching for the cause blindly on a running service.
-
Network schema collection. We inventory interfaces, address plans, routing tables, filtering rules, and how traffic actually flows between segments.
Discrepancies between documentation and real settings surface exactly here — in production they would have turned into loss of connectivity.
-
Addressing and path design. We define prefixes, gateways, default route priorities, name resolution order, and behavior when one of the paths fails. We get a schema where the address and route are assigned deliberately, rather than left as “how it historically developed.”
-
Configuration. We transfer the design into configurations: interfaces, static routes, separate tables for traffic separation, filters. We make changes repeatably and store them in a repository so the test bench, test, and production do not diverge from each other.
-
Test bench verification. We run scenarios: address change, loss of an upstream link, asymmetric return traffic, mixed IPv4 and IPv6 modes. We measure convergence and connectivity recovery time — we go to production only with these numbers.
-
Controlled migration. We switch one node or segment at a time, in an agreed window and with a rollback plan: previous configurations are saved. We confirm service availability with requests from external points, not from the server itself.
-
Documentation handover. We provide the current addressing and routing schema, dependency map, configuration templates, rollback procedure, and a post-change verification checklist. Your engineers can repeat the setup and modify it pointwise without contacting us.
What remains with you: configurations, schemas, procedures
Everything we configure remains in your repository and works without our involvement: you can bring up routing at a new site or restore the previous state using the documents, without searching for the author of the changes. Below is the set of artifacts handed over for each environment.
- Routing configurations — declarative files with interface addresses, gateways, static routes, and separate tables; applied with one command, and the post-reboot state is verified automatically.
- Addressing schema — which prefixes are assigned to segments, where both protocols are enabled, what is given to services, gateways, and service networks, with reserve for expansion.
- Filtering rules — what is allowed between segments and where, tied to services rather than specific hosts: a new machine does not require a policy edit.
- Policy-based routing rules — which traffic goes through which link and how the system behaves when the primary one fails.
- Change procedure — branch, test bench verification, application window, rollback: it is clear in advance which edits are safe during working hours.
- On-call shift instructions — how to distinguish a route problem from a triggered filter in minutes and what data to collect before the engineer connects.
- Connectivity map — flows between sites and external services, internet egress points, and backup paths.
The artifacts sit next to the code and are versioned: the edit history is visible, and the state on any date can be restored. We close the handover with a review session with your team and on-call shift — so the documents do not remain unread.
Case: carrier switchover without connectivity breaks
Before: carrier change as a planned outage
A distributed network of several sites connected to the internet through a single carrier, and the backup link was enabled manually: an engineer noticed degradation, edited the routing table, and restarted sessions.
Switchover took from 12 to 40 minutes, and during all that time established connections broke — exports between sites, client sessions, contractor channels. The backup formally existed, but every carrier change turned into service downtime.
After: switchover in seconds without connection loss
We separated the sites onto two independent carriers and moved routing to a dynamic scheme with availability checks through a separate path.
The route is selected by latency and loss metrics: while both links are alive, traffic goes through the primary one; during degradation it moves to backup in seconds.
The state of active connections is synchronized between border nodes, so returning to the primary link also does not break sessions.
- downtime during switchover: 12–40 minutes → 3–5 seconds;
- loss of established connections: with every carrier change → isolated cases only during planned work;
- false switchovers: several per quarter → zero;
- external perimeter availability: 98.2% → 99.94%.
The numbers are confirmed by two independent sources: Prometheus telemetry and border router logs; external checks from independent points give the same picture. Link resilience stopped depending on a single carrier and on manual actions by the on-call engineer.
How does automatic path rebuilding differ from manual entries?
The first axis is reaction to failure. With manual entries, the routing table remains as the engineer left it: traffic continues going into the failed link until the on-call engineer notices the problem and corrects the configuration.
Automatic rebuilding relies on constant service message exchange between nodes — missing acknowledgments remove the route from active ones, and traffic moves over the backup direction in seconds, without human involvement.
The second axis is network maintenance, that is, the amount of manual work. Static entries are maintained on each node separately: a new subnet, segment migration, second carrier — the edit must be distributed across all configurations and none missed.
A dynamic scheme describes the rules once: nodes themselves report available directions, while weights and metrics set priority. The more points and links, the more noticeable the gap in labor costs.
The qualification of the on-call shift is the third axis. Manual entries require the on-call engineer to keep the topology in their head and calculate the consequences of an edit: an error in an address or mask takes an entire segment down.
Automation moves this work to rules — it is enough to check adjacency state, metrics, and the presence of advertisements. The barrier to entry is lower, and the circle of people capable of safely intervening on a night shift is wider.
Predictability of changes and choice of approach are directly related. A static scheme is transparent but handles growth poorly: relationships between nodes have to be kept in documentation and in one’s head.
An automatic one rebuilds by given rules, and the result of an edit is verified before application on the working environment.
In practice, a hybrid is more often chosen: automation where fast reaction to failure is needed, static routing at fixed boundaries where the route should not change.
Common technical doubts before starting work
Before starting work, it is important for the customer to understand how changes will affect live traffic.
Below are answers to the questions most often asked before Linux routing setup: they concern preserving current rules, regression checks, downtime, and distribution of responsibility.
What will happen to already configured rules?
We first inventory existing routing rules and filtering policies: we collect routing tables, lists in the current kernel branch, and save them to configuration files. New entries are added on top; conflicting ones are transferred by explicit replacement, not deletion.
After the test bench change, the configuration remains reproducible from files, not from manual commands in a session.
How do you verify there are no regressions?
Before edits, we capture a baseline: available routes, response time to key nodes, state of tunnels and balancing. Afterward, we run the same set and compare the result.
Separately, we check path symmetry and behavior when one of the links fails: the route must rebuild, not break the connection.
Is downtime required?
In most scenarios, no. Rules are applied pointwise, without rebooting the node, and changes take effect on active connections gradually.
If the architecture requires a gateway switch, we choose a window of minimal load and keep it short: the client service sees only a brief increase in latency, not a break.
Who is responsible for a failure after implementation?
We fix the setup, addressing schema, and rollback plan in documentation — it is handed over to your team together with the configurations. The warranty support period is included in the work: if a deviation in routes is discovered after the change, we analyze and eliminate it. Responsibility for changes made after handover remains on your side.
Is there a rollback plan?
Yes, and it is verified before, not after, implementation. We save the previous state of routes and tables, prepare a reverse command set, and a configuration rollback point.
If control checks after switchover do not pass, the test bench returns to its original state in minutes — traffic goes along the previous paths.
What next: discuss the task and come back with a plan
To come back with an estimate and implementation plan, rather than general words, we need inputs on the infrastructure: routing on Linux servers is almost always tied to adjacent equipment and to how rules reach the nodes. Below is what is worth sending before discussing the task.
- Current addressing: static addresses or DHCP assignment, how many interfaces and subnets on each node, whether there is dual-stack IPv4/IPv6.
- Who distributes routes: gateway, router, balancer — and where the rules are currently described.
- Target logic: what should travel between subnets, what needs to be isolated, where egress to the outside and address translation are required.
- How rules reach the servers: manually, by configuration templates, or through a management system — the work schema depends on this.
- Work window constraints: allowable downtime, whether there is a backup link during switchover.
- Accesses: test environment for verification, change approval procedure, who accepts the result.
- How you currently check connectivity: tracing, interface error counters, logs.
This set is enough to assemble the work schema and result verification procedure: first a test environment and connectivity checkpoints, then rollout across nodes with rollback capability, then comparison with the original state.
Describe the task — we will come back with an estimate, implementation plan, and a clear acceptance procedure.