Setting Up Hetzner Cloud Load Balancer: Where Fault Tolerance Breaks
A typical picture: one virtual server receives all traffic, the domain points directly to it, and if a second node exists at all, it sits idle.
Load balancer configuration was never performed in such a setup—it is replaced by manual actions, and the single point of failure remains unnoticed until the first outage.
The cost is measured not in minutes of downtime, but in uncompleted transactions. When a node fails, some requests have already been accepted but not processed: the client sees a timeout, payment does not go through, a submitted form disappears. Manual traffic switching adds the on-call engineer’s reaction time to this.
This only becomes clear at the moment of an outage, because before it no one measured the service's fault tolerance.
There are no health checks that remove a node from rotation, no automatic failover of traffic to a live server, and no limit on the number of connections to a single node.
Externally, the system looks healthy: graphs are smooth, latency is normal, and free resources are available. The problem is that the safety margin is not tested in advance—it manifests at the only moment when it can no longer be tested.
We examine how this is addressed: distributing traffic across multiple nodes, health checks with meaningful intervals and trigger thresholds, and the behavior of open sessions when a server is removed from rotation. Next—what a managed cloud-grade load balancer provides, including Hetzner Cloud.
Measurable Effect: Recovery Time, Downtime, Fault Tolerance
After a properly assembled setup, what changes is not the feeling of reliability, but concrete values. Node availability is checked automatically every few seconds: two failed responses in a row—and the node is removed from rotation, traffic goes to healthy ones. Manual intervention when a service falls is no longer required; Hetzner Cloud reacts on its own.
At peak, only those requests that were already being processed on the failed node are lost. The load balancer sends new requests to healthy servers immediately, so the spike in errors is limited to fractions of a second and does not turn into a failure of the entire service line.
What is worth verifying in the report after handover:
- Recovery time — removing a node takes seconds, not tens of minutes of manual edits and restarts.
- Service downtime — planned updates proceed one server at a time; availability is not zeroed out for even a minute.
- Lost requests — the error rate at peak drops from tenths of a percent to short spikes of a couple of seconds.
- Smooth switching — active connections and sessions are not dropped; the user does not see a reload.
- Eliminated class of failures — the failure of a single server stops being an incident and does not require firefighting.
- Live testing — we shut down a node under load and see where requests went and how many errors clients saw.
Balancing Modes and Health Checks: What Fits and When
The choice of scheme is determined by two questions: what the load balancer sees in the traffic and how it understands that a node is alive.
Connection-level balancing distributes flows between nodes without inspecting request contents—it is cheaper in resources and leaves encryption to the application side.
Request-level balancing sees the address, headers, and response code: it can route traffic by paths, maintain sessions, and terminate encryption itself.
Placement in front of the cluster provides a single public entry point; placement inside the network separates layers and hides internal services from external scanning.
A standard port health check only confirms that the process accepts a connection—the application may still be returning errors to the user.
Therefore, the check is configured against an application endpoint response: the response code, response time, and number of attempts determine when a node is removed from rotation and when it is returned.
| Option | When it fits | Limitations |
|---|---|---|
| Connection-based | Long sessions, databases, custom encryption handlers | Does not see path and headers, no routing by addresses |
| Request-based | Web services and APIs, rules by paths and headers | Higher load on the load balancer, depends on correct client headers |
| In front of the cluster | Single entry point, receiving encrypted traffic from outside | The public address requires rate limits and access lists |
| Inside the network | Internal services, layer separation | No direct access from outside; an external entry point or gateway is needed |
Check thresholds define cluster behavior: overly aggressive ones cause false removals and 'flapping' rotation; overly soft ones delay removal of a failed node.
The limitations of a design also show up in details—terminating encryption on the load balancer shifts certificate work onto it, while connection-based balancing does not provide path routing.
In practice, options are combined: the external layer receives traffic, and the internal one distributes it across services.
Implementation Procedure: From Design Audit to Operations Handover
We start the work by recording the current state, not by installing a load balancer: an error in the design manifests not at setup time, but under real traffic, when switching between nodes should already work.
Therefore, we first capture the actual picture, then design the environment, and only after that change what serves users.
- Infrastructure audit. We collect the current architecture: which services are exposed externally, how addresses are distributed, where certificates are stored, and how node availability is currently checked. We separately list points of failure—a single node without redundancy, manual switching, and lack of health monitoring for application servers.
- Load balancing design. We determine the load balancer's placement relative to zones, the composition of target groups, the distribution algorithm, health check intervals and thresholds, and behavior when a node fails. We also decide where encrypted traffic is handled and which redirection rules are needed.
- Network and access preparation. Private networks, filtering rules, engineer and monitoring system permissions—we close all of this before production traffic starts.
- Environment configuration. We configure Hetzner Cloud Load Balancer, target groups, health checks, certificates, and logging. Configuration is stored as code: a change goes through review and is rolled back with a single operation.
- Load testing. We run scenarios for growth in concurrent connections, failure of one node, service degradation, and traffic spikes. We observe how quickly distribution is rebuilt and how latency behaves.
- Commissioning. We switch traffic in stages, with the ability to return to the previous design. We capture latency and error metrics before and after—to see the effect, not assume it.
- Documentation handover. We provide the design, operations and rollback instructions, alert descriptions, and failure response procedure, so the customer's engineer can investigate an incident without us.
After handover, there is short-term support: we address configuration questions and help with changes until the team can work with the environment independently.
Configurations, Health Check Rules, and Handover Documentation
Handover is not a demonstration of a working panel, but a reproducible set of artifacts that remains with your team.
Below is exactly what we hand over for the Hetzner load balancer: configuration, health check rules, network design, access, and an incident runbook—and how to use it without us.
- Load balancer configuration in declarative form, in your repository: routing rules, target groups, connection limits. A change goes through review and is deployed with a single command, not by editing in the panel.
- Service health check rules: path, interval, failure threshold, timeout. Alongside—justification of the values for your traffic profile: how quickly a node is removed from rotation and why exactly that timeframe.
- Network design: private segment, subnets, availability zones, entry points, and load balancer connections to application servers and the database. It shows what is open externally and what is closed.
- Access rights and network rules: engineer roles, key-based access, isolation of the build environment from production. It is clear who can change what and where it is recorded.
- Operations and incident response runbook: how to remove a node from balancing, how to roll back configuration to the previous one, in what sequence to act, and whom to involve.
- Monitoring: metrics for latency, error rate, and number of live nodes with trigger thresholds. A notification arrives before a user complaint.
Case Study: Eliminated Degradation on Node Failure and Manual Switching
Before the Work
The customer had two cloud servers; traffic between them was distributed at the DNS level with a short record TTL.
When a node failed, some clients continued to contact the dead address: the TTL does not expire instantly, and intermediate resolver caches keep the old address for minutes.
The on-call engineer learned about the problem from complaints, brought up a replacement, and manually edited records—6-8 operations per incident and up to 20 minutes of degradation under load.
What We Changed
We moved the entry point from DNS to Hetzner Cloud Load Balancer, declared the nodes in a target group, and configured health checks against an application handler rather than an open port.
The threshold is two failed checks in a row, removing the node from rotation before stopping the service so active connections can complete. We added a deployment template for a new node and automatic attachment to the group.
The Result
- Failover on node failure—12-15 seconds instead of 20 minutes and without human involvement.
- Successful request rate at the moment of the incident—99.9% versus 96%; there were no user session drops during failover.
- Manual on-call operations per incident—zero instead of 6-8: the alert arrives when rotation is already complete.
Cloud Load Balancer or Your Own Server with Balancing: What to Choose
The choice comes down to one question: who keeps the balancing layer operational—us or you.
The managed Hetzner Cloud load balancer is a ready resource with availability checks, target groups, encryption termination, and automatic removal of a failed server from rotation: setup takes hours, and scaling comes down to adding nodes to the group.
There are two limitations. The routing policy is fixed by the provider—custom header rules or complex failure logic cannot be expressed there.
Diagnostics are limited to the available metrics: when requests 'disappear' between the load balancer and the application, you have to investigate indirectly, using backend logs.
Your own balancing node gives full control: arbitrary routing schemes, custom checks, fine-tuned timeouts and queues, and the ability to capture a traffic dump at the moment of failure.
The price is ongoing maintenance: updates, certificate rotation, a fault-tolerant pair of nodes, and an external observer that brings up the second if the first fails.
Responsibility boundaries are the main dividing line. With a managed service, the provider covers platform failure, but we are responsible for target group configuration, correctness of checks, and rollout order.
With your own node, everything falls into our scope: the application, balancing, and availability of the entry point. We fix this boundary in the documentation before work begins, so that during an incident review no one is looking for someone to blame.
The practical criterion is simple: if the routing scheme fits within the standard capabilities of a managed load balancer, your own node is extra maintenance effort with no gain in reliability.
We install our own when there are requirements that cannot be expressed by the platform. Describe your traffic scheme—we will come back with an estimate and a plan.
Technical Concerns: Checks, Limits, Migration Without Downtime
Before starting, an engineer usually checks three things: how liveness checks are configured, what the limits on concurrent connections are, and whether a running project can be switched without a maintenance window.
The fault-tolerance design depends on these decisions, so in the Hetzner Cloud environment we discuss them before work, not after.
How is a node liveness check configured?
The check path, polling interval, response timeout, and number of allowed consecutive failures are parameters, not constants. The load balancer removes a node from rotation after a series of failed responses and returns it when the check passes again.
Overly aggressive thresholds cause false positives during spikes; overly soft ones keep traffic on a problematic node longer. We calibrate by load profile.
What about connection limits?
A limit on concurrent connections is set for each target server: beyond it, new requests are not opened even if the node is alive. We calculate headroom from the real profile—with persistent connections, the channel is occupied longer than with short requests.
If you hit the limit, we add nodes to the pool or raise the limit: the first path is more resilient, the second is faster.
Can migration be done without downtime?
Yes: we bring up the load balancer in the new environment, add nodes to the pool, and wait until checks confirm readiness—only then do we switch traffic. We move the domain record with caching reduced in advance, and issue the certificate for the new address before that moment.
The old environment remains in reserve; rollback is returning the record, not restoring from backups.
Let's Discuss the Task and Come Back with an Estimate and Work Plan
To come back with a work estimate rather than generalities, one conversation and an export of the current configuration are enough for us. Below is what we request as input and what we deliver as output; on your side, this creates no obligations.
- Initial data. List of services behind the load balancer, their ports and protocols, domains, TLS requirements, expected and peak load—this is enough to understand the scope.
- Access for review. Read access to the cloud project or a configuration export: firewall rules, private networks, target groups, availability check parameters.
- Current design review. We show how traffic flows now, where the point of failure is, and why a second copy of the service cannot be brought up without manually editing addresses.
- Risk list. As a separate list: connection drops during deployment, lack of rollback, ports open to the outside, certificate expiration, a single application node.
- Implementation plan. Step by step: network and target groups, availability checks, TLS termination, traffic switching, load testing, rollback runbook.
- Work timeline. Estimate by stages and total duration: basic turnkey setup fits within one business day; migration of live traffic—a separate item with a window.
- What remains with you. Configurations, traffic flow design, switching checklist, and a set of metrics that show the load balancer's state.