Why the default HAProxy setup breaks under load
The default HAProxy setup is a set of default values that is enough in a staging environment and insufficient at peak.
The first bottleneck is connection timeouts that are not set explicitly: while a client holds an inactive connection and the application server takes longer than usual to respond, processing slots are exhausted before the traffic is.
The queue grows, response time increases avalanche-like, and load balancing begins to distribute not requests but delays.
The absence of node health checks is the second landmine. By default, the load balancer often checks only port availability: an application can return errors but remain in rotation and receive its share of traffic.
One degraded node drags down the entire pool, and the problem is discovered through user complaints rather than metrics.
The third is a single entry point into the system as a point of failure. One process, one address, manual configuration edits: an update or restart means active sessions are dropped, and failure of the load balancer itself takes down the entire service.
Statistics are not collected in this case, so incident analysis happens after the fact, using application logs.
We'll look at how this is addressed: explicit timeouts matched to the traffic profile, health checks based on application response, entry point redundancy, and monitoring that shows degradation before it becomes an outage.
What changes after reworking the configuration and enabling monitoring
What we change in the configuration is visible in three metrics: peak response time, the share of error responses, and the time it takes the system to return to normal after a node fails.
At peak, traffic stops hitting a single server: the balancing algorithm selects a node by the number of active connections rather than round-robin, the number of simultaneous sessions per node is limited, and connections to applications are reused instead of opening a new one for each request.
Timeouts are set according to the actual distribution of processing time, so a slow but live node has time to respond instead of being cut off prematurely, and the queue at the entry point does not grow avalanche-like.
HAProxy maintains a separate health check for each node: interval, threshold of consecutive failures, and conditions for returning to rotation.
This closes a class of failures that was the main source of 5xx before the rework — a degraded node that responds but slowly continued to receive its share of traffic and corrupted responses for some users.
Now a node is removed from rotation before it starts returning errors, retries are sent only for requests safe to retry, and the rest receive an explicit response instead of a hanging connection.
Recovery time after a node failure is determined by the check interval and failure threshold — this is a configurable value, not a matter of luck.
The node re-enters rotation gradually: first it receives a small share of traffic, and only if response time remains normal does it return to full capacity.
The backup node is brought into operation when the primary pool has degraded entirely, so a burst of failures does not turn into a complete outage.
Load monitoring relies on access logs and counters: processing time per request, traffic distribution across nodes, error share, queue wait time, and the state of each server.
From this data it is clear whether a specific node has degraded or overall traffic has increased, and alerts are set from the observed load profile rather than by eye.
A few days after enabling it, it becomes clear which thresholds have become unnecessary and where headroom is needed — the configuration is refined based on facts, not assumptions.
HAProxy setup: balancing options and their limitations
A balancing scheme is not a single directive but a set: distribution algorithm, health checks, backup circuit, and entry point. A mistake at any level looks the same — some requests go to a non-working node and hang there until the timeout expires.
Four schemes and their boundaries
| Option | When it fits | Limitations |
|---|---|---|
| Distribution by connections (roundrobin, leastconn) | Homogeneous nodes, short requests | Long connections skew the load; a newly added node takes the entire flow |
With active health checks (option httpchk) |
The service is alive but returns errors | Checking a separate address does not see database degradation; frequent polling causes false removals |
With a backup circuit (backup) |
A critical service with limited capacity headroom | The backup is enabled only when the primary fails completely; its degradation goes unnoticed until an outage |
| With external distribution at the entry point (DNS record, upstream node) | Multiple entry points, geography | Record caching and removal delay keep traffic on a failed node for minutes |
Before choosing, we check three things: how many seconds it takes for a node to be removed from rotation, what happens if the load balancer itself fails, and where session state lives.
If sessions are sticky and a node goes away, the user loses their cart; if the check looks only at an open port, degradation remains invisible. We select the scheme for the traffic profile, not by default.
How implementation proceeds: from scheme analysis to handover
The order of work is the same for a new implementation and for rebuilding an already running load balancer: first we record the actual state, then we assemble the target scheme and test it in a staging circuit, and only after that do we touch the production circuit based on an individual calculation.
- Infrastructure audit. We map services, addresses, and ports, and look at where the load balancer sits relative to the perimeter and protection. We analyze statistics and logs: where requests are lost, how the session lives, and what edits were made manually on the servers.
- Target scheme. We decide what is balanced at the connection level and what requires header inspection, how load is distributed, and where session state is stored. We agree on the scheme before the first line of configuration.
- Configuration in staging. We deploy HAProxy in a test circuit with the same server roles. Here we also configure statistics collection, logs, and request rate limits.
- Load testing. We run a profile close to peak: response time, distribution across servers, behavior as the number of connections grows, and with slow clients. We make edits while it is a staging environment, not users.
- Failure testing. We take servers down one by one, test health checks and automatic node recovery. Here we also record the change rollback scenario.
- Traffic cutover. We switch in stages: first a share of requests, then the entire flow, while monitoring errors and latency. If metrics go out of bounds, rollback takes minutes.
- Handover to the on-call shift. We hand over the scheme, configuration, standard operating procedures, and incident analysis. The on-call engineer, together with ours, goes through a configuration reload and a drill failure.
What remains with you: configurations, diagrams, policies
The load balancer remains in your environment, which means everything needed for its operation and edits must remain with your team, not in correspondence with us.
That is why we hand over not “files for memory” but a set of artifacts by which an on-call engineer can bring up the environment, change a route, and roll back an unsuccessful edit without our help.
We write each document for a person seeing the system for the first time: where to look, what to check after a change, and which signal to react to first.
- Working configurations — HAProxy files for all environments, with comments on non-obvious decisions: why this response timeout was chosen, how queues are distributed, and what is reserved for peak loads.
- Traffic flow diagram — where traffic comes from, where encryption terminates, which nodes sit behind the load balancer, and which places remain points of failure. Reconstructing the picture takes minutes, not hours of log analysis.
- Access policies — rules for who and from which addresses can access the control panel and statistics, how changes in the production environment are restricted, and who is authorized to approve them.
- Rollback instructions — step-by-step return to the previous configuration in a single operation, a checklist after the return, and the procedure if the rollback did not help.
- Change versioning — configurations in a version control system, each edit as a separate changeset with a description: it is visible who, when, and why a route or rule was changed.
- Monitoring policy — which metrics we collect, at which thresholds an alert is triggered, where it goes, and who is on call.
- Handover procedure — accounts, roles, and permissions are issued to your people, and our access is closed immediately after handover.
Case study: moved traffic to a backup circuit without downtime
Before the work, inbound traffic was served by a single balancing node: encrypted traffic termination, binding of client sessions to a specific server, and backend health checks all converged on it.
Any planned configuration update meant restarting the process — according to logs, this caused up to 400 dropped sessions per maintenance window and a spike of 502 responses at the entry point. A backup circuit existed, but it was switched to manually and only in an emergency.
We moved the entry point to a pair of HAProxy nodes with a shared floating address: the configuration is assembled from a single source and rolled out as a whole, and a seamless reload passes listening sockets to the new process without breaking already established connections.
Session stickiness was removed — state is moved to shared storage, so any node can serve a request. Switching between circuits is now controlled by health checks, not by an on-call engineer.
What the measurements showed
We tested in two ways: by running a load profile captured from production traffic, and by controlled node shutdown during business hours.
The median response time at the entry point decreased almost threefold, the share of responses longer than one second fell from 7% to 0.3%, and 5xx responses — from 2.4% to 0.02%.
There are no more dropped sessions during maintenance: switching to the backup circuit fits within one check interval, and the user does not notice it. Switching the circuit back is the same operation with the same result.
How does a network load balancer differ from an application load balancer?
The network layer works with a stream of connections: the load balancer sees the address and port, selects a server by algorithm, and does not interfere with the traffic further.
Packet contents are not parsed, so latency is minimal, and behavior is the same for HTTP, DBMS, and long-lived connections.
There is one limitation: the balancing unit is the entire connection, and if a client holds a single channel, different types of requests within it cannot be routed separately by destination.
The application layer parses the application protocol and sees the method, path, headers, and cookies.
This enables request routing: one pool of servers serves static content, a second serves the API, a third serves service addresses, and rules are changed at the entry point without edits on the servers themselves.
It also handles cookie-based session stickiness and precise health checks: a node returning an error on the service address is removed from rotation, even though at the network level it would appear alive.
Encryption termination is a separate decision with its own cost. When encryption is terminated at the entry point, certificates and connection parameters live in one place, internal services receive plaintext traffic, and routing rules see the content in full.
End-to-end encryption to the server keeps traffic closed to the load balancer, but routing then relies only on metadata, and certificate renewal falls on every node.
In practice, the modes coexist: HAProxy handles some services at the network level and HTTP load at the application level, and the mode is set separately for each server pool.
Common technical concerns before starting work
The engineering team usually has the same concerns: what happens if a node fails, how quickly can we roll back, who is responsible for certificates, and will we have to catch degradation through user complaints.
We discuss these points before the start so the scheme remains maintainable rather than “works as long as you don’t touch it.”
Below are short answers in expandable blocks. If your question is not here, we will add it to the task analysis and come back with an estimate and work plan.
How quickly will you know that a node has dropped out?
The load balancer checks not only port availability but also the application’s readiness to respond — via a separate health address. A node that fails the check a specified number of times in a row is removed from rotation within seconds, without waiting for complaints.
We record check thresholds, intervals, and connection timeouts in the configuration and align them with your load profile: a threshold that is too strict causes false removals, while one that is too lenient keeps a dead node for too long.
What if the load balancer itself becomes a single point of failure?
We set up two nodes in hot standby mode with a shared virtual address: the active one serves traffic, the second holds state and takes over if connectivity is lost.
Failover takes fractions of a second and requires no manual action — sessions and connection tables are preserved through synchronization.
What about certificates and their renewal?
Encryption is terminated on the load balancer, certificates are renewed automatically, and the chain is verified before rollout to production. Certificate replacement happens without breaking active connections: first the new one is picked up, then the old one is phased out.
Can changes be rolled back and an incident analyzed?
The configuration lives in a version control system, and rollout goes through syntax validation and a smooth reload without dropping connections. Rollback is returning to the previous revision in a single operation.
For incident analysis, we collect logs with timestamps, response codes, and response time for each node: they show where degradation began and which node caused it.
Let’s discuss your scheme and come back with a work plan
To make the estimate substantive rather than “by eye,” we need a minimum of source data.
Send the current traffic flow diagram, the load balancer configuration, the list of services behind it, and availability requirements — that is enough to see bottlenecks and come back with a work plan.
We do not take on “doing everything” instead of you: some tasks are better left to the internal team, and we will reflect that in the plan as well.
Then we move through stages, and at each one it is clear what is happening and why.
- Source data: entry points, domains, certificates, target services, and routing rules.
- Analysis of the current scheme: how traffic is distributed, where the single point of failure is, and what is already configured in HAProxy.
- Implementation plan: changes step by step with verification after each — from configuration edits to traffic cutover.
- Maintenance window: we agree on an interval with minimal impact on users and a ready rollback to the previous configuration.
- Monitoring: metrics, alerts, and a load balancer dashboard — so the problem is seen before requests come in.
- Timeline estimate: duration of each stage and the overall schedule with points where your approval is needed.
- Handover: operating instructions and analysis of typical situations with your team.
Describe the task in free form — we will come back with an estimate, plan, and scope of work.