HAProxy Ingress Controller: Bottlenecks of Load Balancing in a Cluster
The HAProxy Ingress Controller almost always ends up being the only entry point into the cluster: all external traffic passes through it, and its failure means all services are unavailable at once.
Under steady load this goes unnoticed, but as soon as long-lived connections appear — streaming video, persistent communication channels, streaming calls between services — the picture changes.
The controller stops being a transparent intermediary and turns into a bottleneck that determines how many requests actually reach the pods.
The first weak point is routing of long-lived connections. By default, the load balancer distributes requests among pods, but when a release is updated, pods are recreated, and already established sessions are cut off midstream.
Without proper connection draining and balancing by the number of active streams, rather than only by new requests, clients experience drops on every rollout.
A separate challenge is stream balancing in the cluster: for TCP load, some of the usual HTTP-level routing rules simply do not apply, and the decision has to be made at the entry layer.
The second is traffic observability.
If you do not collect per-backend distribution metrics, response-time statistics, and the failure rate, degradation looks like “users sometimes cannot open it.” The engineer does not see that one pod is taking almost the entire stream while the rest are idle.
We look at how this is addressed: how to remove the single point of failure at the entry, configure balancing of streaming connections, and gain observability into request distribution.
What a Dedicated Entry Layer Delivers: Measurable Effect
The first measurable effect is the share of successful connections.
When the entry layer holds the connection and waits for the application’s response, drops during a new version rollout and node restarts do not reach the client: the share of successful requests moves from 97–98% to 99.9%. Failures remain in metrics rather than in user complaints.
The second is failover switching time. Health checks remove an unresponsive node from rotation within seconds, traffic moves to healthy nodes, and no on-call intervention is required.
Before this, such a failure meant minutes of unavailability for some users and manual troubleshooting.
The third is behavior under peak load. The entry layer limits the number of simultaneous connections to each application and queues the excess, so a burst does not turn into a cascading failure: nodes receive exactly as much as they can process.
At the same time, everything unrelated to business logic is offloaded from applications — encryption, response compression, static content delivery.
The fourth is an eliminated class of failures where a single node handles all traffic and its failure brings down the service. Entry points are made redundant, rotation state is restored automatically, and restarting an individual node stops being an event. We build such a layer on HAProxy and move routing, health checks, and metrics onto it.
We use these values to accept the work: if the share of successful connections has not increased, switching time has not decreased, and applications are still spending resources on auxiliary tasks, the task is not solved.
Ingress Operating Modes: Comparison of Approaches
The entry layer can be kept inside the cluster or moved in front of it — this determines who owns the external address, where encryption terminates, and how load is distributed across worker nodes.
The HAProxy controller works in both modes, so the choice is dictated not by the tool but by topology. Below are three schemes with their maintenance cost.
| Option | When it fits | Limitations of the option |
|---|---|---|
| Built-in entry layer | One cluster, application traffic, a single address is needed with route updates directly from cluster objects | Availability depends on cluster nodes; the external address changes when nodes are moved; there is no separate layer for filtering attacks before the cluster |
| External load balancer in front of the cluster | Several clusters or environments behind one address, node liveness checks, offloading part of the load before the entry | A second configuration layer that is synchronized with internal routes; the load balancer itself is a point of failure, so a standby instance is needed |
| Hybrid routing scheme | Separation of streaming and application traffic: databases, message brokers, and streaming services go separately, web services go through the application entry | Two layers on the request path complicate incident analysis; rules are duplicated, and a unified ownership scheme is needed |
You should start with the question of how many environments must be served behind one address and whether there is traffic that is not a web request. If so, the hybrid scheme turns out to be cheaper to operate than trying to route everything through a single entry layer.
Implementation Procedure: From Audit to Handover to Operations
The customer usually comes with a symptom: the entry layer has stopped coping, some requests time out, and during a release update traffic disappears completely for a few seconds. Then the search for the culprit begins — application, network, or load balancer.
We structure the work so that every stage has a verifiable artifact rather than a verbal “it got better.”
-
Traffic scheme audit. We build a map: which services receive external traffic, where end-to-end TCP is needed, where HTTP is sufficient, and what timeouts and connection limits are currently set. The result is a list of bottlenecks linked to a specific route.
-
Entry layer design. We determine the number of controller replicas, placement across availability zones, role distribution among them, the certificate scheme, and the procedure for updates without downtime. The result is a scheme and configuration specification.
-
Controller configuration. We move routing rules, backend health checks, connection limits, and retry policies into cluster resource definitions. The result is configuration in a version control system, reproducible from scratch.
-
Monitoring and alerts. We display balancing metrics and backend status on dashboards and set up alerts for a growing error rate and node dropouts. The result is that failures are visible before users report them.
-
Fault tolerance testing. We shut down replicas and entire zones one by one, recording switching time and the number of lost requests. The result is a test report with actual values.
-
Load testing. We run a profile close to peak, with long hold times and sharp growth. The result is limit values for requests per second and degradation points.
-
Handover to operations. We provide the update and rollback procedure, access scheme, instructions for the on-call shift, and analysis of typical failures. The result is that the customer’s team can change the configuration and restore the layer on its own.
Configurations, Schemes, and Access Policies
After implementation, the customer’s team is left not with a “configured load balancer” but with a set of artifacts that allow the entry layer to be maintained and reproduced without the author.
The configuration lives in a repository under version control, changes go through review, and they are applied identically in test and production environments.
- Entry layer configuration — files and manifests in the repository, split by environment; no rule is edited by hand on a live node.
- Traffic flow scheme — the path from the external address to the container: where encryption terminates, internal ports, timeouts, session stickiness to one instance, readiness probes.
- Routing map — domains, paths, headers, and rule priorities, with an indication of which service sits behind which rule.
- Access policies and roles — role model, separation of permissions between the on-call engineer, developer, and administrator, and an action log.
- Maintenance procedure — how to connect a new service or change a certificate, how to update rules without downtime, and what is checked afterward.
- Rollback instructions — return to the previous configuration version in one operation, indicating recovery time and the responsible person.
- Entry layer monitoring — a set of metrics, triggering thresholds, and the on-call procedure when they are exceeded.
Case Study: Moving a Production Workload to a Dedicated Entry Layer
Before: The Entry Layer Shared Nodes with Applications
External traffic went through the same node pool where applications and background jobs ran. During rolling updates, entry processes terminated together with open connections — clients saw drops on long downloads and streaming requests.
The share of lost connections during an update reached 1.7%, and a node failure switched traffic in 60–90 seconds.
A separate item was support requests: 10–15 per month, some with the same wording, “everything dropped during the update.” Engineers were troubleshooting incidents instead of working on development tasks.
What Was Changed
The entry layer was moved to a separate node pool and built on the HAProxy Ingress Controller.
Health checks were configured with a short interval and failure threshold, and the timeout was selected for the load profile: long requests are not cut off, and an unhealthy node is quickly removed from balancing.
Before an update, the node is put into drain mode — new connections do not go to it, and active ones complete gracefully.
We validate the configuration before applying it, so an error does not reach production. The error rate per route, response time, and number of active connections are displayed in common monitoring — an incident is visible before users report it.
After: The Numbers
The share of lost connections during updates is zero across several dozen consecutive updates, including nighttime windows with peak load. Failover switching for a node failure takes 2–4 seconds.
Support requests about the entry layer decreased from 10–15 to 1–2 per month, and those are about access and certificates. Applying configuration and rolling back to the previous version takes seconds instead of minutes.
What to Do If Another Load Balancer Is Already Deployed in the Cluster
If another entry controller is already running in the cluster, you do not need to remove it immediately — both schemes can coexist peacefully.
The HAProxy Ingress Controller is deployed as a separate release with its own IngressClass, while existing routes continue to be served by the previous controller: it only takes objects that reference it.
External traffic arrives at a separate address, so the old and new schemes do not interfere with each other.
Next comes parallel operation of the two schemes on service routes and one production route. We compare behavior using metrics: response codes, backend response time, connection distribution across pods, and reaction to node failure.
We separately test non-standard scenarios — TLS termination, long-lived connections, WebSocket, and TCP traffic bypassing HTTP.
We shift the traffic share using weights on the external load balancer or via DNS with a short TTL: 5, 10, 25, 50, 100 percent.
Managing by weight rather than by switching ensures a transition without downtime: at any moment you can see how many requests go to the new scheme and how it responds to them, while a route always exists on at least one side.
We prepare the rollback plan before starting the shift and test it in practice, not on paper.
The old controller remains deployed, route manifests stay in version control, and returning the weight to zero and the route class to its previous value takes seconds and does not require rebuilding the configuration.
The duration of the transition is determined not by cluster size but by the number of non-standard scenarios: custom checks, external services, non-standard annotations. Usually this is a few weeks of steady operation without stopping the service.
Technical Doubts Before Starting Work
Before starting, the customer’s engineers check five things: where time is lost at the entry, what happens to long sessions, how unavailable nodes are filtered out, whether the real client address reaches the application, and whether HAProxy can be moved to production traffic without downtime.
Where does latency at the entry layer come from?
The load balancer holds the connection itself and selects a node based on current load. If the queue and number of simultaneous connections are not accounted for in the selection rules, some requests wait in the queue — we set the algorithm for the traffic profile.
Are long connections dropped — chats, downloads, streaming?
By default, everything that stays silent longer than the timeout is closed, and to the user this looks like a random disconnect. We agree on hold intervals and check frequency: an active session stays alive, while a stuck one releases the resource.
How does the entry layer know a node is unavailable?
Active health checks plus analysis of response codes in live traffic. The former catch a process failure, the latter catch a node that responds but returns errors. A faulty instance is removed from rotation and brought back gradually.
Will the real client address reach the application?
Only if you pass an auxiliary header with the original address from the entry layer to the application. Otherwise, the load balancer’s address is visible in logs and on the backend: rate limits, audit, and regional rules break.
Can we move to a new entry layer without stopping the service?
Yes. The new layer is brought up in parallel, traffic is switched using records in the name resolution system, and we keep the old one in reserve until metrics confirm success. Rollback is returning the records and takes minutes.
Let’s Discuss Your Traffic Scheme and Come Back with a Plan
For the assessment to be specific rather than an “industry average,” we need two things: your flow scheme and a list of environment constraints.
Describe what is already running, what is only planned, and which requirements are strict — for access, for preserving the client address, and for the change window.
With this set, we will analyze where ordinary HTTP balancing is sufficient and where end-to-end TCP mode with connection-level proxying is needed.
Then we will come back with scheme options and a work plan: what we do in the first stage, what is reasonable to move later, where the forks are, and what each of them costs in operations. Below is a short list that covers almost all questions even before the call.
- Flow scheme: traffic sources, entry points, services behind the load balancer, external consumers.
- Environment constraints: where we are hosted — cloud, own machines, or hybrid; what is available in terms of addresses and ports.
- Modes: where path and header parsing is needed, and where preserving the original client address for the application is needed.
- Encryption: where certificates are issued and renewed, and whether an end-to-end channel to the application is needed.
- Observability: which metrics and logs the on-call shift needs and where to send them.
- Rollback requirements: acceptable downtime and how far back to return to the previous configuration version.
- Who on your side accepts the work and approves changes in the production environment.
In response, you will receive an assessment and work plan, scheme options with the pros and cons of each, and the composition of the deliverables — configurations, flow schemes, the maintenance procedure, and deployment documentation for HAProxy Ingress Controller.
Describe the task in free form: send the scheme and constraints, then we will have a call and align on the details.