Nginx L4/L7 Load Balancing Setup

Practical Nginx L4/L7 Load Balancing Scenarios

Imagine your online store is slow, half the requests timeout, and monitoring shows one server overloaded at 90% while two others idle. This is a classic misconfigured load balancing scenario. Typical picture: backend servers have different capacities (e.g., one with 4 vCPU, two with 2 vCPU), and round-robin sends equal requests, overloading the weaker nodes. This leads to cascading latency increases — TTFB spikes to 3-5 seconds, and some requests fail due to connection limits. Industrial L7 load balancing with Nginx using least_connections and weights solves the problem.

In practice, we encountered a project where round-robin caused response times of 10 seconds. After switching to least_conn with weights, load balanced and response times dropped by 40%. According to our measurements, least_connections works 2-3 times more efficiently than round-robin under varying request durations. With an average order of 3000 RUB, losing every tenth customer costs 900,000 RUB per year.

This article covers real technical challenges encountered in practice — from session loss to SSL passthrough — and how to overcome them.

Why L4/L7 Load Balancing is Critical for High-Load Projects

Failure of one server can bring down the entire service. Without proper health checks and failover, users get 502 errors. Moreover, suboptimal request distribution increases response time 2-3 times, directly impacting conversion. L4 (TCP/UDP) load balancing works for any protocol but ignores request content. L7 (HTTP/HTTPS) offers flexibility: routing by URL, headers, cookies. Together they cover any scenario.

When to Use L4 vs L7?

The choice depends on the task. L4 load balancing (via the stream module) is indispensable for services that don't require protocol analysis: databases (MySQL, PostgreSQL), Redis, SSH. It adds minimal latency (~1 ms). L7 is suitable for HTTP applications requiring sticky sessions, canary deployments, or A/B testing. For example, an online store with a shopping cart — L7 with IP hash or cookie-based sticky. For microservices architecture, we often combine: L4 for balancing traffic between pods, L7 for internal routing.

Problems We Solve

• Uneven load distribution. Round-robin is not always fair: under varying request loads (some requests heavier), servers get unevenly loaded. We use least_connections or weighted least_connections to compensate.
• Session loss on backend failure. Without sticky sessions, a user may land on a different server, losing cart data. We implement cookie-based sticky or hash $cookie_SERVERID consistent.
• Incorrect health checks. If passive checks fail, traffic goes to a down server. We configure max_fails and fail_timeout, and for critical systems, active health checks via Nginx Plus or external monitoring.
• SSL passthrough for L4. Proxying HTTPS without decryption requires ssl_preread + SNI routing. This non-obvious setting is often overlooked.

How We Do It: Stack and Configuration

We use Nginx Open Source (stable version) or Nginx Plus for enterprise features. Projects follow a GitOps approach: configs stored in Git, deployed via Ansible or Terraform. Below is a working configuration example for a typical online store project.

L7 HTTP Load Balancing

# /etc/nginx/conf.d/upstream.conf
upstream app_rr {
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
    keepalive 64;
}

upstream app_lc {
    least_conn;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080 weight=2;
    server 10.0.1.12:8080;
}

upstream app_ip_hash {
    ip_hash;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080 down;
}

upstream app_random {
    random two least_conn;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

Comparison of Load Balancing Algorithms

L4 vs L7 Load Balancing Comparison

Health Checks and Failover

upstream app_backend {
    server 10.0.1.10:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8080 backup;
    keepalive 64;
    keepalive_requests 1000;
    keepalive_time 1h;
}

server {
    listen 443 ssl http2;
    server_name app.example.com;
    ssl_certificate /etc/letsencrypt/live/app.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/app.example.com/privkey.pem;

    location / {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        proxy_read_timeout 60s;
        proxy_next_upstream error timeout http_500 http_502 http_503;
        proxy_next_upstream_tries 2;
        proxy_next_upstream_timeout 10s;
    }
}

proxy_next_upstream is a powerful but dangerous tool. We enable http_500 only after careful analysis: if the request was POST and partially completed, retrying may duplicate the operation. So for idempotent methods (GET, PUT) — yes, for POST — with caution.

Cookie-based Sticky Sessions

upstream app_sticky {
    hash $cookie_SERVERID consistent;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

L4 TCP Load Balancing via Stream Module

stream {
    log_format basic '$remote_addr [$time_local] ' '$protocol $status $bytes_sent $bytes_received ' '$session_time';
    access_log /var/log/nginx/stream-access.log basic;

    upstream mysql_cluster {
        least_conn;
        server 10.0.1.20:3306;
        server 10.0.1.21:3306;
        server 10.0.1.22:3306 backup;
    }

    server {
        listen 3306;
        proxy_pass mysql_cluster;
        proxy_connect_timeout 5s;
        proxy_timeout 300s;
    }

    upstream https_backends {
        server 10.0.1.10:443;
        server 10.0.1.11:443;
    }

    server {
        listen 443;
        proxy_pass https_backends;
        ssl_preread on;
    }
}

For upstream monitoring, we use stub_status + Prometheus exporter — providing metrics like nginx_up, nginx_connections_active, etc.

upstream app_cluster {
    least_conn;
    server 10.0.1.10:8080 max_fails=5 fail_timeout=60s;
    server 10.0.1.11:8080 max_fails=5 fail_timeout=60s;
    server 10.0.1.12:8080 backup;
    keepalive 128;
}

server {
    listen 80 default_server;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    ssl_certificate /etc/nginx/ssl/cert.pem;
    ssl_certificate_key /etc/nginx/ssl/key.pem;

    location / {
        proxy_pass http://app_cluster;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Our Work Process

1. Analysis — gather architecture, current load metrics, determine algorithm and number of upstream servers.
2. Design — design configuration, agree on health checks, timeouts, retry policies.
3. Implementation — configure Nginx, deploy via GitOps (Ansible/Terraform), test on staging.
4. Testing — load testing, failover verification, monitoring.
5. Deployment — blue-green or canary rollout with zero downtime.

Timeline and Cost

A standard project (up to 5 upstream servers) takes 1 to 3 days. Cost is calculated individually — depends on configuration complexity, need for sticky sessions, and monitoring integration. Order load balancing setup from professionals — leave a request on the website.

What's Included

• Nginx L4/L7 load balancing configuration with chosen algorithm.
• Health checks and failover.
• Cookie-based sticky sessions (if needed).
• Monitoring (stub_status + Prometheus, or Datadog setup).
• Configuration documentation and operational procedures.
• Training your team in basics of support.
• 2 weeks of post-deployment support.

Guarantee and Experience

Our engineers hold NGINX and Kubernetes certifications and have over 50 load balancing projects under their belt. The configuration is backed by an SLO with guaranteed fault tolerance.

Get an expert consultation — contact us to discuss your architecture. We will help choose the right algorithm and set up monitoring.

Source: Load balancing