Setting Up Nginx Reverse Proxy: Caching, Load Balancing, SSL

Typical scenario: 1000 RPS and backend drowning

A heavily loaded application (1000 RPS) quickly hits database limits or CPU bottlenecks due to repeated computations. Backends on Node.js or Python struggle with each request, especially if there are repetitive SQL queries or heavy templates. Response times may hit 500ms, and P99 skyrockets. A properly configured reverse proxy with caching solves multiple issues: SSL termination, static file serving, response caching, and load balancing. We often see projects where the backend directly serves every request, and nginx is only used as an SSL terminator — a direct path to overload.

Our engineers with 10+ years of experience have tuned hundreds of such setups. Result: response time drops by 70% for cacheable requests, backend load reduces multiple times. Comparison: nginx cache serves in microseconds vs. Redis/Memcached in milliseconds (microseconds vs. milliseconds). The cost of such a setup is from a few hours of work, while server resource savings can reach thousands of dollars per month. For example, after implementing caching, you can reduce the number of servers from 5 to 2, saving $3000 monthly.

Basic reverse proxy configuration

upstream backend {
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    keepalive 32;
}

server {
    listen 443 ssl http2;
    server_name example.com;
    ssl_certificate /etc/ssl/certs/example.com.fullchain.pem;
    ssl_certificate_key /etc/ssl/private/example.com.key;

    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        proxy_buffering on;
        proxy_buffer_size 4k;
        proxy_buffers 8 4k;
    }
}

keepalive 32 and proxy_http_version 1.1 with Connection "" are mandatory for persistent connections. Without them, each request opens a new TCP connection, leading to thousands of TIME_WAIT sockets on a loaded server. TLS handshake takes ~2ms, while serving from cache is under 0.1ms, so persistent connections are critical.

Caching backend responses

http {
    proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=backend_cache:10m max_size=100m inactive=60m use_temp_path=off;
    server {
        location /api/ {
            proxy_cache backend_cache;
            proxy_cache_key "$scheme$host$request_uri";
            proxy_cache_valid 200 302 10m;
            proxy_cache_valid 404 1m;
            proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
            proxy_cache_lock on;
            add_header X-Cache-Status $upstream_cache_status;
            proxy_pass http://backend;
        }
    }
}

proxy_cache_use_stale is one of the most important parameters. On a backend error, nginx serves stale cache instead of a 502. updating protects against the Thundering Herd: while one request refreshes the cache, others get the old value. According to the official nginx documentation, this is key for high-load systems. Add the X-Cache-Status header to see HIT, MISS, STALE, or EXPIRED.

How to verify caching is working?

Add add_header X-Cache-Status $upstream_cache_status; and look for HIT, MISS, STALE, etc. Also observe the drop in response time and backend load. P99 latency should drop from 500ms to 50ms. For monitoring, use Prometheus metrics from nginx stub_status.

Which headers affect caching?

The backend controls caching via HTTP headers. If the backend returns Cache-Control: no-cache, nginx does not cache the response. For forced invalidation, use the purge API:

location ~ /purge(/.*) { allow 127.0.0.1; deny all; proxy_cache_purge backend_cache "$scheme$host$1"; } 

Use curl: curl -X PURGE http://localhost/purge/api/products/123.

Caching static files and serving directly

Static files (jpg, css, js, etc.) can be cached or served directly from nginx, bypassing the backend. Direct serving reduces backend CPU load by 30–50%. Example for hashed static files:

location /static/ {
    alias /var/www/app/public/static/;
    expires 1y;
    add_header Cache-Control "public, immutable";
    access_log off;
    gzip_static on;
}

immutable tells the browser: the file will never change, do not make conditional requests.

Why use nginx for caching instead of backend cache?

Process of work

1. Audit: analyze current configuration, logs, metrics via stub_status or Prometheus. Identify bottlenecks where caching brings most impact.
2. Design: choose cache size, zones, invalidation rules. Calculate keys_zone and max_size based on average response size (e.g., 10M zone stores ~80,000 keys).
3. Implementation: deploy reverse proxy with caching, load balancing, SSL. Use Ansible or Terraform for repeatable deployment.
4. Tuning: adjust timeouts, buffers, keepalive according to backend characteristics. For example, for a backend with long responses, increase proxy_read_timeout to 120s.
5. Testing: load test with wrk or k6. Target: P99 latency < 100ms for cacheable requests.

Pricing is determined individually after auditing your infrastructure and load patterns. Contact us — we will prepare a proposal tailored to your needs.

What's included

• Full configuration documentation and operation manual.
• Access to a repository with Ansible roles or Terraform modules.
• Team training (2 hours): how to clear cache, read logs, tune parameters.
• 30-day warranty on the setup.

Typical problem: caching authenticated requests

By default, nginx caches responses without considering the Authorization header. One user may receive a response intended for another. Solution: add proxy_cache_bypass $http_authorization; or include Authorization in the cache key: proxy_cache_key "$scheme$host$request_uri$http_authorization";. Ensure this does not compromise security.

How to properly tune proxy_cache_use_stale?

proxy_cache_use_stale defines when to serve stale cache when backend is problematic. The value updating is key for high-load systems. It protects against the Thundering Herd effect: nginx allows one request to update the cache, while others get the stale value without hitting the backend. We recommend always using updating in combination with error timeout.

Order professional setup — our engineers with 10+ years of experience will help deploy a production-ready reverse proxy for your project. Get a consultation today: we will analyze your infrastructure and propose the optimal configuration.