Varnish Cache Setup for HTTP Acceleration

Varnish Cache Setup for HTTP Acceleration

Note: when the backend responds in 300-500 ms but the user expects a page in 50 ms, a regular nginx cache doesn't provide the needed speedup. Varnish Cache solves this at the kernel level: it stores responses in RAM and processes tens of thousands of requests per second on standard hardware. For example, at 10,000 RPS load, Varnish can deliver cached responses in 1-2 ms, while nginx with cache takes 5-10 ms.

We configure Varnish end-to-end for projects with any architecture — from media sites to high-load APIs. In our practice, there are cases with hit rates above 95% and backend load reduction of 80%. One project — an e-commerce store with 500,000 products — after implementing Varnish reduced product page load time from 1 second to 30 ms, and monthly cloud resource costs dropped from $3000 to $500.

Clients receive not only the configuration but also operational documentation and team training. In this article, we'll cover typical problems: improper cookie caching, ignoring grace mode, and lack of purge API.

Architecture: Varnish in front of nginx

Client → Varnish :80 → nginx :8080 → Backend :3000 

Varnish cannot terminate SSL, so in production, nginx or HAProxy is placed in front for HTTPS termination:

Client → nginx :443 (SSL termination) → Varnish :80 → nginx :8080 → Backend 

Or use hitch — a TLS proxy specifically for pairing with Varnish.

Why Varnish instead of nginx cache?

Nginx is an excellent web server and reverse proxy, but its caching system is limited. Varnish is designed exclusively for caching: flexible VCL language, precise TTL control, grace and stale modes, built-in ban mechanism, and detailed statistics. In projects with hit rates below 90% and cache size over 1 GB, Varnish provides a 2-5x speed advantage over nginx.

VCL: Configuration Language

VCL (Varnish Configuration Language) is a C-like language, compiled to native code on load. This provides speed but requires understanding Varnish's state machine: vcl_recv → vcl_hash → vcl_hit/vcl_miss → vcl_fetch → vcl_deliver.

Basic configuration /etc/varnish/default.vcl

vcl 4.1;
import std;

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 5s;
    .first_byte_timeout = 60s;
    .between_bytes_timeout = 30s;
}

sub vcl_recv {
    # Remove port from Host for correct cache key
    set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");

    # Do not cache POST, PUT, DELETE
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # Do not cache authenticated requests
    if (req.http.Authorization || req.http.Cookie ~ "session") {
        return (pass);
    }

    # Remove marketing parameters from URL for better hit rate
    set req.url = regsuball(req.url, "([?&])(utm_[a-z]+|fbclid|gclid)=[^&]+", "\1");
    set req.url = regsub(req.url, "[?&]$", "");

    return (hash);
}

sub vcl_hash {
    hash_data(req.url);
    hash_data(req.http.Host);

    # Separate cache for mobile/desktop if needed
    if (req.http.X-Device-Type) {
        hash_data(req.http.X-Device-Type);
    }

    return (lookup);
}

sub vcl_backend_response {
    # Cache 404 for 1 minute to avoid hitting backend constantly
    if (beresp.status == 404) {
        set beresp.ttl = 1m;
        set beresp.grace = 30s;
    }

    # Grace period: serve stale cache while refreshing
    set beresp.grace = 30s;

    # Do not cache if backend says no-cache
    if (beresp.http.Cache-Control ~ "no-cache|no-store|private") {
        set beresp.uncacheable = true;
        set beresp.ttl = 120s;
        return (deliver);
    }

    # Default caching for 10 minutes
    if (!beresp.http.Cache-Control) {
        set beresp.ttl = 10m;
    }

    return (deliver);
}

sub vcl_deliver {
    # Add debug header
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS";
    }

    # Remove internal Varnish headers
    unset resp.http.X-Varnish;
    unset resp.http.Via;

    return (deliver);
}

if (req.url ~ "wp-admin|wp-login|ajax") { return (pass); } 

Case study: e-commerce store with 500,000 products

The client complained about slow product page loading (over 1 second). The cause was nginx cache not caching responses due to session cookies. We implemented Varnish with a rule to ignore session cookies for guest users and configured a grace period of 30 seconds. Hit rate increased from 45% to 93%, response time dropped to 30 ms.

How to configure Varnish for maximum hit rate?

To achieve hit rate 90%+, follow these rules:

• Clean URL from marketing parameters (utm, fbclid).
• Ignore session cookies for unauthenticated users.
• Use granular TTLs: hours for static, minutes for content.
• Enable grace mode so the backend doesn't block response during refresh.

How to configure Varnish: step-by-step guide

1. Install Varnish from the repository (varnish-cache.org).
2. Configure the backend in default.vcl (host and port of your web server).
3. Define vcl_recv rules: exclude POST, authenticated requests, admin URLs.
4. Configure vcl_backend_response: TTL, grace, no-cache handling.
5. Add PURGE support: ACL and rule in vcl_recv.
6. Enable monitoring: varnishstat and integration with Prometheus.

Purge: forced cache invalidation

acl purge_acl {
    "localhost";
    "127.0.0.1";
    "10.0.0.0"/8;
}

sub vcl_recv {
    if (req.method == "PURGE") {
        if (!client.ip ~ purge_acl) {
            return (synth(405, "Not Allowed"));
        }
        return (purge);
    }
}

Invalidate a specific URL: curl -X PURGE http://localhost/api/products/123. For mass invalidation by pattern, use varnishadm ban req.url ~ "^/api/products".

Monitoring

# Real-time statistics
varnishstat

# View request logs
varnishlog -i ReqURL,RespStatus,ObjTTL

# Hit rate over last 10 seconds
varnishstat -f MAIN.cache_hit,MAIN.cache_miss -1

Target cache hit rate for media sites and e-commerce is 85%+. If below 60%, either there is too much personalized content, or cookies are blocking caching too aggressively. After implementing Varnish, we also reduced CDN costs: in one project, savings amounted to $1500 monthly.

Process and timeline

The cost is calculated individually after an audit — contact us to evaluate your project. Get a consultation from an engineer on web application acceleration.

What's included

• VCL configuration tailored to your project
• Purge API setup (ban mechanism by URL and patterns)
• SSL integration (nginx or hitch)
• Monitoring and alerts (Prometheus + Grafana dashboard)
• Operations documentation and cache invalidation procedure
• Team training (1-2 hours)
• Configuration warranty — 30 days after commissioning

Our experience and results

We have been working with Varnish for over 5 years, configuring cache for projects with loads up to 50,000 RPS. Typical results: hit rate 92-98%, backend load reduction of 80%, average response time decrease from 200-400 ms to 5-20 ms. All engineers are certified in Varnish and Kubernetes. Read more about VCL in the Varnish Book.

Typical mistakes in Varnish configuration

• Caching all URLs: ignoring POST, authenticated requests, admin panels. Correct: only GET/HEAD without session.
• Too large TTL: for dynamic sites (e-commerce) optimal is 1-10 minutes.
• Missing purge API: without a ban mechanism, cache invalidation becomes a manual operation.
• Ignoring grace: when backend fails, Varnish should serve stale cache, otherwise errors occur.

If your project requires acceleration — contact us. We will evaluate the architecture and propose a solution.