Configuring a Caching Forward Proxy with Access Control
Consider this: when dozens of servers in a VPC simultaneously fetch packages from external repositories, traffic saturates the channel, and each repeated request incurs extra egress costs. Our team of certified engineers has 7+ years of experience in launching production proxies and over 40 successful projects implementing Squid. We configure the proxy as a caching forward proxy that solves this problem: it serves data from a local cache, reducing external traffic by 60-80%. For a typical 50-server cluster, this can save over $2,000 per month in egress fees. Additionally, the server provides full control—domain whitelists, port blocking, and HTTPS inspection via SSL Bump. The solution works for small clusters and enterprise infrastructures with thousands of hosts. We guarantee a hit rate of at least 70% within two weeks. Request an audit of your current infrastructure—we will determine potential traffic savings.
Problems We Solve
External Channel Congestion
Each server in a private network independently downloads the same packages (apt, pip, npm, Docker images). Without a proxy, this amounts to tens of gigabytes of duplicate traffic monthly. Squid with ufs cache on SSD or rock on RAM-disk serves files from local storage, reducing external link load by 5–10 times.
Lack of Access Control
Servers in an isolated subnet should not have direct internet access. The proxy via ACL (dstdomain, url_regex) restricts allowed resources. We use whitelists, whitelist files, and integration with Active Directory via ntlm_auth for granular authentication.
HTTPS Traffic Inspection
Without SSL Bump, you cannot see what is transmitted inside HTTPS connections. Our proxy with bump mode decrypts traffic, allowing caching of secured resources and blocking malicious content. However, we apply bump only when justified—for critical repositories or according to security policy.
How We Configure the Proxy: A Case Study
From our practice: a client with 50 virtual machines in Yandex Cloud, all updating from external repositories. Stack: Ubuntu 22.04, Squid 6.6, Terraform for deployment. Our engineer:
- Deployed a Squid instance with a 200 GB SSD disk for cache (3.5 GHz CPU, 16 GB RAM, 1 Gbps network).
- Configured squid.conf with ACLs for private subnets and a whitelist file of 30 domains (repositories, Docker Hub, GitLab).
- Enabled cache_dir ufs /var/spool/squid 50000 16 256 (50 GB cache) and maximum_object_size 200 MB.
- Connected squidclient mgr:info for hit rate monitoring.
- Set up logging to access.log and rotation with logrotate.
- Distributed scripts for http_proxy configuration on clients (apt, Docker, systemd).
Result: hit rate 73% in the second week, external traffic reduced from 1.5 TB to 400 GB per month. Savings—over 50% on egress costs.
Why Our Proxy Over a Custom Nginx Cache?
| Criteria | Squid | Nginx as Caching Proxy |
|---|---|---|
| CONNECT support | Native for forward proxy | Requires ngx_http_proxy_connect module |
| SSL Bump | Built-in, flexible | Not available in official build |
| Domain-based ACL | dstdomain, url_regex | Via map + regex |
| HTTPS caching | Full with bump | Pass-through only |
| Monitoring | squidclient, SNMP, Prometheus | stub_status, Prometheus |
| Performance | 10–30 thousand requests/sec per core | Up to 100 thousand requests/sec per core (without SSL Bump) |
Conclusion: Squid wins in access control and traffic inspection; Nginx wins in pure static caching at high RPS. We use Squid when a full-featured forward proxy is required.
Optimizing the Server for High Loads
- Use cache_dir rock on SSD—faster than ufs, less fragmentation.
- Increase cache_mem to 30-50% of RAM, but not exceeding maximum_object_size_in_memory (usually 1-4 MB).
- Enable memory_pools_limit and prefix_write_cache to reduce fragmentation.
- Configure workers (multithreading) for Squid 6+: workers 4 and cpu_affinity_map.
- Use collapse_forwarding to merge concurrent requests for the same URL.
- Supports up to 10,000 concurrent connections with proper tuning.
For more configuration parameters, see the Squid configuration manual.
Typical Configuration Mistakes
- Forgetting to initialize the cache (squid -z)—the server won't start.
- Not specifying http_access allow localhost—the instance cannot connect to itself.
- Too small cache_dir—cache fills quickly, hit rate drops.
- SSL Bump without installing CA on clients—browsers complain about the certificate.
Work Process
| Stage | Duration | Result |
|---|---|---|
| Infrastructure audit | 1–2 days | Report on traffic, load, requirements |
| Configuration design | 1 day | Network diagram, config files, access policies |
| Turnkey deployment and setup | 2–4 days | Working proxy with ACL, cache, SSL Bump (if needed) |
| Integration with client servers | 1–2 days | http_proxy, apt, Docker, CI/CD configuration |
| Monitoring and optimization | 3–5 days | Prometheus + Grafana setup for Squid, cache tuning |
What's Included in the Work
- Documentation: configuration files, network diagram, operation instructions.
- Monitoring system setup (Prometheus + Grafana for Squid).
- Administrator training: working with ACL, log analysis, cache management.
- Technical support at startup: 7 days after going live.
- Basic setup starts at $1,500; pricing adjusted after audit.
Cost is calculated individually after the audit. It includes configuration, testing, documentation, and startup support.
Squid is a mature, reliable solution for caching and access control that we implement in infrastructures of any scale. Traffic savings, transparent monitoring, and flexible security policies pay off in the first months. For more information, see Wikipedia. If you want to assess the potential for your project, contact us—we'll send a sample configuration for your scenario. Reach out for a preliminary estimate of your project.