Rate limiting in nginx: where the server breaks without limits
A burst of requests rarely arrives evenly. More often it is a single client, a runaway catalog crawl script, or an external bot that opens hundreds of parallel connections and does not release them until the timeout expires.
The server is formally alive: the process is running, the port is listening, but the workers are occupied by a single source.
The worker pool is a shared and finite resource. When all available workers are waiting for a response from the database or an external service on heavy requests, new requests queue for acceptance.
The queue grows faster than it can be drained, and response time for other clients stretches from a fraction of a second to tens of seconds.
Then a cascade kicks in: users do not wait and leave, while automatic retries add load. In the logs it looks the same — connection drops under peak with unchanged code, a growing share of errors and warnings about a shortage of free workers. The cause is not hardware but exhaustion of the worker pool and accumulated waits.
What request rate limiting at the entry point provides
Response time stops growing under peak
A rule on the entry proxy (nginx) works earlier than the application: excess traffic receives a short rejection and does not occupy worker processes.
The queue does not accumulate, so stable response time holds even at peak — during the newsletter hour or campaign launch instead of growing severalfold. Thresholds are set according to the load profile taken from real traffic, not after an incident.
Password brute-force and mass catalog scraping are cut off
A separate rule for the login form counts not only frequency but also the share of failed attempts from a single address — a series of hundreds of errors does not turn into brute-force.
For the catalog, the threshold is softer: cutting off parsers leaves people full access, while a script from a single address gets only a portion of pages per minute. Search engine bots go through a separate list and operate without limits.
One client's failure does not bring down the others
Client isolation is needed where an office or partner export sits behind a single address: without it, one burst occupies all worker processes and brings down the service for everyone.
With separation by address, access key, and session, only the offender hits the limit, while the rest continue to work. This eliminates an entire class of failures — not "the server went down" but "a specific source exhausted its quota."
Rate limiting in nginx: protection levels and their limitations
Limits are placed at several levels, and each covers its own class of overload. The closer the counter is to the client, the cheaper the rejection: a request cut off at the external proxy reaches neither the application nor the database.
Where to place the limit
| Level | When it fits | Limitations |
|---|---|---|
| External proxy before the entry point | Mass bursts, scanners, brute-force on login forms | Only the client address is visible: behind an office network the quota is shared by everyone, and distributed traffic bypasses it |
| Entry nginx: rate limit and concurrent connection limit | A workable option for a website and API | The zone and accounting key must be coordinated; on several nodes without shared storage the counters are independent, and the total limit is diluted |
| Application or gateway | Limits by account, token, role, or specific route | Counted after request parsing — more expensive; rules diverge from the load balancer config |
The accounting key is the main choice: client address, address taking into account the real source behind the proxy, session, or token. The more precise the key, the fewer false positives on shared addresses.
What limits do not solve
Rate limiting is about legitimate load and rough bursts, not about protection as a whole. It will not stop slow low-intensity brute-force from many addresses and will not save you from an expensive request that fits within the quota but loads the database. It also will not distinguish a bot from a client if the key is only the address.
Therefore limits work in conjunction with caching, timeouts, request body size limits, and log analysis: each covers its own area, and individually none of them provides resilience.
Stages of implementing limits: from audit to procedures
We start with an audit of incoming traffic: we analyze reverse proxy logs, find the routes and sources with the main volume, peak intervals, and the share of service requests.
From this we build a load profile — the rate the service sustains with headroom, and the points where degradation begins.
- Incoming traffic audit — logs, routes, sources, peak intervals.
- Load profile — what rate the service can handle and what rate leads to increased response time.
- Zone and threshold scheme — where to place the limit: at cluster entry, on a specific service, or on an expensive route, with different thresholds for authenticated and anonymous clients.
- Rule configuration — the limits themselves, response codes, headers with remaining quota, behavior when exceeded.
- Testing on a staging environment — we run the rules on a copy of the environment to see exactly who the limit restricts.
- Load run and handover of procedures — we hold the peak, compare metrics with the plan, and give the team a document with thresholds and the procedure for revising them.
How testing under peak works
The run repeats the load profile from the audit, including short bursts from a single source. We verify that the limit triggers before response time grows for others, and that legitimate traffic does not fall under the threshold — otherwise the restriction hits your own users.
What is needed on your side
Access to service logs and metrics, a window for testing on a copy of the environment, and a contact for the on-call engineer who will confirm the thresholds. The rest is our work: rules, the run, procedures, and analysis of results.
What remains with you: configurations, zone schemes, and procedures
We hand over not a "configured server" but a working package: nginx request rate limiting configurations, a zone and threshold scheme, a list of trusted addresses, on-call instructions, and an incident response procedure.
Everything is stored in the customer's repository, changes go through review, and are rolled out via the usual delivery process.
Contents of the document package
- Annotated configurations: which key the limit is counted by (address, token, cookie) and which zones are declared.
- Zone and threshold scheme — a table "zone → limit → burst → response code", so values change deliberately rather than randomly.
- List of trusted addresses and subnets: allowlists for monitoring, payment callbacks, integrations, and office networks.
- On-call instructions: how to distinguish a legitimate burst from an attack and how to enable a gentler mode.
- Response procedure: who removes the limit, who is notified, when the threshold is raised, and when it is cut more strictly.
- Load run scenarios — they confirm that legitimate peaks pass while rough brute-force gets rejected.
- Limit change log: date, reason, who approved it — so it is clear where the current value came from.
How to maintain this without us
Thresholds live in a single file and can be changed without rewriting the logic. The instructions and procedure are updated together with the scheme, so a new engineer gets up to speed with the system in an evening, not a week.
Case study: how an online store got through a sale peak
A peak sale is the day when an online store's infrastructure is truly tested: traffic grows severalfold within minutes, and everything that failed is immediately visible.
Below are figures from monitoring exports before and after implementing request rate limiting on the entry node on nginx.
What it was like before
At the moment the campaign started, the entry node accepted the entire flow indiscriminately: the share of successful requests fell to 62%, and response time at peak reached 9 seconds — the shopper closed the catalog before receiving a response.
Part of the traffic consisted of bots: they crawled the catalog and filled shopping carts, occupying all application worker processes.
Server errors became 40 times more numerous than on a normal day, and this was dealt with only after the campaign, using logs.
What it became after
Limits were separated by level: a strict threshold per source address for search and checkout, a soft one — with queueing and delay — for the catalog and images, and a separate profile for search engine bots.
The share of successful requests at peak rose to 99.4%, response time stayed within 1.2 seconds, and the number of server errors dropped by an order of magnitude — with the same shopper traffic.
Triggers are visible in the monitoring export by rule name, so a false block is removed with a single edit, without restarting the service.
Thresholds were selected based on the real traffic profile and revised after each load rehearsal; the configuration is versioned together with the rest of the infrastructure, and the result is reproduced at the next sale.
How a global limit differs from a limit per specific address
A global limit sets a single ceiling for the entire service: the accounting key is a constant, so all requests fall into a common counter. This mode catches a uniform excess flow and incidents when traffic grew due to a client failure or an integration error.
But it misses skew: if one address takes nine tenths of the quota, the others receive rejections even though the total threshold is formally not exceeded.
A limit per client address counts each source separately, and one source exceeding it does not hit the others. Here frequent polling, automated data collection, and a forgotten test environment with repeated requests are visible.
But distributed traffic from hundreds of addresses passes through such a mode: each stays within its own quota, while the total load on the service remains destructive.
The choice of limiting key depends on what we are protecting: for an external interface, the address is used; for internal calls, the client identifier or route; for expensive operations, a combination of attributes.
Alongside this, request accounting modes are configured: average rate separately and allowed burst separately. Otherwise, a legitimate application launch that immediately sends a series of requests is discarded just like an attack.
In practice, both modes are kept together: the global one acts as a fuse for the entire service, while the per-address limit protects against a specific source.
What the limit lets through also depends on what counts as a single request: if requests to cacheable routes are not counted, the threshold can be lowered without risk to live users.
Technical doubts: client key, bursts, and allowlists
Five doubts that come up before request rate limiting — each has a proven solution.
What should be the accounting key: address, token, or route?
The base key is the client address, but behind a proxy it is substituted, so we take the real address from the forwarding header.
For the API, the key is built from the token; for heavy routes, from the pair "address + route": a burst in search does not bring down the entry point.
How do you distinguish a false positive from real load?
First, a monitoring mode: the rules count requests and write to the log but do not reject. Within a week you can see what share of clients receives a rejection and on which routes. Most often these are pages with resource loading — we adjust the key or burst in a targeted way.
Will ordinary visitors start getting rejections?
Calm browsing produces single requests per second; we set the limit an order of magnitude higher. The burst allows a short series, so a page with a dozen files loads completely. Automated systems receive rejections, not a human.
Can it be enabled on a live service without downtime?
The configuration is applied by reloading; already open connections are not dropped. We introduce the rule first on one route or part of the nodes, look at the rejection share and response time, then roll it out. Rollback is removing the rule with the same reload.
What about trusted sources?
We put monitoring, health checks, and partner integrations into separate zones with high thresholds rather than fully excluding them. A trusted address does not become a hole: if there is an anomaly, the limit still triggers. The list lives next to the rules and goes through review on every change.
Let's discuss your load and come back with an estimate and plan
A discussion of a request rate limiting task goes faster when there are facts about the load, not just a description of symptoms.
We review them together with the system topology and agree on what level to place the limits at: at the entry before the proxy, inside it by route, or in the application itself.
It is useful to prepare for the conversation:
- peak profile: daily and weekly maximums, bursts from mailings, share of anonymous and service traffic;
- system diagram: where the load balancer and proxy are, how many application nodes, whether there is a cache and queue;
- list of public entry points and methods that cannot be limited in the same way;
- incident history: when and on what it went down, which requests looked like brute-force or mass scraping;
- expected growth: planned launches, seasonality, new integrations, and mobile clients;
- access to the staging environment to test rules before deploying to production.
This review forms the basis for an estimate of the scope of work and a stage-by-stage implementation plan. We start with a monitoring mode: the rules count exceedances but do not block, and it is visible who is affected.
Then we enable rejection with clear response codes and headers, and set separate thresholds for critical routes. The next steps are a pilot on one service, measuring the share of rejected requests and response time, and rolling out to the remaining nodes.