Rate limiting protects capacity, controls cost and blunts abuse. It also breaks legitimate integrations, punishes users behind shared addresses, and creates support load when configured carelessly. The algorithm choice matters less than what you key the limit on, which is the decision most teams make by default rather than deliberately.
01The key matters more than the algorithm
Limiting by IP address is the default and the worst general choice. Corporate networks, universities, mobile carriers and VPNs put thousands of legitimate users behind one address, so a per-IP limit either throttles them collectively or is set so high it stops constraining anyone. Meanwhile a determined attacker rotates addresses cheaply.
Key on identity wherever you have it: the API key, the account, the authenticated user. That aligns the limit with who is actually consuming resources and makes it meaningful. Reserve IP-based limits for unauthenticated endpoints where you have nothing better, and set them generously as a blunt safety net rather than a precise control.
- Prefer account or API key over IP for anything authenticated.
- Limit per tenant as well as per user - one tenant should not exhaust shared capacity.
- Apply separate, tighter limits to expensive endpoints.
- Never key solely on something the client controls, like a header they set.
02Choosing an algorithm
Fixed windows are simplest and have a well-known flaw: a client can send a full quota at the end of one window and another immediately at the start of the next, producing double the intended rate across the boundary. For most internal uses that is tolerable; for cost control it is not.
Token bucket is the usual right answer. It permits bursts up to the bucket size while enforcing a sustained average rate, which matches how legitimate clients actually behave - mostly idle, occasionally busy. Sliding window log is the most accurate and the most expensive in memory, worth it only where precision genuinely matters.
| Algorithm | Bursts | Cost | Use when |
|---|---|---|---|
| Fixed window | Allows boundary doubling | Very low | Simple internal limits |
| Sliding window counter | Smoothed approximation | Low | Good general default |
| Token bucket | Controlled, configurable | Low | Public APIs; matches real usage |
| Leaky bucket | None - smooths output | Low | Protecting a fixed-capacity downstream |
| Sliding window log | Exact | High memory | Strict billing or compliance limits |
03Tell the client what happened
A bare 429 with no information forces integrators to guess, and they guess badly - usually by retrying immediately, which makes the problem worse. Returning the limit, the remaining allowance, the reset time and a Retry-After header lets a well-behaved client self-regulate.
This turns rate limiting from an adversarial control into a coordination mechanism. Most clients hitting limits are not abusive; they are a legitimate integration written by someone who had no way to know your thresholds. Publishing limits in documentation and returning them in headers eliminates most of the resulting support burden.
04Distributed limiting has a coordination cost
Across multiple instances, per-instance counters mean the effective limit is your configured value multiplied by instance count, and it changes when you autoscale. Shared state in Redis fixes the accuracy and adds a network round trip to every request plus a dependency whose failure needs a defined behaviour.
Decide that failure behaviour explicitly. Failing open means abuse passes through during a Redis outage; failing closed means your API goes down when the limiter does. For most systems failing open is correct - the limiter protects against excess, and an outage of the protection should not become an outage of the service. For billing-critical limits, the opposite may hold.
05What rate limiting does not solve
Rate limits constrain volume from a single key. They do very little against distributed abuse across many accounts, credential stuffing that stays under the threshold per account, scraping that is slow and patient, or a single expensive request that consumes enormous resources on its own.
Those need different controls: cost-based limiting where expensive operations consume more quota than cheap ones, anomaly detection on behaviour patterns rather than counts, proof of work or challenges for unauthenticated abuse, and hard query complexity limits on flexible interfaces like GraphQL. Rate limiting is one layer, and treating it as the whole defence is a common and expensive assumption.
- Weight limits by operation cost, not just request count.
- Add complexity limits to GraphQL or any flexible query interface.
- Detect behavioural anomalies, not only threshold breaches.
- Layer challenges for unauthenticated endpoints under abuse.
Topics
Marcus Hale
Lead Architect · SyncTrix
Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.
Building something like this?
SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.