Rate limiting exists to keep one caller from degrading service for everyone else. A single global limit does not do that - it either sits high enough to be irrelevant or low enough to throttle your largest customer during their busiest hour. Useful limits are per-caller, per-endpoint-cost, and communicated clearly enough that a well-behaved client can adapt.
01Choose the algorithm for the traffic you actually get
Fixed windows are simplest and have an obvious flaw: a client can send a full window's allowance at the end of one window and again at the start of the next, producing double the intended rate at the boundary. Sliding windows fix that at the cost of more state.
Token buckets are usually the best fit for APIs because they permit bursts while bounding the sustained rate, which matches how real integrations behave - mostly idle, occasionally syncing a batch. Leaky buckets smooth output instead, which is what you want in front of a fragile downstream system rather than at your edge.
| Algorithm | Allows bursts | State cost | Good for |
|---|---|---|---|
| Fixed window | At boundaries, badly | Lowest | Rough protection only |
| Sliding window log | No | Highest | Strict fairness, low volume |
| Sliding window counter | Slightly | Low | General purpose |
| Token bucket | Yes, bounded | Low | Most public APIs |
| Leaky bucket | No, smooths | Low | Protecting a fragile downstream |
02Not all requests cost the same
Counting requests treats a primary-key lookup and a report spanning a year of data as equivalent, which is obviously wrong and leads to limits set for the worst case. That punishes every client doing cheap, high-volume work.
Assign a cost to each endpoint and spend from a budget instead. A trivial read costs one, a search costs ten, a bulk export costs a hundred. Clients doing sensible things get far more throughput, and the expensive endpoints get protected properly rather than incidentally.
03Tell the client what is happening
A bare 429 with no context leaves a client guessing, and guessing usually means retrying immediately, which makes everything worse. Return the limit, the remaining budget, the reset time and a Retry-After header, and a competent client will back off correctly without any support conversation.
The headers also let clients self-regulate before hitting the wall. Integrations that can see they are at eighty percent of budget will slow down on their own, which is a considerably better outcome for both sides than a wall of rejections at the top of the hour.
| Header | Meaning | Example |
|---|---|---|
| RateLimit-Limit | Budget for the window | 1000 |
| RateLimit-Remaining | Budget left | 247 |
| RateLimit-Reset | Seconds until refill | 1800 |
| Retry-After | On 429 only, when to retry | 30 |
04Separate the tenant from the endpoint
Two limits working together handle most real situations. A per-tenant budget stops one customer consuming shared capacity. A per-endpoint limit stops any caller hammering the one expensive operation that hurts your database, regardless of how much budget they have left.
Add a third for unauthenticated traffic, keyed on IP and set considerably lower. Most abusive traffic never authenticates, and treating anonymous requests with the same generosity as a paying customer is how you fund someone else's scraping.
05Decide what happens at the limit
Rejecting is not the only option. Queueing briefly and serving late is often better for a batch integration that does not care about latency. Degrading - serving cached or reduced results - is better for a read path. Reserve hard rejection for writes and genuinely expensive operations.
Whichever you choose, exempt your own health checks and internal callers explicitly, and make sure the limiter fails open rather than closed. A rate limiter whose Redis has gone down should not take your API with it, and that is a surprisingly common way to turn a cache outage into a full outage.
Topics
Priya Iyer
Staff Engineer · SyncTrix
Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.
Building something like this?
SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.
Talk to an engineer