Rate Limiting Strategies for APIs: Token Bucket vs Sliding Window Trade-offs
A website can receive thousands of requests without being under attack, while a much smaller stream of requests can put an important service at risk. Rate limiting helps applications control how frequently clients can access a resource. The challenge is choosing rules that reduce abuse without unnecessarily blocking legitimate visitors.
In this article
1. What rate limiting is designed to do
Rate limiting places a boundary on how frequently a client can perform an operation. An API might allow a certain number of requests per minute, while an authentication endpoint might limit repeated sign-in failures. A public search service could use a different policy from a password-reset form because the cost and risk of each operation are different.
The rule normally combines a limit, a measurement period or refill rate, and a way to identify the client being limited. Depending on the application, that identifier could be an authenticated account, an API key, an IP address, or a combination of signals.
Each choice has consequences. An IP address is easy to obtain, but several people may share one address through a corporate network, mobile carrier, or public Wi-Fi. An account identifier is useful for authentication limits, but attackers can distribute attempts across many accounts. A well-designed policy therefore considers what the endpoint does and how the client can be identified reliably.
Four common algorithms are fixed window, sliding window, token bucket, and leaky bucket. They all control request volume, but they differ in how they track time, handle bursts, and use memory.
2. Fixed-window counters
A fixed-window limiter divides time into blocks and counts requests inside each block. For example, an application might permit 60 requests per client during each one-minute interval. Once the counter reaches 60, additional requests are rejected until the next window begins.
This method is straightforward to implement. The server generally needs a counter and a window identifier, which makes fixed windows attractive for simple APIs and systems where low overhead matters.
The weakness is the boundary between consecutive windows. Imagine that a client sends 60 requests just before one window ends, then sends another 60 immediately after the next window starts. Both groups fit their respective counters, even though 120 requests may arrive within a very short period.
Example: a limit of 60 requests per minute does not guarantee that no more than 60 requests can arrive during every possible 60-second interval. With fixed windows, a burst near the reset boundary can exceed that amount.
This does not make fixed-window limiting useless. The boundary effect may be acceptable for low-risk operations, analytics endpoints, or approximate usage quotas. It becomes more important when a brief burst can overwhelm a service or create an unacceptable number of authentication attempts.
3. Sliding-window algorithms
A sliding window evaluates recent activity relative to the current time rather than relying exclusively on a fixed calendar boundary. This gives the limiter a more consistent view of how much traffic has arrived during the preceding interval.
Sliding-window log
A sliding-window log records request timestamps. When a new request arrives, the limiter removes timestamps outside the lookback period and counts the remaining entries. If the count has reached the limit, the new request is rejected.
This approach can enforce a precise rolling-window limit, provided the implementation handles concurrent requests correctly. Its main drawback is storage: every recorded request consumes state until it falls outside the window.
For a high-volume service, storing and cleaning up many timestamps can be considerably more expensive than maintaining a small counter. The approach may still be appropriate when precise rolling-window enforcement is worth that cost.
Sliding-window counter
A sliding-window counter uses counts from adjacent fixed windows to estimate recent traffic. It weights the previous window's count according to how much of that window overlaps the current rolling interval, then combines it with the current window's count.
This reduces the boundary effect without storing every timestamp. It is an approximation rather than an exact record of every request in the rolling period, but it can offer a useful balance between accuracy and storage overhead.
When a sliding window is useful
Consider a public API that allows 100 requests per minute. A sliding-window policy helps prevent a client from exploiting a minute boundary to send two full quotas almost back-to-back. A log provides precise counting, while a counter-based version reduces the amount of state that must be stored.
4. Token bucket
Token bucket controls traffic by tracking a supply of tokens. A bucket has a maximum capacity, and tokens are replenished at a configured rate. Each accepted request consumes a token. When the bucket is empty, the limiter rejects or delays additional requests until more tokens become available.
Suppose a bucket holds 20 tokens and refills at one token per second. After a sufficiently long idle period, a client can make up to 20 requests in a short burst. Once those tokens are used, the client must wait for the bucket to refill before continuing at the sustained rate.
This is useful for APIs where legitimate bursts are normal. A mobile application may make several calls after reconnecting to the network, or a dashboard may request multiple resources when a user opens it. Token bucket can accommodate those bursts while still limiting the long-term request rate.
Capacity and refill rate must be configured together. A high refill rate permits sustained traffic, while a large capacity permits a larger burst. Increasing the capacity does not increase the long-term refill rate, but it does allow more requests to pass at once.
Token bucket can also be used on sensitive endpoints. It is not automatically a poor choice for login protection. The important questions are whether its burst allowance is acceptable, how it is combined with other limits, and whether repeated failures trigger additional controls.
Remember: token bucket controls the rate over time while intentionally permitting bursts. It does not, by itself, identify malicious clients or determine whether a request is legitimate. Those decisions require the surrounding application policy.
5. Leaky bucket
Leaky bucket is another way to regulate traffic, but its behavior depends on the implementation. In a common queue-based version, incoming requests enter a queue and leave at a controlled rate. When the queue is full, additional requests are dropped or rejected.
This can smooth a burst of incoming traffic before it reaches a downstream service. For example, a system that processes jobs at a predictable pace may queue excess work instead of allowing a sudden spike to overwhelm its workers.
There is also a counter-based form of leaky-bucket rate limiting that tracks a level which decreases over time. As new requests arrive, that level increases; requests are rejected when it exceeds a configured threshold. This version does not necessarily queue work for later processing.
The distinction matters because queuing and rejecting requests produce different user experiences. A queue can smooth traffic but may increase latency and consume memory. Immediate rejection protects capacity but requires clients to retry later if appropriate.
Token bucket and leaky bucket are related ideas, but they should not be treated as interchangeable. Token bucket allows accumulated capacity to be spent on a burst. A queue-based leaky bucket instead controls the rate at which queued work is released.
6. Comparing the four approaches
The following comparison highlights typical behavior. Exact storage requirements and burst characteristics depend on how each algorithm is implemented.
| Algorithm | Burst behavior | State required | Common use |
|---|---|---|---|
| Fixed window | Can allow a large burst across a window boundary. | Usually a counter and window expiry. | Simple quotas and approximate traffic limits. |
| Sliding-window log | Enforces a rolling limit when implemented correctly. | Request timestamps within the window. | Use cases needing precise rolling-window counts. |
| Sliding-window counter | Reduces the fixed-window boundary effect. | A small number of counters and window information. | APIs needing a balance of accuracy and efficiency. |
| Token bucket | Permits bursts up to the configured capacity. | Token balance and timing information. | APIs that need to tolerate occasional bursts. |
| Leaky bucket | Can smooth traffic through a queue or enforce a threshold. | Queue state or a changing level. | Traffic shaping and controlled work processing. |
There is no universal winner. A precise sliding-window log may be unnecessary for a low-risk endpoint, while a fixed-window counter may be too permissive for a strict rolling quota. Token bucket is often a practical choice for APIs that need burst tolerance, and queue-based leaky bucket can help when downstream processing must remain steady.
7. Choosing limits for login endpoints
Authentication endpoints need a policy that accounts for both abuse and legitimate mistakes. A user may mistype a password, forget which email address they registered, or sign in through several devices. At the same time, an attacker may automate repeated guesses or distribute attempts across a large list of accounts.
One limit based only on IP address is rarely enough. Many legitimate users can share a public IP address, while a distributed attack may come from many different addresses. A more resilient design can combine several controls:
- Per-account limits: monitor repeated failures against an individual account without exposing whether the account exists.
- Per-IP or network limits: restrict unusually high traffic from one source while accounting for shared networks and trusted proxy configuration.
- Global limits: protect the authentication service itself from a sudden rise in total request volume.
- Progressive controls: introduce additional verification, temporary delays, or other safeguards when suspicious patterns appear.
- Monitoring and alerts: identify distributed attempts and repeated failures that individual counters may not reveal.
Token bucket can be part of this design if its capacity and refill rate are appropriately conservative. A sliding-window counter may be easier to reason about when the requirement is a maximum number of attempts over a rolling interval. The choice should be tested against legitimate login patterns and the threat model rather than made from the algorithm's name alone.
Rate limiting should not be the only defense against password attacks. Strong password storage, multifactor authentication, breached-password checks where appropriate, secure session management, and monitoring all address different parts of the problem.
Avoid permanent account lockouts as a default response. An attacker may deliberately trigger a lockout against someone else's account. Prefer carefully designed limits and recovery procedures that reduce abuse without making it easy to deny legitimate users access.
8. Rate limiting across multiple servers
A limiter that stores counters only in a server's local memory may behave correctly during development and fail when the application is deployed across several servers. Each server sees only the requests that reach it, so a client may receive a separate quota from every server.
For example, imagine an application with four servers, each independently permitting 100 requests per minute for the same client. If requests are distributed evenly and each server maintains its own counter, the client could potentially make close to 400 requests during that period rather than the intended 100.
A shared data store can coordinate counters across servers. Redis is one common option because it supports fast operations and mechanisms for executing updates atomically. Other architectures may use a gateway, a database, or a managed rate-limiting service.
Shared state introduces its own design questions. The counter update must be safe under concurrent requests, keys need appropriate expiry, and the application must decide what happens when the shared store is unavailable. Depending on the endpoint, allowing requests through during a failure may be preferable to blocking every user, or the security requirement may demand a stricter fallback.
Network latency and data-store load also matter. A rate limiter should not become the bottleneck it was designed to prevent. The algorithm's state requirements, request volume, consistency needs, and deployment topology should all influence the implementation.
9. Implementation mistakes to avoid
Even a well-chosen algorithm can fail when its surrounding logic is incorrect. Before deploying a rate limiter, check the following implementation details.
- Trusting arbitrary client-supplied IP headers. Headers such as
X-Forwarded-Forshould only be trusted when they are set or sanitized by a known proxy. Otherwise, clients may be able to manipulate the identifier used for limiting. - Using only one identifier. IP-based limits, account-based limits, and global service limits protect against different patterns. Combining them can provide more coverage than relying on one counter.
- Ignoring concurrency. Separate read and increment operations can race when multiple requests arrive at once. Use atomic operations or transactions appropriate to the storage system.
- Forgetting expiration and cleanup. Old counters and request records should not accumulate indefinitely when they are no longer needed.
- Choosing limits without traffic data. Observe normal usage and test expected bursts before deciding how aggressively to reject requests.
- Failing to communicate retries. When a server returns HTTP 429, the response can include a useful explanation and, where appropriate, a
Retry-Afterheader. - Assuming a 429 response stops all abuse. It communicates that a limit has been exceeded; it does not prevent an attacker from switching accounts, addresses, or endpoints.
Handling HTTP 429 responses
HTTP status code 429, defined in RFC 6585, means that the client has sent too many requests in a given period. An application can use it when a rate limit is exceeded. A Retry-After header may tell a well-behaved client how long to wait before trying again.
The delay should reflect the actual policy. If the limiter can calculate when the client is eligible to retry, returning that information helps legitimate clients avoid repeated requests that will simply be rejected again.
For further details, see the RFC 6585 definition of HTTP 429. For general application-security guidance, the OWASP guidance on blocking brute-force attacks discusses additional protections for authentication systems.
Conclusion
Rate limiting is more than choosing a maximum number of requests. The algorithm determines how that limit behaves around time boundaries, whether legitimate bursts are accepted, how much state must be stored, and how well the policy works when traffic reaches multiple servers.
Fixed windows are simple but can permit bursts around reset points. Sliding windows provide a more consistent view of recent activity, with different storage costs depending on the implementation. Token bucket is useful when bursts are acceptable, while leaky-bucket designs can help regulate the rate at which work reaches downstream systems.
The best implementation starts with the endpoint's actual requirements. A login form, a public data API, and a background job queue do not necessarily need the same policy. Combine an appropriate algorithm with reliable client identification, concurrency-safe state, sensible retry behavior, and monitoring. That is what turns a counter into an effective part of an application's reliability and security controls.
Key Takeaways
- Fixed-window counters are inexpensive but can allow bursts across adjacent windows.
- Sliding-window logs provide precise rolling counts, while sliding-window counters use less state.
- Token bucket allows controlled bursts and limits the sustained rate.
- Leaky-bucket implementations can smooth queued work or enforce a changing threshold.
- Login protection benefits from layered limits rather than relying only on IP addresses.
- Distributed applications need coordinated state if a quota is meant to apply across all servers.
- Atomic updates, trusted proxy configuration, cleanup, and monitoring are essential implementation details.
- HTTP 429 and an appropriate Retry-After value help legitimate clients respond to limits correctly.