DEV Community: Ziad Mohammed

Rate Limiting: How APIs Control Traffic (and Where It Gets Complicated)

Ziad Mohammed — Fri, 21 Aug 2026 13:12:53 +0000

Rate limiting sounds simple. Count requests. Block the extra ones. But when you actually build it, things get complicated fast. This article explains the main ideas, the algorithms, and the real problems that appear in production.

What Is a Rate Limiter?

A rate limiter watches incoming requests and decides: allow this one, or block it?

Every rate limiter needs three things:

Limit: the maximum number of requests allowed. Example: 100 requests.
Window: the time period for that limit. Example: per minute.
Identifier: how the system knows who is sending the request. Example: the user's ID, API key, or IP address.

So a complete rule looks like: "100 requests per minute per user ID."

The identifier matters more than most people think. IP address seems like a natural choice, but it breaks down quickly. A company office with 500 employees may share one IP address. A university network does the same. If you use IP as your identifier, you can accidentally block hundreds of innocent users because one person on that network sent too many requests. API keys and user IDs are much better options when you have them.

The Algorithms

There are five common approaches. Each one works differently and fits different situations.

1. Token Bucket

Think of a bucket that holds tokens. Each user has their own bucket.

The bucket has a maximum size, for example 10 tokens.
Tokens are added at a fixed rate, for example 1 token per second.
Each request uses 1 token.
If the bucket is empty, the request is blocked.

The bucket size and the refill rate are two different settings. The refill rate controls how many requests the user can make over time. The bucket size controls how many they can make at once, in a burst.

A user with a bucket of 10 and a refill rate of 1 per second can send 10 requests in one second, then 1 request every second after that.

Bursts are not a problem with this algorithm. They are the point. Use token bucket when your users sometimes send several requests at once and that is normal behavior.

2. Leaky Bucket

Imagine a bucket with a small hole at the bottom.

Requests come in from the top (like water being poured in).
Requests are processed at a fixed rate through the hole at the bottom.
If too many requests arrive at once, the bucket overflows and those requests are dropped.

This always processes requests at the same rate. No matter how many requests arrive, the output is smooth and steady.

The downside: if a user sends no requests for a while, that idle time is wasted. When they come back, they get no extra allowance for the quiet period. Token bucket saves unused capacity; leaky bucket does not.

Also, requests wait in the queue instead of getting rejected immediately. For web APIs, a fast rejection is usually better than a long wait.

3. Fixed Window Counter

Time is divided into fixed blocks. For example, one block per hour.

Each block has a request limit: 5 requests per minute.
The system counts requests in the current block.
If the count reaches the limit, new requests are blocked.
When the block ends, the counter resets to zero.

This is the easiest algorithm to build and understand.

The known problem: a user can send double their limit in a short time by timing it at the boundary between two blocks. Send 5 requests at 11:59:59, then 5 more at 12:00:01. That is 10 requests in 2 seconds, even though the limit is 5 per minute. This is not a rare edge case. Users who want to send as many requests as possible figure this out.

Use fixed window when your system can handle that occasional double-burst and you want the simplest possible implementation.

4. Sliding Window Log

Instead of resetting a counter every hour, this algorithm keeps a record of the exact time of every request.

When a new request arrives, delete all old records outside the window.
Count the remaining records.
If the count is at the limit, block the request.

This is the most accurate approach. The count is always exact. No boundary problem like fixed window.

The cost is memory. Every request from every user needs a stored timestamp. At high traffic, this adds up fast. Use this when accuracy is critical and traffic is not too high.

5. Sliding Window Counter

This is a lighter version of the sliding window log. Instead of storing every timestamp, it keeps just two numbers: the count from the previous window and the count from the current window.

It estimates the real rolling count with this formula:

rolling_count = previous_count × (1 - elapsed_fraction) + current_count

For example, if 30% of the current hour has passed, it assumes 70% of the previous hour's requests are still inside the rolling window. This is an estimate, not an exact count. The error is usually small.

This is a good middle ground. More accurate than fixed window, much cheaper in memory than sliding window log. It is a good default for most public APIs.

What Happens When a Request Is Blocked

When a request exceeds the limit, there are three ways to respond.

Blocking: reject the request immediately with HTTP 429 Too Many Requests. Include a Retry-After header so the client knows when to try again. This is the simplest and most common approach.

Throttling: instead of rejecting, slow the request down. Make it wait before processing. The request eventually succeeds, just with added delay. The downside is that the server still holds the connection open while waiting.

Shaping: allow the request but give it lower priority. High-traffic users go to the back of the line. Normal users are served first. CDNs use this for users who have exceeded their bandwidth limit.

For most APIs, just use blocking. It is fast, clear, and easy for clients to handle.

The Distributed Problem

Everything above works fine on one server. Most production systems run on multiple servers at the same time.

If each server keeps its own counter in memory, each server applies the limit independently. With 10 servers and a limit of 100 requests per minute, a user can actually send 1000 requests per minute by spreading their traffic across all 10 servers.

The fix is to use a shared counter that all servers read and write together. Redis is the standard tool for this. Every request goes to Redis to check and update the counter. Redis handles the counting atomically, meaning two servers cannot read the same number at the same time and both allow a request that should have been blocked.

The cost is one extra network call on every request. At high traffic, that adds latency.

There is also a question worth deciding before you go to production: what happens if the rate limiter itself crashes? Two options:

Fail open: all requests pass through while the limiter is down. Users are not affected, but your infrastructure is unprotected.
Fail closed: all requests are blocked while the limiter is down. Your infrastructure is safe, but real users cannot access your API.

Neither option is wrong. It depends on what you are protecting. Decide this early.

Where to Put the Rate Limiter

A rate limiter can sit in different places in your system.

API gateway: sits in front of your entire application. Best for blocking abuse before it ever reaches your servers. One configuration applies to everything.

Application middleware: lives inside your service. Needed when different endpoints have different limits, or when the limit depends on something the gateway does not know about, like a user's subscription plan.

For most systems: use the gateway for general protection, use application-level limits where the rules are more specific.

Which Algorithm Should You Use

Algorithm	Best for	Main cost
Token Bucket	Bursty but legitimate traffic	Irregular load on downstream services
Leaky Bucket	Smooth, predictable output rate	Wastes idle capacity, slow rejection
Fixed Window	Simplicity matters, bursts are acceptable	Boundary exploit allows 2x burst
Sliding Window Log	Maximum accuracy, low traffic	High memory usage
Sliding Window Counter	Public APIs, balanced accuracy and cost	Slightly approximate count

The algorithm matters, but the identifier and placement matter just as much. A good algorithm on the wrong identifier still lets abuse through.

Five Caching Strategies, and When Each One Will Burn You

Ziad Mohammed — Mon, 17 Aug 2026 23:12:33 +0000

I have seen Redis added to a system the way people add salt to food, instinctively, before tasting anything. The cache goes in, latency drops, everyone is happy, and then three months later someone files a bug because users are seeing stale data they swear they updated. The cache was the problem. Not because caching is bad, but because the wrong strategy was picked for how the system actually reads and writes.

There are five patterns worth knowing. They are not interchangeable.

What is Cache?

A temporary storage layer that sits between your application and your database. The goal is to avoid repeated expensive database calls by serving frequently accessed data from memory.

There is no universally correct strategy. The right choice depends on whether your system is read-heavy, write-heavy, or whether data consistency is the priority.

1. Cache Aside (Lazy Loading)

Flow:

App checks the cache
On a hit, return data
On a miss, app goes to the DB itself, stores the result in cache, returns it

The app owns all three steps. The cache is passive.

On write: invalidate or update the cache entry. Invalidation is usually safer.

Best for: read-heavy data that does not change constantly (user profiles, product configs, catalog data)

Tradeoffs:

First request after a miss always pays the full DB cost
Simultaneous misses on the same key all hit the DB at once (thundering herd)
Cache-check logic is scattered in application code

2. Write Through

Flow: app writes to cache, cache synchronously writes to DB, response returns only after DB confirms

Best for: low write frequency with strict read freshness requirements (admin panels, financial dashboards)

Tradeoffs:

Higher write latency because of two sequential synchronous writes
Cache pollution: data that is never read still gets cached on every write

3. Read Through

Flow:

App reads from cache only, never touches DB directly
On a miss, the cache layer fetches from DB, populates itself, returns data

The cache owns the DB interaction, not the app. This simplifies application code.

Best for: read-heavy workloads where you want to keep DB access out of application code (newsfeeds, product listings)

Tradeoffs:

Writes typically bypass the cache and go straight to DB, so cached data goes stale immediately after a write
Staleness window is controlled by TTL, which may not be acceptable for all data types
Cold start always pays the full DB cost

4. Write Back (Write Behind)

Flow:

App writes to cache
Cache acknowledges success immediately
Cache flushes to DB asynchronously later

Best for: write-heavy workloads where throughput matters more than durability (analytics, counters, logging)

Tradeoffs:

Data loss risk if cache crashes before flushing. How much you lose depends on cache persistence config (Redis with no persistence can lose everything since the last flush)
Not suitable for financial, order, or any data where losing a write is unacceptable

Difference from Write Through: Write Back acknowledges before the DB write. Write Through waits for the DB to confirm. Write Back trades durability for speed. Write Through trades speed for consistency.

5. Write Around

Flow:

App writes directly to DB, cache is skipped entirely
On read, app checks cache first
On miss, fetches from DB and populates cache

Best for: data that is written once and read infrequently (audit logs, exported reports, archived records)

Tradeoffs:

First read after any write is always a cache miss because the cache was never populated at write time
Increased read latency until the data gets cached through normal read traffic

Difference from Cache Aside: Cache Aside can update or invalidate the cache on write. Write Around deliberately skips the cache on every write, assuming the written data will not be read back soon.

The summary that actually holds up

Cache Aside and Read Through are for read-heavy systems, but they handle cache population differently and the write behavior of Read Through is easy to overlook. Write Through and Write Back are both write-aware strategies, but they are opposites in terms of what they trade off: Write Through trades write speed for consistency, Write Back trades durability for speed. Write Around is for data you write but rarely read back.

The mistake is not reaching for one of these. The mistake is not thinking about what your system actually does before picking one.