Greg

Posted on Nov 9

API Rate Limiting: How to Test It and How to Design It

#api #apisec #security

Rate limiting is usually discussed as an availability feature. In APIs it is also a security control — for brute force, enumeration, scraping, and cost abuse. Most implementations are weaker than the team believes.

Every API has a rate limit. Some have it in the gateway. Some have it in the application. Some have it in a design document from 2022. The question is not whether a limit exists, but whether it is enforced, scoped correctly, and tested.

This guide covers both sides: how to design limits that hold under real abuse, and how to test the ones you have.

Why rate limiting is a security control

Rate limiting is often filed under "reliability." In API security it does four jobs:

Slows brute force and credential stuffing. Without limits, a login endpoint is an oracle.
Blocks enumeration. Sequential IDs, email existence checks, and password reset flows become expensive to probe.
Prevents cost abuse. SMS, email, AI tokens, third-party APIs, and expensive queries all have a price per call.
Protects availability. One client should not be able to degrade service for everyone else.

A missing limit is not a theoretical weakness. It turns every other endpoint into a resource the attacker can use at scale.

Part 1: How to design limits that work

1. Scope by identity, not just by IP

IP-based limits are the most common and the least effective. Attackers rotate IPs, use proxies, or share a NAT with thousands of legitimate users.

Use a hierarchy:

Per API key / token — the primary identity for machine clients.
Per user — for interactive sessions.
Per tenant — so one customer cannot exhaust a shared quota or hide behind many users.
Per IP — as a coarse backstop for unauthenticated traffic.
Global — to protect the platform from aggregate overload.

Different endpoints need different scopes. Login should be limited per account and per IP. Expensive exports should be limited per tenant and per user.

2. Use cost-based limits for expensive operations

Not all requests are equal. A GET /health is cheap; a report generation or an AI completion is not. A flat "100 requests per minute" either throttles cheap calls unnecessarily or lets expensive ones through.

Assign a cost to operations and limit by budget:

GET  /items           cost 1
POST /search          cost 5
POST /reports/export  cost 50
POST /ai/completion   cost 100

A tenant with a budget of 1,000 units per minute can make many cheap calls or a few expensive ones. This aligns limits with actual resource consumption.

3. Choose the right algorithm

Fixed window — simple, but allows bursts at window boundaries (2× the limit around the boundary).
Sliding window — smoother, slightly more state.
Token bucket — allows controlled bursts; good default for APIs.
Leaky bucket — smooths output; good for protecting downstream services.
Concurrency limit — caps in-flight requests; essential for expensive, long-running operations.

For most APIs, a token bucket per identity plus a concurrency limit per tenant covers the common cases.

4. Make limits visible and actionable

Return:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1712345678

Clients that know when to retry behave better. Clients that get an opaque 429 retry immediately and make the problem worse.

5. Distinguish throttling from blocking

A limit breach is usually temporary. Return 429, not 403. Reserve blocking (and account lockout) for repeated abuse, and make it time-bound and observable.

6. Apply limits at the right layer

Gateway / edge — coarse per-IP and per-key limits, DDoS protection, global quotas.
Application — identity-aware limits, cost-based budgets, business rules.
Database / downstream — concurrency limits, timeouts, connection pool caps.

A single layer is rarely enough. The gateway does not know your business logic; the application does not see the whole traffic picture.

7. Protect the limiter itself

If the rate limiter uses Redis, it becomes a critical dependency. Plan for:

Redis outages (fail open or fail closed — decide deliberately),
hot keys and sharding,
clock skew,
distributed counters and atomic operations.

A limiter that fails open under load is not a control. A limiter that fails closed can take down the API. Choose per endpoint.

8. Handle distributed and multi-region setups

In-memory counters per instance do not enforce a global limit. With N instances, the effective limit is N × the configured value. Use a shared store or a gateway that aggregates.

9. Don't trust client-supplied identity for limits

Limits keyed on X-Tenant-Id or a query parameter are bypassable by changing the value. Derive the key from the authenticated identity and validate any routing header against it.

10. Log and alert on limit breaches

A spike in 429s can mean:

a client bug (retry storm),
a legitimate customer hitting a plan limit,
an attack.

Distinguish these with identity, endpoint, and pattern data. Alert on sustained breaches and on many distinct identities hitting the same endpoint.

Part 2: How to test rate limiting

Test 1. Verify the limit exists

Send a burst of requests and record the status sequence:

200, 200, ..., 200, 429, 429, ...

If you never see 429, either the limit is very high or it is not enforced. Check whether the limit is enforced per instance rather than globally by sending requests to different instances.

Test 2. Find the actual threshold

Start below the documented limit and increase. Compare the observed threshold with the documentation and the plan tier. A mismatch is a finding: customers are either over-paying or over-consuming.

Test 3. Test scope

Send requests with:

the same token from different IPs,
different tokens from the same IP,
different users in the same tenant,
the same user across tenants,
unauthenticated requests from rotating IPs.

If the limit is per IP only, the token/IP matrix will reveal it. If it is per instance, parallel requests to multiple instances will exceed it.

Test 4. Test bypass attempts

spoofed X-Forwarded-For, X-Real-IP, Forwarded headers,
different casing or encoding of paths (/api/items, /api//items, /api/items/),
query parameters that change the cache key but not the handler,
HTTP/2 multiplexing and connection reuse,
changing tenant_id, account_id, or user_id in the body,
version switching (/v1 vs /v2),
GraphQL batching and aliases,
WebSocket messages that bypass HTTP limits.

Test 5. Test expensive operations separately

A generic limit may protect GET endpoints and leave exports, reports, AI calls, or file processing unbounded. Measure cost per request, not just count.

Test 6. Test failure behavior

What happens when the limiter's store is unavailable? Does the API fail open (no limits) or closed (all requests rejected)? Both can be valid, but the choice should be deliberate and documented.

Test 7. Test the response contract

Is Retry-After present and accurate?
Do X-RateLimit-* headers match the actual behavior?
Does the error body explain the limit without leaking internals?
Does the client have a machine-readable error code?

Test 8. Test recovery

After the window resets, does the API recover immediately? Are there lingering locks or counters that keep a client blocked longer than advertised?

Test 9. Test the limits you cannot see

Some limits are business-level: "one free trial per organization," "maximum 10 exports per day," "no more than 3 password resets per hour." Test these flows end-to-end, not just at the HTTP layer.

A minimal test script

import asyncio, httpx

async def burst(url, headers, n=200):
    async with httpx.AsyncClient() as client:
        results = []
        for _ in range(n):
            r = await client.get(url, headers=headers)
            results.append(r.status_code)
        return results

results = asyncio.run(burst(
    "https://api.example.com/v1/items",
    {"Authorization": "Bearer <token>"},
))
print(results.count(200), results.count(429))

Run it against two tokens, two IPs (if you have test egress), and two instances. The pattern of 429s tells you what the limiter is actually keyed on.

Common mistakes

Only per IP. Rotating IPs defeats it; NAT punishes legitimate users.
Only on login. Enumeration, exports, and AI endpoints stay open.
In-memory counters per instance. The real limit is N × configured.
Trusting X-Forwarded-For. Clients can set it unless your edge overwrites it.
No limit on expensive operations. Cheap requests are throttled; costly ones are not.
No Retry-After. Clients retry immediately and amplify the load.
Limits that never reset. A bug can lock a customer out permanently.
Silent dropping. Requests disappear without a 429, making debugging impossible.
Limits keyed on client-supplied tenant IDs. Trivial to bypass.
No monitoring. You cannot tell an attack from a retry storm.

Where continuous testing fits

Rate limits are a classic drift target: a limit is raised to unblock a customer, a proxy rule is relaxed, a new instance is deployed without the shared counter, or a new expensive endpoint ships without a budget. All of these look fine in code review and only show up under load.

This is part of OWASP API4 (Unrestricted Resource Consumption), and it is one of the checks in LiveAPISec's deterministic core: burst tests on selected endpoints, comparison of results between scans, and a configuration fingerprint that flags when rate-limit behavior changes. The point is not a one-time "yes, a limit exists" — it is knowing that the limit still exists, at the right scope, after the next deploy.

pipx install liveapisec
liveapisec scan --help

Checklist

Design

[ ] Limits are scoped per token, user, tenant, and IP as appropriate.
[ ] Expensive operations use cost-based budgets or concurrency limits.
[ ] Limits are enforced globally, not per instance.
[ ] 429 responses include Retry-After and rate-limit headers.
[ ] The limiter's failure mode (open/closed) is deliberate.
[ ] Limit keys derive from authenticated identity, not client input.
[ ] Breaches are logged and alerted with identity and endpoint context.

Testing

[ ] Verify the limit exists and find the real threshold.
[ ] Test across tokens, IPs, users, tenants, and instances.
[ ] Attempt bypasses via headers, paths, versions, and batching.
[ ] Test expensive endpoints separately.
[ ] Test limiter failure and recovery.
[ ] Test business-level limits end-to-end.

Key takeaways

Rate limiting is a security control, not just a reliability feature.
Scope matters more than the number: per-IP-only limits are mostly theater.
Cost-based limits protect the resources that actually cost money.
In-memory, per-instance counters do not enforce a global limit.
Test for bypasses, not just for the presence of 429.
Return useful errors and monitor breaches — a spike is either a bug, a customer, or an attack.

DEV Community