API Design & Rate Limiting: Building APIs That Scale Without Breaking

How to design REST APIs that scale: HTTP methods, status codes, pagination, rate limiting (token bucket, sliding window), JWT auth, and security best practices.

---

The API That Became a Weapon

In 2021, a developer at a major cloud provider wrote a blog post about an outage that cost the company an estimated $10 million in lost revenue. The root cause wasn't a hardware failure or a DDoS attack. It was a single internal API that was designed without rate limits. A misconfigured internal service began calling the API in a retry loop — 10,000 requests per second, each timing out and retrying immediately. Within minutes, the API was down. Cascading failures took out three dependent services. By the time engineers identified the source, customers had been unable to access their data for 4 hours.

The API had no rate limiting, no circuit breaker, and no Retry-After header to tell the client to back off. It was, in retrospect, an accident waiting to happen — and it happened.

This guide explains how to design APIs that scale gracefully: how to use HTTP methods and status codes correctly, how to implement rate limiting that protects your infrastructure without punishing legitimate users, how to handle authentication with JWTs, and how to avoid the security mistakes that turn APIs into attack surfaces. Whether you're building your first REST API or hardening a production service that handles millions of requests, the same principles apply.

---

REST API Design: Resources, Not Actions

The most common mistake in API design is treating endpoints like function calls. A poorly designed API looks like a list of actions: /getUser, /createOrder, /deleteProduct. A well-designed API looks like a map of resources: /users, /orders, /products. The difference matters because HTTP already provides the verbs.

REST (Representational State Transfer) models your application as a set of resources identified by URLs, and uses HTTP methods to perform operations on those resources. The four methods you'll use 95% of the time:

Method	Purpose	Idempotent	Example
GET	Retrieve a resource	Yes	GET /users/42 returns user 42
POST	Create a new resource	No	POST /users creates a new user
PUT	Replace a resource entirely	Yes	PUT /users/42 replaces user 42
PATCH	Update part of a resource	No	PATCH /users/42 updates some fields
DELETE	Remove a resource	Yes	DELETE /users/42 removes user 42

Idempotency means making the same request multiple times produces the same result as making it once. GET, PUT, and DELETE are idempotent: deleting user 42 five times leaves the database in the same state as deleting it once. POST is not: five POST /users requests create five different users.

This matters for reliability. Networks fail. Clients timeout. If a POST request times out, the client doesn't know whether the server processed it — retrying might create a duplicate. The fix is an idempotency key : a unique client-generated token (a UUID works well) sent in a header like Idempotency-Key: <uuid>. The server stores the key with the result and returns the cached result for duplicate keys.

URL design guidelines:

• Use nouns, not verbs: /users not /getUsers
• Use plurals for collections: /orders not /order
• Nest for relationships: /users/42/orders for user 42's orders
• Use query parameters for filtering: /orders?status=active&limit=20
• Keep URLs lowercase with hyphens: /order-items not /orderItems

---

HTTP Status Codes: The Language Clients Speak

HTTP status codes are how your API communicates what happened. Using them correctly means clients can respond appropriately without parsing response bodies. The categories:

Range	Category	When to use
2xx	Success	The request succeeded
3xx	Redirection	The resource moved (rare in APIs)
4xx	Client error	The request was malformed or unauthorized
5xx	Server error	Something went wrong on your end

The codes you'll use most often:

• 200 OK — standard success for GET, PUT, PATCH, DELETE
• 201 Created — POST that created a new resource (include a Location header pointing to the new resource)
• 204 No Content — success with no response body (common for DELETE)
• 400 Bad Request — the request was malformed (missing required field, invalid type)
• 401 Unauthorized — authentication failed (missing or invalid credentials)
• 403 Forbidden — authenticated but not allowed to access this resource
• 404 Not Found — the resource doesn't exist
• 409 Conflict — the request conflicts with the current state (duplicate email, version mismatch)
• 422 Unprocessable Entity — the request was well-formed but semantically invalid (validation errors)
• 429 Too Many Requests — rate limit exceeded (include Retry-After header)
• 500 Internal Server Error — something went wrong on your end (don't expose details)

The 401 vs 403 distinction trips up many developers. 401 means "I don't know who you are" (authentication). 403 means "I know who you are, but you can't do this" (authorization). A user without a valid token gets 401. A regular user trying to access an admin endpoint gets 403.

429 Too Many Requests deserves its own section — it's the rate limit response, and how you implement it determines whether your API survives a traffic spike or a misbehaving client.

---

Rate Limiting: Protecting Your Infrastructure

Rate limiting is the practice of controlling how many requests a client can make in a given time window. It protects your API from abuse, prevents a single tenant from monopolizing resources, and ensures fair access across all users.

The challenge is doing this without frustrating legitimate users. A developer integrating with your API will hit rate limits during testing. A batch job running at midnight will hit rate limits. A user with a flaky network that retries failed requests will hit rate limits. Good rate limiting distinguishes between a client that's slightly over its limit and one that's attacking your API.

The Three Algorithms

Fixed window is the simplest approach: allow N requests per window (e.g., 1,000 per hour), reset the counter at the start of each window. The implementation is trivial — a counter per client that resets on a timer:

counter[client_id] += 1
if counter[client_id] > LIMIT:
    return 429

The weakness is the edge effect : a client that sends 1,000 requests at 10:59 and another 1,000 at 11:01 has sent 2,000 requests in 2 minutes, despite the "1,000 per hour" limit. The fixed window doesn't see that the two bursts overlap.

Sliding window log stores every request timestamp for every client and counts how many fall within the current window. It's precise but expensive: storing millions of timestamps per client is memory-intensive. A common optimization is the sliding window counter , which combines the current window's count with the previous window's count, weighted by how far into the current window you are:

$$count = current_count \times (1 - \frac{time_into_window}{window_size}) + previous_count$$

This approximates the sliding window without storing every timestamp, at the cost of allowing slightly more than the limit at window boundaries.

Token bucket is the most flexible algorithm and the one used by most production APIs (including GitHub and Stripe). Imagine a bucket that holds a maximum of B tokens. Tokens are added at a constant rate r per second (e.g., 10 tokens per second). Each request consumes one token. If the bucket is empty, the request is rejected with 429.

The token bucket handles bursts naturally: a client that's been idle for a minute has accumulated 60 tokens and can burst to 60 requests immediately. But sustained traffic cannot exceed the refill rate. The algorithm is:

$$tokens = min(B, tokens + r \times \Delta t) - 1$$

If tokens >= 0, the request is allowed. If tokens < 0, it's rejected.

The API Rate Limit & Cost Calculator computes utilization, throttled requests, and monthly cost for any of these algorithms given your limit, load, and price per request. Use it to estimate whether your current limits can handle a 10× traffic spike, or to plan the cost of a rate-limited batch job.

Communicating Limits to Clients

A rate limit without communication is just an error. Good APIs tell clients three things in every response:

1. What the limit is : X-RateLimit-Limit: 1000
2. How many requests remain : X-RateLimit-Remaining: 847
3. When the limit resets : X-RateLimit-Reset: 1687305600 (Unix timestamp)

When the limit is exceeded, return 429 Too Many Requests with a Retry-After header that tells the client how long to wait before retrying:

HTTP/1.1 429 Too Many Requests
Retry-After: 60
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1687305600

{
  "error": "rate_limit_exceeded",
  "message": "You have exceeded the rate limit of 1000 requests per hour.",
  "retry_after": 60
}

The Retry-After header is in seconds (or an HTTP date). A well-behaved client reads this header and waits before retrying — instead of retrying immediately and making the problem worse.

Practical implementation: Most production APIs implement rate limiting at the API gateway or load balancer level (Kong, NGINX, AWS API Gateway, Cloudflare) rather than in application code. This offloads the work and applies limits before requests reach your application servers.

---

Authentication: Knowing Who's Calling

Every API needs to answer two questions: who is this? (authentication) and are they allowed to do this? (authorization). The most common approaches:

API Keys

The simplest approach: give each client a unique key, and require it in every request (usually via header: Authorization: Bearer <key> or X-API-Key: <key>). The server looks up the key to identify the client and check permissions.

API keys are easy to implement but have a weakness: they don't expire. If a key is leaked (committed to a GitHub repo, logged in an error message, intercepted over an unencrypted connection), anyone who has it can impersonate that client. Mitigations: require HTTPS, allow clients to rotate keys, and set expiration dates on keys.

JWT (JSON Web Tokens)

A JWT is a self-contained token that encodes claims (user ID, permissions, expiration) in a JSON payload, signed by the server. The client sends the token in the Authorization: Bearer <token> header. The server verifies the signature and extracts the claims — no database lookup required.

A JWT has three parts, each Base64URL-encoded and separated by dots:

eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjo0MiwiZXhwIjoxNjg3MzA1NjAwfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

• Header (eyJhbGciOiJIUzI1NiJ9): algorithm and token type
• Payload (eyJ1c2VyX2lkIjo0MiwiZXhwIjoxNjg3MzA1NjAwfQ): claims — user_id, expiration, roles
• Signature (SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c): HMAC of header+payload with server secret

The critical claim is exp (expiration). A JWT without an expiration is valid forever — if leaked, it's a permanent access token. Set short expirations (15 minutes to 1 hour) and use refresh tokens for longer sessions.

JWT ≠ encryption. The payload is signed, not encrypted — anyone who intercepts the token can read the claims (that's why you should never put secrets in a JWT). The signature only guarantees that the token was issued by someone with the server secret and hasn't been tampered with. The JWT Decoder parses tokens client-side, showing the header, payload, and signature — useful for debugging authentication flows.

OAuth 2.0

OAuth 2.0 is a delegation protocol: it lets a user grant a third-party application limited access to their resources without sharing their password. The flow: the user clicks "Log in with Google," Google asks for permission, and issues an access token that the third-party app can use on the user's behalf.

For machine-to-machine APIs (where there's no user to click "Log in"), use the client credentials flow: the client exchanges its client ID and secret for an access token. This is what backend services use to communicate with each other.

---

Security: The Mistakes That Get You Hacked

APIs are the most common attack surface for web applications. The OWASP API Security Top 10 lists the most critical risks. The ones every developer should know:

Broken Object-Level Authorization (BOLA)

The most common API vulnerability. If GET /users/42 returns user 42's data, but the server doesn't check that the authenticated user is user 42 (or an admin), then any authenticated user can access any other user's data by changing the ID in the URL. Every endpoint that takes an ID must check authorization.

Excessive Data Exposure

APIs often return more data than the client needs. If GET /users/42 returns the user's email, phone, address, and hashed password, the client may only display the name — but the sensitive data is now in the browser's network tab, in logs, and in any proxy between client and server. Return only the fields the client needs, and use different response schemas for different roles.

Lack of Resources & Rate Limiting

Without rate limits, an attacker can brute-force passwords, scrape data, or overwhelm your API with requests. This is the failure mode that caused the $10 million outage described earlier. Rate limit authentication endpoints aggressively (5 attempts per minute per IP) and API endpoints per-client.

Mass Assignment

If your API accepts a JSON body and binds it directly to a database model without filtering, an attacker can set fields they shouldn't control. A PATCH /users/42 endpoint that accepts { "role": "admin" } because the request body is bound directly to the model is a mass assignment vulnerability. Whitelist the fields that can be updated.

Security Headers

Every API response should include:

• Strict-Transport-Security: max-age=31536000 — force HTTPS
• X-Content-Type-Options: nosniff — prevent MIME sniffing
• Cache-Control: no-store — don't cache responses with sensitive data

---

Practical Tips for API Development

1. Always version your API. Use URL versioning (/v1/users) or header versioning (Accept: application/vnd.api.v1+json). Never release a breaking change without a version bump.
2. Use pagination for collections. Returning 100,000 records in one response is a memory bomb. Use cursor-based pagination (?cursor=<token>&limit=20) for large datasets, or offset-based (?offset=0&limit=20) for smaller ones.
3. Return consistent error formats. Every error response should have the same structure: { "error": { "code": "...", "message": "...", "details": [...] } }.
4. Log requests, not bodies. Log the method, URL, status code, and response time. Never log request bodies — they may contain passwords or tokens.
5. Use HTTPS everywhere. No exceptions. Even internal APIs should use TLS.
6. Set request size limits. A POST endpoint without a body size limit is vulnerable to memory exhaustion attacks. Limit to what you actually need (1 MB for most APIs).
7. Implement circuit breakers. If a dependent service is failing, stop calling it after N consecutive failures and return a fallback response. This prevents cascading failures.

---

Frequently Asked Questions

Q: What is the difference between REST and GraphQL?

A: REST models your API as resources identified by URLs, with HTTP methods as verbs. GraphQL uses a single endpoint and lets clients specify exactly what data they need in the query. REST is simpler and cacheable; GraphQL is more flexible for complex data requirements.

Q: What is idempotency and why does it matter?

A: Idempotency means making the same request multiple times produces the same result as making it once. GET, PUT, and DELETE are idempotent; POST is not. For non-idempotent operations, use an idempotency key (a UUID) so retries don't create duplicates.

Q: What is the difference between 401 and 403?

A: 401 Unauthorized means authentication failed (missing or invalid credentials) — I don't know who you are. 403 Forbidden means you're authenticated but not allowed to access this resource — I know who you are, but you can't do this.

Q: What is a JWT?

A: A JSON Web Token is a self-contained token that encodes claims (user ID, permissions, expiration) in a JSON payload, signed by the server. The server verifies the signature and extracts the claims without a database lookup. JWTs are signed, not encrypted — never put secrets in them.

Q: What is the best rate limiting algorithm?

A: Token bucket is the most common in production (used by GitHub and Stripe) because it handles bursts naturally while capping sustained traffic. Fixed window is simpler but allows 2× bursts at window boundaries. Sliding window is more precise but more expensive to implement.

Q: What headers should a rate-limited API return?

A: X-RateLimit-Limit (the limit), X-RateLimit-Remaining (requests left), and X-RateLimit-Reset (when the limit resets). On 429 responses, include Retry-After (seconds to wait before retrying).

Q: How do I protect against BOLA?

A: Broken Object-Level Authorization — the most common API vulnerability. Every endpoint that takes an ID must check that the authenticated user is authorized to access that specific resource. Don't rely on clients to only request their own data.

Q: Should API keys expire?

A: Yes. Keys that never expire are a permanent liability if leaked. Set expiration dates (90 days is common) and allow clients to rotate keys. Require HTTPS to prevent interception.

---

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide