DEV Community: Sir Max

API Keys vs OAuth 2.0 vs JWT: How I Actually Choose (After Getting It Wrong Twice)

Sir Max — Sat, 12 Sep 2026 02:26:11 +0000

API Keys vs OAuth 2.0 vs JWT: How I Actually Choose (After Getting It Wrong Twice)

I've shipped authentication three different ways across three different services, and I got it wrong twice. Not "wrong" as in broken — wrong as in needlessly complicated, which is worse, because nobody notices until a new engineer spends a week trying to onboard.

Here's the short version of what I learned: API keys, OAuth 2.0, and JWT are not three options on the same menu. One is a credential, one is an authorization framework, and one is a token format. Comparing them directly is like comparing "a password" to "a login flow" to "a piece of paper."

Once that clicked, the decisions got easy. Let me walk through how I actually choose now.

Start with two questions, not with a technology

Before touching any auth library, answer these:

Who is the caller? A server you control? A third-party app acting for one of your users? A browser holding a session?
Who decides what they're allowed to do? You (static permissions), the user (consent screen), or an external identity provider?

If the answer to (1) is "another backend" and (2) is "me", you want an API key. That's the whole decision. Everything below is just detail.

API keys: the boring answer that's usually right

An API key is a long random string that identifies the caller. That's it. It carries no user identity, no scopes, no expiry unless you add one.

I use API keys when:

machine-to-machine calls happen between services I control
the permission model is coarse ("this key can read, that key can write")
I want to be able to revoke access in one database update

What most teams get wrong is storage. They hash user passwords properly and then store API keys in plaintext, because "it's just a key." If your database leaks, those keys are directly usable. Treat them like passwords: store a hash, show the raw value exactly once at creation.

import hashlib
import secrets

def create_api_key(conn, owner_id: str, scope: str) -> str:
    """Returns the raw key ONCE. Only the hash is persisted."""
    raw = f"sk_{secrets.token_urlsafe(32)}"
    key_hash = hashlib.sha256(raw.encode()).hexdigest()
    conn.execute(
        "INSERT INTO api_keys (key_hash, owner_id, scope, created_at, revoked) "
        "VALUES (?, ?, ?, datetime('now'), 0)",
        (key_hash, owner_id, scope),
    )
    conn.commit()
    return raw

Lookup is a single indexed query on key_hash. No secrets at rest, no encrypted-column gymnastics, and revocation is just flipping a flag.

I also give every key a prefix column (the first 8 characters, stored in plaintext). It's useless to an attacker and it lets you search logs by key without ever seeing the secret. When someone says "which integration is hammering us?", you can answer in ten seconds.

Where API keys fall apart: the moment a request needs to say which user it's for. A key identifies the app, not the person. The usual workaround — passing a user ID alongside the key — means anyone with the key can impersonate anyone. That's where the next option starts.

OAuth 2.0: for when someone else's user is involved

OAuth 2.0 exists to solve one specific problem: letting a third-party application access a user's data without the user handing over their password.

If your service never has third-party apps acting on behalf of your users, you probably don't need OAuth. This is the mistake I made the first time — I built a full authorization-code flow with a consent screen for an internal admin panel where every user was an employee and every permission was static. Six weeks of work that a table of API keys would have replaced.

If you do need it, the parts that matter in practice:

Authorization Code + PKCE is the only flow I'd use for anything user-facing. The implicit flow is dead. Client credentials is fine, but that's machine-to-machine — see the API keys section above and ask whether OAuth buys you anything.
Scope design is your real API design. read is a bad scope. orders:read is a good one. Scopes should map to things a user can understand on a consent screen, because that's exactly where they'll be shown.
Refresh token rotation is not optional. Issue a new refresh token every refresh, invalidate the old one, and treat reuse of a stale refresh token as a compromise signal worth logging loudly.

The cost nobody mentions: OAuth means you now maintain a consent UI, a token store, a revocation path, and a support burden for "why did this app show up in my account?" That's a fair price for third-party access. It's a terrible price for authenticating your own mobile app.

JWT: a format, not an authentication strategy

This is where the second mistake lived.

A JWT is just a signed JSON blob. It is not a protocol, not a session system, and not automatically more secure than an opaque token. The appeal is real: the server can verify a token without a database round trip, because the signature proves the contents.

The trap is what you do after that.

Mistake I made: I used JWTs as the session mechanism and set a 30-day expiry to avoid forcing re-logins. That meant a user who was banned, demoted, or who changed their password still had a valid token for up to a month. There is no "log out everywhere" with a self-contained token unless you build revocation — and the moment you build a revocation list you're doing a database lookup on every request, which was the whole reason you picked JWT.

So here's my rule now:

Situation	What I use	Why
Server-to-server, static permissions	API key (hashed at rest)	Simplest thing that works, instant revocation
Third-party app acting for a user	OAuth 2.0 + PKCE	Built for consent and delegated access
Short-lived token between services	JWT with 5–15 min expiry	Verification without a shared session store
Long-lived user session	Opaque token in a server-side session	Revocation is a single delete
Browser session	HttpOnly, Secure, SameSite cookie	Never let JavaScript touch the credential

The honest summary: JWT is an optimization for verification, not a replacement for state. Use short expiries and accept that revocation happens at the next refresh, or use opaque tokens and keep a session table. Don't use a 30-day JWT and pretend the problem doesn't exist.

The details that actually cause incidents

A few things I now check on every service, regardless of which option won:

Timing-safe comparison. Comparing secrets with == leaks information through response timing. Use hmac.compare_digest or your language's equivalent. It's one line.
Keys in logs. Most auth incidents I've seen weren't database breaches — they were a key printed in a log line, then shipped to a log aggregator with a wider audience than the database. Redact Authorization headers before logging, always.
No auth in query strings. Query strings land in access logs, browser history, and referrer headers. This is the single most common way a working system leaks credentials on day one.
Expiry you can actually enforce. A expires_at column that nothing reads is decoration. If a key can be rotated, rotate it — quarterly, automatically, and make rotation a boring scheduled task rather than a quarterly panic.

The checklist I use now

Is the caller a service I control with static permissions? → API key, hashed at rest, with a plaintext prefix for log searching.
Is a third party acting on behalf of a user? → OAuth 2.0, authorization code + PKCE, narrow scopes, rotating refresh tokens.
Do I need verification without a shared store, on a timescale of minutes? → JWT, short expiry.
Is this a user session that must be revocable immediately? → Opaque token, server-side session table.

Two of my three services ended up with API keys. The one that needed OAuth genuinely needed it. Nobody's implementation was made better by adding a token format it didn't have a problem for.

If you're staring at a design right now and the answer to question one is "another backend," stop reading comparison posts (including this one) and write the hashed-key table. You'll be done before lunch.

ETags and Cache-Control for APIs: The Headers Most Teams Skip

Sir Max — Thu, 10 Sep 2026 08:42:55 +0000

ETags and Cache-Control for APIs: The Headers Most Teams Skip

I once spent a morning staring at a dashboard that made no sense. The service was handling a few thousand requests a minute, and a large slice of the responses were byte-for-byte identical to what the same client had received eight seconds earlier. We were paying CPU, bandwidth, and database connections to say exactly the same thing, over and over.

The fix wasn't a bigger machine. It was three HTTP headers that most API teams never send.

This is the practical version of what I learned: how conditional requests actually work, which cache directives matter for JSON APIs (and which ones silently break your auth), and the one header — Vary — that causes caching bugs you only find in production.

Everything below is plain HTTP. It works in FastAPI, Express, Rails, Go, Django, whatever. If your framework sets these automatically, good — but you should still know what it's setting.

Why caching an API is not caching a web page

When people say "caching," they usually mean one of two very different things:

Server-side caching. You store the computed result (or the database row) somewhere fast — Redis, memcached, an in-process LRU — and skip the expensive work. The response still goes out over the wire in full.
Client-side / intermediary caching. The client (a browser, a mobile app, a CDN, a reverse proxy) keeps a copy of the response and decides not to ask again, or to ask conditionally.

Most teams do #1 and stop. That leaves #2 on the table, and #2 is where the cheap wins are: you save the entire round trip, the serialization, and the bandwidth.

The mechanism for #2 is the conditional request. It's two headers and a status code.

ETag + If-None-Match: the 304 you're not sending

Here's the flow.

Client GETs /v1/users/42. Your server returns the body plus an ETag header — an opaque identifier for that specific representation:

HTTP/1.1 200 OK
Content-Type: application/json
ETag: "a3f9c1e8"

{"id": 42, "name": "Ada", "role": "admin"}

Next time the client needs that resource, it sends the tag back:

GET /v1/users/42
If-None-Match: "a3f9c1e8"

If nothing changed, you return 304 Not Modified with no body:

HTTP/1.1 304 Not Modified
ETag: "a3f9c1e8"

The client reuses its copy. You saved the serialization, the payload, and the network transfer. On a 20 KB JSON document, that's 20 KB you didn't send — and a client that reuses its own copy instead of parsing a new one.

Generating an ETag that doesn't lie

The first trap: people hash the response body after they've built it. That's fine for correctness, but it means you did all the work anyway. The bigger trap is picking a tag that changes when the content doesn't, which turns every conditional request into a miss.

Two approaches that hold up:

Weak ETags from a version column. If your row has updated_at or a monotonic version, use it: ETag: W/"user-42-v17". Cheap, correct, no hashing. The W/ prefix says "semantically equivalent," which is exactly what you mean.
Strong ETags from a content hash. Hash the canonical serialized bytes. Correct, but costs a hash per response — usually fine (sha1 on a few KB is microseconds), and it catches changes your version column missed, such as a joined table updated independently.

If you serve the same resource in multiple formats — JSON and CSV, say — the ETag must differ per representation. Otherwise a client can cache the CSV, send If-None-Match back to your JSON endpoint, and get a 304 for the wrong body.

Cache-Control: the directives that actually matter

ETag makes revalidation cheap. Cache-Control decides when revalidation happens at all.

For APIs the useful set is small:

Directive	What it does	Use it for
`no-store`	Nothing may be cached, ever	Anything with secrets in the body
`no-cache`	May cache, must revalidate every time	Data that changes often but benefits from 304s
`private`	Only the end client may cache, not shared proxies	Per-user responses
`public`	Shared caches (CDN) may store it	Genuinely public reference data
`max-age=N`	Fresh for N seconds, no revalidation	Slowly changing reference data
`s-maxage=N`	`max-age` for shared caches only	CDN tuning
`stale-while-revalidate=N`	Serve stale for N seconds while refreshing	Read-heavy, latency-sensitive endpoints

A sane default for an authenticated JSON API:

Cache-Control: private, no-cache, max-age=0

That reads as: "the client may keep a copy, but every request must be revalidated." Combined with ETag, you get 304s when nothing moved and full bodies only when something did. You get most of the benefit with none of the staleness risk.

Do not put public on anything behind auth. public means any intermediary — corporate proxies, CDNs, your own gateway — may store it and hand it to the next caller. That's how one user's data ends up in another user's response.

The `Vary` header, and why auth breaks caching

This is the one that bites in production.

HTTP caches key responses on the URL by default. But your response often depends on more than the URL — it depends on request headers. Vary tells caches which headers to include in the cache key:

Vary: Accept-Encoding, Accept, Authorization

If you serve different bodies based on Accept (JSON vs protobuf) or on the caller's permission level, and you don't declare that in Vary, a shared cache will happily serve one caller's representation to another. This is a real, known class of data-leak bug, not a theoretical one.

Some practical rules:

For per-user authenticated responses, the simplest safe answer is Cache-Control: private. Shared caches see it and refuse to store. That makes Vary: Authorization largely moot for shared caches — but still declare it if you rely on client-side caches.
Never key a cache on Authorization and mark the response public. That combination is contradictory and different CDNs handle it differently.
Vary: Accept-Encoding is set by most servers automatically. Don't override it away.

Invalidation on writes

Conditional requests only stay honest if the ETag changes exactly when the resource changes. Two patterns work:

Version-based. Bump a version integer, or touch updated_at, on every write — inside the same transaction. The ETag derives from it. Simple, predictable, and it survives multiple app servers because the source of truth is the database row.

Hash-based. The ETag derives from the response content, so it changes automatically. Correct with zero bookkeeping, but you only discover the change after you've built the response — savings on the read path only.

Either way: invalidate on the write that changes the representation, not on a timer. A TTL that expires faster than your data changes just moves the cost around; a TTL slower than your data changes serves stale data.

A minimal FastAPI example

import hashlib
from fastapi import FastAPI, Request, Response, HTTPException
from fastapi.responses import JSONResponse

app = FastAPI()

# Pretend this is a database row
USERS = {42: {"id": 42, "name": "Ada", "role": "admin", "version": 17}}


def make_etag(payload: dict) -> str:
    body = repr(sorted(payload.items())).encode()
    return '"' + hashlib.sha1(body).hexdigest()[:16] + '"'


@app.get("/v1/users/{user_id}")
def get_user(user_id: int, request: Request):
    user = USERS.get(user_id)
    if user is None:
        raise HTTPException(status_code=404)

    etag = make_etag(user)

    # Conditional request: the client already has this representation.
    if request.headers.get("if-none-match") == etag:
        return Response(
            status_code=304,
            headers={"ETag": etag, "Cache-Control": "private, no-cache"},
        )

    return JSONResponse(
        user,
        headers={
            "ETag": etag,
            "Cache-Control": "private, no-cache, max-age=0",
            "Vary": "Accept-Encoding, Authorization",
        },
    )

Two things worth calling out. First, the 304 path still returns the ETag — clients need it to keep their copy validated for next time. Second, Cache-Control: private appears on both paths; a 304 without it can be stored by a shared cache under the wrong assumptions.

What to measure before and after

If you want to know whether this is worth the effort, instrument three numbers:

304 ratio — 304s divided by total conditional requests. Under 20% and your ETag is probably churning on irrelevant fields. Near zero and you're not sending ETag at all.
Bytes out per request — total response bytes divided by request count. Should drop noticeably on read-heavy endpoints.
Origin CPU and DB queries — the server-side win. The 304 path should skip the expensive part of the handler wherever correctness allows.

On a polling-heavy endpoint in our stack — a mobile client checking for updates every few seconds — the 304 path replaced most full responses. The client still polls; it just gets a couple hundred bytes back instead of tens of kilobytes, and the origin skips the joins. Same correctness, much less work. The cheapest request is the one you never had to answer.

The short list

If you only do five things:

Return ETag on GET responses. Weak tags from a version column are enough to start.
Honor If-None-Match and return 304 with the same ETag.
Default authenticated endpoints to Cache-Control: private, no-cache.
Declare Vary for anything the response depends on beyond the URL.
Change the ETag on the write, in the same transaction as the write.

None of this is exotic. It's just headers most teams never get around to sending — and the round trips they keep paying for.

4 Patterns That Keep Your API Online When a Third-Party Service Goes Down

Sir Max — Wed, 09 Sep 2026 02:41:05 +0000

It was a normal Friday afternoon until the tickets started: "Is your API down?" Our status page was green. Our logs were clean. Our servers were fine. The thing that was actually down belonged to a vendor we had no visibility into, no pager for, and no way to fix.

That was the day I stopped treating upstream providers as a background assumption and started treating failure as a design input. If your application calls a third-party API — a payment processor, an AI model provider, a geocoding service — then their outage is your outage. The only question is how much of it leaks to your users.

Here are the four patterns that actually kept us online after that day, with code you can adapt.

Pattern 1: Cut failures off early with timeouts everywhere

The most common outage amplifier is not a slow response. It is no response at all. A connection that hangs open consumes a worker thread, then another, then your whole pool, and suddenly your API is "down" even though every upstream is fine.

Rule one: every outbound call gets a connect timeout and a read timeout, and you set them deliberately rather than relying on defaults.

import httpx

# connect timeout: how long to wait for the TCP/TLS handshake
# read timeout: how long to wait between bytes of the response
client = httpx.Client(timeout=httpx.Timeout(connect=3.0, read=10.0))

# For streaming responses, a single read timeout is not enough.
# You want a per-byte budget, not a per-response budget:
stream_timeout = httpx.Timeout(connect=3.0, read=30.0, write=10.0, pool=3.0)

The number matters less than the existence of the limit. Pick a budget that fits your user-facing latency goal, then make it smaller than your platform's request timeout, so you decide when a call is dead — not the load balancer upstream of you.

Pattern 2: A circuit breaker so you stop hammering a dead provider

Timeouts stop a single request from hanging forever. But if the provider is down for twenty minutes, every request will still wait out its full timeout before failing. Under load, that is a self-inflicted pileup: your users wait, your queues fill, your own error rate spikes.

A circuit breaker fixes that by failing fast once you know the upstream is unhealthy. The state machine is small enough to hold in your head:

Closed: calls go through normally. Failures are counted.
Open: after N consecutive failures, all calls fail immediately without touching the network.
Half-open: after a cooldown, one probe request is allowed through. If it succeeds, the circuit closes; if it fails, it opens again.

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, cooldown=30.0):
        self.failure_threshold = failure_threshold
        self.cooldown = cooldown
        self.failures = 0
        self.state = "closed"      # closed | open | half_open
        self.opened_at = None

    @property
    def allowed(self) -> bool:
        if self.state == "closed":
            return True
        if self.state == "open":
            if time.monotonic() - self.opened_at >= self.cooldown:
                self.state = "half_open"
                return True
            return False
        return True  # half_open: exactly one probe request

    def record_success(self) -> None:
        self.failures = 0
        self.state = "closed"

    def record_failure(self) -> None:
        self.failures += 1
        if self.failures >= self.failure_threshold or self.state == "half_open":
            self.state = "open"
            self.opened_at = time.monotonic()

Wiring it around a call is a few lines:

def call_with_breaker(breaker: CircuitBreaker, fn):
    if not breaker.allowed:
        raise UpstreamUnavailable("failing fast: circuit is open")
    try:
        result = fn()
    except Exception:
        breaker.record_failure()
        raise
    breaker.record_success()
    return result

Two details that mattered in practice. First, count only meaningful failures — a 400 validation error from the provider is not an outage signal and should not trip the circuit; a timeout, connection error, or 5xx should. Second, give each upstream dependency its own breaker. A shared breaker means one flaky endpoint takes down the healthy ones with it.

Pattern 3: Serve the last good response instead of an error page

Fast failure is good. Not failing at all is better. For data that changes slowly — a currency list, a weather forecast, a model's cached output — the most reliable source during an outage is often the last successful response you already have.

The pattern is stale-while-error: serve from cache when it is fresh; when the upstream fails, fall back to the stale entry instead of surfacing the error.

import time

_cache: dict[str, tuple[float, object]] = {}

def get_with_fallback(key: str, fetch, ttl_seconds: float = 300.0):
    now = time.monotonic()
    entry = _cache.get(key)

    if entry and entry[0] > now:
        return entry[1]  # fresh cache: happy path

    try:
        payload = fetch()
    except Exception:
        if entry is not None:
            return entry[1]  # upstream down: stale is better than nothing
        raise

    _cache[key] = (now + ttl_seconds, payload)
    return payload

A real implementation should store an explicit fetched_at timestamp with the payload, log loudly whenever it serves a stale fallback, and expire entries entirely after a much longer absolute horizon — you do not want to serve data from last quarter because the vendor was down for a month. The key idea is that your users should never see a provider's incident page; they should see slightly old data and a healthy API.

Pattern 4: Fail over to a second provider — only where it is safe

If your dependency is a commodity API — several providers offer the same interface, like the OpenAI-compatible chat endpoints many model hosts expose — a secondary provider gives you an escape hatch. The loop is simple:

def call_with_failover(providers, request):
    last_error = None
    for provider in providers:
        try:
            return provider.chat(request)
        except (TimeoutError, httpx.TransportError) as exc:
            last_error = exc
            continue
    raise UpstreamUnavailable(f"all providers failed: {last_error}")

The warning I wish someone had given me earlier: failover is only safe when the operation is read-only or idempotent. If you retry a non-idempotent payment charge on a second provider, you do not get resilience — you get two charges and a very angry customer. For writes, failover is a product decision, not a code change. When in doubt, fail over only for reads, and only across providers that share a compatible contract.

Test the failure modes before the failure

Every one of these patterns rots if it is never exercised. The cheap way to keep them honest is to simulate the outage in tests by monkeypatching the transport:

def test_circuit_opens_after_timeouts(monkeypatch):
    breaker = CircuitBreaker(failure_threshold=3, cooldown=30.0)

    def flaky_call():
        raise httpx.ReadTimeout("simulated upstream hang")

    for _ in range(3):
        with pytest.raises(httpx.ReadTimeout):
            call_with_breaker(breaker, flaky_call)

    assert breaker.state == "open"

    # While open, we fail fast without ever touching the network:
    with pytest.raises(UpstreamUnavailable):
        call_with_breaker(breaker, flaky_call)

The test that caught the most for us: asserting the circuit behavior, not just the HTTP behavior. Anyone can write a test that a timeout raises; the valuable test proves the breaker opened, the fallback served stale data, and the whole chain degraded gracefully instead of collapsing.

The checklist I keep next to every integration

Every outbound call has an explicit connect and read timeout.
Each upstream has its own circuit breaker with a sane threshold and cooldown.
Cacheable data has a stale-while-error fallback with a hard expiry.
Non-idempotent calls never get automatic retries or failover.
The failure path is tested with simulated timeouts, not just happy-path mocks.
When the upstream degrades, my API degrades by design — never by accident.

That Friday outage ended up being a gift, honestly. It forced us to treat failure as a normal state of the system rather than an exception to be handled later. Your dependencies will go down. Your users do not have to know.

Trace Any API Request Across Your Stack: A Practical Guide to Request IDs

Sir Max — Mon, 07 Sep 2026 02:15:54 +0000

Trace Any API Request Across Your Stack: A Practical Guide to Request IDs

It was 2 AM, and a customer's payment had failed. Not in our payment service — the request entered our API, then vanished. Three services later, someone had dropped the ball, and I had no idea who.

The logs told a sad story. Our gateway logged the incoming request. Our billing service logged a failure. Our email service logged a retry. But nothing connected them. I spent four hours stitching log lines together by timestamp and user ID, hoping nothing else happened at the same second.

The fix was embarrassingly simple: request IDs. One header, a few lines of middleware, and every log line in every service suddenly belongs to the same story.

Here's how to do it properly, with code you can actually run.

What a request ID is

A request ID (also called correlation ID or trace ID) is a unique identifier generated when a request first enters your system. It travels with the request through every service, every queue, every retry. Every log line, error report, and response carries it.

When something breaks, you grab the ID from the error message and grep once. Done. No archaeology.

Step 1: Generate it at the edge (and only at the edge)

The rule that matters most: only the entry point generates a new ID. Every downstream service must forward the ID it received, never generate its own.

In FastAPI, that looks like this:

import uuid
from fastapi import FastAPI, Request
from starlette.middleware.base import BaseHTTPMiddleware

REQUEST_ID_HEADER = "X-Request-ID"

class RequestIDMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next):
        request_id = request.headers.get(REQUEST_ID_HEADER)
        # Accept an external ID only if it looks sane.
        # This lets your clients correlate on their side too.
        if not request_id or len(request_id) > 128:
            request_id = str(uuid.uuid4())
        request.state.request_id = request_id
        response = await call_next(request)
        response.headers[REQUEST_ID_HEADER] = request_id
        return response

app = FastAPI()
app.add_middleware(RequestIDMiddleware)

Two details people get wrong:

Always return the ID in the response. Your support team's first question to a user becomes "what error code and request ID did you see?" instead of "tell me exactly what you clicked at 2 AM."
Cap the length of incoming IDs. If you blindly echo a header back, you've created a log injection vector — someone sends a 10 KB header full of newlines and your logs get forged entries. Validate or regenerate.

Step 2: Forward it on every outbound call

Generating the ID is useless if it dies at the service boundary. Every HTTP client needs to pass it along.

With httpx, the clean way is a transport hook or a small helper:

import httpx

def client_with_request_id(request_id: str) -> httpx.AsyncClient:
    return httpx.AsyncClient(
        headers={"X-Request-ID": request_id},
        timeout=10.0,
    )

# In your route handler:
async def charge_customer(request: Request):
    client = client_with_request_id(request.state.request_id)
    async with client:
        resp = await client.post(
            "http://billing-service:8080/v1/charges",
            json={"amount_cents": 4999},
        )

The alternative — writing a manual headers= dict at every call site — fails the moment a developer forgets. One shared helper means you forget once, fix it once.

Step 3: Put the ID in every log line

A request ID that isn't in your logs is a request ID that doesn't exist.

Using structlog (or stdlib logging with a filter), bind the ID once per request and stop thinking about it:

import structlog

logger = structlog.get_logger()

@app.middleware("http")
async def bind_request_id(request: Request, call_next):
    structlog.contextvars.bind_contextvars(
        request_id=request.state.request_id,
        path=request.url.path,
        method=request.method,
    )
    try:
        return await call_next(request)
    finally:
        structlog.contextvars.unbind_contextvars(
            "request_id", "path", "method"
        )

Now every log line in that request's lifecycle — including in functions that never touch the request object — carries the ID:

2026-09-06T14:22:31Z [info] charge initiated request_id=9f2c... path=/v1/charges
2026-09-06T14:22:31Z [error] upstream timeout request_id=9f2c... service=billing

If you're on stdlib logging, a logging.Filter that reads a thread-local or contextvars value works the same way. The mechanism matters less than the invariant: every handler has access to the ID without threading it through every function signature.

Step 4: Carry it through async work and retries

This is where most hand-rolled systems break. If your request enqueues a background job and the job runs five minutes later, the ID must travel with the job payload:

job = {
    "type": "send_receipt",
    "request_id": request.state.request_id,  # not optional
    "charge_id": charge.id,
}

Same for retries. When your retry library fires attempt number three, it should reuse the original ID — a new ID per attempt would scatter one logical operation across three unrelated trace lines. Keep attempt counters in a separate header (X-Retry-Count) if you need them.

Message queues (RabbitMQ, SQS, Kafka) have standard header fields for this. When you consume a message, read the ID from the message headers, bind it to your logging context, and forward it on anything you produce downstream.

What this actually buys you

After we shipped this, a real incident went like this:

Customer reports a failed order with the request ID from their error screen.
I run one grep across our log aggregator.
Twenty lines, in order: gateway → billing → retry → billing again → email → success on the third attempt.

The bug (a flaky connection to our payment provider, masked by a silent timeout) was visible in under a minute. The old process would have taken an hour of timestamp-matching.

The honest numbers from our team: average time-to-understanding for a production error dropped from roughly 45 minutes to under 10. Most of that wasn't clever tooling — it was one header, consistently propagated.

Five rules to remember

Only the edge generates IDs. Downstream services forward, never invent.
Return the ID in the response. Your users become your first-line debuggers.
Validate incoming IDs (length, charset) before echoing them anywhere.
Bind the ID to logging context once — don't thread it through signatures.
Persist it across async boundaries: queues, retries, and scheduled jobs.

If your API doesn't have request IDs yet, this is a two-hour change that pays for itself the first time you debug a distributed failure. Start with the middleware and the response header; add propagation as you touch each service.

The alternative is another 2 AM session, grepping logs by timestamp, hoping nobody else's request interleaved with yours. I've been there. It's not worth it.

Building APIs and writing about the parts nobody warns you about. This article is part of my series on API resilience patterns — feedback and war stories welcome in the comments.

3 API Versioning Strategies That Won't Get You Yelled At (And the One I Regret)

Sir Max — Sun, 06 Sep 2026 02:27:23 +0000

3 API Versioning Strategies That Won't Get You Yelled At (And the One I Regret)

Years ago I shipped a tiny endpoint: POST /users/{id}/status. Simple enough — clients set a user's status, done. Three months later, the product team needed that same call to also carry a reason field, and I made the classic mistake: I changed the request body in place, kept the same URL, and told exactly one customer about it. The other four integrations with auto-retry kept hammering the old shape. At 2am, one of them went down in production and the on-call rotation taught me a lesson I still think about: API versioning isn't about naming conventions. It's about a promise you make to people you've never met.

Here are the three strategies I've used in real services, what each one costs you, and the one approach I now regret ever touching.

Strategy 1: Path Versioning — The Boring Choice That Works

GET /v1/users/42
GET /v2/users/42

This is the versioning scheme everyone understands, because it's visible in every log line, every browser tab, and every Stack Overflow snippet.

What it looks like in FastAPI:

from fastapi import FastAPI, APIRouter

app = FastAPI()

def make_router(version: str):
    return APIRouter(prefix=f"/{version}/users", tags=[f"users-{version}"])

# v1 router
v1 = make_router("v1")

@v1.get("/{user_id}")
def get_user_v1(user_id: int):
    return {"id": user_id, "full_name": "Ada Lovelace"}  # v1 shape

# v2 router — same route, new shape
v2 = make_router("v2")

@v2.get("/{user_id}")
def get_user_v2(user_id: int):
    return {"id": user_id, "name": {"first": "Ada", "last": "Lovelace"}}  # v2 shape

app.include_router(v1)
app.include_router(v2)

The good:

Clients can see the version in a curl command — no hidden state.
CDN and cache keys work naturally (/v1/ and /v2/ never collide).
Debugging is trivial: "which version is the app calling?" → look at the URL.

The bad — and it's real:

Every version is a code fork you maintain forever. v1 doesn't disappear because two customers refuse to migrate. Two years in, you're patching a bug in three versions of the same serializer.
URLs get noisy, and documentation has to explain the same endpoint twice.

When to use it: any public API with third-party consumers — people you've never met, running code you can't see. This is 80% of the APIs I've built, and path versioning is what I'd reach for first every time.

Strategy 2: Media-Type (Header) Versioning — Clean URLs, Hidden Complexity

GET /users/42
Accept: application/vnd.myapi.v2+json

The resource stays at one URL; the client declares which representation it understands via the Accept header. This is what Stripe, GitHub, and a bunch of serious API shops do.

Minimal FastAPI implementation:

from fastapi import FastAPI, Request, HTTPException

app = FastAPI()
LATEST = "v2"

@app.get("/users/{user_id}")
async def get_user(request: Request, user_id: int):
    accept = request.headers.get("accept", "")
    if "vnd.myapi.v2+json" in accept:
        return {"id": user_id, "name": {"first": "Ada", "last": "Lovelace"}}
    if "vnd.myapi.v1+json" in accept or accept == "*/*":
        return {"id": user_id, "full_name": "Ada Lovelace"}
    if not accept:
        # Missing Accept header = oldest stable version, never the newest
        return {"id": user_id, "full_name": "Ada Lovelace"}
    raise HTTPException(status_code=406)

The good:

One URL per resource — no link rot, no duplicated cache entries.
You can ship a v3 that changes only one field's semantics while everything else stays identical.

The bad — the mistake that bit me:

Version is invisible in logs unless you deliberately log the Accept header.
Browsers and casual tooling send */*, so you must define what "no version asked" means.
My real mistake: I defaulted missing Accept to the newest version. Old clients that never sent the header silently received the new shape and broke in subtle ways — no error, just wrong payloads. The rule I follow now: missing Accept → serve the oldest stable version. It's the only choice that can't break anyone who was already working.

When to use it: you have a long-lived public resource model, few breaking changes per year, and clients sophisticated enough to set headers. Use it with the "oldest on missing" rule, or you will eat my 2am lesson.

Strategy 3: Query Parameter Versioning — The One I Regret

GET /users/42?v=2

It looks convenient — no header plumbing, no URL restructuring. I used it once for an internal admin tool with exactly one consumer team, and even there it annoyed me:

Cache keys now include the query string, so CDN and HTTP caches treat ?v=1 and ?v=2 as different resources — which means duplicate storage and cold misses for every version flip.
Shared links and bookmarks silently pin whatever version was current when the link was created.
SEO and API discovery tools get confused by multiple URLs for the same resource.
It encourages "just add a param" thinking, and that's how you end up with ?v=2&expand=all&legacy=true&experimental=1.

Where it's actually fine: internal services with one known consumer, feature-flag-style rollouts, or a private B2B integration where you control both ends. For a public API? I wouldn't do it again.

The Part Nobody Talks About: Deprecation Is a Product Decision

The URL scheme matters less than your exit policy. Breaking a client isn't a technical event — it's a customer-relationship event. My current playbook:

Additive changes never get a version bump. New field? Add it. New endpoint? Add it. Old clients that ignore unknown fields keep working. This one rule eliminates most "breaking" changes before they happen.
Breaking changes get an overlap window of at least 6 months. v2 and v1 run side by side. You eat the maintenance cost — that's the price of the promise.
Deprecation is announced in the response itself, not just a blog post:

HTTP/1.1 200 OK
Content-Type: application/json
Deprecation: true
Sunset: Sat, 31 Jan 2027 23:59:59 GMT
Link: <https://api.example.com/v2/users>; rel="successor-version"

The Sunset header tells automated clients when the old version dies — their monitoring can read it and schedule the migration. A small FastAPI middleware makes this painless:

from fastapi import FastAPI
from starlette.middleware.base import BaseHTTPMiddleware
import datetime as dt

app = FastAPI()

V1_SUNSET = "Sat, 31 Jan 2027 23:59:59 GMT"

class DeprecationHeaders(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        response = await call_next(request)
        if request.url.path.startswith("/v1/"):
            response.headers["Deprecation"] = "true"
            response.headers["Sunset"] = V1_SUNSET
            response.headers["Link"] = '<https://api.example.com/v2/users>; rel="successor-version"'
        return response

app.add_middleware(DeprecationHeaders)

Migration guides with before/after diffs. For each breaking change, I publish the exact payload change and a search-and-replace-level migration note. Sounds obvious — almost nobody does it.

What I'd Pick Again

Public API, unknown consumers → path versioning, additive changes by default, 6+ month overlaps.
Long-lived resource model, few breaks, header-savvy clients → media-type versioning with "oldest on missing Accept".
Internal service, one consumer → query param is tolerable; otherwise skip it.

And the meta-lesson: the real cost of versioning is not choosing a URL scheme — it's maintaining N versions of the same code. So the best versioning strategy is to version as rarely as possible, by making additive changes the default and treating every breaking change like the deployment incident it can become.

If you've ever been woken up by an API client you didn't know existed, you already know what I mean.

gRPC vs REST: What the Benchmarks Don't Tell You (I Ported a Real Service to Find Out)

Sir Max — Thu, 03 Sep 2026 02:23:17 +0000

gRPC vs REST: What the Benchmarks Don't Tell You

I spent the better part of a month porting one real service from REST/JSON to gRPC, and the result surprised me. Not because gRPC was slow — it wasn't — but because the numbers I kept seeing in benchmark posts turned out to measure the wrong thing.

This is the honest before/after: what got faster, what got worse, and how I'd decide differently next time.

The service I moved

A small internal API that serves paginated lists of objects with nested fields. A typical response was 80–200 KB of JSON, built by a Python backend, consumed by a web app and two mobile clients. Nothing exotic. Traffic was moderate, but the list endpoints were chatty — every screen load fired three or four requests, and on mobile networks the payload size was starting to hurt.

The plan: define a .proto, generate clients, and swap the transport while keeping the API surface semantically identical.

What the change actually looked like

The contract-first part is genuinely nice. One file, one source of truth:

syntax = "proto3";

package inventory.v1;

service ItemService {
  rpc ListItems(ListItemsRequest) returns (ListItemsResponse);
  rpc StreamItems(ListItemsRequest) returns (stream Item); // later
}

message ListItemsRequest {
  string cursor = 1;
  int32 page_size = 2;
}

message Item {
  string id = 1;
  string name = 2;
  map<string, string> attributes = 3;
  int64 updated_at = 4;
}

The Python server side:

import grpc
from concurrent import futures
import inventory_pb2, inventory_pb2_grpc

class ItemService(inventory_pb2_grpc.ItemServiceServicer):
    def ListItems(self, request, context):
        items = query_items(cursor=request.cursor, limit=request.page_size)
        return inventory_pb2.ListItemsResponse(
            items=[to_proto(i) for i in items],
            next_cursor=items.next_cursor or "",
        )

server = grpc.server(futures.ThreadPoolExecutor(max_workers=16))
inventory_pb2_grpc.add_ItemServiceServicer_to_server(ItemService(), server)
server.add_insecure_port("[::]:50051")
server.start()
server.wait_for_termination()

The client call is roughly as compact as the old requests.get version:

import grpc
import inventory_pb2, inventory_pb2_grpc

with grpc.insecure_channel("service.internal:50051") as channel:
    stub = inventory_pb2_grpc.ItemServiceStub(channel)
    resp = stub.ListItems(inventory_pb2.ListItemsRequest(page_size=50))
    for item in resp.items:
        print(item.name, item.updated_at)

Nothing about this part was painful. The code generation, the typed messages, the IDE autocomplete on response fields — all of it felt like an upgrade over hand-rolled dictionaries and response.json()["data"]["items"][0]["id"].

What actually got faster

Measured on the same box, same database, realistic payloads:

Payload size: about 65% smaller. Protobuf's binary encoding wins hard on repetitive nested structures. This was the win I was actually chasing, and it delivered.
Serialization cost: lower. Python's json is not slow, but protobuf C-accelerated encoding beat it by a wide margin on the 200 KB responses.
End-to-end latency (p95): about 12% better. That's it. Not 3x, not 10x. Because for a single request over the public internet, the dominant cost is network round-trip and the server's actual work — not the wire format.

The unsung hero was HTTP/2 multiplexing. When a client fires several requests over one connection, they no longer queue behind each other. On mobile that collapsed three sequential round trips into one connection's worth of overlap. That showed up more in "the app feels snappier" than in any single-request benchmark.

Where it hurt — the parts benchmarks skip

Here's the stuff no latency chart warns you about:

1. Debugging gets worse. curl is useless against a gRPC endpoint. You need grpcurl with server reflection enabled, or a GUI client, or carefully crafted test scripts. Every incident now requires one more tool in the chain. When a partner reported "the API is broken," my first step went from one command to three.

2. Browsers can't call it. No browser speaks gRPC natively. Web clients need grpc-web or a translation proxy (Envoy, grpc-gateway). Your frontend devs cannot just open the network tab and hit the endpoint — which means the "REST-like" surface they see is now a generated artifact, and the error messages they read are whatever the proxy chose to emit.

3. My CDN caching silently died. Read-heavy list endpoints used to sit behind an edge cache keyed on URL + query string. gRPC uses POST semantics over a binary body — edge caching of those responses is awkward or impossible. I had to rebuild caching at the application layer. Nobody mentions the caching regression in a benchmark post.

4. Schema discipline becomes a process. Protobuf field numbers must never be reused, and deprecated fields linger for compatibility. That's correct and good — but it means a code review now includes "is this a new field number or a recycled one?" JSON tolerated sloppy additions forever; proto punishes them loudly and immediately.

5. Infra setup has edges. The load balancer had to speak HTTP/2 (h2c or TLS with ALPN). Health checks, timeouts, and retry policies all had to be reconfigured for a binary protocol that doesn't answer a plain GET /healthz the way you'd expect. Adding grpc_health_v1 was easy; remembering to do it was the real cost.

Where gRPC genuinely shines

With all that said, I'd still choose gRPC again — for the right service:

Streaming. Server-streaming is dramatically better than my old pattern of paginated polling for large exports. Built-in backpressure and cancellation beat anything I was hand-rolling with SSE.
Contract-first teams. When five microservices in three languages share a boundary, generated clients eliminate a whole class of "the field name changed in the docs again" bugs.
Internal traffic. When both ends are yours, on your network, with your load balancer — the debugging and browser objections mostly disappear.
Typed errors. google.rpc.Status with rich detail is a real upgrade over guessing which HTTP code maps to which internal failure.

The decision guide I wish I'd had

Choose gRPC when:

Both ends are services you control (internal microservices, service mesh, B2B server-to-server)
You need streaming or very high throughput
Payload size on constrained networks is a measured pain point
You have several language teams that will benefit from codegen

Choose REST + JSON when:

Browsers and third parties are the primary consumers
CDN/edge caching is doing real work for you
Debuggability and simplicity beat wire efficiency
You're a small team and don't want to own the gateway/proxy stack

The middle path is popular for good reason: keep gRPC between your internal services, and expose REST/OpenAPI at the public edge through a gateway that translates. You pay for the proxy, but you keep curl-able, cache-able public endpoints.

Bottom line

Benchmarks compare bytes. Production compares developer days. My honest numbers: payload −65%, p95 latency −12%, first-week debugging friction up noticeably, CDN caching gone for those endpoints. gRPC is not "faster REST" — it's a different tool with different sharp edges.

Next time I'm choosing a transport, the question I ask first isn't "which is faster?" It's "who consumes this API, and where does the latency actually live?" That question would have saved me a month.

Building and running APIs for a living. Writing about what I learn along the way.

Streaming Responses with Server-Sent Events: A Practical Guide (with Python Code)

Sir Max — Wed, 02 Sep 2026 02:42:14 +0000

Streaming Responses with Server-Sent Events: A Practical Guide (with Python Code)

When I built my first "live" feature — a notification feed that had to update the moment new data arrived — I did what most people do: I polled. Every 10 seconds, the client hit the notifications endpoint and hoped something changed.

It worked. It also wasted bandwidth, hammered my database with identical queries, and added 5–10 seconds of latency to every update. When I switched to Server-Sent Events (SSE), the same feature became one HTTP connection, instant updates, and a client that reconnects on its own.

Here's how SSE works, when to use it (and when not to), and a working FastAPI + JavaScript implementation you can copy.

What SSE actually is

Server-Sent Events is a standard (part of HTML5) that lets a server push data to a client over a single plain HTTP connection. The server keeps the response open and writes events as they happen.

The wire format is deceptively simple. The Content-Type is text/event-stream, and each event looks like this:

event: progress
id: 3
data: {"step": 3, "percent": 60}

That's it. event: names the type, id: gives a sequence number used for reconnection, and data: is the payload. A blank line separates events. Because it's just HTTP, it works through firewalls and proxies without any protocol upgrade — unlike WebSockets, which need a ws:// handshake and a persistent bidirectional socket.

SSE vs WebSocket: which one do you need?

Short version: if the server needs to push to the client, SSE is usually enough. If the client also needs to push back (chat, multiplayer games, collaborative editing), you need WebSockets.

Concern	SSE	WebSocket
Direction	Server → client only	Bidirectional
Protocol	Plain HTTP	ws:// upgrade handshake
Auto-reconnect	Built into EventSource	You build it
Event IDs	Built-in (`id:`)	You build it
Firewalls/proxies	Works like a normal request	Sometimes blocked or buffered
Binary data	Text only	Yes
Code complexity	Low	Moderate

For dashboards, live feeds, notifications, progress bars, and token streaming, SSE is the simpler tool. I've watched teams adopt WebSockets for a "live status bar" and end up writing a custom reconnection loop, a heartbeat timer, and a load balancer config they're afraid to touch. SSE gave me all of that for free.

A working server in FastAPI

FastAPI makes this clean. Here's an endpoint that streams job progress:

import asyncio
import json
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse

app = FastAPI()

async def event_stream(request: Request):
    # Resume from the last event the client saw, if any
    last_id = request.headers.get("last-event-id")
    start = int(last_id) + 1 if last_id else 1

    for i in range(start, 6):
        # Respect client disconnect
        if await request.is_disconnected():
            break
        payload = {"step": i, "percent": i * 20}
        # SSE format: event name, id, data, blank line
        yield f"event: progress\nid: {i}\ndata: {json.dumps(payload)}\n\n"
        await asyncio.sleep(1)

    yield "event: done\ndata: {}\n\n"

@app.get("/progress")
async def progress(request: Request):
    return StreamingResponse(
        event_stream(request),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no",   # disable nginx buffering
            "Connection": "keep-alive",
        },
    )

Three details that matter:

request.is_disconnected() — if the client closes the tab, the generator stops instead of running forever for a ghost connection.
X-Accel-Buffering: no — if nginx sits in front, it buffers responses by default and your "real-time" feed arrives in 4KB chunks several seconds late. This header disables buffering for the stream.
Cache-Control: no-cache — proxies must never cache a never-ending response.

The last-event-id check is the resume trick: when EventSource reconnects, it automatically sends the last id: it received, so you can continue from where the client left off instead of replaying everything.

The client: about ten lines of JavaScript

const feed = document.getElementById("feed");
const source = new EventSource("/progress");

source.addEventListener("progress", (e) => {
  const data = JSON.parse(e.data);
  feed.textContent = `Step ${data.step}: ${data.percent}%`;
});

source.addEventListener("done", () => {
  source.close();
  feed.textContent = "All done!";
});

The browser handles reconnection for you. If the connection drops, EventSource retries automatically with built-in backoff, and it sends the last id: it saw on each retry.

Pitfalls I actually hit

1. Buffering killed my first demo. It worked instantly on localhost. Deployed behind nginx — 15-second stalls. Default proxy_buffering on collects the whole stream before forwarding anything. The header above fixed it. Test behind your real proxy, not just localhost.

2. Mobile networks drop idle connections. Cellular connections can time out after 30–60 seconds of silence. If nothing happens for a while, send a comment line every 15–30 seconds to keep it alive:

: heartbeat

Comments are ignored by the client but keep the connection from being killed.

3. Reconnect storms after a deploy. When I restarted the server, every connected client reconnected at once and hammered the endpoint simultaneously. EventSource's backoff helps, but for a large fanout you want graceful connection draining on your server, and a small random delay if you control the client side. Also: don't return 500 on a disconnect — clients will retry and amplify the problem.

4. Auth is whatever your app already uses — with one catch. Cookies work for same-origin. For cross-origin you can use credentials or pass a token as a query parameter. But note that EventSource cannot set custom headers — if your API only accepts Authorization headers, you'll need a fetch-based SSE reader or a WebSocket instead.

When NOT to use SSE

Client → server messages (chat, games): SSE is one-way.
Binary data: SSE is text-only. Send a URL, not the bytes.
Very high fanout (10k+ concurrent connections): one open connection per user is expensive. Consider multiplexing or a push service.
You need precise control over reconnection: EventSource's retry behavior is opaque. If you need fine-grained control, roll your own with fetch + ReadableStream.

Final thoughts

SSE has been around for over a decade and is still the most underrated tool in the streaming toolbox. It's one plain HTTP response, supported by every browser, with reconnection built in, and it takes about thirty lines with FastAPI. If your use case is server-to-client events — notifications, progress, live updates, token streaming — try SSE before you reach for a WebSocket.

Why I Switched My API from Offset to Cursor Pagination (and When You Shouldn't)

Sir Max — Mon, 24 Aug 2026 02:21:42 +0000

Why I Switched My API from Offset to Cursor Pagination (and When You Shouldn't)

Two years ago I shipped my first public API with the simplest pagination I could think of:

GET /api/items?page=1&per_page=20

It worked. For about four months. Then the dataset crossed a few hundred thousand rows, a customer reported "missing" orders, and I spent a weekend learning the hard way why offset pagination quietly breaks at scale.

Here's what I learned, with working code for both approaches and the exact numbers that convinced me to switch.

The two ways to paginate

Offset pagination skips a number of rows:

SELECT * FROM orders ORDER BY id LIMIT 20 OFFSET 4000;

Cursor pagination (a.k.a. keyset pagination) remembers the last item you saw and asks for "everything after it":

SELECT * FROM orders WHERE id > 4000 ORDER BY id LIMIT 20;

Same result set, completely different behavior once your data grows or changes.

Problem #1: Offset gets slow, fast

This is the one everyone knows. In PostgreSQL, MySQL, and SQLite, OFFSET 100000 doesn't skip 100,000 rows — the database still reads them, then discards them. The query cost grows linearly with the offset.

I benchmarked this on a 1.2M-row table in PostgreSQL 15 (cold cache, standard btree index on id):

Query	Response time
`LIMIT 20 OFFSET 0`	3 ms
`LIMIT 20 OFFSET 10,000`	45 ms
`LIMIT 20 OFFSET 100,000`	310 ms
`LIMIT 20 OFFSET 1,000,000`	2.4 s

Meanwhile the cursor version stayed flat:

Query	Response time
`WHERE id > 0 LIMIT 20`	3 ms
`WHERE id > 400,000 LIMIT 20`	3 ms
`WHERE id > 950,000 LIMIT 20`	3 ms

An index seek is O(log n). A big offset is O(offset). At page 50,000 your users are waiting two and a half seconds for a page that should take milliseconds.

Problem #2: Offset double-counts and skips under writes

This is the one that bit me, and it's sneakier.

Imagine 100 items, sorted newest-first. A user loads page 1 (items 1-20). Between their request and their click on "next", three new items arrive. Page 2 with OFFSET 20 now returns items that were originally on page 1 — the user sees duplicates. Reverse it (items being deleted) and they see gaps, or worse, think your API is broken and loses data.

I had a customer's sync job ingest the same 20 orders twice because of exactly this. It took three support emails to figure out it was our pagination, not their code.

Cursor pagination doesn't have this problem: the cursor is the last row you actually received, so "everything after this" is always correct, no matter how many rows were inserted or deleted in between.

The cursor implementation, in FastAPI

The simplest version uses the primary key:

from fastapi import FastAPI, Query
from sqlalchemy import select
from sqlalchemy.orm import Session

app = FastAPI()

@app.get("/orders")
def list_orders(
    db: Session,
    cursor: int | None = Query(default=None, description="ID of the last order from the previous page"),
    per_page: int = Query(default=20, ge=1, le=100),
):
    stmt = select(Order).order_by(Order.id.desc()).limit(per_page + 1)
    if cursor is not None:
        stmt = stmt.where(Order.id < cursor)
    rows = db.scalars(stmt).all()

    has_more = len(rows) > per_page
    rows = rows[:per_page]

    next_cursor = rows[-1].id if has_more and rows else None
    return {
        "data": [{"id": o.id, "amount": o.amount} for o in rows],
        "next_cursor": next_cursor,
        "has_more": has_more,
    }

The trick: fetch per_page + 1 rows. If you got more than you asked for, there's a next page — that's your has_more, no extra count query needed. The cursor is the ID of the last row you returned.

Client side it looks like this:

cursor = None
while True:
    params = {"per_page": 100}
    if cursor:
        params["cursor"] = cursor
    resp = requests.get("https://api.example.com/orders", params=params).json()
    process(resp["data"])
    if not resp["has_more"]:
        break
    cursor = resp["next_cursor"]

When you need a composite cursor

id alone only works if you're sorting by id. The moment you sort by something else (say created_at), plain WHERE created_at > X has a bug: ties. Two orders created in the same millisecond, and one of them silently disappears.

The standard fix is a composite cursor — the sort key plus the primary key:

WHERE (created_at, id) < (:cursor_created_at, :cursor_id)
ORDER BY created_at DESC, id DESC
LIMIT 21

In Python:

@app.get("/orders")
def list_orders(
    db: Session,
    cursor_created_at: str | None = Query(default=None),
    cursor_id: int | None = Query(default=None),
    per_page: int = 100,
):
    stmt = select(Order).order_by(Order.created_at.desc(), Order.id.desc()).limit(per_page + 1)
    if cursor_created_at and cursor_id:
        stmt = stmt.where(
            (Order.created_at, Order.id) < (cursor_created_at, cursor_id)
        )
    ...

The tuple comparison is the important part: it breaks ties deterministically, so no row is ever skipped or duplicated. Make sure you have a matching composite index (created_at DESC, id DESC) or this will be slow.

When you should keep offset pagination

Cursor pagination is not a free win. Be honest about the tradeoffs:

You need "jump to page 47". Cursors don't support that — there's no page number. Admin dashboards, spreadsheets, and "go to page N" UIs all want offset.
Your dataset is small and static. A 5,000-row config table with stable data doesn't need cursor. Offset is simpler to explain and debug.
You're paginating in memory or in a cache where offset is already O(1).
Sorting by arbitrary columns (name, price, last_updated) makes cursors awkward — you end up encoding whole tuples in opaque strings, which is fine, but it's real complexity.

My rule of thumb now: cursor is the default for any endpoint that can grow beyond ~10k rows or sees writes during pagination. Offset is for small, stable, or admin-only data.

What the response looks like

Don't return an opaque cursor string you then have to reverse-engineer in support tickets. Return something self-describing:

{
  "data": [...],
  "pagination": {
    "next_cursor": "eyJpZCI6MTIzNDV9",
    "has_more": true
  }
}

If you encode the cursor, base64 a small JSON object with a version field:

import base64, json

def encode_cursor(cursor_id: int, created_at: str) -> str:
    payload = json.dumps({"v": 1, "id": cursor_id, "created_at": created_at})
    return base64.urlsafe_b64encode(payload.encode()).decode()

def decode_cursor(cursor: str) -> dict:
    raw = base64.urlsafe_b64decode(cursor.encode())
    return json.loads(raw)

Version the payload ("v": 1) so you can change the format later without breaking in-flight paginators.

The bottom line

Offset pagination failed me in two ways: it got slow at scale, and it silently corrupted my pagination stream under concurrent writes. Cursor pagination fixed both, cost me one afternoon of migration, and the response format actually got simpler (next_cursor beats page + total_pages in every client I've written).

If you're building a public API today, start with cursor pagination. Your first customer with a large dataset will thank you — and you won't have to explain why their sync job ingested duplicate orders.

I build and operate APIs for a living. This is the kind of thing I wish someone had told me before I shipped my first one.

Building a Token Bucket Rate Limiter in Python: A Step-by-Step Guide

Sir Max — Fri, 21 Aug 2026 02:11:01 +0000

Building a Token Bucket Rate Limiter in Python: A Step-by-Step Guide

A few months ago, one of our internal API endpoints started getting hammered by a misconfigured client. It wasn't an attack — just a loop bug that fired requests every 200ms instead of every 2 seconds. By the time we noticed, the endpoint was serving 429s to everyone else and our database connection pool was exhausted.

The fix wasn't a bigger server. It was a rate limiter.

In this post I'll show you how to build a token bucket rate limiter from scratch in Python — no third-party dependencies, about 40 lines of code, and easy to drop into FastAPI, Flask, or any WSGI app. Even if you end up using a library, understanding the algorithm will help you configure it correctly.

Why a token bucket and not a fixed window?

The naive approach is a fixed window: allow N requests per minute, reset the counter when the minute ends. It's simple, but it has a well-known flaw — the boundary problem. If a client sends 100 requests at 0:59 and another 100 at 1:01, they've sent 200 requests in 2 seconds, even though you allowed only 100 per minute.

A token bucket smooths this out. It allows bursts up to a defined capacity, while enforcing a steady long-term rate. That's exactly how most production APIs (and cloud load balancers) behave: you get a burst budget, but the average rate stays capped.

The algorithm in plain English

Think of a bucket that holds tokens:

The bucket has a capacity (the maximum number of tokens it can hold).
Tokens are added at a fixed rate — say 10 tokens per second.
Each request removes one token (or more, if a request is expensive).
If the bucket is empty, the request is rejected with 429.

Because the bucket starts full, a client can fire off capacity requests instantly (the burst), then settles into the steady rate after that.

The implementation

Here's a thread-safe token bucket in pure Python:

import threading
import time


class TokenBucket:
    def __init__(self, rate: float, capacity: int):
        self.rate = rate          # tokens added per second
        self.capacity = capacity  # maximum burst size
        self.tokens = float(capacity)
        self.updated_at = time.monotonic()
        self._lock = threading.Lock()

    def consume(self, tokens: int = 1) -> bool:
        with self._lock:
            now = time.monotonic()
            elapsed = now - self.updated_at
            # Refill lazily: compute how many tokens accumulated since last check
            self.tokens = min(self.capacity, self.tokens + elapsed * self.rate)
            self.updated_at = now

            if self.tokens >= tokens:
                self.tokens -= tokens
                return True
            return False

Two details matter here:

time.monotonic() instead of time.time(). Wall-clock time jumps (NTP adjustments, manual clock changes) can make your limiter suddenly allow a huge burst or block everyone. Monotonic time only moves forward, which is what you want for measuring elapsed time.
Lazy refill. We don't run a background timer that adds tokens every second. We just compute how many tokens should have accumulated since the last request. This keeps the limiter cheap — the work happens only when a request arrives.

Wiring it into FastAPI

Dropping it into FastAPI takes a few lines:

from fastapi import FastAPI, HTTPException, Request

from token_bucket import TokenBucket

app = FastAPI()

# Allow 10 req/s sustained, with a burst of 30
bucket = TokenBucket(rate=10, capacity=30)


@app.middleware("http")
async def rate_limit(request: Request, call_next):
    client_id = request.client.host
    if not bucket.consume():
        raise HTTPException(
            status_code=429,
            detail="Too many requests",
            headers={"Retry-After": "1"},
        )
    return await call_next(request)

The Retry-After header is not optional polish — well-behaved clients read it and back off. Without it, clients just retry immediately and make the problem worse. A good limiter tells the caller when it's safe to try again.

Per-user buckets: the key matters more than the algorithm

Limiting by IP is the default, but it's often the wrong granularity. Users behind a corporate NAT share one IP — one heavy user can block everyone else at the office. Users on mobile networks cycle IPs constantly, so a per-IP limiter barely limits them at all.

What worked for us: limit by API key when the client is authenticated, fall back to IP for anonymous traffic. That way a single abusive integration gets throttled without collateral damage to unrelated users.

def client_id_for(request: Request) -> str:
    api_key = request.headers.get("X-Api-Key")
    if api_key:
        return f"key:{api_key}"
    return f"ip:{request.client.host}"

Making it work across multiple workers

The in-memory bucket above has a hidden trap: it's per-process. If you run four Uvicorn workers, each one has its own bucket with its own capacity — your effective limit becomes four times what you configured. For a small internal service that might be fine, but the moment you need a real guarantee, the bucket needs to live somewhere shared.

The pragmatic upgrade is Redis with a fixed window. It's not as smooth as a token bucket, but it's trivial to implement and it's a single shared store:

import time

import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=True)


def allow(client_id: str, limit: int = 100, window: int = 60) -> bool:
    key = f"rl:{client_id}:{int(time.time()) // window}"
    count = r.incr(key)
    if count == 1:
        r.expire(key, window)  # clean up after the window ends
    return count <= limit

Two gotchas with this version:

The window boundary problem I mentioned earlier still exists — that's the trade-off you accept for simplicity.
Set the TTL on the first increment (count == 1), not on every request, or you'll extend the window for active users indefinitely.

If you need distributed token-bucket semantics, the standard approach is a Lua script with sorted sets — but honestly, for most APIs the fixed window above is enough, and it's much easier to reason about.

Pitfalls I've hit so you don't have to

Rate limiting the wrong layer. Middleware that runs before auth can't key on the API key. Either parse the key in the middleware, or put the limiter inside the auth dependency.
Forgetting health checks and internal endpoints. You will eventually rate-limit your own health check and take down a load balancer's view of your service. Whitelist internal routes.
Too-large buckets. A capacity of 1000 with a rate of 10/s means the first 1000 requests in a burst all pass. If that's not what you want, keep the burst small and let the rate do the work.
No test coverage for the boundary. Write a test that fires capacity + 1 requests and asserts exactly one 429. It catches regressions the moment they happen.

When to use a library instead

For production, you'll probably reach for something like limits or slowapi (or your framework's built-in support). That's the right call — they handle edge cases I haven't covered here. But now you know what they're doing under the hood: the knobs they expose (rate, burst, storage backend) map directly to the concepts above. When a library's default settings feel wrong for your traffic, you'll know which parameter to touch.

Wrap-up

Rate limiting is one of those things that's invisible until it isn't. The good news is the core idea fits in ~40 lines: a token bucket, a lock, and a monotonic clock. Start with the in-memory version for a single-process service, switch to the Redis fixed window when you scale out, and key by user, not just by IP.

Your future self — and the person who gets paged at 2am — will thank you.

Time Zones in APIs: 5 Rules That Would Have Saved Me 3 Days of Debugging

Sir Max — Wed, 19 Aug 2026 02:06:44 +0000

A few years ago I shipped a booking feature that passed every test I ran locally. Then a user in Berlin wrote in: "My booking is showing the wrong time." Then one in Sydney. Then one in São Paulo.

Three days later I found the bug. I had been storing local server time as a naive datetime, and doing date arithmetic in Python where the server happened to run at UTC+0 — until a container rebuild picked up a different default timezone and everything silently shifted by eight hours.

Nothing crashed. No error was thrown. The data was just wrong, everywhere, retroactively.

Here are the five rules I now follow on every project. They would have saved me that weekend.

Rule 1: Store everything in UTC

Your database should hold instants in time, not wall-clock readings. An instant is "2026-08-19T07:30:00Z". A wall-clock reading is "7:30 AM" — which means a different instant in New York than it does in London.

In PostgreSQL, this is the difference between timestamptz and timestamp:

-- Good: timestamptz stores an absolute instant, stored internally as UTC
CREATE TABLE bookings (
  id           bigint PRIMARY KEY,
  starts_at    timestamptz NOT NULL,
  created_at   timestamptz NOT NULL DEFAULT now()
);

-- Bad: timestamp stores whatever you give it, no zone context
-- "2026-08-19 07:30:00" is ambiguous and will bite you later

timestamptz doesn't actually store a timezone — it normalizes everything to UTC and remembers the instant. The timezone you see is applied at display time. That's exactly what you want.

The single exception: if the user's intent is "9:00 AM every day, local to them" (an alarm, a recurring meeting), store the wall-clock time and the timezone identifier as separate columns. More on that in Rule 4.

Rule 2: Never use naive datetimes in application code

Python lets you shoot yourself in the foot with surprising politeness. datetime.utcnow() sounds like it gives you UTC — and it does, numerically — but it returns a naive object with no tzinfo attached. The moment you compare it against an aware datetime, you get a TypeError, or worse, silent wrongness in a library that assumes a zone.

from datetime import datetime, timezone

# Bad: naive, no timezone attached
now = datetime.utcnow()          # deprecated in 3.12 for exactly this reason

# Good: explicitly aware
now = datetime.now(timezone.utc)

# Convert a stored instant into a user's local time
from zoneinfo import ZoneInfo
berlin = ZoneInfo("Europe/Berlin")
local_time = now.astimezone(berlin)

Use zoneinfo (stdlib since Python 3.9) and pass IANA identifiers like "Europe/Berlin" or "America/Sao_Paulo". Never store or hardcode offsets like +02:00, because offsets change — Berlin is +01:00 in winter and +02:00 in summer. An offset is a snapshot; a timezone is a rulebook.

Rule 3: Use ISO 8601 on the wire

Pick one wire format and enforce it everywhere. ISO 8601 with an explicit offset (or a Z for UTC) is unambiguous and parseable by every language you'll touch.

# Serialize an instant correctly
import json
from datetime import datetime, timezone

def to_json(dt: datetime) -> str:
    return dt.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")

now = datetime.now(timezone.utc)
payload = json.dumps({"created_at": to_json(now)})
# {"created_at": "2026-08-19T07:30:00.123456Z"}

The worst format you can choose is one that looks fine but drops the offset — "2026-08-19 07:30:00". Every consumer will guess a timezone, and they'll guess differently. I've watched two services written in different languages disagree about "now" by a full day because one assumed UTC and the other assumed server-local.

Two practical notes:

Use Z for UTC, not +00:00. A few JSON parsers treat the former specially and the latter as a generic offset, and JavaScript's Date.parse has historically been inconsistent with offsets.
For JavaScript, don't hand-roll date parsing. Use new Date("2026-08-19T07:30:00Z") (ISO with Z is safe) and avoid "2026/08/19" or any locale-specific string, which varies by browser.

Rule 4: Separate "instant" from "intent"

This is the rule that separates people who've been burned from people who haven't.

Most timestamps are instants: when a record was created, when a payment went through. Store those as UTC (timestamptz), display them in the viewer's local zone.

But some timestamps are intents: a user wants their report generated "every Monday at 9 AM", or a meeting "at 3 PM my time". If you naively convert "3 PM in New York" to UTC and store only that, the meeting will drift by an hour every time daylight saving starts or ends.

Store both:

CREATE TABLE recurring_reports (
  id            bigint PRIMARY KEY,
  schedule_time time NOT NULL,          -- 09:00:00, wall-clock intent
  timezone      text NOT NULL,          -- 'America/New_York', IANA name
  last_run_at   timestamptz             -- the last actual instant it ran
);

schedule_time + timezone captures what the user meant. last_run_at captures what actually happened. Mixing the two is where recurring-event bugs live.

Rule 5: Test around DST transitions

If your test suite only ever runs in one timezone and never crosses a daylight-saving boundary, you are testing the one scenario that's guaranteed not to break.

import pytest
from datetime import datetime, timezone
from zoneinfo import ZoneInfo

def localize(dt_utc: datetime, tz_name: str) -> datetime:
    return dt_utc.astimezone(ZoneInfo(tz_name))

def test_dst_spring_forward():
    # 2026-03-29 01:30 UTC in Berlin -> 03:30 (clocks jumped from 02:00 to 03:00)
    instant = datetime(2026, 3, 29, 1, 30, tzinfo=timezone.utc)
    assert localize(instant, "Europe/Berlin").hour == 3

def test_dst_fall_back():
    # 2026-10-25 01:30 UTC in Berlin -> 02:30 (clocks fell back)
    instant = datetime(2026, 10, 25, 1, 30, tzinfo=timezone.utc)
    assert localize(instant, "Europe/Berlin").hour == 2

Add a couple of these to your CI matrix — run the whole suite once with TZ=America/Sao_Paulo and once with TZ=Pacific/Auckland — and you'll catch most timezone bugs before a customer in another hemisphere does.

The checklist I run before shipping anything that touches time

Every datetime column is timestamptz (or the equivalent in your stack), or a deliberate wall-clock + timezone pair.
No naive datetime anywhere in app code — datetime.now(timezone.utc), never utcnow().
One wire format: ISO 8601 with Z for UTC.
Recurring events store intent (time + IANA zone) separately from instants.
CI runs the test suite under at least two timezones, including a DST boundary.

None of this is exotic. It's the kind of boring discipline that feels like overkill right up until the weekend you lose to a bug that never throws an error. Learn it once, from my mistake, instead of from your own.

Stop Returning Bare 500s: A Practical Guide to RFC 9457 Problem Details

Sir Max — Tue, 18 Aug 2026 02:07:56 +0000

Stop Returning Bare 500s: A Practical Guide to RFC 9457 Problem Details

A few months ago I was integrating with a third-party payments API. Everything worked in their sandbox. The moment I flipped to production, a call returned HTTP 200 with this body:

{ "success": false, "msg": "error" }

No status code, no error type, no field hint. Just "error". It took me and their support team a full day to discover the real cause: my request had a transaction amount with one extra decimal place than their schema allowed.

That day cost me more than any bug I've written myself. And it wasn't a hard problem to prevent — the API just didn't tell me what was wrong in a way a machine (or a tired human) could act on.

Here's the thing: error responses are part of your API's contract, not an afterthought. If your happy path is well documented but your error path is a black box, integrators will burn hours, then quietly move to a competitor.

The problem with "just return a message"

Most APIs fall into one of three failure modes:

The bare 500. A generic server error with an empty body or an HTML error page. The client learns nothing.
The 200-that-isn't. Everything returns 200, and the real status is buried in a payload like { "code": 500 }. This breaks caching, retries, and every HTTP-aware tool in the chain.
The inconsistent shape. Sometimes {"error": "..."}, sometimes {"message": "..."}, sometimes {"detail": "..."}. Clients end up writing defensive parsers for all of them.

All three share the same root cause: there's no agreed shape for an error, so every endpoint invents its own.

Enter RFC 9457: Problem Details

RFC 9457 (the successor to the older RFC 7807) defines a standard JSON format for HTTP error responses. The magic is its simplicity — a problem detail looks like this:

{
  "type": "https://api.example.com/problems/insufficient-funds",
  "title": "Insufficient funds",
  "status": 402,
  "detail": "The account does not have enough balance to cover this transaction.",
  "instance": "/transactions/8f2a1c/attempts/41",
  "balance": 12.50,
  "required": 19.99
}

The fields:

type — a URI identifying the kind of problem. Machine-readable, stable over time.
title — a short, human-readable summary (should not change between occurrences of the same type).
status — the HTTP status code, duplicated for convenience.
detail — a human-readable explanation specific to this occurrence.
instance — a URI pointing to the specific resource that caused the problem.

Crucially, you can add extension members — like balance and required above — to carry domain-specific data. That's where the real power lives.

The response must use the Content-Type: application/problem+json header, which tells clients "this is a structured error, parse it accordingly."

A working example in FastAPI

Here's the naive way most Python services handle errors today:

from fastapi import FastAPI, HTTPException

app = FastAPI()

@app.post("/orders")
def create_order(order: dict):
    if order.get("quantity", 0) <= 0:
        raise HTTPException(status_code=400, detail="bad quantity")
    ...

The client gets {"detail": "bad quantity"}. That's fine for a human, but a client that wants to react programmatically has to string-match "bad quantity" — fragile and impossible to localize.

Let's replace it with Problem Details. First, a small exception class and a handler:

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

class Problem(Exception):
    def __init__(self, type_uri, title, status, detail, **extensions):
        self.type_uri = type_uri
        self.title = title
        self.status = status
        self.detail = detail
        self.extensions = extensions

app = FastAPI()

@app.exception_handler(Problem)
async def problem_handler(request: Request, exc: Problem):
    body = {
        "type": exc.type_uri,
        "title": exc.title,
        "status": exc.status,
        "detail": exc.detail,
    }
    body.update(exc.extensions)
    return JSONResponse(
        status_code=exc.status,
        content=body,
        headers={"Content-Type": "application/problem+json"},
    )

Now the endpoint can raise precise, typed errors:

@app.post("/orders")
def create_order(order: dict):
    qty = order.get("quantity", 0)
    if qty <= 0:
        raise Problem(
            type_uri="https://api.example.com/problems/invalid-quantity",
            title="Invalid quantity",
            status=422,
            detail="quantity must be a positive integer",
            field="quantity",
            received=qty,
        )

And a client gets:

{
  "type": "https://api.example.com/problems/invalid-quantity",
  "title": "Invalid quantity",
  "status": 422,
  "detail": "quantity must be a positive integer",
  "field": "quantity",
  "received": 0
}

The field and received extensions let a frontend highlight the exact input box without parsing prose.

Add a trace id for supportability

One extension I add to every error is a correlation id, so a support ticket can be traced back to a log line:

import uuid

@app.exception_handler(Problem)
async def problem_handler(request: Request, exc: Problem):
    body = {
        "type": exc.type_uri,
        "title": exc.title,
        "status": exc.status,
        "detail": exc.detail,
        "trace_id": request.state.trace_id,  # set by middleware
    }
    body.update(exc.extensions)
    return JSONResponse(
        status_code=exc.status,
        content=body,
        headers={"Content-Type": "application/problem+json"},
    )

When a user reports "it failed again," they can paste the trace_id and you jump straight to the logs. This single habit has cut our debugging time on integration issues by more than half.

Four pitfalls that still bite people

1. Leaking internals. A 500 that includes your stack trace, SQL query, or framework version is a gift to attackers. Log the internals, but return a generic problem detail for unexpected errors:

@app.exception_handler(Exception)
async def unhandled_handler(request: Request, exc: Exception):
    logger.exception("unhandled error", extra={"trace_id": request.state.trace_id})
    return JSONResponse(
        status_code=500,
        content={
            "type": "https://api.example.com/problems/internal-error",
            "title": "Internal server error",
            "status": 500,
            "detail": "Something went wrong. Reference this id with support.",
            "trace_id": request.state.trace_id,
        },
        headers={"Content-Type": "application/problem+json"},
    )

2. Returning 200 for errors. This breaks retries (a client can't tell transient from permanent), breaks caches, and confuses every monitoring tool you own. Use real status codes — that's what they're for.

3. Renaming your type URIs. The type field is your machine-readable contract. Once integrators code against invalid-quantity, renaming it is a breaking change. Treat it like an API version — version the URI (/problems/v2/invalid-quantity) if you must change its meaning.

4. Skipping the Content-Type header. Without application/problem+json, generic clients treat your body as plain JSON and lose the semantic "this is an error" signal. Some HTTP libraries will also fail to parse the body into a typed object.

The payoff

Standardizing on Problem Details didn't make our APIs slower or harder to build — it made them predictable. Integrators stopped pinging me with "what does this error mean?" because the answer was in the response. Support tickets started including a trace_id instead of a screenshot of a red toast.

If you only take one thing from this: write your error responses with the same care you write your success responses. Your integrators are your users too.

Have you hit an API with infuriatingly opaque errors? I'd love to hear the worst one in the comments.

The 3 Things That Make Webhooks Actually Work (With Python Code)

Sir Max — Mon, 17 Aug 2026 02:08:35 +0000

Webhooks look simple from the outside. Your service makes an HTTP POST to a customer's URL when something happens, and... that's it, right?

Then you run one in production and learn the hard way that webhooks are a distributed systems problem wearing a friendly costume. The customer's endpoint goes down. A request times out. The same event gets delivered twice. An attacker starts sending fake events to your endpoint.

I've built webhook delivery for two different products, and both times the naive version (just POST and forget) broke within a week. This post walks through the three things that make webhooks actually work — retries, signatures, and idempotency — with code you can copy.

The three hard problems

Delivery is not guaranteed. The receiver's server is down, their firewall drops the request, they deploy mid-flight. A single POST has maybe a 99% success rate, which sounds fine until you realize that means roughly one in a hundred events is silently lost.
Duplication and ordering. If you retry a failed delivery, you might deliver the same event twice. And if two workers deliver events concurrently, the receiver can see them out of order.
Security. Anyone who can reach the receiver's endpoint can POST a fake event. The receiver needs to verify the event actually came from you.

Let me handle each one with real code.

1. Retries with exponential backoff

A retry loop is table stakes, but the details matter. You want exponential backoff with jitter, a cap on attempts, and a cap on total time.

import time
import random
import requests

MAX_ATTEMPTS = 8
BASE_DELAY = 5  # seconds

def deliver_with_retries(url, payload, signature_header):
    attempt = 0
    while attempt < MAX_ATTEMPTS:
        try:
            resp = requests.post(
                url,
                json=payload,
                headers={"X-Signature": signature_header},
                timeout=10,
            )
            if resp.status_code < 500:
                # 2xx/3xx/4xx: don't retry client errors
                return resp.status_code
            # 5xx: server error, retry below
        except requests.exceptions.RequestException:
            # network error or timeout, retry below
            pass

        attempt += 1
        if attempt == MAX_ATTEMPTS:
            return None  # give up, log for manual review

        # exponential backoff with jitter
        delay = BASE_DELAY * (2 ** (attempt - 1))
        jitter = random.uniform(0, delay * 0.3)
        time.sleep(delay + jitter)

    return None

Two decisions in here matter more than the loop itself:

Only retry 5xx and network errors, never 4xx. If the receiver says 400 or 401, that's their bug, not a transient failure. Retrying a 400 just spams their logs and doesn't fix anything.
Add jitter. If a receiver goes down and every sender retries on the exact same schedule, they all hammer the endpoint at once the moment it comes back. Jitter spreads the retries out.

A real system would use a queue and a background worker instead of time.sleep, but the logic is identical — I kept it inline so it's easy to read.

2. Signatures so the receiver trusts you

The receiver needs to answer one question: did this request really come from the sender? The standard answer is an HMAC signature over the raw request body, using a shared secret.

import hashlib
import hmac
import json

def sign_payload(secret: bytes, payload: dict) -> str:
    body = json.dumps(payload, separators=(",", ":"), sort_keys=True)
    return hmac.new(secret, body.encode(), hashlib.sha256).hexdigest()

And on the receiver's side:

def verify_signature(secret: bytes, raw_body: bytes, header: str) -> bool:
    expected = hmac.new(secret, raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, header)

Three details that trip people up:

Sign the raw bytes, not the re-serialized JSON. JSON has no canonical form — whitespace and key order can differ. If the receiver parses the body and re-serializes it before verifying, the signature won't match. Sign the exact bytes that went over the wire.
Use hmac.compare_digest, not ==. == short-circuits and leaks timing information; compare_digest runs in constant time.
Rotate secrets. If a customer's secret leaks, they need to be able to revoke and rotate it without breaking everyone else.

3. Idempotency so duplicate delivery is harmless

Retries mean duplicates are inevitable. The fix is an idempotency key: every event carries a unique ID, and the receiver ignores events it has already processed.

import sqlite3

def handle_event(event_id: str, handler):
    # atomic claim: INSERT succeeds only if the ID is new
    conn = sqlite3.connect("events.db")
    try:
        conn.execute(
            "INSERT INTO processed_events (event_id) VALUES (?)", (event_id,)
        )
        conn.commit()
    except sqlite3.IntegrityError:
        return  # already processed, skip
    handler()

The INSERT with a unique constraint is the whole trick — it gives you an atomic "only process this once" guarantee even if two workers race on the same event.

What actually goes wrong in production

These are bugs I've personally shipped (and then fixed):

Out-of-order delivery. With concurrent workers and retries, event #2 can arrive before event #1. If the receiver assumes order, add a sequence field to the payload so they can detect and buffer out-of-order events.
Retry storms. A receiver with a slow endpoint backs up thousands of events, and your retries make it worse. Cap concurrent deliveries per receiver, and drop events into a dead-letter queue after N failed attempts instead of retrying forever.
No timeout on the receiving side. Set a connect timeout and a read timeout. A receiver that hangs the connection for 60 seconds is worse than one that fails fast.

Wrap-up

If you're adding webhooks to your API, ship all three from day one: retries with backoff and jitter, HMAC signatures over the raw body, and idempotency keys. Retrofitting any of them after customers have built against your API is twice the work.

And if you're consuming webhooks, verify the signature and deduplicate by event ID before you touch your database. Your sender will eventually deliver the same event twice — plan for it.

DEV Community: Sir Max

API Keys vs OAuth 2.0 vs JWT: How I Actually Choose (After Getting It Wrong Twice)

API Keys vs OAuth 2.0 vs JWT: How I Actually Choose (After Getting It Wrong Twice)

Start with two questions, not with a technology

API keys: the boring answer that's usually right

OAuth 2.0: for when someone else's user is involved

JWT: a format, not an authentication strategy

The details that actually cause incidents

The checklist I use now

ETags and Cache-Control for APIs: The Headers Most Teams Skip

ETags and Cache-Control for APIs: The Headers Most Teams Skip

Why caching an API is not caching a web page

ETag + If-None-Match: the 304 you're not sending

Generating an ETag that doesn't lie

Cache-Control: the directives that actually matter

The Vary header, and why auth breaks caching

Invalidation on writes

A minimal FastAPI example

What to measure before and after

The short list

4 Patterns That Keep Your API Online When a Third-Party Service Goes Down

Pattern 1: Cut failures off early with timeouts everywhere

Pattern 2: A circuit breaker so you stop hammering a dead provider

Pattern 3: Serve the last good response instead of an error page

Pattern 4: Fail over to a second provider — only where it is safe

Test the failure modes before the failure

The checklist I keep next to every integration

Trace Any API Request Across Your Stack: A Practical Guide to Request IDs

Trace Any API Request Across Your Stack: A Practical Guide to Request IDs

What a request ID is

Step 1: Generate it at the edge (and only at the edge)

Step 2: Forward it on every outbound call

Step 3: Put the ID in every log line

Step 4: Carry it through async work and retries

What this actually buys you

Five rules to remember

3 API Versioning Strategies That Won't Get You Yelled At (And the One I Regret)

3 API Versioning Strategies That Won't Get You Yelled At (And the One I Regret)

Strategy 1: Path Versioning — The Boring Choice That Works

Strategy 2: Media-Type (Header) Versioning — Clean URLs, Hidden Complexity

Strategy 3: Query Parameter Versioning — The One I Regret

The Part Nobody Talks About: Deprecation Is a Product Decision

What I'd Pick Again

gRPC vs REST: What the Benchmarks Don't Tell You (I Ported a Real Service to Find Out)

gRPC vs REST: What the Benchmarks Don't Tell You

The service I moved

What the change actually looked like

What actually got faster

Where it hurt — the parts benchmarks skip

Where gRPC genuinely shines

The decision guide I wish I'd had

Bottom line

Streaming Responses with Server-Sent Events: A Practical Guide (with Python Code)

Streaming Responses with Server-Sent Events: A Practical Guide (with Python Code)

What SSE actually is

SSE vs WebSocket: which one do you need?

A working server in FastAPI

The client: about ten lines of JavaScript

Pitfalls I actually hit

When NOT to use SSE

Final thoughts

Why I Switched My API from Offset to Cursor Pagination (and When You Shouldn't)

Why I Switched My API from Offset to Cursor Pagination (and When You Shouldn't)

The two ways to paginate

Problem #1: Offset gets slow, fast

Problem #2: Offset double-counts and skips under writes

The cursor implementation, in FastAPI

When you need a composite cursor

When you should keep offset pagination

What the response looks like

The bottom line

Building a Token Bucket Rate Limiter in Python: A Step-by-Step Guide

Building a Token Bucket Rate Limiter in Python: A Step-by-Step Guide

Why a token bucket and not a fixed window?

The algorithm in plain English

The implementation

Wiring it into FastAPI

Per-user buckets: the key matters more than the algorithm

Making it work across multiple workers

Pitfalls I've hit so you don't have to

The `Vary` header, and why auth breaks caching