DEV Community: Sir Max

Streaming Responses with Server-Sent Events: A Practical Guide (with Python Code)

Sir Max — Wed, 02 Sep 2026 02:42:14 +0000

Streaming Responses with Server-Sent Events: A Practical Guide (with Python Code)

When I built my first "live" feature — a notification feed that had to update the moment new data arrived — I did what most people do: I polled. Every 10 seconds, the client hit the notifications endpoint and hoped something changed.

It worked. It also wasted bandwidth, hammered my database with identical queries, and added 5–10 seconds of latency to every update. When I switched to Server-Sent Events (SSE), the same feature became one HTTP connection, instant updates, and a client that reconnects on its own.

Here's how SSE works, when to use it (and when not to), and a working FastAPI + JavaScript implementation you can copy.

What SSE actually is

Server-Sent Events is a standard (part of HTML5) that lets a server push data to a client over a single plain HTTP connection. The server keeps the response open and writes events as they happen.

The wire format is deceptively simple. The Content-Type is text/event-stream, and each event looks like this:

event: progress
id: 3
data: {"step": 3, "percent": 60}

That's it. event: names the type, id: gives a sequence number used for reconnection, and data: is the payload. A blank line separates events. Because it's just HTTP, it works through firewalls and proxies without any protocol upgrade — unlike WebSockets, which need a ws:// handshake and a persistent bidirectional socket.

SSE vs WebSocket: which one do you need?

Short version: if the server needs to push to the client, SSE is usually enough. If the client also needs to push back (chat, multiplayer games, collaborative editing), you need WebSockets.

Concern	SSE	WebSocket
Direction	Server → client only	Bidirectional
Protocol	Plain HTTP	ws:// upgrade handshake
Auto-reconnect	Built into EventSource	You build it
Event IDs	Built-in (`id:`)	You build it
Firewalls/proxies	Works like a normal request	Sometimes blocked or buffered
Binary data	Text only	Yes
Code complexity	Low	Moderate

For dashboards, live feeds, notifications, progress bars, and token streaming, SSE is the simpler tool. I've watched teams adopt WebSockets for a "live status bar" and end up writing a custom reconnection loop, a heartbeat timer, and a load balancer config they're afraid to touch. SSE gave me all of that for free.

A working server in FastAPI

FastAPI makes this clean. Here's an endpoint that streams job progress:

import asyncio
import json
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse

app = FastAPI()

async def event_stream(request: Request):
    # Resume from the last event the client saw, if any
    last_id = request.headers.get("last-event-id")
    start = int(last_id) + 1 if last_id else 1

    for i in range(start, 6):
        # Respect client disconnect
        if await request.is_disconnected():
            break
        payload = {"step": i, "percent": i * 20}
        # SSE format: event name, id, data, blank line
        yield f"event: progress\nid: {i}\ndata: {json.dumps(payload)}\n\n"
        await asyncio.sleep(1)

    yield "event: done\ndata: {}\n\n"

@app.get("/progress")
async def progress(request: Request):
    return StreamingResponse(
        event_stream(request),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no",   # disable nginx buffering
            "Connection": "keep-alive",
        },
    )

Three details that matter:

request.is_disconnected() — if the client closes the tab, the generator stops instead of running forever for a ghost connection.
X-Accel-Buffering: no — if nginx sits in front, it buffers responses by default and your "real-time" feed arrives in 4KB chunks several seconds late. This header disables buffering for the stream.
Cache-Control: no-cache — proxies must never cache a never-ending response.

The last-event-id check is the resume trick: when EventSource reconnects, it automatically sends the last id: it received, so you can continue from where the client left off instead of replaying everything.

The client: about ten lines of JavaScript

const feed = document.getElementById("feed");
const source = new EventSource("/progress");

source.addEventListener("progress", (e) => {
  const data = JSON.parse(e.data);
  feed.textContent = `Step ${data.step}: ${data.percent}%`;
});

source.addEventListener("done", () => {
  source.close();
  feed.textContent = "All done!";
});

The browser handles reconnection for you. If the connection drops, EventSource retries automatically with built-in backoff, and it sends the last id: it saw on each retry.

Pitfalls I actually hit

1. Buffering killed my first demo. It worked instantly on localhost. Deployed behind nginx — 15-second stalls. Default proxy_buffering on collects the whole stream before forwarding anything. The header above fixed it. Test behind your real proxy, not just localhost.

2. Mobile networks drop idle connections. Cellular connections can time out after 30–60 seconds of silence. If nothing happens for a while, send a comment line every 15–30 seconds to keep it alive:

: heartbeat

Comments are ignored by the client but keep the connection from being killed.

3. Reconnect storms after a deploy. When I restarted the server, every connected client reconnected at once and hammered the endpoint simultaneously. EventSource's backoff helps, but for a large fanout you want graceful connection draining on your server, and a small random delay if you control the client side. Also: don't return 500 on a disconnect — clients will retry and amplify the problem.

4. Auth is whatever your app already uses — with one catch. Cookies work for same-origin. For cross-origin you can use credentials or pass a token as a query parameter. But note that EventSource cannot set custom headers — if your API only accepts Authorization headers, you'll need a fetch-based SSE reader or a WebSocket instead.

When NOT to use SSE

Client → server messages (chat, games): SSE is one-way.
Binary data: SSE is text-only. Send a URL, not the bytes.
Very high fanout (10k+ concurrent connections): one open connection per user is expensive. Consider multiplexing or a push service.
You need precise control over reconnection: EventSource's retry behavior is opaque. If you need fine-grained control, roll your own with fetch + ReadableStream.

Final thoughts

SSE has been around for over a decade and is still the most underrated tool in the streaming toolbox. It's one plain HTTP response, supported by every browser, with reconnection built in, and it takes about thirty lines with FastAPI. If your use case is server-to-client events — notifications, progress, live updates, token streaming — try SSE before you reach for a WebSocket.

Why I Switched My API from Offset to Cursor Pagination (and When You Shouldn't)

Sir Max — Mon, 24 Aug 2026 02:21:42 +0000

Why I Switched My API from Offset to Cursor Pagination (and When You Shouldn't)

Two years ago I shipped my first public API with the simplest pagination I could think of:

GET /api/items?page=1&per_page=20

It worked. For about four months. Then the dataset crossed a few hundred thousand rows, a customer reported "missing" orders, and I spent a weekend learning the hard way why offset pagination quietly breaks at scale.

Here's what I learned, with working code for both approaches and the exact numbers that convinced me to switch.

The two ways to paginate

Offset pagination skips a number of rows:

SELECT * FROM orders ORDER BY id LIMIT 20 OFFSET 4000;

Cursor pagination (a.k.a. keyset pagination) remembers the last item you saw and asks for "everything after it":

SELECT * FROM orders WHERE id > 4000 ORDER BY id LIMIT 20;

Same result set, completely different behavior once your data grows or changes.

Problem #1: Offset gets slow, fast

This is the one everyone knows. In PostgreSQL, MySQL, and SQLite, OFFSET 100000 doesn't skip 100,000 rows — the database still reads them, then discards them. The query cost grows linearly with the offset.

I benchmarked this on a 1.2M-row table in PostgreSQL 15 (cold cache, standard btree index on id):

Query	Response time
`LIMIT 20 OFFSET 0`	3 ms
`LIMIT 20 OFFSET 10,000`	45 ms
`LIMIT 20 OFFSET 100,000`	310 ms
`LIMIT 20 OFFSET 1,000,000`	2.4 s

Meanwhile the cursor version stayed flat:

Query	Response time
`WHERE id > 0 LIMIT 20`	3 ms
`WHERE id > 400,000 LIMIT 20`	3 ms
`WHERE id > 950,000 LIMIT 20`	3 ms

An index seek is O(log n). A big offset is O(offset). At page 50,000 your users are waiting two and a half seconds for a page that should take milliseconds.

Problem #2: Offset double-counts and skips under writes

This is the one that bit me, and it's sneakier.

Imagine 100 items, sorted newest-first. A user loads page 1 (items 1-20). Between their request and their click on "next", three new items arrive. Page 2 with OFFSET 20 now returns items that were originally on page 1 — the user sees duplicates. Reverse it (items being deleted) and they see gaps, or worse, think your API is broken and loses data.

I had a customer's sync job ingest the same 20 orders twice because of exactly this. It took three support emails to figure out it was our pagination, not their code.

Cursor pagination doesn't have this problem: the cursor is the last row you actually received, so "everything after this" is always correct, no matter how many rows were inserted or deleted in between.

The cursor implementation, in FastAPI

The simplest version uses the primary key:

from fastapi import FastAPI, Query
from sqlalchemy import select
from sqlalchemy.orm import Session

app = FastAPI()

@app.get("/orders")
def list_orders(
    db: Session,
    cursor: int | None = Query(default=None, description="ID of the last order from the previous page"),
    per_page: int = Query(default=20, ge=1, le=100),
):
    stmt = select(Order).order_by(Order.id.desc()).limit(per_page + 1)
    if cursor is not None:
        stmt = stmt.where(Order.id < cursor)
    rows = db.scalars(stmt).all()

    has_more = len(rows) > per_page
    rows = rows[:per_page]

    next_cursor = rows[-1].id if has_more and rows else None
    return {
        "data": [{"id": o.id, "amount": o.amount} for o in rows],
        "next_cursor": next_cursor,
        "has_more": has_more,
    }

The trick: fetch per_page + 1 rows. If you got more than you asked for, there's a next page — that's your has_more, no extra count query needed. The cursor is the ID of the last row you returned.

Client side it looks like this:

cursor = None
while True:
    params = {"per_page": 100}
    if cursor:
        params["cursor"] = cursor
    resp = requests.get("https://api.example.com/orders", params=params).json()
    process(resp["data"])
    if not resp["has_more"]:
        break
    cursor = resp["next_cursor"]

When you need a composite cursor

id alone only works if you're sorting by id. The moment you sort by something else (say created_at), plain WHERE created_at > X has a bug: ties. Two orders created in the same millisecond, and one of them silently disappears.

The standard fix is a composite cursor — the sort key plus the primary key:

WHERE (created_at, id) < (:cursor_created_at, :cursor_id)
ORDER BY created_at DESC, id DESC
LIMIT 21

In Python:

@app.get("/orders")
def list_orders(
    db: Session,
    cursor_created_at: str | None = Query(default=None),
    cursor_id: int | None = Query(default=None),
    per_page: int = 100,
):
    stmt = select(Order).order_by(Order.created_at.desc(), Order.id.desc()).limit(per_page + 1)
    if cursor_created_at and cursor_id:
        stmt = stmt.where(
            (Order.created_at, Order.id) < (cursor_created_at, cursor_id)
        )
    ...

The tuple comparison is the important part: it breaks ties deterministically, so no row is ever skipped or duplicated. Make sure you have a matching composite index (created_at DESC, id DESC) or this will be slow.

When you should keep offset pagination

Cursor pagination is not a free win. Be honest about the tradeoffs:

You need "jump to page 47". Cursors don't support that — there's no page number. Admin dashboards, spreadsheets, and "go to page N" UIs all want offset.
Your dataset is small and static. A 5,000-row config table with stable data doesn't need cursor. Offset is simpler to explain and debug.
You're paginating in memory or in a cache where offset is already O(1).
Sorting by arbitrary columns (name, price, last_updated) makes cursors awkward — you end up encoding whole tuples in opaque strings, which is fine, but it's real complexity.

My rule of thumb now: cursor is the default for any endpoint that can grow beyond ~10k rows or sees writes during pagination. Offset is for small, stable, or admin-only data.

What the response looks like

Don't return an opaque cursor string you then have to reverse-engineer in support tickets. Return something self-describing:

{
  "data": [...],
  "pagination": {
    "next_cursor": "eyJpZCI6MTIzNDV9",
    "has_more": true
  }
}

If you encode the cursor, base64 a small JSON object with a version field:

import base64, json

def encode_cursor(cursor_id: int, created_at: str) -> str:
    payload = json.dumps({"v": 1, "id": cursor_id, "created_at": created_at})
    return base64.urlsafe_b64encode(payload.encode()).decode()

def decode_cursor(cursor: str) -> dict:
    raw = base64.urlsafe_b64decode(cursor.encode())
    return json.loads(raw)

Version the payload ("v": 1) so you can change the format later without breaking in-flight paginators.

The bottom line

Offset pagination failed me in two ways: it got slow at scale, and it silently corrupted my pagination stream under concurrent writes. Cursor pagination fixed both, cost me one afternoon of migration, and the response format actually got simpler (next_cursor beats page + total_pages in every client I've written).

If you're building a public API today, start with cursor pagination. Your first customer with a large dataset will thank you — and you won't have to explain why their sync job ingested duplicate orders.

I build and operate APIs for a living. This is the kind of thing I wish someone had told me before I shipped my first one.

Building a Token Bucket Rate Limiter in Python: A Step-by-Step Guide

Sir Max — Fri, 21 Aug 2026 02:11:01 +0000

Building a Token Bucket Rate Limiter in Python: A Step-by-Step Guide

A few months ago, one of our internal API endpoints started getting hammered by a misconfigured client. It wasn't an attack — just a loop bug that fired requests every 200ms instead of every 2 seconds. By the time we noticed, the endpoint was serving 429s to everyone else and our database connection pool was exhausted.

The fix wasn't a bigger server. It was a rate limiter.

In this post I'll show you how to build a token bucket rate limiter from scratch in Python — no third-party dependencies, about 40 lines of code, and easy to drop into FastAPI, Flask, or any WSGI app. Even if you end up using a library, understanding the algorithm will help you configure it correctly.

Why a token bucket and not a fixed window?

The naive approach is a fixed window: allow N requests per minute, reset the counter when the minute ends. It's simple, but it has a well-known flaw — the boundary problem. If a client sends 100 requests at 0:59 and another 100 at 1:01, they've sent 200 requests in 2 seconds, even though you allowed only 100 per minute.

A token bucket smooths this out. It allows bursts up to a defined capacity, while enforcing a steady long-term rate. That's exactly how most production APIs (and cloud load balancers) behave: you get a burst budget, but the average rate stays capped.

The algorithm in plain English

Think of a bucket that holds tokens:

The bucket has a capacity (the maximum number of tokens it can hold).
Tokens are added at a fixed rate — say 10 tokens per second.
Each request removes one token (or more, if a request is expensive).
If the bucket is empty, the request is rejected with 429.

Because the bucket starts full, a client can fire off capacity requests instantly (the burst), then settles into the steady rate after that.

The implementation

Here's a thread-safe token bucket in pure Python:

import threading
import time


class TokenBucket:
    def __init__(self, rate: float, capacity: int):
        self.rate = rate          # tokens added per second
        self.capacity = capacity  # maximum burst size
        self.tokens = float(capacity)
        self.updated_at = time.monotonic()
        self._lock = threading.Lock()

    def consume(self, tokens: int = 1) -> bool:
        with self._lock:
            now = time.monotonic()
            elapsed = now - self.updated_at
            # Refill lazily: compute how many tokens accumulated since last check
            self.tokens = min(self.capacity, self.tokens + elapsed * self.rate)
            self.updated_at = now

            if self.tokens >= tokens:
                self.tokens -= tokens
                return True
            return False

Two details matter here:

time.monotonic() instead of time.time(). Wall-clock time jumps (NTP adjustments, manual clock changes) can make your limiter suddenly allow a huge burst or block everyone. Monotonic time only moves forward, which is what you want for measuring elapsed time.
Lazy refill. We don't run a background timer that adds tokens every second. We just compute how many tokens should have accumulated since the last request. This keeps the limiter cheap — the work happens only when a request arrives.

Wiring it into FastAPI

Dropping it into FastAPI takes a few lines:

from fastapi import FastAPI, HTTPException, Request

from token_bucket import TokenBucket

app = FastAPI()

# Allow 10 req/s sustained, with a burst of 30
bucket = TokenBucket(rate=10, capacity=30)


@app.middleware("http")
async def rate_limit(request: Request, call_next):
    client_id = request.client.host
    if not bucket.consume():
        raise HTTPException(
            status_code=429,
            detail="Too many requests",
            headers={"Retry-After": "1"},
        )
    return await call_next(request)

The Retry-After header is not optional polish — well-behaved clients read it and back off. Without it, clients just retry immediately and make the problem worse. A good limiter tells the caller when it's safe to try again.

Per-user buckets: the key matters more than the algorithm

Limiting by IP is the default, but it's often the wrong granularity. Users behind a corporate NAT share one IP — one heavy user can block everyone else at the office. Users on mobile networks cycle IPs constantly, so a per-IP limiter barely limits them at all.

What worked for us: limit by API key when the client is authenticated, fall back to IP for anonymous traffic. That way a single abusive integration gets throttled without collateral damage to unrelated users.

def client_id_for(request: Request) -> str:
    api_key = request.headers.get("X-Api-Key")
    if api_key:
        return f"key:{api_key}"
    return f"ip:{request.client.host}"

Making it work across multiple workers

The in-memory bucket above has a hidden trap: it's per-process. If you run four Uvicorn workers, each one has its own bucket with its own capacity — your effective limit becomes four times what you configured. For a small internal service that might be fine, but the moment you need a real guarantee, the bucket needs to live somewhere shared.

The pragmatic upgrade is Redis with a fixed window. It's not as smooth as a token bucket, but it's trivial to implement and it's a single shared store:

import time

import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=True)


def allow(client_id: str, limit: int = 100, window: int = 60) -> bool:
    key = f"rl:{client_id}:{int(time.time()) // window}"
    count = r.incr(key)
    if count == 1:
        r.expire(key, window)  # clean up after the window ends
    return count <= limit

Two gotchas with this version:

The window boundary problem I mentioned earlier still exists — that's the trade-off you accept for simplicity.
Set the TTL on the first increment (count == 1), not on every request, or you'll extend the window for active users indefinitely.

If you need distributed token-bucket semantics, the standard approach is a Lua script with sorted sets — but honestly, for most APIs the fixed window above is enough, and it's much easier to reason about.

Pitfalls I've hit so you don't have to

Rate limiting the wrong layer. Middleware that runs before auth can't key on the API key. Either parse the key in the middleware, or put the limiter inside the auth dependency.
Forgetting health checks and internal endpoints. You will eventually rate-limit your own health check and take down a load balancer's view of your service. Whitelist internal routes.
Too-large buckets. A capacity of 1000 with a rate of 10/s means the first 1000 requests in a burst all pass. If that's not what you want, keep the burst small and let the rate do the work.
No test coverage for the boundary. Write a test that fires capacity + 1 requests and asserts exactly one 429. It catches regressions the moment they happen.

When to use a library instead

For production, you'll probably reach for something like limits or slowapi (or your framework's built-in support). That's the right call — they handle edge cases I haven't covered here. But now you know what they're doing under the hood: the knobs they expose (rate, burst, storage backend) map directly to the concepts above. When a library's default settings feel wrong for your traffic, you'll know which parameter to touch.

Wrap-up

Rate limiting is one of those things that's invisible until it isn't. The good news is the core idea fits in ~40 lines: a token bucket, a lock, and a monotonic clock. Start with the in-memory version for a single-process service, switch to the Redis fixed window when you scale out, and key by user, not just by IP.

Your future self — and the person who gets paged at 2am — will thank you.

Time Zones in APIs: 5 Rules That Would Have Saved Me 3 Days of Debugging

Sir Max — Wed, 19 Aug 2026 02:06:44 +0000

A few years ago I shipped a booking feature that passed every test I ran locally. Then a user in Berlin wrote in: "My booking is showing the wrong time." Then one in Sydney. Then one in São Paulo.

Three days later I found the bug. I had been storing local server time as a naive datetime, and doing date arithmetic in Python where the server happened to run at UTC+0 — until a container rebuild picked up a different default timezone and everything silently shifted by eight hours.

Nothing crashed. No error was thrown. The data was just wrong, everywhere, retroactively.

Here are the five rules I now follow on every project. They would have saved me that weekend.

Rule 1: Store everything in UTC

Your database should hold instants in time, not wall-clock readings. An instant is "2026-08-19T07:30:00Z". A wall-clock reading is "7:30 AM" — which means a different instant in New York than it does in London.

In PostgreSQL, this is the difference between timestamptz and timestamp:

-- Good: timestamptz stores an absolute instant, stored internally as UTC
CREATE TABLE bookings (
  id           bigint PRIMARY KEY,
  starts_at    timestamptz NOT NULL,
  created_at   timestamptz NOT NULL DEFAULT now()
);

-- Bad: timestamp stores whatever you give it, no zone context
-- "2026-08-19 07:30:00" is ambiguous and will bite you later

timestamptz doesn't actually store a timezone — it normalizes everything to UTC and remembers the instant. The timezone you see is applied at display time. That's exactly what you want.

The single exception: if the user's intent is "9:00 AM every day, local to them" (an alarm, a recurring meeting), store the wall-clock time and the timezone identifier as separate columns. More on that in Rule 4.

Rule 2: Never use naive datetimes in application code

Python lets you shoot yourself in the foot with surprising politeness. datetime.utcnow() sounds like it gives you UTC — and it does, numerically — but it returns a naive object with no tzinfo attached. The moment you compare it against an aware datetime, you get a TypeError, or worse, silent wrongness in a library that assumes a zone.

from datetime import datetime, timezone

# Bad: naive, no timezone attached
now = datetime.utcnow()          # deprecated in 3.12 for exactly this reason

# Good: explicitly aware
now = datetime.now(timezone.utc)

# Convert a stored instant into a user's local time
from zoneinfo import ZoneInfo
berlin = ZoneInfo("Europe/Berlin")
local_time = now.astimezone(berlin)

Use zoneinfo (stdlib since Python 3.9) and pass IANA identifiers like "Europe/Berlin" or "America/Sao_Paulo". Never store or hardcode offsets like +02:00, because offsets change — Berlin is +01:00 in winter and +02:00 in summer. An offset is a snapshot; a timezone is a rulebook.

Rule 3: Use ISO 8601 on the wire

Pick one wire format and enforce it everywhere. ISO 8601 with an explicit offset (or a Z for UTC) is unambiguous and parseable by every language you'll touch.

# Serialize an instant correctly
import json
from datetime import datetime, timezone

def to_json(dt: datetime) -> str:
    return dt.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")

now = datetime.now(timezone.utc)
payload = json.dumps({"created_at": to_json(now)})
# {"created_at": "2026-08-19T07:30:00.123456Z"}

The worst format you can choose is one that looks fine but drops the offset — "2026-08-19 07:30:00". Every consumer will guess a timezone, and they'll guess differently. I've watched two services written in different languages disagree about "now" by a full day because one assumed UTC and the other assumed server-local.

Two practical notes:

Use Z for UTC, not +00:00. A few JSON parsers treat the former specially and the latter as a generic offset, and JavaScript's Date.parse has historically been inconsistent with offsets.
For JavaScript, don't hand-roll date parsing. Use new Date("2026-08-19T07:30:00Z") (ISO with Z is safe) and avoid "2026/08/19" or any locale-specific string, which varies by browser.

Rule 4: Separate "instant" from "intent"

This is the rule that separates people who've been burned from people who haven't.

Most timestamps are instants: when a record was created, when a payment went through. Store those as UTC (timestamptz), display them in the viewer's local zone.

But some timestamps are intents: a user wants their report generated "every Monday at 9 AM", or a meeting "at 3 PM my time". If you naively convert "3 PM in New York" to UTC and store only that, the meeting will drift by an hour every time daylight saving starts or ends.

Store both:

CREATE TABLE recurring_reports (
  id            bigint PRIMARY KEY,
  schedule_time time NOT NULL,          -- 09:00:00, wall-clock intent
  timezone      text NOT NULL,          -- 'America/New_York', IANA name
  last_run_at   timestamptz             -- the last actual instant it ran
);

schedule_time + timezone captures what the user meant. last_run_at captures what actually happened. Mixing the two is where recurring-event bugs live.

Rule 5: Test around DST transitions

If your test suite only ever runs in one timezone and never crosses a daylight-saving boundary, you are testing the one scenario that's guaranteed not to break.

import pytest
from datetime import datetime, timezone
from zoneinfo import ZoneInfo

def localize(dt_utc: datetime, tz_name: str) -> datetime:
    return dt_utc.astimezone(ZoneInfo(tz_name))

def test_dst_spring_forward():
    # 2026-03-29 01:30 UTC in Berlin -> 03:30 (clocks jumped from 02:00 to 03:00)
    instant = datetime(2026, 3, 29, 1, 30, tzinfo=timezone.utc)
    assert localize(instant, "Europe/Berlin").hour == 3

def test_dst_fall_back():
    # 2026-10-25 01:30 UTC in Berlin -> 02:30 (clocks fell back)
    instant = datetime(2026, 10, 25, 1, 30, tzinfo=timezone.utc)
    assert localize(instant, "Europe/Berlin").hour == 2

Add a couple of these to your CI matrix — run the whole suite once with TZ=America/Sao_Paulo and once with TZ=Pacific/Auckland — and you'll catch most timezone bugs before a customer in another hemisphere does.

The checklist I run before shipping anything that touches time

Every datetime column is timestamptz (or the equivalent in your stack), or a deliberate wall-clock + timezone pair.
No naive datetime anywhere in app code — datetime.now(timezone.utc), never utcnow().
One wire format: ISO 8601 with Z for UTC.
Recurring events store intent (time + IANA zone) separately from instants.
CI runs the test suite under at least two timezones, including a DST boundary.

None of this is exotic. It's the kind of boring discipline that feels like overkill right up until the weekend you lose to a bug that never throws an error. Learn it once, from my mistake, instead of from your own.

Stop Returning Bare 500s: A Practical Guide to RFC 9457 Problem Details

Sir Max — Tue, 18 Aug 2026 02:07:56 +0000

Stop Returning Bare 500s: A Practical Guide to RFC 9457 Problem Details

A few months ago I was integrating with a third-party payments API. Everything worked in their sandbox. The moment I flipped to production, a call returned HTTP 200 with this body:

{ "success": false, "msg": "error" }

No status code, no error type, no field hint. Just "error". It took me and their support team a full day to discover the real cause: my request had a transaction amount with one extra decimal place than their schema allowed.

That day cost me more than any bug I've written myself. And it wasn't a hard problem to prevent — the API just didn't tell me what was wrong in a way a machine (or a tired human) could act on.

Here's the thing: error responses are part of your API's contract, not an afterthought. If your happy path is well documented but your error path is a black box, integrators will burn hours, then quietly move to a competitor.

The problem with "just return a message"

Most APIs fall into one of three failure modes:

The bare 500. A generic server error with an empty body or an HTML error page. The client learns nothing.
The 200-that-isn't. Everything returns 200, and the real status is buried in a payload like { "code": 500 }. This breaks caching, retries, and every HTTP-aware tool in the chain.
The inconsistent shape. Sometimes {"error": "..."}, sometimes {"message": "..."}, sometimes {"detail": "..."}. Clients end up writing defensive parsers for all of them.

All three share the same root cause: there's no agreed shape for an error, so every endpoint invents its own.

Enter RFC 9457: Problem Details

RFC 9457 (the successor to the older RFC 7807) defines a standard JSON format for HTTP error responses. The magic is its simplicity — a problem detail looks like this:

{
  "type": "https://api.example.com/problems/insufficient-funds",
  "title": "Insufficient funds",
  "status": 402,
  "detail": "The account does not have enough balance to cover this transaction.",
  "instance": "/transactions/8f2a1c/attempts/41",
  "balance": 12.50,
  "required": 19.99
}

The fields:

type — a URI identifying the kind of problem. Machine-readable, stable over time.
title — a short, human-readable summary (should not change between occurrences of the same type).
status — the HTTP status code, duplicated for convenience.
detail — a human-readable explanation specific to this occurrence.
instance — a URI pointing to the specific resource that caused the problem.

Crucially, you can add extension members — like balance and required above — to carry domain-specific data. That's where the real power lives.

The response must use the Content-Type: application/problem+json header, which tells clients "this is a structured error, parse it accordingly."

A working example in FastAPI

Here's the naive way most Python services handle errors today:

from fastapi import FastAPI, HTTPException

app = FastAPI()

@app.post("/orders")
def create_order(order: dict):
    if order.get("quantity", 0) <= 0:
        raise HTTPException(status_code=400, detail="bad quantity")
    ...

The client gets {"detail": "bad quantity"}. That's fine for a human, but a client that wants to react programmatically has to string-match "bad quantity" — fragile and impossible to localize.

Let's replace it with Problem Details. First, a small exception class and a handler:

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

class Problem(Exception):
    def __init__(self, type_uri, title, status, detail, **extensions):
        self.type_uri = type_uri
        self.title = title
        self.status = status
        self.detail = detail
        self.extensions = extensions

app = FastAPI()

@app.exception_handler(Problem)
async def problem_handler(request: Request, exc: Problem):
    body = {
        "type": exc.type_uri,
        "title": exc.title,
        "status": exc.status,
        "detail": exc.detail,
    }
    body.update(exc.extensions)
    return JSONResponse(
        status_code=exc.status,
        content=body,
        headers={"Content-Type": "application/problem+json"},
    )

Now the endpoint can raise precise, typed errors:

@app.post("/orders")
def create_order(order: dict):
    qty = order.get("quantity", 0)
    if qty <= 0:
        raise Problem(
            type_uri="https://api.example.com/problems/invalid-quantity",
            title="Invalid quantity",
            status=422,
            detail="quantity must be a positive integer",
            field="quantity",
            received=qty,
        )

And a client gets:

{
  "type": "https://api.example.com/problems/invalid-quantity",
  "title": "Invalid quantity",
  "status": 422,
  "detail": "quantity must be a positive integer",
  "field": "quantity",
  "received": 0
}

The field and received extensions let a frontend highlight the exact input box without parsing prose.

Add a trace id for supportability

One extension I add to every error is a correlation id, so a support ticket can be traced back to a log line:

import uuid

@app.exception_handler(Problem)
async def problem_handler(request: Request, exc: Problem):
    body = {
        "type": exc.type_uri,
        "title": exc.title,
        "status": exc.status,
        "detail": exc.detail,
        "trace_id": request.state.trace_id,  # set by middleware
    }
    body.update(exc.extensions)
    return JSONResponse(
        status_code=exc.status,
        content=body,
        headers={"Content-Type": "application/problem+json"},
    )

When a user reports "it failed again," they can paste the trace_id and you jump straight to the logs. This single habit has cut our debugging time on integration issues by more than half.

Four pitfalls that still bite people

1. Leaking internals. A 500 that includes your stack trace, SQL query, or framework version is a gift to attackers. Log the internals, but return a generic problem detail for unexpected errors:

@app.exception_handler(Exception)
async def unhandled_handler(request: Request, exc: Exception):
    logger.exception("unhandled error", extra={"trace_id": request.state.trace_id})
    return JSONResponse(
        status_code=500,
        content={
            "type": "https://api.example.com/problems/internal-error",
            "title": "Internal server error",
            "status": 500,
            "detail": "Something went wrong. Reference this id with support.",
            "trace_id": request.state.trace_id,
        },
        headers={"Content-Type": "application/problem+json"},
    )

2. Returning 200 for errors. This breaks retries (a client can't tell transient from permanent), breaks caches, and confuses every monitoring tool you own. Use real status codes — that's what they're for.

3. Renaming your type URIs. The type field is your machine-readable contract. Once integrators code against invalid-quantity, renaming it is a breaking change. Treat it like an API version — version the URI (/problems/v2/invalid-quantity) if you must change its meaning.

4. Skipping the Content-Type header. Without application/problem+json, generic clients treat your body as plain JSON and lose the semantic "this is an error" signal. Some HTTP libraries will also fail to parse the body into a typed object.

The payoff

Standardizing on Problem Details didn't make our APIs slower or harder to build — it made them predictable. Integrators stopped pinging me with "what does this error mean?" because the answer was in the response. Support tickets started including a trace_id instead of a screenshot of a red toast.

If you only take one thing from this: write your error responses with the same care you write your success responses. Your integrators are your users too.

Have you hit an API with infuriatingly opaque errors? I'd love to hear the worst one in the comments.

The 3 Things That Make Webhooks Actually Work (With Python Code)

Sir Max — Mon, 17 Aug 2026 02:08:35 +0000

Webhooks look simple from the outside. Your service makes an HTTP POST to a customer's URL when something happens, and... that's it, right?

Then you run one in production and learn the hard way that webhooks are a distributed systems problem wearing a friendly costume. The customer's endpoint goes down. A request times out. The same event gets delivered twice. An attacker starts sending fake events to your endpoint.

I've built webhook delivery for two different products, and both times the naive version (just POST and forget) broke within a week. This post walks through the three things that make webhooks actually work — retries, signatures, and idempotency — with code you can copy.

The three hard problems

Delivery is not guaranteed. The receiver's server is down, their firewall drops the request, they deploy mid-flight. A single POST has maybe a 99% success rate, which sounds fine until you realize that means roughly one in a hundred events is silently lost.
Duplication and ordering. If you retry a failed delivery, you might deliver the same event twice. And if two workers deliver events concurrently, the receiver can see them out of order.
Security. Anyone who can reach the receiver's endpoint can POST a fake event. The receiver needs to verify the event actually came from you.

Let me handle each one with real code.

1. Retries with exponential backoff

A retry loop is table stakes, but the details matter. You want exponential backoff with jitter, a cap on attempts, and a cap on total time.

import time
import random
import requests

MAX_ATTEMPTS = 8
BASE_DELAY = 5  # seconds

def deliver_with_retries(url, payload, signature_header):
    attempt = 0
    while attempt < MAX_ATTEMPTS:
        try:
            resp = requests.post(
                url,
                json=payload,
                headers={"X-Signature": signature_header},
                timeout=10,
            )
            if resp.status_code < 500:
                # 2xx/3xx/4xx: don't retry client errors
                return resp.status_code
            # 5xx: server error, retry below
        except requests.exceptions.RequestException:
            # network error or timeout, retry below
            pass

        attempt += 1
        if attempt == MAX_ATTEMPTS:
            return None  # give up, log for manual review

        # exponential backoff with jitter
        delay = BASE_DELAY * (2 ** (attempt - 1))
        jitter = random.uniform(0, delay * 0.3)
        time.sleep(delay + jitter)

    return None

Two decisions in here matter more than the loop itself:

Only retry 5xx and network errors, never 4xx. If the receiver says 400 or 401, that's their bug, not a transient failure. Retrying a 400 just spams their logs and doesn't fix anything.
Add jitter. If a receiver goes down and every sender retries on the exact same schedule, they all hammer the endpoint at once the moment it comes back. Jitter spreads the retries out.

A real system would use a queue and a background worker instead of time.sleep, but the logic is identical — I kept it inline so it's easy to read.

2. Signatures so the receiver trusts you

The receiver needs to answer one question: did this request really come from the sender? The standard answer is an HMAC signature over the raw request body, using a shared secret.

import hashlib
import hmac
import json

def sign_payload(secret: bytes, payload: dict) -> str:
    body = json.dumps(payload, separators=(",", ":"), sort_keys=True)
    return hmac.new(secret, body.encode(), hashlib.sha256).hexdigest()

And on the receiver's side:

def verify_signature(secret: bytes, raw_body: bytes, header: str) -> bool:
    expected = hmac.new(secret, raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, header)

Three details that trip people up:

Sign the raw bytes, not the re-serialized JSON. JSON has no canonical form — whitespace and key order can differ. If the receiver parses the body and re-serializes it before verifying, the signature won't match. Sign the exact bytes that went over the wire.
Use hmac.compare_digest, not ==. == short-circuits and leaks timing information; compare_digest runs in constant time.
Rotate secrets. If a customer's secret leaks, they need to be able to revoke and rotate it without breaking everyone else.

3. Idempotency so duplicate delivery is harmless

Retries mean duplicates are inevitable. The fix is an idempotency key: every event carries a unique ID, and the receiver ignores events it has already processed.

import sqlite3

def handle_event(event_id: str, handler):
    # atomic claim: INSERT succeeds only if the ID is new
    conn = sqlite3.connect("events.db")
    try:
        conn.execute(
            "INSERT INTO processed_events (event_id) VALUES (?)", (event_id,)
        )
        conn.commit()
    except sqlite3.IntegrityError:
        return  # already processed, skip
    handler()

The INSERT with a unique constraint is the whole trick — it gives you an atomic "only process this once" guarantee even if two workers race on the same event.

What actually goes wrong in production

These are bugs I've personally shipped (and then fixed):

Out-of-order delivery. With concurrent workers and retries, event #2 can arrive before event #1. If the receiver assumes order, add a sequence field to the payload so they can detect and buffer out-of-order events.
Retry storms. A receiver with a slow endpoint backs up thousands of events, and your retries make it worse. Cap concurrent deliveries per receiver, and drop events into a dead-letter queue after N failed attempts instead of retrying forever.
No timeout on the receiving side. Set a connect timeout and a read timeout. A receiver that hangs the connection for 60 seconds is worse than one that fails fast.

Wrap-up

If you're adding webhooks to your API, ship all three from day one: retries with backoff and jitter, HMAC signatures over the raw body, and idempotency keys. Retrofitting any of them after customers have built against your API is twice the work.

And if you're consuming webhooks, verify the signature and deduplicate by event ID before you touch your database. Your sender will eventually deliver the same event twice — plan for it.

I Ran REST, gRPC, and GraphQL in Production — Here's What I'd Pick Again

Sir Max — Sun, 16 Aug 2026 02:06:34 +0000

Over the last few years I've ended up running all three of the big API styles in production at the same time. Not because I wanted to — because different parts of the system pulled me in different directions. A public REST API for third-party developers, gRPC for the internal service-to-service mesh, and a GraphQL layer sitting on top of both for our own frontend.

This is what I learned actually running them side by side, including the parts the docs don't tell you.

The setup, in one sentence each

REST — resources over HTTP, JSON bodies, status codes carry meaning. Boring, predictable, and every tool on earth understands it.
gRPC — binary protocol over HTTP/2, contracts defined in .proto files, code generated for you. Fast and strict.
GraphQL — a single endpoint where the client asks for exactly the fields it wants. Flexible, but that flexibility cuts both ways.

Where each one genuinely shines

REST: when other people call your API

If third-party developers are your audience, REST is almost always the right answer. The moment you expose an API to people you don't control, the priority shifts from raw performance to "can a stranger figure this out with curl and a README?"

Every HTTP client, every monitoring tool, every load balancer, every browser devtools speaks REST natively. That compatibility is worth more than any latency win.

curl -X POST https://api.example.com/v1/orders \
  -H "Content-Type: application/json" \
  -d '{"items": [{"sku": "a-100", "qty": 2}]}'

Status codes do the error handling for you. A 429 tells the client to slow down. A 409 says "conflict, retry with new state". Half your documentation writes itself the day you start, because the conventions are already understood.

gRPC: when your own services talk to each other

Inside your own infrastructure, you control both sides of the wire. That's where gRPC wins. Protobuf serialization is dramatically smaller than JSON, HTTP/2 multiplexing removes head-of-line blocking, and the generated client/server code means you never hand-write a request parser again.

service InventoryService {
  rpc ReserveStock(ReserveRequest) returns (ReserveResponse);
}

message ReserveRequest {
  string sku = 1;
  int32 quantity = 2;
}

The thing people miss about gRPC is that the code generation is the feature. You write the .proto, run one command, and both the server stub and the client are generated in every language you use. Renaming a field becomes a compile error instead of a runtime surprise.

GraphQL: when the frontend and backend drift apart

GraphQL shines when the team building the UI is separate from the team building the backend, and you want to ship frontend features without a backend deploy for every one.

query {
  order(id: "42") {
    status
    total
    items {
      name
      quantity
    }
  }
}

The frontend gets exactly the shape it needs — no more eight REST calls to render one screen, no more over-fetching thirty fields when you need three.

What the docs don't tell you

gRPC is annoying to debug

With REST, when something breaks you can usually reproduce it in a browser or a single curl. With gRPC you need a dedicated tool like grpcurl, and the binary payload means you can't just eyeball a response. Your on-call flow gets more friction. That's a real, recurring cost — not a one-time setup.

GraphQL caching is genuinely hard

REST gives you nice cache keys for free, because the key is just the URL. GraphQL gives you one URL for everything and a POST body that differs on every request. HTTP caching basically stops working, and you end up building application-level caching — or reaching for persisted queries — that you never needed with REST.

REST's "flexibility" becomes a tax

REST is the least opinionated of the three, and that shows up as inconsistency. Field naming, pagination style, error shapes — every new endpoint risks inventing its own convention unless you enforce one. gRPC and GraphQL both hand you a schema that does this enforcement for you.

Versioning and breaking changes

REST handles evolution with URL versioning — /v1, /v2 — simple, but it means you keep maintaining old versions long after you'd like to retire them. gRPC bakes field numbers and deprecation rules into protobuf itself, so additive changes are safe and breaking changes are loud at compile time. GraphQL sidesteps the problem almost entirely: fields are additive by default, and you can mark a field @deprecated while old clients keep working.

The practical result: I spend the least time worrying about versioning with GraphQL, and the most with REST, because old /v1 endpoints have a way of accumulating forever. Every one of them is a small maintenance debt you'll be paying off for years.

A rough decision guide

Your situation	Pick
Third-party / public API	REST
Internal service-to-service, high traffic	gRPC
One team, own frontend and backend	GraphQL
Mobile app on slow networks	GraphQL
Simple internal tool, low traffic	REST — don't over-engineer
Streaming / real-time	gRPC

What I'd pick again

If I had to restart today with the same system: REST for anything public, gRPC for the internal mesh, and skip GraphQL until the frontend team actually feels the pain.

That last part is the real lesson. I adopted GraphQL partly because it was popular, not because I had a concrete problem it solved. The caching complexity and the extra tooling were real costs, and for a while they weren't paying for themselves. Now that the frontend team is genuinely large and moves fast, GraphQL earns its keep — but I should have waited until that was actually true.

The boring truth: most of the time REST is fine, gRPC is a great upgrade for the internal path once you measure the traffic, and GraphQL is a power tool for a specific problem, not a default.

Pick the tool for the job you actually have, not the one you read about on Hacker News.

3 Ways to Make Your API Requests Idempotent (With Working Code)

Sir Max — Sat, 15 Aug 2026 02:06:22 +0000

3 Ways to Make Your API Requests Idempotent (With Working Code)

We lost a customer payment last year because of a retry. A third-party gateway timed out on our first charge attempt, so our client retried — and the customer got billed twice. It wasn't a huge amount, but the refund process, the support ticket, and the customer's trust took weeks to repair.

The root cause wasn't the gateway. It was us: our endpoint wasn't idempotent. We assumed every request happens exactly once. In a world of flaky networks, that assumption is wrong.

Here's what I learned, and the three patterns I now reach for whenever a request can be retried.

Why "retry" and "exactly once" don't mix

Networks fail in ways that make it impossible to know whether a request reached the server:

The server processed the request, but the response got lost on the way back.
The server is still processing when the client's timeout fires.
A load balancer retried the request internally without telling you.

When the client retries, the server may process the same logical operation twice. For a GET, that's usually harmless. For a POST /payments, it's a double charge.

The general fix is to make operations idempotent: performing the same operation multiple times has the same effect as performing it once.

Pattern 1: Idempotency keys

The most common pattern is an idempotency key — a client-generated unique identifier that the server uses to deduplicate retries.

import uuid

# In-memory store for this example. Use Redis or Postgres in production.
_store = {}

def process_payment(payment_id, amount, idempotency_key):
    if idempotency_key in _store:
        return _store[idempotency_key]  # replay the cached result
    # ... actually charge the card here ...
    result = {"payment_id": payment_id, "status": "succeeded"}
    _store[idempotency_key] = result
    return result

The client generates the key once per logical operation and reuses it on every retry:

key = uuid.uuid4().hex  # generated once, kept across retries
process_payment("pmt_123", 49.99, key)  # first attempt
process_payment("pmt_123", 49.99, key)  # retry — returns the cached result

The server stores the key-to-response mapping. On a duplicate, it returns the stored result instead of re-executing the work.

A few details I got wrong the first time:

The key must be generated by the client, not the server. The whole point is that the server can't distinguish a new request from a retry without it.
Guard against races. Two retries can arrive at the same time. A naive "check then insert" has a gap where both threads see the key as missing. Use an atomic insert (or a lock) so only one side effect happens.
Return the same response, not just a success flag. If the first attempt returned a 201 with a payment ID, the retry should return that same 201 and ID — not a 409 conflict or an empty 200.

Pattern 2: Make the operation naturally idempotent

Some operations don't need a key if you design the semantics to be idempotent from the start.

PUT is the classic example. A PUT /users/42 that sets the full representation is idempotent by definition — sending it twice sets the same state twice.

For "create" operations, derive a stable identifier from the request itself instead of generating a random ID on every call:

import hashlib

def create_user(email, name):
    user_id = "user_" + hashlib.sha256(email.encode()).hexdigest()[:16]
    # INSERT ... ON CONFLICT (id) DO UPDATE SET name = EXCLUDED.name
    return {"id": user_id, "email": email, "name": name}

Because the ID is a deterministic hash of the email, sending the same request twice produces the same user — no duplicate rows. This is a form of upsert, and it works well for records with a natural unique key.

The tradeoff: you need a natural key (email, SKU, external ID). If you don't have one, fall back to idempotency keys.

Pattern 3: Optimistic concurrency

Idempotency keys protect against retries. But what about two different requests that both want to modify the same record? That's where optimistic concurrency helps — and it pairs well with the first two patterns.

Send a version (or updated_at timestamp) with the request, and reject the update if the version doesn't match:

def update_article(article_id, body, expected_version):
    row = db.get(article_id)
    if row["version"] != expected_version:
        return {"error": "conflict", "current_version": row["version"]}, 409
    db.update(article_id, body=body, version=row["version"] + 1)
    return {"version": row["version"] + 1}, 200

This doesn't make a single request idempotent on its own, but it prevents a common source of duplicate side effects: two writers clobbering each other. Combined with idempotency keys, it covers most of the failure modes I've actually hit in production.

The pitfalls that will bite you

A few things I learned the hard way:

Expire your idempotency records. Don't store keys forever. Keep them for 24 hours — that covers virtually all realistic retry windows — then delete them. Otherwise you've built a slow memory leak.
Scope keys per endpoint and per user. The same key on a different endpoint should not collide. Namespace it: user123:payments:abc....
Handle the "stale key" case. If a client reuses a key for a different payload, that's a bug. Detect it and return a 422 instead of silently returning the old result.
Test with real concurrency. Fire 50 simultaneous requests with the same key and confirm only one side effect happens. A unit test that calls the function twice sequentially will miss the race.

What I'd tell my past self

The whole idempotency-key idea felt like over-engineering until the double charge happened. Now I treat it like input validation: something you add by default to any endpoint that has a side effect.

If your API mutates state and a client might retry, add an idempotency key. It's maybe 30 lines of code. The alternative is a support ticket titled "you charged me twice."

Build a Production-Ready Rate Limiter in 40 Lines of Python

Sir Max — Fri, 14 Aug 2026 02:10:23 +0000

Build a Production-Ready Rate Limiter in 40 Lines of Python

I've shipped rate limiters three times in my career. The first one broke production. The second one made our biggest customer angry. The third one — finally — worked without anyone noticing it was there.

That last part is the goal. A good rate limiter is invisible. It protects your API without ever getting in the way of legitimate traffic.

Here's what I learned, plus working code you can actually use.

The mistake everyone makes first

Your first instinct is a fixed window:

def is_allowed(user_id: str, limit: int, window: int = 60) -> bool:
    key = f"rl:{user_id}:{int(time.time() // window)}"
    count = redis.incr(key)
    redis.expire(key, window, nx=True)
    return count <= limit

This looks correct. In a 60-second window, each user can make limit requests. The counter resets every minute.

The problem is window boundaries. A user can make limit requests in the last second of one window, then limit more in the first second of the next. That's 2 * limit requests in about two seconds.

If your limit is 100 requests/minute, a burst of 200 is usually fine. But if your limit is 10 requests/minute and you're protecting an expensive LLM endpoint, that double-burst is exactly what takes you down.

Sliding window: the fix that's worth the extra lines

Instead of resetting the counter at a boundary, count only the requests in the last 60 seconds.

import time
import redis

def rate_limited(user_id: str, limit: int, window: int) -> bool:
    """Return True if the request should be allowed."""
    now = time.time()
    key = f"rl:{user_id}"
    pipe = redis.pipeline()

    # Drop entries older than the window
    pipe.zremrangebyscore(key, 0, now - window)
    # Add the current request, scored by its timestamp
    pipe.zadd(key, {str(now): now})
    # Count what's left
    pipe.zcard(key)
    # Expire the key so we don't leak memory
    pipe.expire(key, window)

    _, _, count, _ = pipe.execute()
    return count <= limit

This is a sliding window log implemented with a Redis sorted set. Each request is a member scored by its timestamp. Old entries get dropped, and we count what remains.

It's precise, and it's O(n) in the number of requests per window — which is fine for almost every real-world API.

The nuance nobody writes about

A rate limiter that just returns 429 is a bad rate limiter. Real users hit limits for three reasons:

They're legitimate but busy — a batch job, a retry loop
They made a mistake — bad pagination, missing caching
They're abusing your API

You can't tell these apart by counting alone. What you can do is communicate.

Always send these headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1720000000

And in the 429 response body, tell them when to retry:

{
  "error": "rate_limited",
  "retry_after_seconds": 18
}

The first version I shipped returned a bare 429 Too Many Requests with no body. Support tickets followed. It took me a full day to realize users were retrying instantly in a loop, which made the problem worse.

Three mistakes I still see in the wild

1. Using 503 instead of 429. A 503 means "the server is down." It triggers alerting, retries with backoff, and occasionally auto-scaling. A 429 means "you're asking too fast." These are different problems with different fixes. If your rate limiter returns 503, your on-call engineer is going to hate you.

2. Rate limiting on the wrong key. Rate limit per user, not per IP. NAT means a whole office can share one IP, so a per-IP limit will throttle a classroom full of students using your API legitimately. If you do limit by IP, read the X-Forwarded-For header — and never trust it blindly, because clients can set it.

3. Hard-coding limits in the code. Limits are a product decision, not a code decision. Read them from config. A customer paying for a higher tier shouldn't need a code deploy to get it.

Token bucket: when you want to smooth, not just block

A sliding window answers one question: "has this user made too many requests in the last N seconds?" That's enough for most cases.

A token bucket answers a different question: "can this user burst briefly, as long as their long-term average stays under control?" Tokens refill at a steady rate, and a request costs one token. The bucket can hold a small surplus, so a user who's been idle can burst.

import time

def token_bucket_allows(user_id, capacity, refill_rate):
    key = f"tb:{user_id}"
    now = time.time()
    # Lua keeps the read-modify-write atomic
    script = """
    local tokens = tonumber(redis.call('get', KEYS[1]) or ARGV[1])
    if tokens < 1 then return 0 end
    redis.call('set', KEYS[1], tokens - 1)
    return 1
    """
    ...

I've genuinely needed this exactly twice. Both times it was a write-heavy endpoint where clients legitimately burst (uploading a batch of records) but had to average out over an hour. If you can't name that scenario for your API, stick with the sliding window. It's simpler and easier to reason about.

Putting it together

Here's a minimal decorator that wires the sliding-window function into any function:

from functools import wraps

class RateLimited(Exception):
    def __init__(self, retry_after: int):
        self.retry_after = retry_after
        super().__init__(f"rate limited, retry in {retry_after}s")

def rate_limit(limit: int, window: int):
    def decorator(fn):
        @wraps(fn)
        def wrapper(*args, **kwargs):
            user_id = kwargs.get("user_id", "anonymous")
            if not rate_limited(user_id, limit, window):
                raise RateLimited(retry_after=window)
            return fn(*args, **kwargs)
        return wrapper
    return decorator

@rate_limit(limit=100, window=60)
def expensive_endpoint(user_id: str):
    return {"ok": True, "user": user_id}

The point isn't the exact code — it's the model. Count precisely, communicate clearly, and make the limit configurable.

What I'd do differently next time

Start with sliding window; skip fixed window entirely. The extra complexity is one Redis call.
Ship the headers on day one. They cost nothing and save you support tickets.
Log every 429. Those logs are your early warning that a customer is about to churn because their integration is stuck in a retry loop.

Rate limiting feels like an infrastructure detail until the moment it isn't. The moment it isn't is usually 3 AM.

3 Testing Habits That Caught Bugs Before My Users Did

Sir Max — Thu, 13 Aug 2026 02:16:31 +0000

Last month a user reported that our checkout endpoint charged them twice. The code looked fine in review. Every test was green. The bug only appeared when two requests hit the same endpoint within the same millisecond — a race condition between the "check if order exists" query and the "insert order" query.

That single bug cost a refund, a support ticket, and an afternoon of my time. It also reminded me of something I already knew but kept ignoring: tests that only cover the happy path don't catch much.

Since then I've adopted three habits that have caught bugs before my users did. They're not fancy. They're just specific.

Habit 1: Write the failure test before the success test

When I add a feature, I used to start by writing the test that proves it works. Now I start with the test that proves it breaks the right way.

The reason is simple: the success test tells you the feature does what you want. The failure test tells you it doesn't do what you don't want — and in production, "what you don't want" is usually where the bugs live.

Here's a concrete example. Imagine a function that parses a comma-separated list of IDs:

def parse_ids(raw: str) -> list[int]:
    """Parse a comma-separated string into a list of ints."""
    return [int(x) for x in raw.split(",")]

The happy-path test looks like this:

def test_parse_ids_valid_input():
    assert parse_ids("1,2,3") == [1, 2, 3]

But what happens with "1,2,"? With "1,,2"? With ""? With "1, 2, 3" (spaces)?

The failure tests are where the real design decisions show up:

import pytest

def test_parse_ids_trailing_comma():
    assert parse_ids("1,2,") == [1, 2]  # or should it raise?

def test_parse_ids_empty_string():
    with pytest.raises(ValueError):
        parse_ids("")

def test_parse_ids_spaces():
    # whitespace tolerance is a product decision, not an afterthought
    assert parse_ids("1, 2, 3") == [1, 2, 3]

Writing these forces me to decide what "invalid" means before I ship it. The bug that motivated this habit was a CSV import that silently dropped the last row whenever the file had a trailing newline — because I never wrote a test for that exact case.

Habit 2: Use property-based testing for the edge cases you can't think of

You can't enumerate every edge case by hand. That's the whole problem.

Property-based testing flips the approach: instead of writing specific inputs and expected outputs, you describe a property that should always hold, and the framework generates hundreds of random inputs trying to break it.

The hypothesis library makes this almost free in Python:

from hypothesis import given, strategies as st

@given(st.text())
def test_parse_ids_never_crashes(s: str):
    # whatever the input, the function must either return a list
    # of ints or raise a clean ValueError — never a raw TypeError
    try:
        result = parse_ids(s)
        assert all(isinstance(x, int) for x in result)
    except ValueError:
        pass  # clean rejection is fine

On the first run, Hypothesis found that parse_ids("1.5") raises ValueError (from int("1.5")), which is fine — but it also probed inputs I would never have written by hand, like strings with Unicode digits and whitespace-only bodies. None of them were bugs in this case, but the point stands: it tested inputs I would never have thought to write.

The one time property-based testing paid for itself instantly: we had a function that rounded currency amounts. I wrote the property "rounding twice equals rounding once" — and Hypothesis found a value in under a second where it didn't hold. That was a real rounding bug in a money path, caught before it reached a customer.

The habit isn't "always use Hypothesis." It's "find the one property that must always be true, and let the machine try to break it."

Habit 3: Every bug becomes a regression test

This is the cheapest, highest-value habit on the list, and it took me embarrassingly long to adopt.

The rule: when a bug reaches production (or even just code review), the first commit is a failing test that reproduces it. The second commit fixes it.

The test stays in the suite forever. If the bug ever comes back, the test catches it instantly.

Here's the double-charge bug from the intro, turned into a regression test:

import threading

def test_no_double_charge_under_concurrency():
    results = []

    def charge():
        # simulate two simultaneous checkouts for the same order
        results.append(checkout(order_id=42, amount=1990))

    threads = [threading.Thread(target=charge) for _ in range(2)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()

    # exactly one charge should succeed; the other should be rejected
    assert len([r for r in results if r.status == "charged"]) == 1

This test is flaky-prone if checkout() doesn't actually use a lock or a database unique constraint — which is exactly the point. Writing it forced me to add a UNIQUE constraint on (order_id) in the database, which is the real fix. The test just makes sure that fix never regresses.

Why this habit matters more than the others: it turns every production incident into permanent institutional knowledge. Six months from now, when someone refactors the checkout code, the test will still be there. They won't have to remember what happened — the test remembers.

What I'd tell my past self

None of these habits require new tooling or a big rewrite. They're about where I point my attention:

Test the failures first — decide what "invalid" means before you ship it.
Find the one invariant that must always hold — and let property-based testing try to break it.
Every bug becomes a test — turn incidents into permanent knowledge.

The double-charge bug happened because I only tested the happy path. It cost an afternoon. The regression test I wrote afterward has run thousands of times since and will catch that class of bug forever.

Cheap insurance. Start with habit three — it's the one that pays off fastest.

4 Docker Compose Patterns That Saved Our Team Hours of Debugging

Sir Max — Tue, 11 Aug 2026 02:09:18 +0000

4 Docker Compose Patterns That Saved Our Team Hours of Debugging

I spent most of last year running a half-dozen services on a single VPS. Database, API gateway, a few workers, Redis, the usual stack. Docker Compose was the obvious choice — but my early configs were a mess of trial and error.

Here are four patterns I learned the hard way. Each one solved a real problem that cost me at least an evening of frustrated debugging.

1. Healthchecks That Actually Work

My first healthcheck looked like this:

# ❌ Too aggressive — restarts the container before it's ready
healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
  interval: 5s
  timeout: 3s
  retries: 3
  start_period: 5s

The problem: my app took 15–20 seconds to warm up (DB migrations, cache loading). Compose gave it 5 seconds, then killed it. I'd get a cascade of depends_on failures and every container restarting in a loop.

The fix is simpler than you think — make start_period match reality:

# ✅ Give the app time to actually start
healthcheck:
  test: ["CMD-SHELL", "wget -q -O- http://localhost:3000/health || exit 1"]
  interval: 10s
  timeout: 5s
  retries: 5
  start_period: 30s  # This is the key line

The start_period is how long Compose waits before counting failures. Set it to your app's actual warmup time. For a Node.js app with migrations, 30 seconds is a safe default. For a Go binary, 5 seconds might be enough.

Pro tip: Use CMD-SHELL instead of CMD. The shell form handles pipes and redirects. The array form doesn't — which is why curl often fails silently when your endpoint returns a redirect.

2. Profiles: One File, Two Environments

I used to maintain two separate compose files:

docker-compose.yml for production
docker-compose.dev.yml for development

Every change had to be synced. I missed a port mapping once and spent an hour wondering why my local debugger couldn't connect.

Docker Compose profiles solve this cleanly:

services:
  api:
    build: .
    ports: ["3000:3000"]
    environment:
      - NODE_ENV=production

  # Dev-only services
  adminer:
    image: adminer
    profiles: ["dev"]
    ports: ["8080:8080"]

  mailhog:
    image: mailhog/mailhog
    profiles: ["dev"]
    ports: ["1025:1025", "8025:8025"]

  # Prod-only overrides
  api-dev:
    extends:
      service: api
    profiles: ["dev"]
    volumes: ["./src:/app/src"]
    environment:
      - NODE_ENV=development

Now I just flip a flag:

# Production
docker compose up -d

# Development (all dev tools)
docker compose --profile dev up -d

All the core services start in both environments. Dev tools (Adminer, MailHog, hot-reload) only come up with --profile dev. One file, no duplication, no drift.

3. `extends` for Dry Compose Files

Here is a pattern I wish I'd discovered sooner. Instead of repeating environment variables across services:

# ❌ Repeated env vars everywhere
services:
  worker-1:
    environment:
      - DB_HOST=postgres
      - DB_PORT=5432
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - LOG_LEVEL=info
  worker-2:
    environment:
      - DB_HOST=postgres
      - DB_PORT=5432
      - REDIS_HOST=redis
      # ... same thing again

Use extends with a base service:

# ✅ Define once, reuse everywhere
x-base-env: &base-env
  DB_HOST: postgres
  DB_PORT: 5432
  REDIS_HOST: redis
  REDIS_PORT: 6379
  LOG_LEVEL: info

services:
  worker-1:
    environment:
      <<: *base-env
      WORKER_NAME: processor-1

  worker-2:
    environment:
      <<: *base-env
      WORKER_NAME: processor-2

This is YAML anchors, not a Docker feature — but Docker Compose supports them fully. I use anchors for environment blocks, volume mounts, and even entire service definitions when I need multiple similar containers.

4. Wait-For-It Without Custom Scripts

The classic problem: your API starts before the database is ready. Connection refused. Container restarts. Log spam.

The old solution was a wait-for-it.sh script in every container. The modern solution is built into Compose:

services:
  api:
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy

  postgres:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 3s
      retries: 5

  redis:
    image: redis:7
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5

The condition: service_healthy tells Compose: "Do not start this container until that container passes its healthcheck." No custom scripts, no sleep loops, no guesswork.

What These Patterns Share

Looking back, all four patterns solve the same underlying problem: making startup order predictable. Most Docker Compose debugging sessions start with "which container failed first?" Once you can answer that reliably, the rest is just configuration.

I still keep a docker-compose.debug.yml override file for those 2 a.m. incidents — it cranks up log levels, exposes extra ports for profiling, and adds a tty: true so I can docker attach when things really go sideways. But the four patterns above eliminated about 80% of my late-night debugging sessions.

If you have your own Compose patterns that saved you, drop them in the comments. I am always looking for new tricks.

3 Things I Wish I Knew Before Building with LLM APIs

Sir Max — Mon, 10 Aug 2026 02:15:30 +0000

3 Things I Wish I Knew Before Building with LLM APIs

I've spent the last year shipping features backed by LLM APIs — GPT, Claude, Gemini, and a handful of open-source models via Ollama. Along the way I burned through a lot of credits, broke production a couple of times, and learned things the hard way.

Here are three lessons I wish someone had told me before I started.

1. Token counting is not optional — it's the whole job

Early on, I treated token limits as a "nice to have" — something you throw a rough character count at and hope for the best. That worked fine until a user pasted a 40KB log file into a chat window and my backend silently truncated their context, giving answers that looked confident but were completely wrong.

What I do now: Every request goes through a token counter before it touches the LLM.

import tiktoken

def count_tokens(text: str, model: str = "gpt-4") -> int:
    enc = tiktoken.encoding_for_model(model)
    return len(enc.encode(text))

For multi-model setups, I built a small abstraction that maps model names to their tokenizers. claude-* uses Anthropic's claude-tokenizer, Gemini uses google-generativeai, and for Ollama models I fall back to a conservative estimate based on character count.

The real insight: You need two limits — a hard cutoff for the full context window, and a softer "reserved output" budget so the model has room to actually respond. I set aside 25% of the context window for output tokens. If your input + reserved output > context limit, trim the input — not the other way around.

This one change eliminated about 90% of the "why is the AI giving incomplete answers?" bug reports.

2. Streaming is table stakes, but structured output is the hard part

Everyone knows you should stream LLM responses — users don't want to stare at a spinner for 15 seconds. That's the easy part.

The hard part is what to do with the stream. If you're calling an LLM from a backend that needs to return JSON to a frontend, you can't just stream raw tokens. You need structured output on the backend side, and you need it fast.

The pattern that works for me:

import asyncio
import json

async def stream_and_parse(prompt: str, schema: dict):
    """Stream LLM response, parse JSON at the end."""
    buffer = []

    async for chunk in llm_stream(prompt):
        buffer.append(chunk)
        # Send raw text to frontend via WebSocket
        await ws.send(chunk)

    full_text = "".join(buffer)
    # Parse the structured result from completed text
    return json.loads(full_text)

But here's the gotcha: not all LLMs are equally good at JSON output. Claude is great at it. Gemini 2.0 Flash is fast but occasionally drops closing braces. GPT-4o handles it well but sometimes wraps JSON in markdown fences that you need to strip.

My rule of thumb: Always validate the parsed JSON against your schema. If parsing fails, retry with a system prompt that says "Return ONLY valid JSON, no markdown fences, no explanations." That second attempt fixes 95% of failures.

For production, I also log every JSON parse failure with the raw response — nothing is more frustrating than a silent parse error that turns into a NoneType has no attribute three callers later.

3. Fallback is not a feature — it's architecture

My first LLM integration looked like this:

user → my API → OpenAI → response

When OpenAI had an outage (and it did, twice in six months), my entire feature was dead. Users got timeout errors with no explanation.

What I run now:

user → my API → primary provider (Claude)
              → fallback #1 (GPT-4o) — different provider
              → fallback #2 (Gemini Flash) — cheap, always available

But it's not enough to just chain fallbacks. You need circuit breakers:

import time
from functools import wraps

class CircuitBreaker:
    def __init__(self, failure_threshold=3, reset_timeout=60):
        self.failures = 0
        self.threshold = failure_threshold
        self.reset_timeout = reset_timeout
        self.last_failure_time = 0

    def record_failure(self):
        self.failures += 1
        self.last_failure_time = time.time()

    def is_open(self):
        if self.failures >= self.threshold:
            if time.time() - self.last_failure_time < self.reset_timeout:
                return True  # Circuit is open — skip this provider
            self.failures = 0  # Reset timeout passed, try again
        return False

If Claude returns three consecutive 5xx errors, the circuit breaker opens for 60 seconds. Requests skip directly to GPT-4o instead of waiting for timeouts. When the breaker resets, it tests the water with a single request before fully reopening.

The lesson: LLM APIs are external dependencies. Treat them like you'd treat a database connection or a third-party payment service. They will fail, and your users shouldn't notice.

The takeaway

Building with LLMs isn't hard because the models are complex — it's hard because the infrastructure around them needs to be resilient. Token management, structured parsing, and provider fallback are not optional extras after launch. They're the stuff you need on day one.

These three patterns have been running in production for months now. They're not particularly clever or novel. They're just battle-tested.

Got your own LLM API war stories? I'd love to hear them in the comments.