DEV Community: Sir Max

3 Things Nobody Tells You About Building Public APIs

Sir Max — Wed, 22 Jul 2026 02:10:42 +0000

3 Things Nobody Tells You About Building Public APIs

Last year, I spent six months building and maintaining a public API. I thought I knew what I was getting into — I'd built internal APIs for years. REST endpoints, authentication, some docs. How different could "public" be?

Very different. Here are three things that blindsided me.

1. Rate Limiting Is 20% Code, 80% Edge Cases

When I first added rate limiting, I thought I was done in an afternoon. A token bucket per API key, Redis-backed, 100 requests per minute. Ship it.

Then the emails started.

"Why am I getting 429s? I'm only making 50 requests per minute."

Turns out, their backend had 12 worker processes, each with its own connection pool. From our perspective, they were 12 separate clients arriving nearly simultaneously. The token bucket drained in seconds.

"Your rate limiter broke our retry logic."

Their retry library used exponential backoff — but all 12 workers retried at the same time, creating thundering-herd 429 loops that consumed their entire quota in retries alone.

What I actually had to build:

import time
import hashlib
from collections import defaultdict
from threading import Lock

class RateLimiter:
    """
    A sliding-window rate limiter that groups by client fingerprint
    rather than raw IP or API key alone.
    """
    def __init__(self, window_seconds=60, max_requests=100):
        self.window = window_seconds
        self.max_req = max_requests
        self.buckets = defaultdict(list)
        self.lock = Lock()

    def fingerprint(self, api_key: str, user_agent: str, 
                    x_forwarded_for: str) -> str:
        """Group requests from the same deployment, not just the same key."""
        components = [api_key]
        # Truncate user-agent to group same SDK versions
        ua_base = user_agent[:50] if user_agent else "unknown"
        components.append(ua_base)
        return hashlib.sha256("|".join(components).encode()).hexdigest()[:16]

    def allow(self, fingerprint: str) -> bool:
        now = time.time()
        with self.lock:
            # Prune old entries
            self.buckets[fingerprint] = [
                t for t in self.buckets[fingerprint] 
                if now - t < self.window
            ]
            if len(self.buckets[fingerprint]) >= self.max_req:
                return False
            self.buckets[fingerprint].append(now)
            return True

The key insight: rate limit by logical client, not by connection. One API key might represent a distributed system with dozens of pods. You need to account for that.

Also, return Retry-After headers and distinguish between "you personally are too fast" (429) and "the system is overloaded" (503). Your users will thank you.

2. Documentation Rots Faster Than Code

I wrote thorough API docs during the first month. OpenAPI spec, examples for every endpoint, even a Postman collection.

Three months later, a user opened an issue: "The POST /users example returns a 201, but I'm getting a 400."

The docs still showed the v1 request format. We'd added a tenant_id field in v2 and made it required. Nobody updated the examples.

What I learned:

Documentation isn't a deliverable — it's a process. Here's what works:

# openapi.yaml — generate docs from this, don't write them separately
paths:
  /users:
    post:
      summary: Create a user
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateUserRequest'
            examples:
              basic:
                value:
                  name: "alice"
                  tenant_id: "t_4a7b2c"
      responses:
        '201':
          description: User created
          content:
            application/json:
              examples:
                basic:
                  value:
                    id: "u_9f3e1a"
                    name: "alice"

Three rules that saved me:

One source of truth. Your OpenAPI spec IS your documentation. Tools like Swagger UI or Scalar render it for free. Never maintain separate docs.
Test your examples. Write integration tests that literally run the request bodies from your spec examples and assert the responses match. If the example breaks, your CI catches it before the user does.
Version your docs alongside your code. Every time you add a field or change a response shape, update the examples in the same commit. Review it in the PR. If the reviewer can't understand the API change from the diff, neither can your users.

3. Your Users Will Use Your API in Ways You Never Imagined

This one surprised me the most. I designed my API for backend-to-backend communication. Standard JSON, RESTful endpoints, bearer tokens.

Within weeks, I saw:

Someone built a Google Sheets plugin that called our API from Apps Script. They expected CSV responses because "JSON parsing in Google Sheets is annoying." Fair point — we added ?format=csv.
A mobile developer was polling our status endpoint every 3 seconds. Not for any malicious reason — they genuinely didn't know about webhooks. We added webhook support and documentation, and their polling dropped to zero.
Someone was using our search endpoint as their primary database query layer. They'd built their entire product around our API's GET /items?q=... endpoint — not realizing it was optimized for full-text search, not structured queries. When we changed the relevance scoring algorithm, their app broke.

The lesson: You don't control how your API gets used. You can only observe, adapt, and communicate.

Practical steps that helped:

Log usage patterns, not just errors. Which endpoints get hit together? What query parameters are most common? These are signals about what your users actually need.
Read your own access logs like product research. When I noticed ?format=csv appearing in URL patterns before we'd built it, it was because users were manually appending it, hoping it would work. It didn't — but it told us what to build.
Deprecate gently. When you need to change behavior, add a Deprecation: true header and a Sunset: <date> header first. Blog about it. Email your users. Give them weeks, not hours.

# Add sunset headers to every response
@app.middleware("http")
async def add_deprecation_headers(request, call_next):
    response = await call_next(request)
    if request.url.path.startswith("/v1/"):
        response.headers["Deprecation"] = "true"
        response.headers["Sunset"] = "Sat, 01 Nov 2026 00:00:00 GMT"
        response.headers["Link"] = (
            '</v2/users>; rel="successor-version"'
        )
    return response

What I'd Do Differently

If I were starting another public API today, I'd do three things from day one:

Ship with rate limiting, not add it later. Retrofitting rate limiting into an API that already has users is 10x harder than building it in from the start. Users build assumptions around your API's behavior — changing those assumptions is painful.
Treat your OpenAPI spec like source code. It goes in the same repo, gets reviewed in the same PRs, and gets tested in the same CI pipeline. Your API is only as good as its documentation.
Watch your users, don't guess what they want. Access logs are the most honest feedback you'll ever get. People vote with their requests.

Building a public API taught me that the technical part — the endpoints, the auth, the JSON — is maybe 30% of the work. The other 70% is everything around it: the docs, the rate limiting, the deprecation strategy, the user communication.

Nobody tells you that upfront.

Have you built or maintained a public API? What surprised you? I'd love to hear about it in the comments.

I Stopped Grepping Logs and Started Querying Them — My 3-Step Structured Logging Setup

Sir Max — Tue, 21 Jul 2026 02:09:24 +0000

I Stopped Grepping Logs and Started Querying Them — My 3-Step Structured Logging Setup

Two months ago, our production API went down at 2 AM. The on-call engineer spent 40 minutes grepping through 200 MB of plaintext logs before finding the root cause: a database connection pool exhaustion triggered by a slow upstream service.

The worst part? The information was in the logs the whole time. We just couldn't find it fast enough.

That night, I promised myself I'd fix our logging. Here's the 3-step setup that now catches issues in under a minute — with real code.

Step 1: Stop Writing Logs as Strings, Start Writing JSON

The old way:

2026-07-21 02:14:03 INFO [worker-3] POST /api/orders 500 1247ms database_error
2026-07-21 02:14:03 INFO [worker-7] POST /api/orders 200 89ms
2026-07-21 02:14:04 WARN [worker-3] Connection pool exhausted (active=20, max=20, waiting=3)

To find how many 500s happened in the last hour? Grep for 500, count lines, hope the format is consistent. To correlate "connection pool exhausted" with the endpoint that caused it? Good luck — those two lines aren't even adjacent.

Here's the same information, structured:

{"timestamp":"2026-07-21T02:14:03.421Z","level":"INFO","worker":"worker-3","event":"request_completed","method":"POST","path":"/api/orders","status":500,"duration_ms":1247,"error":"database_error","request_id":"req-abc123"}
{"timestamp":"2026-07-21T02:14:03.822Z","level":"INFO","worker":"worker-7","event":"request_completed","method":"POST","path":"/api/orders","status":200,"duration_ms":89,"request_id":"req-abc124"}
{"timestamp":"2026-07-21T02:14:04.155Z","level":"WARN","worker":"worker-3","event":"connection_pool_exhausted","active":20,"max":20,"waiting":3,"request_id":"req-abc123"}

Now I can query with jq:

# All 500 errors in the last hour
jq 'select(.status == 500 and .timestamp >= "2026-07-21T01:14:00Z")' app.log

# Find all logs for a specific request
jq 'select(.request_id == "req-abc123")' app.log

# Count errors by endpoint
jq -s 'group_by(.path) | map({path: .[0].path, errors: [.[] | select(.status >= 500)] | length})' app.log

Step 2: Use structlog (Python), Not print()

Here's the setup that took 20 minutes and changed everything:

import structlog
import logging

# Configure structlog to output JSON
structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.dev.ConsoleRenderer()  # Pretty output in dev
        if __import__('os').environ.get('ENV') != 'production'
        else structlog.processors.JSONRenderer()
    ],
    wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
    context_class=dict,
    logger_factory=structlog.PrintLoggerFactory(),
)

logger = structlog.get_logger()

Now every log entry carries context:

# Bind request-scoped context once
log = logger.bind(request_id="req-abc123", user_agent="curl/8.4")

log.info("auth_check_passed", user_id=42, method="api_key")
log.info("db_query_completed", query="SELECT orders", rows=15, duration_ms=12)
log.warning("connection_pool_exhausted", active=20, max=20, waiting=3)

The request_id is automatically injected into every log line from that scope. No more guessing which lines belong together.

One thing I learned the hard way: always use structlog.processors.UnicodeDecoder() in the processor chain. Without it, log entries with non-ASCII characters (think user-submitted names with accents or emoji) crash the JSON serializer silently. Add it right after the TimeStamper:

structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.UnicodeDecoder(),  # <-- save yourself a headache

Step 3: Add a /health Endpoint That Tells the Truth

Our old health check:

@app.get("/health")
async def health():
    return {"status": "ok"}

This returns 200 even when the database is unreachable and Redis is on fire. It lied to our load balancer for three months before we found out.

The new one:

@app.get("/health")
async def health():
    checks = {}

    # Check database
    try:
        await db.execute("SELECT 1")
        checks["database"] = "ok"
    except Exception as e:
        checks["database"] = f"error: {e}"
        logger.error("health_check_failed", component="database", error=str(e))

    # Check Redis
    try:
        await redis.ping()
        checks["redis"] = "ok"
    except Exception as e:
        checks["redis"] = f"error: {e}"
        logger.error("health_check_failed", component="redis", error=str(e))

    all_ok = all(v == "ok" for v in checks.values())
    status_code = 200 if all_ok else 503

    return JSONResponse(
        content={"status": "ok" if all_ok else "degraded", "components": checks},
        status_code=status_code,
    )

Now our load balancer actually knows when to stop sending traffic. And the structured error logs from this endpoint — with component and error fields — make it trivially searchable:

jq 'select(.event == "health_check_failed") | {component, error, timestamp}' app.log

What Changed (Numbers)

Before	After
40 min to find root cause at 2 AM	Under 2 min
200 MB plaintext logs / day	60 MB structured JSON (gzip compresses better)
Guessing which lines belong to one request	`request_id` links everything
`grep 500 app.log	wc -l`
"Everything is fine" health check	Real component-level health

The disk savings surprised me. Structured JSON looks bigger, but since identical keys repeat across lines, gzip compression is much more effective. Our daily log volume dropped by 70% after switching.

What I'd Do Differently Next Time

Don't log request bodies by default. We logged full POST bodies for debugging and accidentally logged a credit card number. Use a middleware that redacts known sensitive fields before the logger sees them.
Standardize event names. "request_completed", "db_query_completed", "auth_check_passed" — pick a naming convention and stick to it. Inconsistent event names make queries useless.
Log context, not narratives. Don't write "The user tried to log in but the password was wrong." Write {"event": "login_failed", "reason": "invalid_password"}. Let the dashboard compose the story.

Structured logging won't win you a promotion, but it will save you from 2 AM panic-grepping. That's a win in my book.

What's your logging setup? Still grepping, or have you switched to something better? Let me know in the comments.

The 4 HTTP Status Codes Every API Developer Gets Wrong (And What Actually Works)

Sir Max — Mon, 20 Jul 2026 02:13:16 +0000

The 4 HTTP Status Codes Every API Developer Gets Wrong (And What Actually Works)

I've reviewed hundreds of API designs over the past few years, and one pattern keeps showing up: developers consistently misuse the same handful of HTTP status codes. It's not laziness — it's that most of us learned from frameworks that picked sensible defaults for us, and we never stopped to question them.

Here are the four most commonly misused status codes, why the "obvious" choice is wrong, and what you should use instead.

1. 200 OK for Everything (Then an Error in the Body)

This is the granddaddy of all API mistakes. GraphQL made it famous, but I see it everywhere:

HTTP/1.1 200 OK

{
  "success": false,
  "error": "Invalid email address",
  "code": "VALIDATION_ERROR"
}

Why it's wrong: HTTP status codes exist so that infrastructure can make decisions without parsing your body. Load balancers, CDNs, monitoring tools, and client libraries all look at the status code first. When your error responses return 200, you break:

Retry logic (clients retry 5xx, not 2xx)
Circuit breakers (they open on error rates, which you're hiding)
Caching layers (they cache 200s aggressively)
Health checks (your service looks healthy even when everything is failing)
Log aggregation (your error rate metric becomes meaningless)

What to use instead:

Scenario	Correct Code
Validation failure	`400 Bad Request`
Missing resource	`404 Not Found`
Auth required	`401 Unauthorized`
Permission denied	`403 Forbidden`
Business rule violation	`409 Conflict` or `422 Unprocessable Entity`

Here's what a proper error response looks like:

HTTP/1.1 422 Unprocessable Entity

{
  "error": "validation_failed",
  "message": "Email address is invalid",
  "details": [
    {
      "field": "email",
      "reason": "not_a_valid_email_format",
      "value": "not-an-email"
    }
  ]
}

Your monitoring dashboard will thank you.

2. 404 Not Found for Permission Denied

I see this one constantly in multi-tenant APIs. A user requests a resource that exists but doesn't belong to them, and the API returns:

HTTP/1.1 404 Not Found
{"error": "Resource not found"}

Why it's wrong: 404 tells the client "this thing doesn't exist." But it does exist — the user just isn't allowed to see it. The distinction matters:

A 404 is safe to log as a dead link or a typo.
A 403 is a security event. Someone tried to access something they shouldn't.

If you return 404 for permission errors, your security monitoring can't distinguish between "someone followed a broken link" and "someone is probing for resources they shouldn't see."

What to do:

# Instead of this:
if resource.owner_id != current_user.id:
    raise HTTPException(status_code=404)  # Wrong!

# Do this:
if resource.owner_id != current_user.id:
    raise HTTPException(status_code=403, detail="You don't have access to this resource")

# Or, to prevent information disclosure about what resources exist:
if not resource or resource.owner_id != current_user.id:
    raise HTTPException(status_code=404)  # Okay if you intentionally hide existence

The nuance: If you're deliberately preventing enumeration attacks — where an attacker probes IDs to discover what exists — 404 is acceptable as a security measure. But be intentional about it. Don't do it just because your ORM's get_or_404 helper made it the path of least resistance.

3. 500 Internal Server Error for User Mistakes

A client sends bad JSON, and your server logs this:

ERROR: JSONDecodeError at line 1 column 45

Then returns:

HTTP/1.1 500 Internal Server Error

Why it's wrong: 500 means "something went wrong on the server." But malformed input is a client problem. The server is working fine. You've just trained your on-call engineer to wake up for a problem the client caused.

Every major framework has this gotcha. Express, FastAPI, Rails — they all let unhandled parse errors bubble up to 500 by default.

What to do:

# FastAPI example
from fastapi import Request
from fastapi.responses import JSONResponse

@app.exception_handler(RequestValidationError)
async def validation_handler(request: Request, exc: RequestValidationError):
    return JSONResponse(
        status_code=400,
        content={
            "error": "invalid_request",
            "message": "The request body could not be parsed",
            "details": exc.errors()
        }
    )

The rule is simple: if the server could fix it by asking the client to send different data, it's a 4xx. If the server operator needs to fix something, it's a 5xx. Bad JSON falls squarely in the first category.

4. 201 Created When Nothing Was Created

I blame tutorial culture for this one. Every REST API tutorial shows:

POST /users → 201 Created

So developers return 201 for every POST. But sometimes a POST doesn't create anything new:

POST /api/orders/42/cancel
HTTP/1.1 201 Created  ← wait, what?

Why it's wrong: 201 includes a Location header pointing to the new resource. If you're not returning a Location header that the client can GET, you probably shouldn't be using 201.

What to use instead:

POST Result	Correct Code	Header
New resource created	`201 Created`	`Location: /resource/123`
Action processed, nothing new	`200 OK`	—
Action accepted, processing later	`202 Accepted`	—
Action completed, see other resource	`303 See Other`	`Location: /other-resource`

# Cancel an order — nothing is created
POST /api/orders/42/cancel
HTTP/1.1 200 OK
{"status": "cancelled", "refund_id": "ref_789"}

# Submit a report that'll be generated later
POST /api/reports
HTTP/1.1 202 Accepted
{"report_id": "rep_456", "status": "queued"}

A Quick Self-Audit for Your API

Pull your API logs from the last 24 hours and run this checklist:

Search for 200 + "error" or "success": false in the response body. That's your 200-for-errors problem.
Search for all 500 responses and check what caused them. How many were bad client input that should have been a 4xx?
Search for 404 and check how many are permission issues you're hiding. Are you leaking information about what resources exist?
Search for 201 and verify every single one returns a Location header. If not, it should probably be a 200 or 202.

I ran this audit on our own API last quarter and found that 22% of our 500s were actually 4xx errors in disguise. Fixing that one thing dropped our on-call alert noise by almost a quarter.

The Bigger Picture

HTTP status codes aren't pedantry. They're the interface between your API and everything that sits in front of it — proxies, caches, CDNs, monitoring, client SDKs. When you get them right, your infrastructure works with you instead of against you.

The next time you're about to type status_code=200, take two seconds to ask: "What actually happened here, and who needs to know about it?"

Your on-call engineer — and your users — will thank you.

What's the worst HTTP status code abuse you've seen in the wild? I once encountered an API that returned 418 I'm a Teapot for rate limiting. It was creative, but debugging it at 3 AM was not fun.

Why Your API Needs Idempotency Keys (And How to Implement Them in 10 Minutes)

Sir Max — Sun, 19 Jul 2026 02:06:15 +0000

Why Your API Needs Idempotency Keys (And How to Implement Them in 10 Minutes)

Last month, a user of our payment API got charged three times for a single transaction. The culprit? A flaky mobile network that retried the same POST request three times before the client finally got a 200 response. The server processed all three.

If you've ever built an API that handles money, inventory, or anything stateful, you've probably encountered this problem. The solution is simpler than you think: idempotency keys.

What Is Idempotency?

An operation is idempotent if performing it multiple times produces the same result as doing it once. In HTTP terms:

GET — naturally idempotent (fetching a resource doesn't change it)
PUT — naturally idempotent (setting a resource to state X gives the same result every time)
POST — not idempotent (creating a new resource each time)
DELETE — technically idempotent (deleting something that's already deleted is still… deleted)

The problem lives in POST. Every time a client sends a POST, a new resource gets created. If the client retries because of a network timeout, you get duplicates.

The Idempotency Key Pattern

Here's the core idea: the client generates a unique key for each intended operation and sends it as a header. The server uses this key to detect duplicates.

Client → POST /payments
         Idempotency-Key: 4f8b3a2c-1d9e-4e7f-a3b1-c6d8e9f0a1b2
         Body: {"amount": 49.99, "currency": "USD"}

Server → 201 Created (first time)

Client → POST /payments  (network retry — same key!)
         Idempotency-Key: 4f8b3a2c-1d9e-4e7f-a3b1-c6d8e9f0a1b2
         Body: {"amount": 49.99, "currency": "USD"}

Server → 200 OK  (returns the original response, no duplicate!)

That's it. The magic is that the server remembers "I've already processed this key" and returns the cached response instead of creating a duplicate.

Implementation in 10 Minutes

Here's a production-ready implementation. We'll use Redis for storage since it's fast and has built-in TTL support.

Step 1: The Middleware (FastAPI)

import hashlib
import json
from fastapi import Request, HTTPException
import redis.asyncio as redis
from functools import wraps

redis_client = redis.Redis(host="localhost", port=6379, decode_responses=True)

IDEMPOTENCY_TTL = 86400  # 24 hours

def idempotent(func):
    @wraps(func)
    async def wrapper(*args, **kwargs):
        request: Request = kwargs.get("request") or args[0]
        idempotency_key = request.headers.get("Idempotency-Key")

        if not idempotency_key:
            raise HTTPException(
                status_code=400,
                detail="Idempotency-Key header is required"
            )

        # Check if we've seen this key before
        cached = await redis_client.get(f"idem:{idempotency_key}")

        if cached:
            cached_data = json.loads(cached)
            return JSONResponse(
                content=cached_data["body"],
                status_code=cached_data["status"],
                headers=cached_data.get("headers", {})
            )

        # Key doesn't exist — process the request
        response = await func(*args, **kwargs)

        # Cache the response before returning
        response_data = {
            "status": response.status_code,
            "body": json.loads(response.body.decode()),
            "headers": dict(response.headers),
        }
        await redis_client.setex(
            f"idem:{idempotency_key}",
            IDEMPOTENCY_TTL,
            json.dumps(response_data)
        )

        return response

    return wrapper

Step 2: Use It on Your Endpoint

@app.post("/payments")
@idempotent
async def create_payment(request: Request):
    body = await request.json()
    payment = await process_payment(body["amount"], body["currency"])
    return JSONResponse(
        content={"id": payment.id, "status": payment.status},
        status_code=201,
    )

That's it. Every POST to /payments now requires an Idempotency-Key header, and duplicates return the exact same response.

Handling Concurrent Requests

There's one edge case: what if two requests with the same key arrive at the same time? Without a lock, both might pass the cache check, both process the payment, and both try to write the response.

The fix: atomic SET NX (set if not exists). Redis supports this natively:

# Replace the simple GET + SET with an atomic operation
lock_key = f"idem:lock:{idempotency_key}"

# Try to acquire the lock
acquired = await redis_client.setnx(lock_key, "processing")
if not acquired:
    # Another request is processing this key — wait and retry
    for _ in range(30):  # 30 × 100ms = 3 second timeout
        await asyncio.sleep(0.1)
        cached = await redis_client.get(f"idem:{idempotency_key}")
        if cached:
            return json.loads(cached)["body"]
    raise HTTPException(status_code=409, detail="Request is still processing")

try:
    # Process the request...
    pass
finally:
    # Always clean up the lock
    await redis_client.delete(lock_key)

What I Learned the Hard Way

I've implemented this pattern across three different systems. Here's what the docs don't tell you:

1. Key generation matters more than you think

Don't let clients generate random UUIDs and call it a day. The key should be derived from the intent of the request — a hash of (user_id + operation_type + idempotent_payload). If a user accidentally sends two different amounts with the same key, you want to reject the second one, not silently return the first.

# Better approach: validate that the payload matches
async def wrapper(*args, **kwargs):
    ...
    body_hash = hashlib.sha256(
        json.dumps(body, sort_keys=True).encode()
    ).hexdigest()

    if cached:
        if cached_data["body_hash"] != body_hash:
            raise HTTPException(
                status_code=422,
                detail="Idempotency-Key reused with different payload"
            )

2. 24 hours is not always the right TTL

Stripe uses 24 hours. Shopify uses 72. I use 24 hours for payments and 1 hour for everything else. The tradeoff: longer TTL = more Redis memory = better protection against very-late retries. Pick based on your domain.

3. Don't idempotent everything

Not every POST needs this. Read-only operations, analytics events (where duplicates are acceptable), and fire-and-forget notifications don't benefit from idempotency keys. Applying it everywhere adds latency and Redis overhead for no gain.

Use idempotency keys when:

Money is involved (payments, refunds, credits)
Inventory changes (stock decrements, reservation creation)
State transitions (order status changes, account activation)

Skip idempotency keys for:

Logging / analytics events
Search queries
Email/sms notifications (duplicates are annoying but not catastrophic)

The Stripe Standard

If you want to see this pattern in the wild, look at Stripe's API. They pioneered the Idempotency-Key header and their implementation is the gold standard:

Keys expire after 24 hours
POST requests with the same key and different body return a 422
Their client libraries auto-generate keys from (method + path + body hash)
They store the entire response including HTTP status code and headers

This is the pattern I recommend copying directly. It's battle-tested across billions of requests.

Quick Reference: The Copypaste Version

Here's everything you need in one place. Copy, paste, adapt:

Header:    Idempotency-Key: <uuid>
Storage:   Redis with 24h TTL
Strategy:  GET key → if exists return cached → if not, SET NX lock → process → SET response → DELETE lock
Conflict:  Same key + different body → 422
Missing:   No key header → 400

Wrapping Up

Adding idempotency keys to your API takes about 10 minutes of coding and saves you from the kind of 3 AM production incident where a user got charged multiple times and your phone won't stop buzzing.

Start with your payment endpoints. Then expand to anything that mutates state in a way you'd regret seeing twice. Your future self (and your users) will thank you.

Have you implemented idempotency keys in your API? What storage backend did you use? I'd love to hear about alternatives to Redis — let me know in the comments.

Stop Mocking Your APIs — A Practical Guide to Contract Testing That Actually Works

Sir Max — Sat, 18 Jul 2026 02:06:47 +0000

Stop Mocking Your APIs — A Practical Guide to Contract Testing That Actually Works

Last year, we shipped a feature that broke three downstream services. The unit tests passed. The integration tests passed. The mock server returned exactly what we expected. And yet, production was on fire at 2 PM on a Tuesday.

The problem? Our mock server was lying to us.

Here's what I learned, and how we fixed it.

The Mock Server Trap

Mocks are seductive. You write a test:

# test_user_service.py
mock_api.get("/users/42").returns({"id": 42, "name": "Alice", "role": "admin"})

result = user_service.get_user_name(42)
assert result == "Alice"  # ✅ PASS

Everything is green. You deploy. Then you learn that the real API changed name to display_name last month, and nobody told you. Your mock was faithfully returning a field the API no longer sends.

Mocks test what you think the API looks like. They don't test what the API actually looks like.

What Contract Testing Actually Is

Contract testing verifies that two services agree on how they communicate. Think of it like an API prenup: both sides sign off on a shared understanding, and if either side wants to change it, the other gets notified.

There are two flavors:

Approach	Works by	Best for
Consumer-driven	Consumer defines expectations, provider verifies them	Internal microservices
Provider-driven	Provider publishes a spec, consumers verify against it	Public APIs, SDKs

I'll focus on consumer-driven because that's where mocks cause the most damage.

A Real Example: Payment Service

Imagine you're building an order service that calls a payment API. The payment team maintains their own codebase, and you have no control over it.

Step 1: The Consumer Defines What They Need

# order_service/tests/contract/test_payment_contract.py
import pytest
from pact import Consumer, Provider

pact = Consumer("OrderService").has_pact_with(Provider("PaymentService"))

def test_create_payment():
    # "Here's what I expect the payment API to return"
    expected = {
        "payment_id": "pay_abc123",
        "status": "captured",
        "amount": 2999,
        "currency": "usd"
    }

    (pact
        .given("a valid payment request")
        .upon_receiving("a POST to /payments")
        .with_request("POST", "/payments", body={
            "order_id": "order_555",
            "amount": 2999,
            "currency": "usd"
        })
        .will_respond_with(200, body=expected))

    with pact:
        result = create_payment("order_555", 2999, "usd")
        assert result["payment_id"] == "pay_abc123"

This generates a pact file — a JSON document describing what OrderService expects from PaymentService.

Step 2: The Provider Verifies It Can Meet Those Expectations

# payment_service/tests/contract/test_provider.py
import pytest
from pact import Verifier

def test_payment_provider_contracts():
    verifier = Verifier(provider="PaymentService")

    # Run against the REAL PaymentService (not a mock)
    verifier.verify_pacts(
        "./pacts/orderservice-paymentservice.json",
        url="http://localhost:8080"  # ← The actual service
    )

When this runs, it sends real HTTP requests to your running PaymentService. If the response doesn't match what OrderService expects, the test fails — before anyone deploys.

Step 3: Share the Pact

The pact file lives in a shared location (Pact Broker, or even a Git repo). Both teams run their contract tests in CI. When the Payment team wants to rename status to payment_status, they'll see:

FAILED: OrderService expects field 'status' in response body

And they'll either coordinate the change with you or version the API.

Three Things I Wish I Knew When I Started

1. Contract Tests Are NOT a Replacement for E2E Tests

A contract test says "this specific interaction works." It does not say "the entire business flow works." You still need end-to-end tests for critical paths. Contract tests sit between unit tests and E2E tests on the testing pyramid.

     /\
    /E2E\       ← Few, slow, cover full flows
   /------\
  /Contract\    ← Medium count, verify API agreements
 /----------\
/   Unit     \  ← Many, fast, verify logic
--------------

2. Start with Your Most Fragile Integration

Don't contract-test everything on day one. Pick the integration that breaks most often — in my case, it was the payment service. Write contracts for 2-3 key endpoints. Prove the value. Then expand.

We started with 3 contracts. Six months later we had 47. The number of "API changed without telling us" incidents dropped from roughly 2 per month to zero.

3. Pacts Are Not Versioned by Default — Handle It

If you store pact files in Git, tag them with provider versions. Otherwise, when the PaymentService team deploys v2, you're still verifying against v1 pacts from your CI cache.

# In CI, publish with a version tag
pact-broker publish ./pacts \
  --consumer-app-version=$(git rev-parse HEAD) \
  --branch=$(git rev-parse --abbrev-ref HEAD)

The "Can I Deploy?" Question

This is where contract testing earns its keep. With a Pact Broker set up, you can ask:

https://broker.pact.io/can-i-deploy?pacticipant=PaymentService&version=abc123

The broker checks: Does this version of PaymentService satisfy every consumer's expectations? If yes → safe to deploy. If no → here's who you'd break.

No more guessing.

When NOT to Use Contract Testing

You control both sides and can deploy them together. If you're a solo dev with two services in the same repo, contract testing adds overhead with little benefit.
The API is extremely stable. If you're consuming GitHub's v3 REST API, it hasn't changed in years. A simple mock is fine.
You're prototyping. Don't add contracts to a POC that might be deleted in two weeks.

Getting Started (In Under 30 Minutes)

# Install Pact for Python
pip install pact-python

# Or for Node.js
npm install @pact-foundation/pact

# Or for Go / Java / Ruby / Rust — Pact has bindings for everything

Then write your first consumer test (5 minutes), run your provider against it (5 minutes), and share the pact file (2 minutes). The hardest part is convincing another team to run the provider verification in their CI — but once they see it catch a breaking change, they'll be on board.

What Changed for My Team

We went from:

"The payment API changed again and broke staging" — weekly occurrence
"Can we deploy? I don't know, check the mocks" — constant uncertainty
"The integration tests take 45 minutes" — E2E bloat

To:

Breaking API changes caught in CI, before merge
"Can I deploy?" answered by Pact Broker in 2 seconds
Integration test suite down to 12 minutes (contract tests replaced half the E2E scenarios)

No magic. Just agreeing on what the API should look like, and then verifying that agreement automatically.

What's your experience with API testing? Have you tried contract tests, or do you still rely on mocks? I'd love to hear what's worked (or failed spectacularly) in your stack.

How to Add WebSocket Support to Your REST API Without Starting Over

Sir Max — Fri, 17 Jul 2026 02:11:41 +0000

How to Add WebSocket Support to Your REST API Without Starting Over

Your users want real-time updates. Your REST API works fine. Do you really need to rewrite everything?

I faced this exact question six months ago. Our dashboard had a "refresh" button that users clicked obsessively — sometimes 20-30 times a minute during peak hours. The polling was hammering our servers, and the user experience was terrible. But the thought of migrating our entire REST API to WebSockets? That felt like a 3-month project nobody had time for.

Turns out, you don't need to rewrite anything. Here's how we added real-time capabilities alongside our existing REST API in under 200 lines of code.

The Approach: REST + WebSocket Sidecar

Instead of replacing REST endpoints, we added a thin WebSocket layer that sits next to them. REST handles CRUD operations as before. WebSocket handles real-time events. They share the same business logic and database — no duplication.

Client ←→ REST API (CRUD)     ←→ Business Logic ←→ DB
Client ←→ WebSocket (events)  ←→ Event Bus     ←→ DB

The key insight: WebSocket is for notifications, not data fetching. When something changes, the server pushes a lightweight event ("order #42 status changed to shipped"). The client then fetches the full data via REST if it needs details. This keeps your WebSocket messages small and your REST endpoints unchanged.

Step 1: Add a WebSocket Endpoint to Your Existing Server

Here's the FastAPI version (the same pattern works for Express, Go net/http, or any framework):

# server.py — add this to your existing FastAPI app

import asyncio
import json
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.staticfiles import StaticFiles
from typing import Set

app = FastAPI()

# --- Your existing REST routes stay untouched ---
@app.get("/api/orders/{order_id}")
async def get_order(order_id: int):
    return await db.fetch_order(order_id)

@app.post("/api/orders")
async def create_order(order: OrderCreate):
    new_order = await db.insert_order(order)
    # 🔥 This is the only new line — broadcast the event
    await ws_manager.broadcast("order_created", {"id": new_order.id})
    return new_order

# --- New: WebSocket manager (50 lines) ---
class WSManager:
    def __init__(self):
        self.connections: Set[WebSocket] = set()

    async def connect(self, ws: WebSocket):
        await ws.accept()
        self.connections.add(ws)

    def disconnect(self, ws: WebSocket):
        self.connections.discard(ws)

    async def broadcast(self, event_type: str, data: dict):
        message = json.dumps({"type": event_type, "data": data})
        dead = set()
        for ws in self.connections:
            try:
                await ws.send_text(message)
            except Exception:
                dead.add(ws)
        self.connections -= dead

ws_manager = WSManager()

@app.websocket("/ws")
async def websocket_endpoint(ws: WebSocket):
    await ws_manager.connect(ws)
    try:
        while True:
            # Keep connection alive, handle client pings
            data = await ws.receive_text()
            if data == "ping":
                await ws.send_text("pong")
    except WebSocketDisconnect:
        ws_manager.disconnect(ws)

That's it. Your existing REST routes are unchanged. The only addition is ws_manager.broadcast() calls wherever state changes happen.

Step 2: The Client Side — Subscribe, Don't Poll

Here's a tiny client that replaces your polling loop:

// client.js
class RealTimeClient {
    constructor(url) {
        this.ws = new WebSocket(url);
        this.handlers = new Map();

        this.ws.onmessage = (event) => {
            const { type, data } = JSON.parse(event.data);
            const handler = this.handlers.get(type);
            if (handler) handler(data);
        };

        // Reconnect with backoff
        this.ws.onclose = () => {
            setTimeout(() => this.connect(url),
                Math.min(30000, 1000 * 2 ** this.retries++));
        };
    }

    on(eventType, callback) {
        this.handlers.set(eventType, callback);
    }

    // Keep alive
    startHeartbeat(intervalMs = 30000) {
        setInterval(() => {
            if (this.ws.readyState === WebSocket.OPEN) {
                this.ws.send("ping");
            }
        }, intervalMs);
    }
}

// Usage — 10 lines to replace polling
const client = new RealTimeClient("wss://api.example.com/ws");
client.startHeartbeat();

client.on("order_created", (order) => {
    addOrderToUI(order);  // Optimistic update
});

client.on("order_updated", async (data) => {
    // Fetch full details via REST if needed
    const order = await fetch(`/api/orders/${data.id}`).then(r => r.json());
    updateOrderInUI(order);
});

The client subscribes to event types it cares about. No more setInterval(fetchOrders, 3000). No more wasted requests when nothing changed.

The Events We Track

We started with just three event types and expanded over time:

Event	Triggers	Payload
`order_created`	New order placed	`{id, status, total}`
`order_updated`	Status change, edit	`{id, changes: {}}`
`system_alert`	Rate limit hit, error spike	`{level, message}`

Lightweight payloads. If the client needs full details, it hits the REST endpoint — which is already cached and fast because it's unchanged.

Three Things I Wish I Knew Earlier

1. You need a heartbeat. No, really.

WebSocket connections die silently. Firewalls drop idle connections after 60-120 seconds. Load balancers do the same. Without a ping/pong heartbeat, your users will see "connected" but receive no updates — and they won't know why.

We learned this the hard way: our first deployment worked perfectly in development, then broke entirely in production behind AWS ALB, which kills idle WebSocket connections after 60 seconds. A 30-second heartbeat fixed it.

2. Authentication belongs on the WebSocket, not just REST

Don't rely on "they already authenticated via REST." WebSocket connections are separate. Pass a token during the initial handshake:

@app.websocket("/ws")
async def websocket_endpoint(ws: WebSocket):
    # Verify token before accepting
    token = ws.query_params.get("token", "")
    user = await verify_token(token)
    if not user:
        await ws.close(code=4001)
        return
    await ws.accept()
    # Store user context with the connection
    await ws_manager.connect(ws, user_id=user.id)

3. Start with one event type

When we first planned this, we mapped out 23 event types we "needed." Two weeks later, we launched with 3. The other 20 either weren't actually needed, or we added them organically when users asked.

Start with the one event that causes the most polling pain. Ship it. Add more as you learn what users actually want.

Performance: What Changed

After deploying this alongside our existing REST API:

Metric	Before (Polling)	After (WebSocket)
Avg requests/sec	340	85
p99 API latency	180ms	45ms
Dashboard update delay	3 seconds	< 200ms
Server CPU (peak)	72%	38%

The drop in REST traffic was immediate — about 75% reduction because polling was our biggest traffic source. Users got updates faster, servers cost less. Win-win.

When NOT to Do This

WebSocket isn't always the answer:

If your data changes once a day, polling every 5 minutes is simpler and perfectly fine.
If you're on serverless (Lambda/Cloud Functions), WebSockets are awkward. Use AWS API Gateway WebSocket support or consider Server-Sent Events (SSE) instead.
If your team is 1-2 people and polling works, don't over-engineer. The "refresh" button works. Ship features, not architecture.

The Bottom Line

Adding real-time to a REST API doesn't require a rewrite. Add a WebSocket endpoint, broadcast events from your existing mutation handlers, and give clients a simple subscription model. Three event types, a heartbeat, and token auth — that's 90% of what you need.

The hardest part isn't the code. It's resisting the urge to over-design it before you ship.

Have you added real-time features to an existing API? What approach did you take? I'd love to hear what worked (and what didn't) in the comments.

3 Database Query Patterns That Took Our API From 800ms to 50ms

Sir Max — Thu, 16 Jul 2026 02:09:53 +0000

3 Database Query Patterns That Took Our API From 800ms to 50ms

Last month, our /users/search endpoint was taking 800ms on a good day. Users were complaining. The frontend team added a loading spinner (bless them). But I knew the real problem: our database queries were doing way too much work.

Here are the three patterns that changed everything — with real before/after code.

Pattern 1: Stop SELECT * When You Only Need 3 Columns

This one sounds obvious. Everyone knows you shouldn't SELECT *. But in practice? I kept finding it everywhere.

Here's what our user search query looked like:

-- Before: 320ms avg
SELECT * FROM users
WHERE status = 'active'
  AND last_login > NOW() - INTERVAL '30 days'
ORDER BY created_at DESC
LIMIT 50;

The users table had 47 columns. bio alone was a TEXT field averaging 2KB. We were hauling megabytes of data over the wire for columns the API never returned.

-- After: 45ms avg
SELECT id, username, avatar_url, created_at
FROM users
WHERE status = 'active'
  AND last_login > NOW() - INTERVAL '30 days'
ORDER BY created_at DESC
LIMIT 50;

A covering index sealed the deal:

CREATE INDEX idx_users_search ON users (status, last_login, created_at)
INCLUDE (username, avatar_url);

Lesson: Don't just avoid SELECT * — audit what your ORM is actually fetching. Django's only() and SQLAlchemy's load_only() exist for a reason.

Pattern 2: N+1 Queries Are the Silent Killer

Every ORM user has heard "beware the N+1 problem." But here's the thing: it doesn't always show up in development because your local DB has 50 rows.

Our /organizations/{id}/members endpoint fetched an org, then looped through members to get each user's profile:

# Before: 600ms for an org with 30 members
org = db.query(Organization).get(org_id)
members = db.query(Member).filter_by(org_id=org_id).all()

result = []
for member in members:
    user = db.query(User).get(member.user_id)  # N queries!
    result.append({
        "id": member.id,
        "role": member.role,
        "username": user.username,
        "avatar": user.avatar_url,
    })

30 members = 31 queries. Scale that to 200 members and you're looking at 2+ seconds.

The fix took two lines:

# After: 55ms (single query with JOIN)
members = (
    db.query(Member, User)
    .join(User, Member.user_id == User.id)
    .filter(Member.org_id == org_id)
    .all()
)

result = [
    {
        "id": m.Member.id,
        "role": m.Member.role,
        "username": m.User.username,
        "avatar": m.User.avatar_url,
    }
    for m in members
]

If you can't restructure to a JOIN, use selectinload (SQLAlchemy) or prefetch_related (Django) to batch it into 2 queries.

How to catch these: Enable SQL query logging in your dev environment. If you see the same query pattern repeating with different IDs, you've got an N+1.

Pattern 3: Paginate With Keys, Not Offsets

This one was the biggest surprise. Our activity feed endpoint used OFFSET pagination:

-- Before: 180ms on page 1, 2400ms on page 50
SELECT * FROM activities
WHERE user_id = $1
ORDER BY created_at DESC
LIMIT 20 OFFSET 1000;

OFFSET 1000 means the database reads 1020 rows, then throws away the first 1000. On page 50 with 10M rows in the table, the query planner gives up and does a full sequential scan.

Keyset pagination (also called "cursor-based") uses a WHERE clause on the sorted column:

-- After: 8ms on any page
SELECT * FROM activities
WHERE user_id = $1
  AND created_at < $2  -- the cursor
ORDER BY created_at DESC
LIMIT 20;

The API contract changes slightly — you return a next_cursor instead of total_pages:

{
  "data": [...],
  "next_cursor": "2026-07-10T14:32:00Z"
}

Tradeoff: You can't jump to page 50. But honestly — when was the last time you clicked page 50 of anything? Most users never go past page 3. The speed gain is worth it.

The Numbers

Endpoint	Before	After	Improvement
`/users/search`	320ms	45ms	86% faster
`/orgs/{id}/members`	600ms	55ms	91% faster
`/activities` (page 50)	2400ms	8ms	99.7% faster

Total effort: one afternoon of profiling, one morning of refactoring.

What I'd Do Differently Next Time

Profile in production first. My local DB had 500 rows. Production had 10 million. The N+1 issue was invisible locally.
Add query time metrics to your APM. A dashboard showing p50/p95/p99 per endpoint would have caught this months earlier.
Write a linter rule for SELECT *. Seriously. Two hours of writing a SQL lint rule saved me from ever finding this pattern in code review again.

The database is usually the bottleneck. But it's also usually the easiest thing to fix once you know where to look.

How I Shrunk Our Docker Images from 1.2GB to 85MB Using Multi-Stage Builds

Sir Max — Wed, 15 Jul 2026 02:11:43 +0000

How I Shrunk Our Docker Images from 1.2GB to 85MB Using Multi-Stage Builds

Last month, a junior dev on my team pushed a Docker image to our registry. The CI pipeline sat there for 11 minutes. When I checked, the image was 1.2 GB. For a Python web app with 3 dependencies.

I've been there. You probably have too. The apt-get install creep. The forgotten pip install cache. The "I'll clean it up later" that never happens.

Here's how I fixed it — and how you can too.

The Problem With Fat Images

Every Docker image bloats for the same reasons:

Build dependencies leak into runtime. You need gcc to compile a package, but you don't need it to run the app.
Package manager caches. apt-get update downloads index files you'll never use again.
Layers are additive. Each RUN line creates a new layer. Delete a file in layer 5? It's still sitting in layer 3, eating space.
"Just use ubuntu:latest" as a base image. That's 77 MB before you even start.

The fix isn't adding more rm -rf commands. It's multi-stage builds.

The Before: A Typical Bloated Dockerfile

Here's a real Python app Dockerfile I found in production:

FROM python:3.11

RUN apt-get update && apt-get install -y \
    gcc g++ make \
    libpq-dev libffi-dev \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
RUN pip install -e .

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Result: 1.18 GB

The python:3.11 image alone is ~920 MB. Add gcc, g++, make, and compiled wheels, and you're past 1 GB. The app itself? Maybe 15 MB of actual code and dependencies.

The Fix: Multi-Stage Builds

Multi-stage builds let you use one image for building and another for running. Here's the pattern:

# Stage 1: Build
FROM python:3.11-slim AS builder

RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc g++ make libpq-dev libffi-dev \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Stage 2: Runtime
FROM python:3.11-slim

# Only copy what we actually need from the builder
COPY --from=builder /root/.local /root/.local

# Install only runtime system deps
RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq5 \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY . .

ENV PATH="/root/.local/bin:$PATH"

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Result: 210 MB — already 5.6x smaller.

What changed:

Switched from python:3.11 (~920 MB) to python:3.11-slim (~120 MB)
Build tools (gcc, g++, make) stay in the builder stage — never reach the final image
pip install --user keeps packages isolated, making COPY --from=builder clean
--no-install-recommends on apt skips unnecessary packages

Taking It Further: Alpine and Distroless

210 MB is better, but we can go further.

Option 1: Alpine Base

FROM python:3.11-alpine AS builder

RUN apk add --no-cache gcc musl-dev libffi-dev postgresql-dev make

WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

FROM python:3.11-alpine

RUN apk add --no-cache libpq
COPY --from=builder /root/.local /root/.local
WORKDIR /app
COPY . .

ENV PATH="/root/.local/bin:$PATH"
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Result: ~85 MB

Trade-off: Alpine uses musl libc instead of glibc. If you're compiling C extensions (looking at you, psycopg2), test thoroughly before deploying.

Option 2: Distroless (for Go/Rust apps)

If you're building a compiled language, distroless is even more extreme:

FROM golang:1.22-alpine AS builder

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -ldflags="-w -s" -o /app/server .

FROM gcr.io/distroless/static-debian12

COPY --from=builder /app/server /server
ENTRYPOINT ["/server"]

Result: ~14 MB for a full Go web server.

No shell. No package manager. No ls, no cat, no bash. Just your binary and a CA certificate bundle.

The Numbers: Before and After

Strategy	Image Size	Build Time	Security Surface
`python:3.11`	1.18 GB	4 min	High (780+ packages)
`python:3.11-slim` + multi-stage	210 MB	3 min	Medium (120+ packages)
`python:3.11-alpine` + multi-stage	85 MB	2.5 min	Low (40+ packages)

That's a 93% size reduction. Our CI pipeline went from 11 minutes to 90 seconds — mostly because pulling and pushing 85 MB is a lot faster than 1.2 GB.

Three Gotchas I Hit (So You Don't Have To)

1. Alpine's DNS Resolution

python:3.11-alpine doesn't include nss by default, which breaks DNS resolution in some network setups. If your app can't resolve hostnames in Kubernetes, add:

RUN apk add --no-cache nss

This adds ~2 MB but saves hours of debugging.

2. `pip install --user` and PATH

If you forget ENV PATH="/root/.local/bin:$PATH", your runtime container won't find uvicorn, gunicorn, or any CLI tool installed via pip. The build succeeds. The deploy fails at runtime. Ask me how I know.

3. COPY Order Matters

Docker caches layers. Put COPY requirements.txt . and pip install before COPY . .. Otherwise, changing one line of source code invalidates the pip layer cache, and you're reinstalling dependencies on every build.

The Real ROI

The image size isn't just vanity metrics. Smaller images mean:

Faster deploys. When Kubernetes pulls a new pod image, 85 MB pulls in seconds. 1.2 GB can take minutes.
Lower registry costs. Most registries charge by storage. AWS ECR is $0.10/GB/month. Across 50 images, that adds up.
Smaller attack surface. Fewer packages = fewer CVEs. An alpine image has roughly 40 packages. A full ubuntu image has hundreds.
Happier developers. Nobody wants to wait 10 minutes for a CI run.

What To Do Today

Open your biggest Dockerfile
Run docker images | grep your-app and note the size
Switch the base image from :latest to :slim or :alpine
Split build and runtime into separate stages
Copy only what you need with COPY --from=builder

The first time I did this, it took 20 minutes. The payoff was 93% smaller images, forever.

Have a Docker optimization story? Drop a comment — I'm always looking for new tricks.

The 5-Minute Health Check That Saved My Team 40 Hours of Debugging

Sir Max — Tue, 14 Jul 2026 02:08:11 +0000

The 3 AM Wake-Up Call

Two years ago I got paged at 3 AM. The monitoring dashboard was screaming that our payment service was down. I logged into the server, checked the process list — the service was running. Checked the port — it was listening. Checked the logs — requests were coming in and responses were going out.

But payments were failing. Why? Because the database connection pool had silently exhausted, and every request was timing out after 30 seconds.

Here's the infuriating part: we had a /health endpoint. It returned 200 OK with {"status": "alive"}. The load balancer was happy. The process supervisor was happy. Customers were not.

That night I learned the difference between a liveness check and a health check — and it cost us 40 hours of debugging over the following month to untangle the cascading failures.

Liveness ≠ Health

Most teams conflate these two things:

	Liveness Check	Health Check
Answers	"Is the process running?"	"Can the process do its job?"
Checks	Port is open, HTTP responds	DB connection, upstream APIs, disk space, queue depth
Used by	Orchestrator (K8s, systemd)	Load balancer, monitoring, on-call dashboards
When it fails	Restart the container	Stop routing traffic to it

The /health endpoint that returns {"status": "ok"} regardless of whether the database is reachable? That's a liveness check masquerading as a health check. It will lie to your load balancer and make bad situations worse.

The Pattern: 3-Tier Health Checks

After that 3 AM incident I standardized on three tiers:

# health.py — drop this into any service
import time
import json
from typing import Dict, Any

class HealthChecker:
    """3-tier health check for any service."""

    def __init__(self):
        self._checks = {}
        self.start_time = time.time()

    def register(self, name: str, check_fn, tier: str = "critical"):
        """Register a health check function.

        Tiers:
        - critical: service cannot function without this (DB, config)
        - degraded: service is impaired (cache miss, secondary API)
        - informational: nice to know (queue depth, disk usage)
        """
        self._checks[name] = {"fn": check_fn, "tier": tier}

    def run(self) -> Dict[str, Any]:
        results = {}
        overall = "healthy"

        for name, check in self._checks.items():
            try:
                ok, detail = check["fn"]()
                results[name] = {
                    "status": "pass" if ok else "fail",
                    "detail": detail,
                    "tier": check["tier"]
                }
                if not ok and check["tier"] == "critical":
                    overall = "unhealthy"
                elif not ok and check["tier"] == "degraded" and overall == "healthy":
                    overall = "degraded"
            except Exception as e:
                results[name] = {
                    "status": "fail",
                    "detail": str(e),
                    "tier": check["tier"]
                }
                if check["tier"] == "critical":
                    overall = "unhealthy"

        return {
            "status": overall,
            "uptime_seconds": int(time.time() - self.start_time),
            "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
            "checks": results
        }


# ---- Usage in your FastAPI app ----
from fastapi import FastAPI, Response

app = FastAPI()
health = HealthChecker()

# Register checks — these are the actual functions you'd implement
health.register("database", lambda: check_db_connection(), tier="critical")
health.register("redis_cache", lambda: check_redis(), tier="degraded")
health.register("disk_space", lambda: check_disk("/var/data"), tier="informational")
health.register("upstream_payment_api",
    lambda: check_upstream("https://api.payments.example/health"),
    tier="critical")


@app.get("/health")
def health_endpoint():
    result = health.run()
    status_code = 200 if result["status"] == "healthy" else 503
    return Response(
        content=json.dumps(result),
        status_code=status_code,
        media_type="application/json"
    )


@app.get("/health/live")
def liveness():
    """Simple liveness — used by K8s livenessProbe."""
    return {"status": "alive"}


@app.get("/health/ready")
def readiness():
    """Readiness — used by K8s readinessProbe and load balancer."""
    result = health.run()
    if result["status"] == "unhealthy":
        return Response(content=json.dumps(result), status_code=503)
    return result

The key insight: your load balancer should use /health/ready, not /health/live. When the database is down, the process is alive, but it cannot serve traffic. Route requests elsewhere.

What to Check (and What NOT to Check)

After implementing this pattern across 12 services, here's what I learned about which checks are worth it:

Worth Checking

Check	Tier	Why
Database connection (ping + simple query)	critical	Most common silent failure
Upstream API availability	critical/degraded	Catch cascading failures early
Disk space (more than 10 percent free)	degraded	Prevents the "disk full at 2 AM" surprise
Message queue depth	informational	Early warning for backpressure
Configuration validity	critical	Catch bad deploys before traffic hits them
Redis/memcached	degraded	Without cache, service degrades but still works

NOT Worth Checking

Check	Why Not
Deep application logic	Health check is not an integration test suite
Every single downstream service	Pick what's actually critical for THIS service
Synthetic transactions on every call	Use a separate probe for that (run every 60s, not on every health check request)
Anything that takes over 2 seconds	Health checks are called frequently; slow checks cause cascading timeouts

The Kubernetes Wiring

If you're on Kubernetes, here's how to wire this up:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
      - name: my-service
        # Liveness: is the process alive? Restart if not.
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 30
          timeoutSeconds: 3
          failureThreshold: 3

        # Readiness: can this pod serve traffic? Stop routing if not.
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 2

The liveness probe checks /health/live — it should never fail unless the process is truly stuck. The readiness probe checks /health/ready — it fails when the database is down, and traffic stops being routed to that pod. The pod stays alive, waiting for the database to come back.

This separation is the single most impactful thing you can do for service reliability.

The 5-Minute Implementation Checklist

Create a health.py module — copy the HealthChecker class above
Add /health/live and /health/ready endpoints to your service
Register critical dependencies as tier="critical" checks
Wire up your load balancer to use /health/ready, not /health
Test it: kill your database and verify your service returns 503 and gets drained from the load balancer
Add a dashboard widget showing per-service health status — it'll pay for itself the next time something breaks

One Final Rule

If your health check returns 200 OK but can't actually tell you whether the service is working, you don't have a health check. You have a lie.

A health check that lies is worse than no health check at all — because it gives you false confidence while quietly routing traffic to broken instances.

What does your team's health check actually verify? I'd love to hear what checks have saved you from late-night pages.

How I Stopped Getting Paged at 3 AM: Building Resilient API Clients

Sir Max — Sun, 12 Jul 2026 02:06:58 +0000

The 3 AM Phone Call

Two years ago, I woke up to my phone buzzing at 3:14 AM. Our payment processing API was down — again. The third-party provider had another "unexpected outage," and our system was retrying requests in a tight loop, hammering their already-dead endpoint while our users saw nothing but spinning wheels.

That night cost us 47 abandoned carts and a very angry customer support ticket that started with "I tried 11 times and..."

I swore I'd never get that call again. Here's what I built instead — and what you should build too if your application depends on external APIs.

The Problem: Most API Clients Are Optimistic to a Fault

The default HTTP client in most languages does exactly one thing when a request fails: nothing useful. No retry. No backoff. No circuit breaker. Just a stack trace and a confused user.

Here's what a naive API call looks like in most codebases:

import requests

def charge_customer(amount, token):
    resp = requests.post("https://api.paymentprovider.com/v1/charges", json={
        "amount": amount,
        "token": token
    })
    return resp.json()

This function has three failure modes, and it handles exactly zero of them:

Failure	What Happens	User Sees
Timeout (API hangs)	`requests` waits forever (default: no timeout)	Infinite spinner
5xx error	Exception propagates up	"Something went wrong"
Network blip (transient)	Instant failure, no retry	Frustrating "try again"

Layer 1: Timeouts (The One-Liner That Saves You)

The single highest-ROI change you can make:

# Before: no timeout — can hang forever
resp = requests.post(url, json=payload)

# After: connect timeout + read timeout
resp = requests.post(url, json=payload, timeout=(3.0, 10.0))

The tuple is (connect_timeout, read_timeout). Connect timeout handles DNS/network delays. Read timeout handles slow responses. If your API hasn't sent bytes in 10 seconds, something is wrong — fail fast so your retry logic can kick in.

Real numbers from our system: Adding timeouts reduced average failure recovery time from 45 seconds to 6 seconds. Users stopped rage-quitting during upstream outages.

Layer 2: Retry With Exponential Backoff and Jitter

Not all failures are permanent. Network blips, DNS hiccups, and load balancer reboots last seconds. A retry with the right backoff strategy catches these.

import time
import random

def retry_with_backoff(func, max_retries=3, base_delay=1.0):
    for attempt in range(max_retries + 1):
        try:
            return func()
        except (ConnectionError, Timeout, ServerError) as e:
            if attempt == max_retries:
                raise
            # Exponential backoff with jitter
            delay = base_delay * (2 ** attempt)
            jitter = random.uniform(0, delay * 0.3)
            time.sleep(delay + jitter)

Why jitter matters: If 10,000 clients all retry at exactly 1s, 2s, 4s, you create thundering herds that can knock over a recovering service. Jitter spreads the load. AWS has a great writeup on this in their architecture blog — their internal services all use jittered backoff.

The retry delays look like this:

Attempt 1: ~1.0-1.3 seconds
Attempt 2: ~2.0-2.6 seconds
Attempt 3: ~4.0-5.2 seconds

Total: ~7-9 seconds before giving up. Fast enough that users notice the delay, slow enough to catch most transient failures.

Layer 3: Circuit Breaker (Stop Hitting a Dead Horse)

Retry with backoff works for transient failures. But when a service is down for 30 minutes, retrying every 9 seconds means 200 failed requests per endpoint — wasting resources and potentially making things worse.

Enter the circuit breaker pattern:

import threading
import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, recovery_timeout=30):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failures = 0
        self.last_failure_time = 0
        self.state = "CLOSED"  # CLOSED, OPEN, HALF_OPEN
        self.lock = threading.Lock()

    def call(self, func, fallback=None):
        with self.lock:
            if self.state == "OPEN":
                if time.time() - self.last_failure_time > self.recovery_timeout:
                    self.state = "HALF_OPEN"
                else:
                    return fallback() if fallback else None

        try:
            result = func()
            with self.lock:
                self.failures = 0
                self.state = "CLOSED"
            return result
        except Exception:
            with self.lock:
                self.failures += 1
                self.last_failure_time = time.time()
                if self.failures >= self.failure_threshold:
                    self.state = "OPEN"
            raise

# Usage
breaker = CircuitBreaker(failure_threshold=5, recovery_timeout=30)

def get_product_price(product_id):
    return breaker.call(
        lambda: requests.get(f"https://api.pricing.com/v1/products/{product_id}", timeout=5),
        fallback=lambda: {"price": get_cached_price(product_id)}
    )

Three states:

CLOSED: Normal operation. Requests flow through.
OPEN: Circuit is tripped. All requests immediately return the fallback (or fail fast). No requests reach the downstream service.
HALF_OPEN: After the recovery timeout, one request is allowed through as a probe. If it succeeds, the circuit closes. If it fails, it opens again.

This one pattern eliminated our 3 AM pages. When the payment provider went down, the circuit opened within 5 failures (~45 seconds), and from that point on, every request returned a "payment temporarily unavailable — we'll notify you when it's back" message instead of spinning forever. Support tickets about payment failures dropped from "I tried 11 times" to "I got a clear message and an email when it was back."

Layer 4: Graceful Degradation (The Fallback Chain)

Not every API call is critical. If your product recommendation engine is down, showing a static "popular products" list is better than a blank page.

Build fallback chains:

def get_recommendations(user_id):
    # Tier 1: Personalized ML model
    try:
        return call_recommendation_api(user_id)
    except Exception:
        pass

    # Tier 2: Cached recent recommendations
    cached = get_cached_recommendations(user_id)
    if cached:
        return cached

    # Tier 3: Static popular products
    return get_popular_products(category="default")

Map your API dependencies on a criticality spectrum:

Criticality	Examples	Strategy
Critical (must succeed)	Payment processing, auth	Retry + circuit breaker + alerting
Important (can degrade)	Product search, recommendations	Fallback chain + stale cache
Nice-to-have (optional)	Analytics, logging, A/B tests	Fire-and-forget, async queue

Putting It All Together: A Production-Ready API Client

Here's what a resilient API call looks like after all four layers:

from dataclasses import dataclass
from typing import Optional, Callable
import requests
import time
import random
import threading

@dataclass
class ResilientClient:
    timeout: tuple = (3.0, 10.0)
    max_retries: int = 3
    base_delay: float = 1.0
    circuit_threshold: int = 5
    circuit_recovery: int = 30

    def __post_init__(self):
        self._breakers = {}

    def _get_breaker(self, key: str) -> CircuitBreaker:
        if key not in self._breakers:
            self._breakers[key] = CircuitBreaker(
                failure_threshold=self.circuit_threshold,
                recovery_timeout=self.circuit_recovery
            )
        return self._breakers[key]

    def call(self, method: str, url: str, *,
             fallback: Optional[Callable] = None,
             breaker_key: Optional[str] = None,
             **kwargs) -> requests.Response:

        breaker_key = breaker_key or f"{method}:{url}"
        breaker = self._get_breaker(breaker_key)

        def make_request():
            return requests.request(
                method, url, timeout=self.timeout, **kwargs
            )

        def attempt_with_retry():
            for attempt in range(self.max_retries + 1):
                try:
                    return make_request()
                except (requests.ConnectionError, requests.Timeout) as e:
                    if attempt == self.max_retries:
                        raise
                    delay = self.base_delay * (2 ** attempt)
                    jitter = random.uniform(0, delay * 0.3)
                    time.sleep(delay + jitter)
            raise RuntimeError("unreachable")

        return breaker.call(attempt_with_retry, fallback=fallback)


# Usage
client = ResilientClient()

try:
    resp = client.call("POST", "https://api.payment.com/v1/charges",
                       json={"amount": 1999, "token": "tok_xxx"},
                       fallback=lambda: queue_for_retry(amount=1999))
except Exception:
    # Circuit is open and no fallback — show user a clear message
    show_message("Payment is temporarily unavailable. We'll retry automatically.")

The Results After 6 Months

Since implementing these four layers across our 40+ external API dependencies:

3 AM pages: Zero. Down from 2-3 per month.
Failed payment recovery: 94% of transient failures now resolve automatically via retry (was ~30% with naive retry).
User-facing error rate: Down 82%. Users see fallback content or clear messages instead of blank screens.
Upstream load during outages: Down 96%. Circuit breakers stop the barrage.

The best part? Most of this is 50 lines of code. The CircuitBreaker class is 30 lines. The retry decorator is 15 lines. Timeouts are one parameter.

What I'd Do Differently

Start with timeouts, not retries. I built an elaborate retry system first, then realized I had no timeout set — so retries were waiting for the default infinity-second timeout before even starting. Backwards.
Log every circuit trip. When a circuit opens, log it with the endpoint, failure count, and timestamp. You'll discover patterns — like "the inventory API always goes down Tuesdays at 2 AM during their maintenance window."
Circuit breakers need per-endpoint granularity, not per-service. POST /charges failing doesn't mean GET /products will fail. Key your breakers by endpoint, not by hostname.
Test your fallbacks. Easy to write a fallback function, easy to forget it breaks when the upstream API changes its response format. Run chaos engineering tests monthly.

TL;DR

Four layers, from simplest to most impactful:

Timeouts — 1 line of code, prevents infinite hangs
Retry with jittered backoff — 15 lines, catches transient failures
Circuit breaker — 30 lines, stops hammering dead services
Fallback chain — Business logic, keeps users happy during outages

Build these before you need them. Because at 3 AM, you don't want to be the person debugging a retry loop.

What's your approach to handling API failures? I'd love to hear what patterns have worked for you — drop a comment below.

I Built a 3-Layer Prompt Cache and Cut My LLM Costs by 73%

Sir Max — Sat, 11 Jul 2026 02:07:27 +0000

I was burning through $200/month on LLM API calls. The worst part? I knew at least half of those requests were sending nearly identical prompts over and over.

So I built a cache. Not a Redis one-liner — a three-layer system that catches duplicates at different levels. Here's how it works, and the numbers I got.

The Problem

Every time a user asks "Summarize this document," my backend sends the full system prompt (500+ tokens of instructions) plus the document text to the LLM. Multiply that by 100 requests per hour, and you're paying to send the same system prompt 100 times.

But it's worse than that. Users ask similar questions. "Summarize this" and "Give me a summary" and "TLDR this" are semantically identical. An exact-match cache won't catch those.

And even when the question is totally different, the structure of my prompts is the same — same few-shot examples, same output format instructions, same "you are a helpful assistant" preamble. That's 200+ tokens I'm paying for on every single call.

The Three Layers

I broke caching into three tiers, from cheapest to most expensive:

Layer 1: Exact Match Cache (LRU)

This is your standard key-value cache. Hash the full prompt, check if we've seen it before.

import hashlib
from collections import OrderedDict

class ExactCache:
    def __init__(self, max_size=10000):
        self.cache = OrderedDict()
        self.max_size = max_size

    def _hash(self, prompt: str) -> str:
        return hashlib.sha256(prompt.encode()).hexdigest()

    def get(self, prompt: str):
        key = self._hash(prompt)
        if key in self.cache:
            self.cache.move_to_end(key)  # LRU bump
            return self.cache[key]
        return None

    def set(self, prompt: str, response: str):
        key = self._hash(prompt)
        self.cache[key] = response
        self.cache.move_to_end(key)
        if len(self.cache) > self.max_size:
            self.cache.popitem(last=False)

This catches about 15% of requests in my system — mostly identical queries from different users hitting the same endpoint.

Layer 2: Semantic Cache (Embedding + Cosine)

This is where things get interesting. Instead of requiring an exact match, we compare the meaning of prompts using embeddings.

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity

class SemanticCache:
    def __init__(self, embed_fn, threshold=0.95, max_size=5000):
        self.embed = embed_fn  # your embedding function
        self.threshold = threshold
        self.entries = []  # list of (vector, prompt, response)
        self.max_size = max_size

    def get(self, prompt: str):
        vec = self.embed(prompt)
        if not self.entries:
            return None

        vectors = np.array([e[0] for e in self.entries])
        similarities = cosine_similarity([vec], vectors)[0]
        best_idx = np.argmax(similarities)

        if similarities[best_idx] >= self.threshold:
            return self.entries[best_idx][2]  # response

        return None

    def set(self, prompt: str, response: str):
        vec = self.embed(prompt)
        self.entries.append((vec, prompt, response))
        if len(self.entries) > self.max_size:
            self.entries.pop(0)

I use a lightweight embedding model (all-MiniLM-L6-v2, runs in ~10ms on CPU). Setting the threshold at 0.95 means "Summarize this report" and "Give me a summary of this report" match, but "Write a poem about this report" doesn't. This catches another 22% of requests.

A warning: if your threshold is too low (say 0.85), you'll get false positives — semantically similar but meaningfully different queries getting cached responses. I started at 0.90 and crept up to 0.95 after a few embarrassing incidents.

Layer 3: Prompt Template Cache

Most LLM applications use structured prompts with a fixed template and variable inputs. Instead of sending the full prompt every time, we leverage the provider's prompt caching features. Anthropic's prompt caching saves 90% on the cached portion of the prompt, and many OpenAI-compatible providers offer similar mechanisms.

class TemplateCache:
    def __init__(self, llm_call_fn):
        self.llm_call = llm_call_fn

    def call(self, system_prompt: str, user_input: str):
        # Anthropic: mark system as cache_control block
        # OpenAI-compatible: system message benefits from
        # automatic prefix caching on many providers
        return self.llm_call(system_prompt, user_input)

This layer catches the remaining 36% of token spend — those calls where the specific user input is unique, but the system prompt and template structure are identical to thousands of previous calls.

The Pipeline

All three layers in sequence:

class PromptCachePipeline:
    def __init__(self, embed_fn, llm_call_fn):
        self.exact = ExactCache(max_size=10000)
        self.semantic = SemanticCache(embed_fn, threshold=0.95)
        self.template = TemplateCache(llm_call_fn)

    def call(self, system_prompt: str, user_input: str):
        full_prompt = f"{system_prompt}\n\n{user_input}"

        # Layer 1: Exact match
        cached = self.exact.get(full_prompt)
        if cached:
            return cached, "exact_hit"

        # Layer 2: Semantic match
        cached = self.semantic.get(full_prompt)
        if cached:
            self.exact.set(full_prompt, cached)  # promote to exact
            return cached, "semantic_hit"

        # Layer 3: Call LLM
        response = self.template.call(system_prompt, user_input)

        # Store in all layers
        self.exact.set(full_prompt, response)
        self.semantic.set(full_prompt, response)

        return response, "miss"

Real Numbers

I ran this on a system handling ~50,000 LLM calls per day across five different prompt types (summarization, extraction, classification, Q&A, code review).

Layer	Hit Rate	Token Savings
Exact Cache	15.2%	2.1M tokens/day
Semantic Cache	22.4%	3.8M tokens/day
Template Cache	35.8%	7.2M tokens/day
Total	73.4%	13.1M tokens/day

At GPT-4o pricing ($2.50/1M input tokens), that's about $33/day saved, or $990/month. On a system that was costing $1,350/month, the new bill is roughly $360.

What I'd Do Differently

Three things I learned the hard way:

1. Eviction strategy matters. I started with pure LRU for the semantic cache — bad idea. An LRU cache evicts the least recently used entry, but in an LLM system, some prompts are rare but expensive (long documents). I switched to a weighted eviction that considers both recency and token count. A 2000-token cached response is worth more than a 50-token one.

2. Embedding model choice is critical. I tried OpenAI's text-embedding-3-small first — great quality, but the API latency killed my response times (adding 80-120ms). Switching to a local model (all-MiniLM-L6-v2 via sentence-transformers) brought that down to 8-12ms. The quality difference at a 0.95 threshold was negligible.

3. Don't cache everything. Some prompts should never be cached — anything with time-sensitive data ("what's the weather"), user-specific context ("based on my previous message"), or creative generation ("write a unique story about..."). I added a simple no_cache flag to my pipeline that certain endpoints set.

Try It Yourself

The full implementation (with tests and the weighted eviction strategy) is about 200 lines of Python. You can wire it into an existing LLM pipeline in an afternoon. Start with Layer 1 — the exact match cache — and you'll probably recoup the implementation time in API savings within the first week.

The biggest surprise wasn't the cost savings. It was the latency improvement. A cache hit in Layer 1 returns in under 1ms instead of 800ms+ for an LLM round-trip. Users noticed. Your mileage will vary depending on how repetitive your prompt patterns are, but if you're running any kind of production LLM service, I'd bet at least 30% of your prompts are cacheable.

I Stopped Writing API Reference Docs and Started Writing Cookbooks — Engagement Tripled

Sir Max — Fri, 10 Jul 2026 02:14:27 +0000

I Stopped Writing API Reference Docs and Started Writing Cookbooks — Engagement Tripled

Three years ago I shipped an API that I was genuinely proud of. Clean REST design, consistent error handling, thoughtful pagination. I spent two weeks writing the reference docs — every endpoint, every parameter, every response code. I shipped it and waited for the adoption flood.

Crickets. A dozen signups. Two developers who actually built something. Both emailed me asking questions the docs technically answered.

The problem wasn't the quality of the reference. It was the shape. Here's what I learned.

Nobody reads reference docs cover to cover

Reference documentation is a dictionary. You don't read a dictionary — you look up a word when you're stuck. But when a developer is evaluating whether to use your API, they aren't stuck yet. They're asking one question:

"Can this solve my problem, and how much code do I have to write?"

A dictionary can't answer that. A cookbook can.

The cookbook format that changed everything

I rewrote our docs around task-oriented recipes. Each recipe follows this structure:

Goal — what you'll build (a working feature, not an abstract concept)
Prerequisites — literally what's needed (API key, SDK installed, any setup)
Steps — numbered, copy-paste-ready code blocks
Full working example — the entire script, runnable as-is
What's next — links to related recipes

Here's a concrete example. Old reference-style docs might say:

POST /v1/chat/completions — Creates a model response for the given conversation. Accepts model, messages, temperature, max_tokens.

The cookbook version:

## Recipe: Build a customer support triage bot

**Goal**: A script that reads a customer message, classifies urgency,
and drafts a suggested reply.

**Prerequisites**: API key from dashboard, Python 3.8+

**Step 1**: Install the SDK

bash
pip install acme-ai


**Step 2**: Classify urgency

python
from acme_ai import Client

client = Client(api_key="your-key-here")

urgency = client.chat.create(
model="acme-4",
messages=[
{"role": "system", "content": "Classify as: urgent, normal, or low."},
{"role": "user", "content": "My account was charged twice this month!"}
]
)
print(urgency.choices[0].message.content) # → "urgent"


**Step 3**: Generate a draft reply

python
reply = client.chat.create(
model="acme-4",
messages=[
{"role": "system", "content": "Draft a polite, helpful reply."},
{"role": "user", "content": "My account was charged twice this month!"}
]
)
print(reply.choices[0].message.content)


**Full script**: [See gist](https://gist.github.com/example)

**Next**: [Add context from your database →](/recipes/context-augmentation)

Same endpoint. Completely different experience. The developer can copy, paste, run, and see it work in under 60 seconds.

What happened when I switched

Within three months:

Time-to-first-API-call dropped from ~40 minutes to ~6 minutes. We measured this by instrumenting the gap between account creation and first successful 200 response.
Support tickets about "how do I do X" dropped 60%. Not because developers got smarter — because the answer was already in a recipe.
Word-of-mouth referrals increased. Developers started tweeting "check out this API, their docs are actually good" — a sentence I never heard about our reference docs.

The reference docs didn't go away. They're still there, linked from every recipe for developers who need the fine print. But they went from being the front door to being the appendix.

Three rules for writing cookbook docs

After a few years of iterating, here's what I've learned:

1. Every recipe must end with a working script

Not a snippet. A complete file. If a developer can't run it without filling in blanks, it's not a recipe — it's a suggestion. The "full working example" section is the most important part of the page.

2. Recipes should answer "why would I use this?"

Every recipe title is a job to be done. Not "Using the Webhook API" — that's a feature name. Instead: "Get Real-Time Notifications When an Order Ships." The developer knows immediately whether this recipe is for them.

3. Measure what matters

We track these metrics per recipe page:

Completion rate: what percentage of visitors scroll to the bottom?
Copy-paste rate: how many people click the "copy" button on code blocks?
Error rate on copied code: do people paste our examples and get errors? (We run automated tests on every code block in CI.)

If a recipe has low completion and high copy-paste errors, we rewrite it — regardless of whether anyone filed a ticket.

When NOT to use a cookbook

Cookbook docs aren't always the answer:

Very simple APIs (3 endpoints, no auth complexity). Your "Getting Started" page might be enough. A cookbook might feel like padding.
Internal tools with a captive audience of 5 engineers who already know the domain. Reference + a few examples is fine.
Unstable APIs where endpoints change weekly. Cookbooks rot fast. Stabilize the API first.

But for any public-facing API with a real developer audience, the cookbook approach pays for itself within weeks.

Start small

If the idea of rewriting all your docs into recipes feels overwhelming (it did for me), start with three:

The "hello world" recipe — the simplest possible working example (under 10 lines).
The "90% use case" recipe — what most developers actually come to your API for.
The "aha moment" recipe — the one that makes developers say "oh, THAT's what this is for."

Ship those three, measure the difference, then decide whether to do the rest.

I've been building APIs for a while now, and the single highest-leverage decision I ever made was treating documentation as a product, not an afterthought. Reference docs tell developers what your API can do. Cookbooks show them what they can build.

The second one is what gets people to ship.