3 Things Nobody Tells You About Building Public APIs

Last year, I spent six months building and maintaining a public API. I thought I knew what I was getting into — I'd built internal APIs for years. REST endpoints, authentication, some docs. How different could "public" be?

Very different. Here are three things that blindsided me.

---

1. Rate Limiting Is 20% Code, 80% Edge Cases

When I first added rate limiting, I thought I was done in an afternoon. A token bucket per API key, Redis-backed, 100 requests per minute. Ship it.

Then the emails started.

"Why am I getting 429s? I'm only making 50 requests per minute."

Turns out, their backend had 12 worker processes, each with its own connection pool. From our perspective, they were 12 separate clients arriving nearly simultaneously. The token bucket drained in seconds.

"Your rate limiter broke our retry logic."

Their retry library used exponential backoff — but all 12 workers retried at the same time, creating thundering-herd 429 loops that consumed their entire quota in retries alone.

What I actually had to build:

import time
import hashlib
from collections import defaultdict
from threading import Lock

class RateLimiter:
    """
    A sliding-window rate limiter that groups by client fingerprint
    rather than raw IP or API key alone.
    """
    def __init__(self, window_seconds=60, max_requests=100):
        self.window = window_seconds
        self.max_req = max_requests
        self.buckets = defaultdict(list)
        self.lock = Lock()

    def fingerprint(self, api_key: str, user_agent: str, 
                    x_forwarded_for: str) -> str:
        """Group requests from the same deployment, not just the same key."""
        components = [api_key]
        # Truncate user-agent to group same SDK versions
        ua_base = user_agent[:50] if user_agent else "unknown"
        components.append(ua_base)
        return hashlib.sha256("|".join(components).encode()).hexdigest()[:16]

    def allow(self, fingerprint: str) -> bool:
        now = time.time()
        with self.lock:
            # Prune old entries
            self.buckets[fingerprint] = [
                t for t in self.buckets[fingerprint] 
                if now - t < self.window
            ]
            if len(self.buckets[fingerprint]) >= self.max_req:
                return False
            self.buckets[fingerprint].append(now)
            return True

The key insight: rate limit by logical client, not by connection. One API key might represent a distributed system with dozens of pods. You need to account for that.

Also, return Retry-After headers and distinguish between "you personally are too fast" (429) and "the system is overloaded" (503). Your users will thank you.

---

2. Documentation Rots Faster Than Code

I wrote thorough API docs during the first month. OpenAPI spec, examples for every endpoint, even a Postman collection.

Three months later, a user opened an issue: "The POST /users example returns a 201, but I'm getting a 400."

The docs still showed the v1 request format. We'd added a tenant_id field in v2 and made it required. Nobody updated the examples.

What I learned:

Documentation isn't a deliverable — it's a process. Here's what works:

# openapi.yaml — generate docs from this, don't write them separately
paths:
  /users:
    post:
      summary: Create a user
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateUserRequest'
            examples:
              basic:
                value:
                  name: "alice"
                  tenant_id: "t_4a7b2c"
      responses:
        '201':
          description: User created
          content:
            application/json:
              examples:
                basic:
                  value:
                    id: "u_9f3e1a"
                    name: "alice"

Three rules that saved me:

1. One source of truth. Your OpenAPI spec IS your documentation. Tools like Swagger UI or Scalar render it for free. Never maintain separate docs.
2. Test your examples. Write integration tests that literally run the request bodies from your spec examples and assert the responses match. If the example breaks, your CI catches it before the user does.
3. Version your docs alongside your code. Every time you add a field or change a response shape, update the examples in the same commit. Review it in the PR. If the reviewer can't understand the API change from the diff, neither can your users.

---

3. Your Users Will Use Your API in Ways You Never Imagined

This one surprised me the most. I designed my API for backend-to-backend communication. Standard JSON, RESTful endpoints, bearer tokens.

Within weeks, I saw:

• Someone built a Google Sheets plugin that called our API from Apps Script. They expected CSV responses because "JSON parsing in Google Sheets is annoying." Fair point — we added ?format=csv.
• A mobile developer was polling our status endpoint every 3 seconds. Not for any malicious reason — they genuinely didn't know about webhooks. We added webhook support and documentation, and their polling dropped to zero.
• Someone was using our search endpoint as their primary database query layer. They'd built their entire product around our API's GET /items?q=... endpoint — not realizing it was optimized for full-text search, not structured queries. When we changed the relevance scoring algorithm, their app broke.

The lesson: You don't control how your API gets used. You can only observe, adapt, and communicate.

Practical steps that helped:

1. Log usage patterns, not just errors. Which endpoints get hit together? What query parameters are most common? These are signals about what your users actually need.
2. Read your own access logs like product research. When I noticed ?format=csv appearing in URL patterns before we'd built it, it was because users were manually appending it, hoping it would work. It didn't — but it told us what to build.
3. Deprecate gently. When you need to change behavior, add a Deprecation: true header and a Sunset: <date> header first. Blog about it. Email your users. Give them weeks, not hours.

# Add sunset headers to every response
@app.middleware("http")
async def add_deprecation_headers(request, call_next):
    response = await call_next(request)
    if request.url.path.startswith("/v1/"):
        response.headers["Deprecation"] = "true"
        response.headers["Sunset"] = "Sat, 01 Nov 2026 00:00:00 GMT"
        response.headers["Link"] = (
            '</v2/users>; rel="successor-version"'
        )
    return response

---

What I'd Do Differently

If I were starting another public API today, I'd do three things from day one:

1. Ship with rate limiting, not add it later. Retrofitting rate limiting into an API that already has users is 10x harder than building it in from the start. Users build assumptions around your API's behavior — changing those assumptions is painful.

2. Treat your OpenAPI spec like source code. It goes in the same repo, gets reviewed in the same PRs, and gets tested in the same CI pipeline. Your API is only as good as its documentation.

3. Watch your users, don't guess what they want. Access logs are the most honest feedback you'll ever get. People vote with their requests.

Building a public API taught me that the technical part — the endpoints, the auth, the JSON — is maybe 30% of the work. The other 70% is everything around it: the docs, the rate limiting, the deprecation strategy, the user communication.

Nobody tells you that upfront.

---

Have you built or maintained a public API? What surprised you? I'd love to hear about it in the comments.