Always Up-to-Date API Docs Are Real (And No, It’s Not AI)

API documentation is broken. You've likely seen it: endpoints that don't exist anymore, missing response codes, or a lingering "TODO" in a YAML spec. And that's assuming there's a spec at all.

The core issue? Most developers don't enjoy maintaining documentation. Manually editing OpenAPI specs or annotating endpoints with docstrings is tedious, repetitive, and error-prone.

Modern tools try to ease this pain, but each comes with its own trade-offs.

Where Existing Tools Fall Short

1. Export-Based Tools (Postman, Insomnia, etc.)

Tools like Postman allow you to build collections and export them as OpenAPI specs. This is certainly better than hand-writing YAML, but it's still manual work. You need to remember to update your collections every time you change an endpoint, add parameters, or modify response structures.

The workflow looks like this: change code → update Postman collection → export spec → and hope nothing was missed. It's not error-prone in the traditional sense, but it requires discipline and creates another maintenance burden that developers often skip when deadlines loom. You're always just one forgotten update away from spec drift.

2. Annotation- and Decorator-Based Tools

A step up from manual exports are tools that let you annotate or decorate your routes directly in code. Flask-RESTX, Django REST Framework, and similar libraries fall into this category. You add decorators or docstrings to your endpoints, and the spec gets generated automatically.

from rest_framework import viewsets
from drf_spectacular.utils import extend_schema, OpenApiParameter
from drf_spectacular.types import OpenApiTypes

class UserViewSet(viewsets.ModelViewSet):
    serializer_class = UserSerializer

    @extend_schema(
        summary="List all users",
        description="Returns a paginated list of all users in the system",
        parameters=[
            OpenApiParameter(
                name='search',
                description='Search users by username',
                required=False,
                type=str
            ),
        ],
        responses={
            200: UserSerializer(many=True),
            403: {"description": "Forbidden"}
        },
    )
    def list(self, request, *args, **kwargs):
        return ...

This approach keeps documentation closer to the code, which is good. But it's still fundamentally manual work. If a developer forgets to add or update an annotation, the spec compiles successfully but becomes incorrect. Maybe missing a 422 validation error response or an optional query parameter. The spec generation works perfectly; the human memory doesn't.

3. Auto-Generation from Type Hints (FastAPI and Friends)

The most sophisticated approach uses type hints and models to automatically generate documentation. FastAPI is the poster child here - it introspects your Pydantic models and function signatures to build comprehensive OpenAPI specs with minimal manual input.

from typing import List, Optional
from fastapi import FastAPI, Query

@app.get("/users/", response_model=List[UserResponse])
async def list_users(
    skip: int = Query(0, ge=0, description="Number of records to skip"),
    limit: int = Query(10, ge=1, le=100, description="Max number of records to return"),
    role: Optional[UserRole] = Query(None, description="Filter by user role"),
    is_active: Optional[bool] = Query(None, description="Filter by active status"),
    search: Optional[str] = Query(None, min_length=1, description="Search in username or email")
):
    return ...

This can automate a significant portion of the work and produces impressively detailed documentation. But edge cases still require manual intervention: middleware logic, custom response codes, specific media types, and detailed examples. You're also locked into a specific framework from day one of your API development, which is a significant architectural commitment.

The Uncomfortable Truth

Even with these advanced tools, the problem persists at scale. According to Nordic APIs research, 43% of APIs don't have a public specification at all, and among those that do, 75% of production APIs have variances from their published OpenAPI specs.

It's not because developers are lazy or tools are bad. It's because they're fighting against the fundamental nature of software development: code changes constantly, and any manual process, no matter how streamlined, will eventually fall behind.

Every manual step is a potential point of failure. Every annotation that needs updating is a chance for drift. Every example that needs refreshing is an opportunity for confusion.

A different approach is needed. One that doesn't rely on developers remembering to update documentation. One that can't drift because it's generated from the single source of truth about API behavior.

That source of truth? It's been in codebases all along.

Tests Are Already Your API Specification

Here's a thought experiment: What if you're already writing comprehensive API documentation every day, you just don't realize it?

Think about what your API tests actually do. They:

• Make real HTTP requests to every endpoint
• Send actual request bodies with realistic data
• Include all required headers and authentication
• Receive and validate real responses
• Check different status codes and error conditions

Your test suite is essentially a living, executable specification of your API. It knows exactly how your API behaves because it's constantly verifying that behavior. The only problem? This knowledge is trapped in test code, invisible to the developers who need it most.

The Insight That Changes Everything

What if you could capture all those test requests and responses and transform them into OpenAPI documentation? Not by parsing test code or adding annotations, but by simply observing what actually happens when tests run?

This isn't a pipe dream. It's exactly what vedro-httpx does - an official plugin for the Vedro testing framework that not only makes API testing effortless but also generates OpenAPI specs from your test runs.

Here's the beautiful part: you don't write documentation. You don't annotate anything. You don't maintain separate files. You just write good tests (which you're doing anyway), and documentation appears. Always accurate. Always complete. Always up-to-date.

How It Actually Works

The Interface Pattern

vedro-httpx encourages wrapping external services in lightweight interface classes. This pattern provides three key benefits:

1. Centralized configuration for reusable HTTPX clients (base URL, headers, TLS, timeouts)
2. Clear, type-annotated methods for each endpoint that keep tests readable
3. A unified Response object that Vedro renders clearly when assertions fail

Here's a typical interface:

from vedro_httpx import Response, SyncHTTPInterface

class AuthAPI(SyncHTTPInterface):
    def __init__(self, base_url: str = "http://localhost") -> None:
        super().__init__(base_url)

    def login(self, username: str, password: str) -> Response:
        return self._request("POST", "/auth/login", json={
            "username": username,
            "password": password
        })

With the AuthAPI interface defined, you can use it in test scenarios:

from vedro_fn import scenario
from contexts import registered_user
from interfaces import AuthAPI

@scenario
def login_as_registered_user():
    user = registered_user()
    response = AuthAPI().login(user["username"], user["password"])
    assert response.status_code == 200

Now vedro-httpx tracks every request and response flowing through your tests. The documentation generation works through two straightforward steps.

Step 1: Record Requests

With a simple CLI flag, vedro-httpx records all HTTP requests and responses during test execution:

$ vedro run --httpx-record-requests

After the test run, all request/response data is saved to .vedro/artifacts in HAR format ,  a widely-used standard that's human-readable JSON. HAR files can be opened in browser developer tools, analyzed with tools like Fiddler, or viewed online with tools like Google HAR Analyzer.

The format looks like this:

{
  "log": {
    "entries": [
      {
        "request": {
          "method": "POST",
          "url": "<URL>",
          "queryString": [],
          "headers": ...
        },
        "response": {
          "status": 200,
          "statusText": "OK",
          "headers": ...,
          "content": ...
        }
      }
    ]
  }
}

Step 2: Generate Documentation

Based on the HAR files, vedro-httpx generates a ready-to-use OpenAPI spec with a single command:

$ vedro_httpx .vedro/artifacts > spec.yaml

The generated spec captures everything your tests actually did:

openapi: 3.0.0
info:
  title: API
  version: 1.0.0
servers:
- url: https://chat-api-tutorial.vedro.io/tl3mzuetbb
paths:
  /chats/{chat_id}/messages:
    post:
      summary: Endpoint for POST /chats/{chat_id}/messages
      operationId: post_chats_chat_id_messages
      parameters:
      - name: x-auth-token
        in: header
        required: true
        description: X auth token header
        schema:
          type: string
      responses:
        '200':
          description: OK
        '400':
          description: Bad Request
        '401':
          description: Unauthorized

Render it with your favorite tool:

$ docker run -p 8080:8080 -v $(pwd):/app -e SWAGGER_JSON=/app/spec.yaml swaggerapi/swagger-ui

That's it. Your tests run, and documentation is generated. No manual steps, no drift, no outdated examples.

The Magic? There Is No Magic

This approach works because it's based on a simple truth: your tests are already doing the work. They're already making real requests and receiving real responses. The tool is just capturing that information and presenting it in a different format.

No guessing about what your API does. No forgetting to document edge cases. No drift between documentation and reality. If your tests pass, your documentation is accurate. If your API changes, your tests fail, you fix them, and your documentation updates automatically.

It's not AI trying to understand your code. It's not complex static analysis. It's just recording what actually happens and turning it into documentation. Simple, reliable, and impossible to get wrong.

The Hidden Benefits You Didn't Expect

When you start generating documentation from tests, something interesting happens. You don't just solve the documentation problem; you accidentally improve your entire API development workflow.

Zero Specification Drift (Really, Zero)

This isn't hyperbole. With traditional approaches, drift is inevitable because documentation and code are separate artifacts that must be manually synchronized. With test-generated docs, drift is literally impossible.

Your tests are your specification. Break one, you break the other. The documentation can't lie about your API's behavior because it's generated from actual API behavior. This is a fundamentally different guarantee than "trying really hard to keep docs updated."

Living Documentation That Updates Itself

Imagine documentation that refreshes with every CI run. No "documentation sprints". No tickets titled "Update API docs for Q3 changes." No archeology expeditions to figure out when the docs stopped matching reality.

Set up a simple GitHub Action with automatic publishing to GitHub Pages and your documentation stays current without any human intervention. Every merge to main updates your documentation. Your API docs are as fresh as your latest test run. Documentation becomes a build artifact, not a maintenance burden.

Instant API Coverage Metrics

Here's a benefit that surprises everyone: gaps in your generated documentation reveal untested endpoints. It's an instant, visceral API coverage metric.
Running the generator and notice /users/{id}/preferences isn't in the spec? That endpoint isn't tested. The absence of documentation becomes a todo list for test coverage. You can even fail builds if certain endpoints disappear from the generated spec, catching accidentally untested code before it ships.

Friction-Free Contract Testing

Your generated OpenAPI spec isn't just documentation - it's a contract. Frontend teams can:

• Generate TypeScript clients automatically
• Spin up mock servers with realistic responses
• Validate their requests against your actual API behavior

Partner integrations become smoother too. Instead of maintaining separate "partner documentation", you hand them the same OpenAPI spec your tests generate. They're coding against your API's actual behavior, not your best guess at what it does.

Design Feedback Loop

This might be the most underrated benefit: bad documentation often reveals bad API design, but only after it's too late. With test-generated docs, you see your API's real interface while you're building it.

Endpoint naming inconsistencies jump out. Weird authentication patterns become obvious. Response format discrepancies glare at you from the generated spec. It's like having an API design reviewer that never sleeps and never misses anything.

The Compound Effect

Each of these benefits reinforces the others:

• Better test coverage leads to better documentation →
• Better documentation reveals design issues earlier →
• Better design makes writing tests easier →
• Easier tests mean more coverage

It's a virtuous cycle where improving any part improves the whole. You're not just solving documentation - you're systematically improving API quality.

How to Use It

The process has two stages: recording requests (1) and generating documentation (2). The beauty is that the second stage works with any language or framework. You just need HAR files as input.

If You're Using Vedro and vedro-httpx

This is the simplest path. Just run your existing tests with the recording flag:

$ vedro run --httpx-record-requests

Then generate the spec:

$ vedro_httpx .vedro/artifacts > spec.yaml

If You're Using Another Python Framework

You can still use vedro-httpx in your tests, even if you're not using Vedro as your main testing framework. The HTTP interface pattern works anywhere.

If You're Using Another Language

You need to record requests and responses in HAR format. Several approaches can help you capture this data:

• Playwright allows recording HAR files for browser-based testing
• Browser developer tools can export network activity as HAR
• HTTP proxies like mitmproxy can intercept and export traffic as HAR
• Some HTTP clients have extensions or middleware for HAR export

Once you have HAR files, the vedro_httpx command-line tool can generate OpenAPI specs from them regardless of how they were created.

You could even capture production traffic (carefully!) and generate documentation from real-world usage patterns.

The Bigger Picture

This approach represents a fundamental shift in how to think about documentation. Instead of treating it as a separate artifact that requires maintenance, it recognizes that documentation is just another view of a system's behavior. And behavior is already captured in tests.

Today it's API documentation. Tomorrow? Maybe infrastructure documentation from deployment tests. Configuration documentation from integration tests. Architecture diagrams from service interaction tests. Once you start seeing tests as executable specifications, you see opportunities everywhere.

The tools and techniques are here. The only question is: how much more time do you want to spend manually updating YAML files?