A practical guide for keeping technical documentation synchronized with production systems
Documentation drift is expensive. When your docs say one thing and your API does another, developers lose trust, support tickets pile up, and teams waste time investigating discrepancies. After working with multiple production systems, I've identified five patterns that significantly reduce documentation maintenance burden while improving accuracy.
These patterns work whether you're documenting REST APIs, GraphQL endpoints, or microservices architectures. They're particularly valuable for teams practicing continuous deployment where manual documentation updates can't keep pace with releases.
Pattern 1: Store Documentation Next to Infrastructure Code
The Traditional Approach:
Documentation lives in Confluence or a separate docs repository. Infrastructure code lives in Terraform files. They drift apart within weeks.
The Better Pattern:
Keep documentation in the same repository as your infrastructure code, versioned together.
/api-infrastructure
/terraform
- api-gateway.tf
- lambda-functions.tf
- dynamodb-tables.tf
/docs
- api-reference.md
- authentication.md
- rate-limits.md
/openapi
- spec.yaml
When someone updates API Gateway throttling settings, the documentation change happens in the same pull request. Reviewers catch inconsistencies before merge.
Implementation:
Add a documentation check to your PR template:
## Pull Request Checklist
- [ ] Code changes tested
- [ ] Infrastructure changes applied to staging
- [ ] Documentation updated to reflect changes
- [ ] OpenAPI spec regenerated if endpoints changed
Why It Works:
The person making infrastructure changes is best positioned to document them. They understand what changed and why. Keeping docs adjacent to code removes the friction of "I'll update the docs later."
Pattern 2: Generate Examples from Actual Infrastructure
The Problem:
Manually written examples become outdated. Your docs say "default timeout is 30 seconds" but you changed it to 60 seconds three months ago.
The Pattern:
Write scripts that query your infrastructure and generate documentation examples automatically.
#!/usr/bin/env python3
"""
Generate API documentation examples from deployed AWS resources
Run this in CI/CD after infrastructure deployment
"""
import boto3
import json
from datetime import datetime
def generate_api_gateway_docs():
"""Extract actual configuration from deployed API Gateway"""
client = boto3.client('apigateway')
# Get API Gateway details
apis = client.get_rest_apis()
for api in apis['items']:
api_id = api['id']
# Get actual throttle settings
stages = client.get_stages(restApiId=api_id)
for stage in stages['item']:
throttle_settings = stage.get('throttleSettings', {})
doc_content = f"""
## Rate Limits
Current production settings for {stage['stageName']}:
- **Burst Limit**: {throttle_settings.get('burstLimit', 'N/A')} requests
- **Rate Limit**: {throttle_settings.get('rateLimit', 'N/A')} requests per second
*Last updated: {datetime.now().strftime('%Y-%m-%d %H:%M UTC')}*
*Generated from deployed infrastructure*
"""
# Write to docs file
with open(f'docs/rate-limits-{stage["stageName"]}.md', 'w') as f:
f.write(doc_content.strip())
if __name__ == '__main__':
generate_api_gateway_docs()
Add to CI/CD:
# .github/workflows/update-docs.yml
name: Update Documentation
on:
workflow_dispatch:
schedule:
- cron: '0 2 * * *' # Daily at 2 AM
jobs:
update-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Generate documentation from infrastructure
run: python scripts/generate-docs.py
- name: Commit changes
run: |
git config user.name "Documentation Bot"
git config user.email "bot@example.com"
git add docs/
git diff --quiet && git diff --staged --quiet || git commit -m "docs: update from deployed infrastructure"
git push
Why It Works:
Your documentation reflects production reality because it's generated from production. Impossible for docs to drift when they're extracted from the source of truth.
Pattern 3: Validate Documentation in CI/CD
The Problem:
Someone updates an API endpoint but forgets to update the corresponding documentation. The discrepancy only surfaces when users complain.
The Pattern:
Add automated checks that fail your build when documentation is incomplete or incorrect.
// scripts/validate-docs.js
const fs = require('fs');
const path = require('path');
/**
* Validate that all API endpoints have corresponding documentation
*/
function validateEndpointDocs() {
const errors = [];
// Read OpenAPI specification
const specPath = path.join(__dirname, '../openapi/spec.yaml');
const spec = require('js-yaml').load(fs.readFileSync(specPath, 'utf8'));
// Extract all endpoint paths
const endpoints = Object.keys(spec.paths || {});
// Check each endpoint has documentation
endpoints.forEach(endpoint => {
const methods = Object.keys(spec.paths[endpoint]);
methods.forEach(method => {
const operation = spec.paths[endpoint][method];
// Check for required documentation fields
if (!operation.description || operation.description.length < 50) {
errors.push(
`${method.toUpperCase()} ${endpoint}: Missing or insufficient description`
);
}
if (!operation.responses || Object.keys(operation.responses).length === 0) {
errors.push(
`${method.toUpperCase()} ${endpoint}: No response codes documented`
);
}
// Check for example responses
Object.entries(operation.responses).forEach(([code, response]) => {
if (code === '200' && !response.content?.['application/json']?.example) {
errors.push(
`${method.toUpperCase()} ${endpoint}: Missing example for 200 response`
);
}
});
});
});
return errors;
}
// Run validation
const errors = validateEndpointDocs();
if (errors.length > 0) {
console.error('Documentation validation failed:\n');
errors.forEach(error => console.error(`${error}`));
process.exit(1);
} else {
console.log('All documentation checks passed');
process.exit(0);
}
Add to your CI pipeline:
# In your .github/workflows/ci.yml
- name: Validate API documentation
run: node scripts/validate-docs.js
Why It Works:
Broken builds force teams to fix documentation before deploying. Documentation becomes a first-class citizen alongside tests and code quality checks.
Pattern 4: Version Documentation with Your API
The Problem:
You release API v2 but documentation for v1 disappears. Users still on v1 have no reference.
The Pattern:
Maintain documentation versions that match your API versions.
/docs
/v1
- authentication.md
- endpoints.md
- changelog.md
/v2
- authentication.md
- endpoints.md
- changelog.md
/v3
- authentication.md
- endpoints.md
- changelog.md
Automate version detection:
# scripts/generate-versioned-docs.py
import os
import shutil
from pathlib import Path
def create_version_docs(api_version):
"""
Create documentation for a specific API version
"""
docs_dir = Path('docs')
version_dir = docs_dir / api_version
# Create version directory
version_dir.mkdir(parents=True, exist_ok=True)
# Copy template docs
template_dir = docs_dir / 'templates'
for template in template_dir.glob('*.md'):
content = template.read_text()
# Replace version placeholders
content = content.replace('{{VERSION}}', api_version)
content = content.replace('{{API_URL}}', f'https://api.example.com/{api_version}')
# Write versioned doc
(version_dir / template.name).write_text(content)
print(f"Created documentation for {api_version}")
# Generate docs for all active versions
active_versions = ['v1', 'v2', 'v3']
for version in active_versions:
create_version_docs(version)
Create a version selector in your docs:
<!-- Version selector for documentation site -->
<div class="version-selector">
<label for="api-version">API Version:</label>
<select id="api-version" onchange="switchVersion(this.value)">
<option value="v3" selected>v3 (latest)</option>
<option value="v2">v2 (supported)</option>
<option value="v1">v1 (deprecated)</option>
</select>
</div>
<script>
function switchVersion(version) {
const currentPath = window.location.pathname;
const newPath = currentPath.replace(/\/v\d+\//, `/${version}/`);
window.location.href = newPath;
}
</script>
Why It Works:
Developers can reference documentation matching their deployed API version. No confusion about which features are available in which version.
Pattern 5: Test Your Documentation Examples
The Problem:
Code examples in documentation break but nobody notices until developers try them and fail.
The Pattern:
Extract code examples from documentation and run them as tests.
# tests/test_documentation_examples.py
import pytest
import requests
import re
from pathlib import Path
def extract_code_blocks(markdown_file):
"""Extract Python code blocks from markdown documentation"""
content = Path(markdown_file).read_text()
# Find all Python code blocks
pattern = r'```
python\n(.*?)
```'
code_blocks = re.findall(pattern, content, re.DOTALL)
return code_blocks
def test_authentication_example():
"""Test that authentication example in docs actually works"""
# Extract code from docs
examples = extract_code_blocks('docs/authentication.md')
# Run each example
for example in examples:
# Create isolated namespace
namespace = {'requests': requests}
try:
exec(example, namespace)
# Check that example made successful API call
if 'response' in namespace:
assert namespace['response'].status_code == 200, \
f"Example failed with status {namespace['response'].status_code}"
except Exception as e:
pytest.fail(f"Documentation example failed to execute: {str(e)}")
def test_rate_limit_values_match_infrastructure():
"""Verify rate limits in docs match deployed values"""
# Parse rate limits from documentation
docs_content = Path('docs/rate-limits.md').read_text()
doc_rate_match = re.search(r'Rate Limit.*?(\d+)', docs_content)
assert doc_rate_match, "Could not find rate limit in documentation"
doc_rate = int(doc_rate_match.group(1))
# Query actual API for rate limits
response = requests.get('https://api.example.com/v1/limits')
actual_rate = response.json()['rateLimit']
assert doc_rate == actual_rate, \
f"Documentation says {doc_rate} req/s but API reports {actual_rate} req/s"
Run tests in CI:
# .github/workflows/test-docs.yml
name: Test Documentation
on: [push, pull_request]
jobs:
test-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: pip install pytest requests
- name: Test documentation examples
run: pytest tests/test_documentation_examples.py -v
Why It Works:
Broken examples fail CI before they reach users. Your documentation examples are guaranteed to work because they're tested continuously.
Putting It All Together
These five patterns create a documentation system that maintains itself:
- Co-location ensures changes happen together
- Generation eliminates manual updates for configuration details
- Validation catches incomplete documentation before deployment
- Versioning supports users across all API versions
- Testing guarantees examples actually work
Implement them incrementally. Start with Pattern 1 (co-location) this week. Add Pattern 3 (validation) next sprint. Build toward a fully automated documentation system over time.
The goal isn't perfect documentation on day one. It's building systems that make accurate documentation the path of least resistance.
Stella Oiro is an AWS Community Builder who builds and documents cloud-native systems in production.
Top comments (0)