API Gateway Patterns

Your API gateway is the front door to your entire platform — get it wrong and you're debugging cascading failures at 2 AM. This collection provides battle-tested configurations for Kong, AWS API Gateway, and custom Python/Node gateways covering rate limiting, authentication, request transformation, circuit breaking, and observability. Each pattern includes the config files, deployment notes, and the reasoning behind every design decision.

Key Features

• Kong Declarative Configs — Ready-to-deploy kong.yml files with plugins for rate limiting, JWT auth, request transformation, and IP restriction
• AWS API Gateway Templates — CloudFormation and SAM templates for REST and HTTP APIs with Lambda authorizers and usage plans
• Custom Gateway in Python — Lightweight ASGI gateway using raw httpx for proxying, with middleware patterns for auth, logging, and rate limiting
• Rate Limiting Strategies — Sliding window, token bucket, and fixed window implementations with Redis and in-memory backends
• Circuit Breaker Pattern — Prevent cascade failures with configurable failure thresholds, half-open states, and fallback responses
• Request/Response Transformation — Header injection, body rewriting, and protocol translation (REST-to-gRPC) configs
• Observability Stack — Prometheus metrics, structured JSON logging, and distributed tracing headers for every pattern

Quick Start

1. Choose your gateway platform and navigate to the corresponding directory
2. Copy the example config:

# configs/kong/kong.yml
_format_version: "3.0"
services:
  - name: user-service
    url: http://user-svc:8080
    routes:
      - name: users-route
        paths:
          - /api/v1/users
        strip_path: true
    plugins:
      - name: rate-limiting
        config:
          minute: 100
          policy: redis
          redis_host: redis
          redis_port: 6379
      - name: jwt
        config:
          claims_to_verify:
            - exp

1. For the custom Python gateway:

python -m src.gateway --config configs/custom/gateway.yaml --port 8000

Architecture

api-gateway-patterns/
├── configs/
│   ├── kong/                     # Declarative configs with plugins
│   ├── aws/                      # SAM templates, Lambda authorizer
│   └── custom/                   # Custom gateway YAML configs
├── src/
│   ├── gateway.py                # Custom ASGI gateway core
│   ├── middleware/               # rate_limiter, circuit_breaker, auth, transformer
│   └── backends/                 # Redis and in-memory rate limit stores
├── docs/                         # Overview and pattern documentation
└── templates/
    └── config.yaml

The custom gateway follows a middleware pipeline pattern: each request passes through an ordered chain of middleware (auth -> rate limit -> transform -> proxy -> transform response -> log).

Usage Examples

Rate Limiting with Sliding Window

# src/middleware/rate_limiter.py
import time
from dataclasses import dataclass

@dataclass
class SlidingWindowConfig:
    window_seconds: int = 60
    max_requests: int = 100
    key_func: str = "client_ip"  # or "api_key", "user_id"

class SlidingWindowLimiter:
    def __init__(self, store, config: SlidingWindowConfig):
        self.store = store
        self.config = config

    async def is_allowed(self, key: str) -> tuple[bool, dict]:
        now = time.time()
        window_start = now - self.config.window_seconds
        # Remove expired entries, count remaining
        count = await self.store.count_in_window(key, window_start, now)
        remaining = max(0, self.config.max_requests - count)
        headers = {
            "X-RateLimit-Limit": str(self.config.max_requests),
            "X-RateLimit-Remaining": str(remaining),
            "X-RateLimit-Reset": str(int(now + self.config.window_seconds)),
        }
        if count >= self.config.max_requests:
            return False, headers
        await self.store.record(key, now)
        return True, headers

Circuit Breaker Configuration

# configs/custom/gateway.yaml
services:
  user-service:
    upstream: http://user-svc:8080
    circuit_breaker:
      failure_threshold: 5        # Open circuit after 5 failures
      success_threshold: 3        # Close after 3 successes in half-open
      timeout_seconds: 30         # Time before half-open attempt
      monitored_exceptions:
        - timeout
        - connection_error
        - 502
        - 503
    fallback:
      status: 503
      body: '{"error": "Service temporarily unavailable", "retry_after": 30}'

Configuration

Key	Type	Default	Description
gateway.port	int	8000	Gateway listen port
gateway.workers	int	4	Number of worker processes
rate_limiting.backend	string	"memory"	memory or redis
rate_limiting.default_limit	int	100	Requests per window
rate_limiting.window_seconds	int	60	Window duration
circuit_breaker.enabled	bool	true	Enable circuit breaker
auth.jwt_secret	string	required	YOUR_JWT_SECRET_HERE
auth.jwt_algorithm	string	"HS256"	JWT signing algorithm
logging.format	string	"json"	json or text
tracing.enabled	bool	false	Enable distributed tracing headers

Best Practices

• Layer your rate limits. Global limits at the gateway + per-service limits closer to the backend. A 1000 req/min global limit with 200 req/min per service prevents any single service from consuming the entire budget.
• Always set timeouts on upstream connections. Set connect timeout to 3s and read timeout to 10s as baseline. Without them, a hanging backend ties up gateway workers.
• Use structured logging from day one. JSON logs with request_id, upstream_service, and response_time_ms fields save hours of debugging.
• Externalize auth token validation. Use environment variables or a secrets manager — never embed secrets in gateway config files.

Troubleshooting

Rate limiter not counting requests correctly
If using Redis backend, ensure all gateway instances connect to the same Redis instance. In-memory stores are per-process and don't share state across workers or instances.

Circuit breaker never closes after opening
Check that success_threshold is reachable — in half-open state, only a limited number of requests are forwarded. If your health check endpoint is separate from your API routes, configure the circuit breaker to use it for recovery probes.

JWT validation fails with "invalid signature"
Verify the jwt_secret matches between your auth service and gateway config. Also check jwt_algorithm — if your auth service signs with RS256 but the gateway validates with HS256, every token will fail.

High latency on request transformation
Body rewriting (especially JSON-to-JSON transforms) is CPU-bound. If transformations add >10ms, consider moving complex transformations to a sidecar or the backend service itself.

---

This is 1 of 7 resources in the API Developer Pro toolkit. Get the complete [API Gateway Patterns] with all files, templates, and documentation for $39.

Get the Full Kit →

Or grab the entire API Developer Pro bundle (7 products) for $79 — save 30%.

Get the Complete Bundle →

---

Related Articles

• API Documentation Toolkit
• API Testing Automation
• GraphQL Starter Kit