Building an API Gateway From Scratch in Go
API gateways are one of those infrastructure components that feel intimidating from the outside. You know they handle auth, routing, rate limiting, but the internals are a black box. I decided to fix that by building one from scratch in Go, using nothing but the standard library's net/http. No frameworks. Just code.
This post walks through the design of simple-api-gateway: what it does, how it's structured, and the key implementation decisions along the way.
The Core Idea: A Gateway is Just a Middleware Chain
Before diving in, it's worth naming the central insight: an API gateway is fundamentally a configurable http.Handler. Every feature, auth, rate limiting, tracing, is just middleware wrapped around a reverse proxy.
In Go, that pattern looks like this:
type Middleware func(http.Handler) http.Handler
A middleware takes a handler and returns a new handler that does something before or after calling the next one. Chain enough of these together and you have a gateway.
The Gateway struct implements http.Handler directly:
type Gateway struct {
config *types.Config
router *Router
middleware map[string]Middleware
}
func (gw *Gateway) ServeHTTP(w http.ResponseWriter, r *http.Request) {
route, params := gw.router.Match(r)
if route == nil {
http.NotFound(w, r)
return
}
handler := gw.buildChain(route)
handler.ServeHTTP(w, r)
}
No magic, just a struct with a method. This also means the gateway is fully testable without a live server. You can call ServeHTTP directly in a test.
Routing
I designed the router to handle three path patterns: exact (/users), parameterised (/users/:id), and prefix wildcards (/users/*). Routes are sorted by specificity at startup so more precise matches always win.
func (ro *Router) Match(r *http.Request) (*types.Route, map[string]string) {
for _, route := range ro.routes { // sorted: exact > param > prefix
if matched, params := matchPath(route.Path, r.URL.Path); matched {
if methodAllowed(route.Methods, r.Method) {
return &route, params
}
}
}
return nil, nil
}
One deliberate limitation: :param segments are matched and used for routing, but the extracted values aren't injected into the request context. For the purpose of this project, learning the gateway pattern, that tradeoff was acceptable, but it's a clear gap for production use.
The Middleware Chain
When a route is matched, its configured middleware is assembled into a chain and applied around the upstream proxy:
func (gw *Gateway) buildChain(route *types.Route) http.Handler {
handler := gw.proxy.Forward(route.Target)
// Apply in reverse so the first listed middleware is outermost
for i := len(route.Middleware) - 1; i >= 0; i-- {
name := route.Middleware[i]
if mw, ok := gw.middleware[name]; ok {
handler = mw(handler)
}
}
return handler
}
The order middleware is listed in config.yaml is the order requests traverse the chain, so if you list [tracing, auth, ratelimit, logging, metrics], tracing wraps everything, auth runs next, and so on. Getting this ordering right matters a lot in practice.
This was designed so each built-in is registered at startup and custom middleware can be added before calling Start():
gw := core.NewGateway(cfg)
gw.Register("my-middleware", func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// do something before
next.ServeHTTP(w, r)
// do something after
})
})
gw.Start()
Authentication
The auth middleware validates JWT Bearer tokens and injects verified claims into the request context:
func NewAuthMiddleware(verifier TokenVerifier) Middleware {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := extractBearer(r)
if token == "" {
writeError(w, http.StatusUnauthorized, "missing token")
return
}
claims, err := verifier.Verify(token)
if err != nil {
writeError(w, http.StatusUnauthorized, "invalid token")
return
}
ctx := context.WithValue(r.Context(), claimsKey{}, claims)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
}
The middleware itself doesn't care how a token is verified, that's delegated to a TokenVerifier interface:
type TokenVerifier interface {
Verify(token string) (Claims, error)
}
This separation means swapping in a different authentication backend is a matter of implementing one interface, with no changes to the middleware itself. The active verifier is selected at startup based on the auth.type field in config — currently jwt is the built-in implementation, but plugging in something like a Keycloak OIDC verifier would just mean implementing Verify against Keycloak's token introspection endpoint and setting type: keycloak in your config:
auth:
enabled: true
type: keycloak # swap without touching middleware code
# keycloak-specific config would go here
Downstream handlers retrieve the injected claims with middleware.ClaimsFromContext(ctx), regardless of which verifier produced them.
Rate Limiting
Rate limiting uses a per-IP token bucket from golang.org/x/time/rate:
func (rl *rateLimiter) getLimiter(ip string) *rate.Limiter {
rl.mu.Lock()
defer rl.mu.Unlock()
if lim, ok := rl.clients[ip]; ok {
return lim
}
lim := rate.NewLimiter(rate.Limit(rl.rps), rl.burst)
rl.clients[ip] = lim
return lim
}
This works fine for a single instance but has two limitations worth knowing: the map has no eviction, so memory grows unbounded over time; and there's no shared state across instances, so it breaks under horizontal scaling. A production system would push this to Redis. For a learning project, the simplicity is worth it.
Observability
Three layers of observability are built in.
Logging uses Go 1.21's slog with configurable JSON or text output. Each request gets a structured log entry with method, path, status, and duration.
Metrics are exposed via Prometheus at a configurable path (default /metrics). Three instruments are tracked:
gateway_requests_total{method, path, status}
gateway_request_duration_seconds{method, path}
gateway_requests_in_flight
The /metrics endpoint itself is served before route matching, bypassing all middleware, otherwise you'd need to auth-exempt your own scraper.
Distributed tracing uses OpenTelemetry. The tracing middleware creates a server span for each request and propagates a traceparent header to the upstream, so traces stitch together across service boundaries. Spans are exported to Jaeger over gRPC OTLP.
One subtle gotcha: tracing must be explicitly listed in each route's middleware array. It is not in the internal isKnownMiddleware registry, so if you omit it, there's no warning — requests just won't be traced.
Configuration
Everything is controlled by a single config.yaml, parsed strictly with gopkg.in/yaml.v3's KnownFields — unknown keys are rejected at startup rather than silently ignored:
server:
port: 8080
routes:
- path: /users
methods: [GET]
target: http://localhost:9002
middleware: [tracing, auth, ratelimit, logging, metrics]
auth:
enabled: true
type: jwt
secret: "your-secret"
rate_limit:
enabled: true
requests_per_second: 5
burst: 4
metrics:
prometheus:
enabled: true
path: /metrics
otel:
enabled: true
endpoint: localhost:4317 # host:port only, no scheme
Running It
# Start Jaeger for tracing
docker compose up -d
# Start the gateway + mock upstream services
make run-example
# Generate a dev JWT and hit an endpoint
make get-token
# Run this a couple of time to have some request with tracing
curl -H "Authorization: Bearer <token>" http://localhost:8080/users
# Inspect traces and metrics
open http://localhost:16686 # Jaeger UI
curl http://localhost:8080/metrics
Observability in Action
Here's what the observability stack looks like when the gateway is running.
Prometheus metrics endpoint
Hitting /metrics exposes all three instruments in standard Prometheus text format — request counts broken down by method, path, and status code, latency histograms, and the current in-flight count.
Structured JSON logs
With logging.format: json set in config, every request produces a structured log entry. This makes logs trivially parseable by tools like Loki, Datadog, or any log aggregator.
Jaeger trace view
Each request generates a server span with timing and metadata. Because traceparent is propagated to the upstream, the full trace stitches together across the gateway and the backend service in a single waterfall view.
Jaeger service graph
The service graph shows the call relationships between the gateway and its upstream services, making it easy to see which services are being hit and at what rate.
What I Took Away
Building this clarified something I'd always glossed over: the complexity in production gateways (Kong, Traefik, AWS API Gateway) isn't in the core concept, it's entirely in the operational concerns layered on top. Distributed rate limiting, hot config reloading, circuit breaking, mTLS, dynamic service discovery. The fundamental pattern, a middleware chain in front of a reverse proxy, is surprisingly approachable.
If you want to genuinely understand a piece of infrastructure, build a limited version of it. The gaps in your mental model become impossible to ignore once you're responsible for filling them.
The full source is on GitHub at github.com/realEbi/simple-api-gateway.
Top comments (0)