Building Web Servers from First Principles (Part 7)

In Chapter 6, we built a functional JSON API that can create and read users. But production servers need more than just business logic - they need logging, error handling, performance monitoring, and other cross-cutting concerns.

Today, we'll implement middleware - the backbone of production web servers that handles these concerns automatically for every request.

What We've Built So Far

From previous chapters:

• ✅ Complete JSON API with CRUD operations
• ✅ Dynamic routing and parameter extraction
• ✅ Request body parsing and validation
• ❌ Request/response logging
• ❌ Error recovery and graceful error handling
• ❌ Performance monitoring

Understanding Middleware

Middleware are functions that sit between the incoming request and your handler. They can:

1. Pre-process requests (logging, authentication, validation)
2. Post-process responses (logging, formatting, caching)
3. Handle errors (panic recovery, error formatting)
4. Add functionality (timing, metrics, CORS headers)

Think of middleware as layers of an onion - each request passes through multiple layers before reaching your handler, then back through the layers to send the response.

Middleware Pattern in Go

In Go, middleware follows this pattern:

type Middleware func(HandlerFunc) HandlerFunc

A middleware is a function that:

• Takes a handler function as input
• Returns a new handler function as output
• The returned handler can do work before/after calling the original handler

Building Our First Middleware: Request Logging

Let's start with the simplest middleware - logging incoming requests. Create middleware.go:

package main

import (
    "log"
    "net/http"
    "time"
)

type Middleware func(HandlerFunc) HandlerFunc

func RequestLoggingMiddleware(next HandlerFunc) HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        log.Printf("%s %s", r.Method, r.URL.Path)
        next(w, r)  // Call the actual handler
    }
}

How this works:

1. RequestLoggingMiddleware receives the next handler (next)
2. It returns a new function that logs the request, then calls next
3. The new function has the same signature as a regular handler

Adding Middleware Support to Our Router

We need to modify our router to support middleware. Update router.go:

type Router struct {
    routes      map[string]map[string]HandlerFunc
    middlewares []Middleware  // Add this field
}

func (r *Router) Use(middleware Middleware) {
    r.middlewares = append(r.middlewares, middleware)
}

Now update the ServeHTTP method to apply middleware:

func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
    handler, err := r.resolveRoute(req)

    if err != nil {
        handler = r.notFoundHandler
    }

    // Apply all middlewares in order
    for _, middleware := range r.middlewares {
        handler = middleware(handler)
    }

    handler(w, req)
}

func (r *Router) notFoundHandler(w http.ResponseWriter, req *http.Request) {
    w.WriteHeader(http.StatusNotFound)
    w.Write([]byte("route not found"))
}

Don't forget to add the missing import and remove the log statement from resolveRoute:

import (
    "context"
    "fmt"
    "net/http"  // Remove the log import from here
    "strings"
)

// In resolveRoute function, remove this line:
// log.Printf("%s %s", method, path)

Testing Request Logging Middleware

Update main.go to use our middleware:

func setupRoutes(s *Server) {
    s.Router.Use(RequestLoggingMiddleware)
    s.Router.POST("/echo", echo)
    s.Router.GET("/users/:id", getUsers)
    s.Router.POST("/users", createUser)
}

Test it:

go run .

Make some requests:

curl http://localhost:3000/users/1
curl -X POST http://localhost:3000/users -H "Content-Type: application/json" -d '{"name":"Test","email":"[e-mail]"}'

You should see logs like:

2024/09/14 15:30:45 GET /users/1
2024/09/14 15:30:50 POST /users

Great! Our middleware is working.

Building Timing Middleware

Let's add performance monitoring by measuring how long each request takes:

func TimingMiddleware(next HandlerFunc) HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        next(w, r)  // Execute the handler
        elapsed := time.Since(start)
        log.Printf("Request took %v seconds", elapsed.Seconds())
    }
}

Add it to your routes:

func setupRoutes(s *Server) {
    s.Router.Use(RequestLoggingMiddleware)
    s.Router.Use(TimingMiddleware)
    s.Router.POST("/echo", echo)
    s.Router.GET("/users/:id", getUsers)
    s.Router.POST("/users", createUser)
}

Test it and you'll see timing logs:

2024/09/14 15:31:15 GET /users/1
2024/09/14 15:31:15 Request took 0.000123 seconds

Building Response Logging Middleware

Request logging shows what's coming in, but we also want to see what's going out. This is trickier because we need to capture both the status code and response body.

We need to create a custom ResponseWriter that wraps the original one:

type responseWriter struct {
    http.ResponseWriter
    statusCode int
    body       []byte
}

func (rw *responseWriter) WriteHeader(code int) {
    rw.statusCode = code
    rw.ResponseWriter.WriteHeader(code)
}

func (rw *responseWriter) Write(body []byte) (int, error) {
    rw.body = append(rw.body, body...)
    return rw.ResponseWriter.Write(body)
}

func ResponseLoggingMiddleware(next HandlerFunc) HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        wrapped := &responseWriter{
            ResponseWriter: w,
            statusCode:     http.StatusOK,  // Default status
        }

        next(wrapped, r)  // Pass our wrapped writer to the handler

        log.Printf("%s %s - Status: %d, Body: %s",
            r.Method, r.URL.Path, wrapped.statusCode,
            string(wrapped.body))
    }
}

How this works:

1. We create a custom responseWriter that wraps the original
2. It captures the status code in WriteHeader()
3. It captures the response body in Write()
4. After the handler runs, we log both pieces of information

Add it to your middleware stack:

func setupRoutes(s *Server) {
    s.Router.Use(RequestLoggingMiddleware)
    s.Router.Use(ResponseLoggingMiddleware)
    s.Router.Use(TimingMiddleware)
    // ... routes
}

Add the required imports:

import (
    "log"
    "net/http"
    "runtime/debug"  // We'll need this for the next middleware
    "time"
)

Test it and you'll see detailed response logs:

2024/09/14 15:32:20 GET /users/1
2024/09/14 15:32:20 GET /users/1 - Status: 200, Body: {"data":{"id":1,"name":"John Doe","email":"[e-mail]"}}
2024/09/14 15:32:20 Request took 0.000089 seconds

Building Panic Recovery Middleware

This is the most important middleware for production - it catches panics and prevents your server from crashing:

func PanicRecoveryMiddleware(next HandlerFunc) HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        defer func() {
            if err := recover(); err != nil {
                log.Printf("Panic recovered: %v\n%s", err, debug.Stack())

                w.Header().Set("Content-Type", "application/json")
                w.WriteHeader(http.StatusInternalServerError)
                w.Write([]byte(`{"error": "Internal server error"}`))
            }
        }()

        next(w, r)
    }
}

How this works:

1. defer func() runs after the handler completes (or panics)
2. recover() catches any panic that occurred
3. We log the error and stack trace for debugging
4. We send a proper JSON error response to the client
5. The server stays running!

Let's test panic recovery by adding a panic to one of our handlers. Update the echo handler in main.go:

func echo(w http.ResponseWriter, r *http.Request) {
    var payload EchoPayload

    panic("I am panicking")  // Add this line to test panic recovery

    err := readBody(r, &payload)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte(err.Error()))
        return
    }

    w.Write([]byte(fmt.Sprintf("You said '%s'", payload.Message)))
}

Add panic recovery to your middleware stack (it should be first to catch panics from other middleware):

func setupRoutes(s *Server) {
    s.Router.Use(PanicRecoveryMiddleware)
    s.Router.Use(RequestLoggingMiddleware)
    s.Router.Use(ResponseLoggingMiddleware)
    s.Router.Use(TimingMiddleware)
    s.Router.POST("/echo", echo)
    s.Router.GET("/users/:id", getUsers)
    s.Router.POST("/users", createUser)
}

Test the panic recovery:

curl -X POST http://localhost:3000/echo -H "Content-Type: application/json" -d '{"message":"test"}'

You should see:

1. Client receives: {"error": "Internal server error"}
2. Server logs: Detailed panic information with stack trace
3. Server keeps running instead of crashing!

Understanding Middleware Order

The order of middleware matters! They form a chain:

Request → Panic Recovery → Request Logging → Response Logging → Timing → Handler
                ↓              ↓                ↓              ↓
Response ← Panic Recovery ← Request Logging ← Response Logging ← Timing ← Handler

Best practices for middleware order:

1. Panic Recovery - First, to catch panics from all other middleware
2. Logging - Early, to log all requests (even ones that error)
3. Authentication - Before business logic
4. Business Logic Handlers - Last

What We've Accomplished

We now have:

• ✅ Request Logging (see what requests come in)
• ✅ Response Logging (see what responses go out)
• ✅ Timing Middleware (performance monitoring)
• ✅ Panic Recovery (graceful error handling)
• ✅ Middleware Chain (composable, reusable middleware)
• ✅ Custom ResponseWriter (intercepting response data)
• ✅ Production-Ready Error Handling (no server crashes)

Testing the Complete Middleware Stack

Remove the panic from the echo handler and test everything:

func echo(w http.ResponseWriter, r *http.Request) {
    var payload EchoPayload

    // Remove or comment out: panic("I am panicking")

    err := readBody(r, &payload)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte(err.Error()))
        return
    }

    w.Write([]byte(fmt.Sprintf("You said '%s'", payload.Message)))
}

Test various endpoints:

# Test echo
curl -X POST http://localhost:3000/echo -H "Content-Type: application/json" -d '{"message":"Hello middleware!"}'

# Test user creation
curl -X POST http://localhost:3000/users -H "Content-Type: application/json" -d '{"name":"Alice","email":"[e-mail]"}'

# Test user retrieval
curl http://localhost:3000/users/1

# Test 404
curl http://localhost:3000/nonexistent

You should see comprehensive logging for every request:

2024/09/14 15:35:10 POST /echo
2024/09/14 15:35:10 POST /echo - Status: 200, Body: You said 'Hello middleware!'
2024/09/14 15:35:10 Request took 0.000045 seconds

Comparing to Real Frameworks

Our Approach:

s.Router.Use(RequestLoggingMiddleware)
s.Router.Use(TimingMiddleware)
s.Router.GET("/users/:id", getUsers)

Gin Framework:

r.Use(gin.Logger())
r.Use(gin.Recovery())
r.GET("/users/:id", getUsers)

Echo Framework:

e.Use(middleware.Logger())
e.Use(middleware.Recover())
e.GET("/users/:id", getUsers)

The concepts are identical! We've built the same middleware system that production frameworks use.

What We've Accomplished in This Series

Across all 7 chapters, we've built a complete web server from scratch:

✅ Foundation (Chapters 1-2)

• HTTP server that listens for requests
• Basic routing with switch statements

✅ Architecture (Chapters 3-4)

• Custom router with clean separation of concerns
• Dynamic routing with URL parameters

✅ API Features (Chapters 5-6)

• Query parameter handling
• JSON request/response processing
• CRUD operations with data persistence

✅ Production Readiness (Chapter 7)

• Comprehensive logging
• Error recovery and graceful error handling
• Performance monitoring
• Middleware architecture

Final Thoughts

You now understand exactly what happens when your frontend calls an API:

1. Request hits the server (Chapter 1)
2. Router matches the path (Chapters 3-4)
3. Middleware processes the request (Chapter 7)
4. Handler parses data and does business logic (Chapters 5-6)
5. Middleware processes the response (Chapter 7)
6. Response returns to the client

This is the same flow in Express, Gin, Django, Rails, or any web framework - the abstractions are different, but the fundamentals are identical.

---

Final Challenge: Try building additional middleware for:

• CORS handling (for frontend applications)
• Rate limiting (prevent abuse)
• Authentication (protect certain routes)
• Request ID tracking (trace requests across services)

You now have all the tools to build production-ready web servers from first principles!

---

Series Complete! 🎉

You've built a complete web server with routing, JSON APIs, and production-ready middleware. The concepts you've learned apply to any web framework or language - you now understand the fundamentals beneath all web development. Let me know in the comments if this really helped you.