Load Balancer + Rate Limiter Lab: HAProxy Stick-Tables for Connection Rate Limiting

Why Rate-Limit at the Load Balancer

Application-level rate limiting catches abuse after the request has already consumed backend resources -- TCP connection, TLS handshake, request parsing. HAProxy stick-tables let you shed load at the edge before any of that happens.

The Docker Compose Lab

# docker-compose.yml
version: '3.8'
services:
  haproxy:
    image: haproxy:2.9
    ports:
      - "80:80"
      - "8404:8404"  # stats
    volumes:
      - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    depends_on:
      - backend1
      - backend2
      - backend3

  backend1:
    build: ./backend
    environment:
      - BACKEND_ID=1
  backend2:
    build: ./backend
    environment:
      - BACKEND_ID=2
  backend3:
    build: ./backend
    environment:
      - BACKEND_ID=3

The backend is a minimal Go HTTP server:

// backend/main.go
package main

import (
    "fmt"
    "net/http"
    "os"
    "time"
)

func main() {
    id := os.Getenv("BACKEND_ID")
    http.HandleFunc("/api/data", func(w http.ResponseWriter, r *http.Request) {
        // Simulate 20ms of work
        time.Sleep(20 * time.Millisecond)
        w.Header().Set("Content-Type", "application/json")
        fmt.Fprintf(w, `{"backend":"%s","status":"ok"}`, id)
    })
    http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(200)
        fmt.Fprint(w, "ok")
    })
    http.ListenAndServe(":8080", nil)
}

HAProxy Config with Stick-Table Rate Limiting

# haproxy.cfg
global
    log stdout format raw local0

defaults
    mode http
    log global
    timeout connect 5s
    timeout client 30s
    timeout server 30s
    option httplog

frontend http_front
    bind *:80

    # Stick-table: track request rate per source IP
    # 100k entries, 30s TTL, track HTTP request rate over 10s window
    stick-table type ip size 100k expire 30s store http_req_rate(10s)

    # Track the client
    http-request track-sc0 src

    # Deny if more than 50 requests in 10s (5 req/s sustained)
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 50 }

    # Add rate limit headers for observability
    http-response set-header X-RateLimit-Remaining %[sc0_http_req_rate(10s),sub(50)] if { sc_http_req_rate(0) gt 0 }

    default_backend app_servers

backend app_servers
    balance leastconn
    option httpchk GET /healthz
    http-check expect status 200

    server backend1 backend1:8080 check inter 3s fall 3 rise 2
    server backend2 backend2:8080 check inter 3s fall 3 rise 2
    server backend3 backend3:8080 check inter 3s fall 3 rise 2

frontend stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 5s

The key section is the stick-table. type ip size 100k creates an in-memory table keyed by source IP. store http_req_rate(10s) tracks a sliding window of request rates. The http-request deny line checks if the rate exceeds 50 requests per 10 seconds and returns 429 before the request ever reaches a backend.

Benchmarking with wrk

Start the lab and run wrk at increasing concurrency:

docker compose up -d

# Baseline: 10 concurrent connections
wrk -t4 -c10 -d30s http://localhost/api/data
# Expected: all 200s, ~20ms avg latency

# Over limit: 100 concurrent connections
wrk -t4 -c100 -d30s http://localhost/api/data
# Expected: mix of 200s and 429s

# Heavy load: 500 concurrent connections
wrk -t4 -c500 -d30s -L http://localhost/api/data
# Expected: mostly 429s, but p99 latency stays low
#   because HAProxy rejects before forwarding

The -L flag prints the full latency distribution. Compare p50, p99, and p999 across the three runs. Without stick-table rate limiting, the 500-connection test would show p99 latency climbing as backends queue requests. With it, p99 stays flat because excess requests never reach backends.

Checking the Stick-Table Live

HAProxy exposes stick-table contents via the stats socket or the stats page at :8404/stats. You can also query it from the command line:

# Show all tracked IPs and their request rates
docker compose exec haproxy \
  socat stdio /var/run/haproxy/admin.sock <<< "show table http_front"

This outputs each tracked IP with its current request rate, which is useful for debugging why a specific client is getting denied.

Key Differences from Application-Level Rate Limiting

1. No backend resources consumed: HAProxy denies at the connection level before proxying
2. Shared state by default: one HAProxy instance sees all traffic, no Redis needed for single-LB deployments
3. No Retry-After header: stick-tables do not natively calculate when the window resets, so clients get a bare 429. Add application-level rate limiting behind this for richer error responses.
4. Coarser granularity: stick-tables work on IP, not on user identity or API key. Use them as the first layer, with app-level per-user limits behind them.

This two-tier approach matches what Day 19 recommended: coarse limits at the edge, fine-grained limits in the application.

Comments

[Abhinav Sharma]

Would be useful to include the 200 and 429 counts beside the latency numbers in each run. That would make it easier to see how much work the backend actually handled as concurrency increased.