Reverse Proxy: The Architecture Pattern Every Backend Engineer Should Master

Introduction: Why Reverse Proxies Matter in Modern Architecture

When you access a website, send a message on WhatsApp, or stream video on Netflix, you're interacting with a reverse proxy—even if you don't realize it. A reverse proxy is the invisible guardian standing between your request and the backend servers that process it. It's not just infrastructure plumbing; it's a critical architectural decision that affects scalability, security, performance, and reliability.

If you're building backend systems at scale, understanding reverse proxies isn't optional—it's essential. Companies like Netflix, Uber, Amazon, and Stripe rely on reverse proxies to handle millions of concurrent requests, distribute load across thousands of servers, and protect their infrastructure from attacks.

This guide explores reverse proxies from first principles, covers popular implementations (Nginx, HAProxy, Envoy), shows you how to build one in Java, and demonstrates production patterns used by companies handling massive traffic.

---

Part 1: Understanding Reverse Proxies

Forward Proxy vs. Reverse Proxy

These terms confuse many engineers. Let's clarify:

Forward Proxy (what you think of as a "proxy"):

• Sits between the client and the internet
• Client explicitly tells it where to go
• Used for: privacy, caching, content filtering
• Example: Corporate firewall, VPN

Client → Forward Proxy → Internet/Server

Reverse Proxy (what we're discussing):

• Sits between the internet and your backend servers
• Client doesn't know it's there—it looks like it's talking to one server
• Used for: load balancing, security, caching, SSL termination
• Example: Nginx, HAProxy, Envoy

Client → Reverse Proxy → Internal Servers

Core Responsibilities of a Reverse Proxy

A reverse proxy wears many hats:

1. Request Routing: Direct incoming requests to the right backend service
2. Load Balancing: Distribute load across multiple servers
3. SSL/TLS Termination: Decrypt HTTPS, talk to backends in plain HTTP
4. Caching: Store responses to avoid hitting backends unnecessarily
5. Compression: Reduce response size for faster transmission
6. Rate Limiting: Protect backends from abuse
7. Request/Response Modification: Add headers, rewrite URLs, etc.
8. Health Checking: Monitor backend health, remove unhealthy servers
9. Circuit Breaking: Protect backends when they fail
10. Authentication/Authorization: Validate requests before reaching backends

The Problem a Reverse Proxy Solves

Imagine you have a web application with 10 backend servers. Without a reverse proxy:

• Clients need to know about all 10 servers (how?)
• If one server dies, clients still try to connect to it
• Scaling requires updating client configurations
• HTTPS encryption/decryption happens on every backend
• No central point to enforce rate limits or authentication
• Backend IPs are exposed to the internet

With a reverse proxy:

• Clients connect to ONE IP address
• Proxy automatically handles dead servers
• Scaling is transparent—add servers, proxy routes to them
• SSL termination happens once, not 10 times
• Rate limiting, auth, and logging happen in one place
• Backend IPs stay private

---

Part 2: Architecture and Patterns

Reverse Proxy Topologies

Single Reverse Proxy:

                     ┌─────────────┐
                     │   Clients   │
                     └──────┬──────┘
                            │
                     ┌──────┴──────┐
                     │Reverse Proxy│
                     └──────┬──────┘
              ┌─────────────┼─────────────┐
              │             │             │
         ┌────┴────┐   ┌───┴────┐   ┌───┴────┐
         │Backend 1│   │Backend 2│   │Backend 3│
         └─────────┘   └────────┘   └────────┘

Problem: Single point of failure

Solution: Redundant proxies

---

Highly Available Reverse Proxies (Active-Active):

                     ┌─────────────┐
                     │   Clients   │
                     └──────┬──────┘
                            │
                    ┌───────┴───────┐
                    │  Load Balancer│ (DNS/anycast)
                    └───┬───────┬───┘
              ┌─────────┘       └─────────┐
         ┌────┴──────────┐          ┌────┴──────────┐
         │Reverse Proxy 1│          │Reverse Proxy 2│
         └────┬──────────┘          └────┬──────────┘
              │                          │
         ┌────┴─────────────────────────┴───┐
         │     Backend Services             │
         └─────────────────────────────────┘

Advantage: No single point of failure

Used by: Netflix, Google, Facebook

---

Service Mesh Pattern (Envoy Sidecars):

Each service has a local Envoy proxy sidecar
Traffic flows: Service A → Envoy → Network → Envoy → Service B

Benefits:
- Observability (metrics at proxy layer)
- Mutual TLS encryption between services
- Sophisticated routing policies
- Retries, timeouts, circuit breaking

Used by: Kubernetes ecosystems with Istio, Linkerd

Load Balancing Algorithms

How does a reverse proxy decide which backend to send a request to?

Round Robin

• Send requests to servers in sequence: 1, 2, 3, 1, 2, 3...
• Simple, fair distribution
• Problem: Doesn't account for server capacity

// Round robin pseudocode
int nextServerIndex = requestCount % serverCount;
Server selected = servers[nextServerIndex];

Least Connections

• Send request to server with fewest active connections
• Good when requests have varying duration
• More complex to track state

// Least connections
Server selected = servers.stream()
    .min(Comparator.comparing(Server::getActiveConnections))
    .orElse(defaultServer);

IP Hash / Consistent Hashing

• Hash client IP to determine server
• Same client always goes to same server (session affinity)
• Good for: stateful applications, caching
• Problem: Adding servers redistributes hash space

int serverIndex = clientIpHash % serverCount;
Server selected = servers[serverIndex];

Weighted Distribution

• Assign capacity weights to servers
• Powerful server gets more traffic: 60%, 30%, 10%
• Perfect for heterogeneous hardware

// If server 1 has weight 60, server 2 has weight 30, server 3 has weight 10
// 60% of requests go to server 1, etc.

Least Response Time

• Send to server with lowest average response time
• Adapts to actual performance
• More computational overhead

---

Part 3: Building a Reverse Proxy in Java

Simple Reverse Proxy with Spring Boot

Let's build a basic reverse proxy to understand the mechanics:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.web.reactive.function.client.WebClient;
import org.springframework.web.reactive.function.client.ExchangeStrategies;
import org.springframework.web.server.WebFilter;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import reactor.core.publisher.Mono;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;

@SpringBootApplication
public class ReverseProxyApplication {

    private static final List<String> BACKENDS = List.of(
        "http://backend-1:8081",
        "http://backend-2:8081",
        "http://backend-3:8081"
    );

    private static final AtomicInteger COUNTER = new AtomicInteger(0);

    public static void main(String[] args) {
        SpringApplication.run(ReverseProxyApplication.class, args);
    }

    @Bean
    public WebClient webClient() {
        ExchangeStrategies strategies = ExchangeStrategies.builder()
            .codecs(configurer -> configurer.defaultCodecConfig()
                .maxInMemorySize(500 * 1024)) // 500KB buffer
            .build();

        return WebClient.builder()
            .exchangeStrategies(strategies)
            .responseTimeout(java.time.Duration.ofSeconds(30))
            .build();
    }

    @Bean
    public WebFilter reverseProxyFilter(WebClient webClient) {
        return (exchange, chain) -> {
            String backendUrl = selectBackend();
            String forwardUrl = backendUrl + exchange.getRequest().getURI().getRawPath();

            // Add query string if present
            String query = exchange.getRequest().getURI().getRawQuery();
            if (query != null && !query.isEmpty()) {
                forwardUrl += "?" + query;
            }

            // Log the proxy operation
            System.out.println("Proxying: " + exchange.getRequest().getURI() 
                + " → " + forwardUrl);

            return webClient
                .method(exchange.getRequest().getMethod())
                .uri(forwardUrl)
                .headers(httpHeaders -> {
                    // Copy client headers to backend request
                    httpHeaders.putAll(exchange.getRequest().getHeaders());
                    // Add X-Forwarded headers (important for logging)
                    httpHeaders.add("X-Forwarded-For", 
                        exchange.getRequest().getRemoteAddress().getAddress().getHostAddress());
                    httpHeaders.add("X-Forwarded-Proto", 
                        exchange.getRequest().getURI().getScheme());
                })
                .body(exchange.getRequest().getBody())
                .retrieve()
                .toEntity(byte[].class)
                .flatMap(responseEntity -> {
                    // Copy response status and headers
                    exchange.getResponse().setStatusCode(responseEntity.getStatusCode());
                    exchange.getResponse().getHeaders()
                        .putAll(responseEntity.getHeaders());

                    // Write response body
                    if (responseEntity.getBody() != null) {
                        return exchange.getResponse()
                            .writeWith(Mono.just(exchange.getResponse()
                                .bufferFactory()
                                .wrap(responseEntity.getBody())));
                    }
                    return exchange.getResponse().setComplete();
                })
                .onErrorResume(error -> {
                    exchange.getResponse().setStatusCode(HttpStatus.BAD_GATEWAY);
                    return exchange.getResponse().setComplete();
                });
        };
    }

    private String selectBackend() {
        // Round-robin load balancing
        int index = Math.abs(COUNTER.getAndIncrement()) % BACKENDS.size();
        return BACKENDS.get(index);
    }
}

Advanced: Reverse Proxy with Health Checking

@Service
public class BackendHealthChecker {

    private final WebClient webClient;
    private final List<BackendServer> servers;
    private final ScheduledExecutorService scheduler;

    public BackendHealthChecker(WebClient webClient) {
        this.webClient = webClient;
        this.servers = new CopyOnWriteArrayList<>();
        this.scheduler = Executors.newScheduledThreadPool(1);

        // Initialize servers
        servers.add(new BackendServer("http://backend-1:8081"));
        servers.add(new BackendServer("http://backend-2:8081"));
        servers.add(new BackendServer("http://backend-3:8081"));

        startHealthChecks();
    }

    private void startHealthChecks() {
        scheduler.scheduleAtFixedRate(
            this::checkAllServers,
            5,  // Initial delay
            10, // Period
            TimeUnit.SECONDS
        );
    }

    private void checkAllServers() {
        servers.parallelStream()
            .forEach(server -> checkServer(server));
    }

    private void checkServer(BackendServer server) {
        long startTime = System.currentTimeMillis();

        webClient.get()
            .uri(server.url + "/health")
            .retrieve()
            .toBodilessEntity()
            .timeout(java.time.Duration.ofSeconds(5))
            .subscribe(
                response -> {
                    long responseTime = System.currentTimeMillis() - startTime;
                    server.setHealthy(true);
                    server.setLastResponseTime(responseTime);
                    server.setLastHealthCheckTime(System.currentTimeMillis());
                },
                error -> {
                    server.setHealthy(false);
                    System.err.println("Health check failed for " + server.url 
                        + ": " + error.getMessage());
                }
            );
    }

    public BackendServer selectHealthyBackend() {
        List<BackendServer> healthyServers = servers.stream()
            .filter(BackendServer::isHealthy)
            .collect(Collectors.toList());

        if (healthyServers.isEmpty()) {
            throw new RuntimeException("No healthy backends available");
        }

        // Least response time among healthy servers
        return healthyServers.stream()
            .min(Comparator.comparing(BackendServer::getLastResponseTime))
            .orElseThrow();
    }

    public static class BackendServer {
        private final String url;
        private volatile boolean healthy = true;
        private volatile long lastResponseTime = 0;
        private volatile long lastHealthCheckTime = 0;

        BackendServer(String url) {
            this.url = url;
        }

        // Getters and setters...
    }
}

Rate Limiting in the Reverse Proxy

@Component
public class RateLimitingFilter implements WebFilter {

    private final LoadingCache<String, AtomicLong> requestCounts;
    private static final int MAX_REQUESTS_PER_MINUTE = 100;

    public RateLimitingFilter() {
        // Cache that expires after 1 minute
        this.requestCounts = CacheBuilder.newBuilder()
            .expireAfterWrite(1, TimeUnit.MINUTES)
            .build(new CacheLoader<String, AtomicLong>() {
                @Override
                public AtomicLong load(String key) {
                    return new AtomicLong(0);
                }
            });
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
        String clientIp = exchange.getRequest().getRemoteAddress().getAddress().getHostAddress();
        AtomicLong counter = requestCounts.getUnchecked(clientIp);

        if (counter.incrementAndGet() > MAX_REQUESTS_PER_MINUTE) {
            exchange.getResponse().setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
            exchange.getResponse().getHeaders().add(
                "Retry-After", "60"
            );
            return exchange.getResponse().setComplete();
        }

        return chain.filter(exchange);
    }
}

Connection Pooling for Backend Communication

@Configuration
public class HttpClientConfiguration {

    @Bean
    public HttpClient httpClient() {
        return HttpClient.create()
            .secure(spec -> spec.sslContext(
                SslContextBuilder.forClient()
                    .build()
            ))
            .tcpConfiguration(tcpClient -> tcpClient
                .connectTimeoutMillis(5000)  // 5 second connection timeout
                .doOnConnected(conn -> conn
                    .addHandlerLast(new ReadTimeoutHandler(10, TimeUnit.SECONDS))
                    .addHandlerLast(new WriteTimeoutHandler(10, TimeUnit.SECONDS)))
                .connectionProvider(
                    ConnectionProvider.builder("pool")
                        .maxConnections(500)        // Max concurrent connections
                        .maxIdleTime(Duration.ofSeconds(20))
                        .maxLifeTime(Duration.ofMinutes(30))
                        .pendingAcquireMaxCount(500)
                        .pendingAcquireTimeout(Duration.ofSeconds(45))
                        .build()
                ))
            .responseTimeout(Duration.ofSeconds(30));
    }

    @Bean
    public WebClient webClient(HttpClient httpClient) {
        return WebClient.builder()
            .clientConnector(new ReactorClientHttpConnector(httpClient))
            .build();
    }
}

---

Part 4: Production Reverse Proxies

Nginx: The Industry Standard

Installation:

# macOS
brew install nginx

# Ubuntu
sudo apt-get install nginx

# Start
nginx

Basic Configuration:

upstream backend_servers {
    server backend1.example.com:8080;
    server backend2.example.com:8080;
    server backend3.example.com:8080;
}

server {
    listen 80;
    server_name api.example.com;

    # Redirect HTTP to HTTPS
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name api.example.com;

    # SSL certificates
    ssl_certificate /etc/ssl/certs/example.com.crt;
    ssl_certificate_key /etc/ssl/private/example.com.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Gzip compression
    gzip on;
    gzip_types text/plain application/json application/javascript;
    gzip_min_length 1000;

    # Rate limiting
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
    limit_req zone=api_limit burst=20 nodelay;

    location / {
        # Pass request to backend
        proxy_pass http://backend_servers;

        # Important headers for backend logging
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Request-ID $request_id;

        # Timeouts
        proxy_connect_timeout 5s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;

        # Buffering
        proxy_buffering on;
        proxy_buffer_size 4k;
        proxy_buffers 8 4k;

        # Error handling
        proxy_intercept_errors on;
        error_page 502 503 504 /50x.html;
    }

    # Health check endpoint
    location /health {
        return 200 "healthy\n";
        add_header Content-Type text/plain;
    }
}

Load Balancing with Weights:

upstream weighted_backends {
    server backend1.com:8080 weight=5;  # 50% traffic
    server backend2.com:8080 weight=3;  # 30% traffic
    server backend3.com:8080 weight=2;  # 20% traffic
}

HAProxy: Advanced Load Balancing

Configuration:

global
    maxconn 4096
    log 127.0.0.1 local0
    log 127.0.0.1 local1 notice

defaults
    log     global
    mode    http
    timeout connect 5000
    timeout client  50000
    timeout server  50000

frontend web_in
    bind *:80
    redirect scheme https code 301 if !{ ssl_fc }

frontend web_https
    bind *:443 ssl crt /etc/ssl/example.com.pem
    default_backend backend_servers

    # Rate limiting
    stick-table type ip size 100k expire 30s store http_req_rate(10s)
    http-request track-sc0 src
    http-request deny if { sc_http_req_rate(0) gt 100 }

backend backend_servers
    balance leastconn  # Least connections
    option httpchk GET /health HTTP/1.1\r\nHost:\ example.com

    server backend1 backend1.com:8080 check inter 2s rise 3 fall 2
    server backend2 backend2.com:8080 check inter 2s rise 3 fall 2
    server backend3 backend3.com:8080 check inter 2s rise 3 fall 2

    # Connection pooling
    http-reuse safe

Envoy: The Cloud-Native Choice

YAML Configuration:

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 10000
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          http_filters:
          - name: envoy.filters.http.router
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: backend_service
                  timeout: 30s
                  retry_policy:
                    retry_on: "5xx"
                    num_retries: 3

  clusters:
  - name: backend_service
    connect_timeout: 1s
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: backend_service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: backend1.com
                port_value: 8080
        - endpoint:
            address:
              socket_address:
                address: backend2.com
                port_value: 8080
        - endpoint:
            address:
              socket_address:
                address: backend3.com
                port_value: 8080
    health_checks:
    - timeout: 1s
      interval: 10s
      unhealthy_threshold: 2
      healthy_threshold: 2
      http_health_check:
        path: "/health"

---

Part 5: Best Practices and Optimization

1. Always Use X-Forwarded Headers

Backends need to know the real client IP. The proxy must forward:

// In your reverse proxy
httpHeaders.add("X-Forwarded-For", clientIp);
httpHeaders.add("X-Forwarded-Proto", scheme);
httpHeaders.add("X-Forwarded-Host", host);
httpHeaders.add("X-Request-ID", UUID.randomUUID().toString());

In your backend Spring Boot app:

@RestController
public class UserController {

    @GetMapping("/users")
    public ResponseEntity getUsers(
        @RequestHeader(value = "X-Forwarded-For", required = false) String clientIp,
        @RequestHeader(value = "X-Request-ID", required = false) String requestId
    ) {
        // Use clientIp for logging and rate limiting
        // Use requestId to trace request through system
        return ResponseEntity.ok("client: " + clientIp);
    }
}

2. Implement Circuit Breaking

If backends are unhealthy, fail fast:

@Service
public class CircuitBreaker {

    private final AtomicInteger consecutiveFailures = new AtomicInteger(0);
    private volatile boolean circuitOpen = false;
    private volatile long lastFailureTime = 0;
    private static final int FAILURE_THRESHOLD = 5;
    private static final long RECOVERY_TIMEOUT = 30000; // 30 seconds

    public <T> T execute(Supplier<T> supplier) throws Exception {
        if (circuitOpen) {
            // Check if recovery time has elapsed
            if (System.currentTimeMillis() - lastFailureTime > RECOVERY_TIMEOUT) {
                circuitOpen = false;
                consecutiveFailures.set(0);
            } else {
                throw new CircuitBreakerOpenException("Circuit is open");
            }
        }

        try {
            T result = supplier.get();
            // Success: reset failures
            consecutiveFailures.set(0);
            return result;
        } catch (Exception e) {
            int failures = consecutiveFailures.incrementAndGet();
            lastFailureTime = System.currentTimeMillis();

            if (failures >= FAILURE_THRESHOLD) {
                circuitOpen = true;
                throw new CircuitBreakerOpenException(
                    "Circuit opened after " + failures + " failures"
                );
            }
            throw e;
        }
    }
}

3. Optimize Connection Pooling

Every proxy needs efficient connection management:

@Configuration
public class PoolingConfiguration {

    @Bean
    public ConnectionProvider connectionProvider() {
        return ConnectionProvider.builder("custom")
            .maxConnections(1000)          // Total connections
            .maxIdleTime(Duration.ofSeconds(20))
            .maxLifeTime(Duration.ofMinutes(10))
            .pendingAcquireMaxCount(100)   // Queue size
            .pendingAcquireTimeout(Duration.ofSeconds(30))
            .build();
    }
}

4. Caching Strategies

Cache responses to reduce backend load:

@Component
public class CachingFilter implements WebFilter {

    private final Map<String, CachedResponse> cache = 
        new ConcurrentHashMap<>();

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
        String cacheKey = getCacheKey(exchange);

        // Check cache
        CachedResponse cached = cache.get(cacheKey);
        if (cached != null && !cached.isExpired()) {
            return serveCachedResponse(exchange, cached);
        }

        // Call backend
        return chain.filter(exchange)
            .doFinally(signalType -> {
                // Cache response if cacheable
                if (isCacheable(exchange)) {
                    cache.put(cacheKey, 
                        new CachedResponse(exchange, 
                        getTTL(exchange)));
                }
            });
    }
}

5. Monitoring and Observability

Track proxy metrics:

@Component
public class ProxyMetrics {

    private final MeterRegistry meterRegistry;
    private final Timer proxyLatency;
    private final Counter backendErrors;

    public ProxyMetrics(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
        this.proxyLatency = Timer.builder("proxy.request.latency")
            .publishPercentiles(0.5, 0.95, 0.99)
            .register(meterRegistry);
        this.backendErrors = Counter.builder("proxy.backend.errors")
            .register(meterRegistry);
    }

    public void recordProxyRequest(long durationMs, 
                                   int statusCode, 
                                   String backend) {
        proxyLatency.record(Duration.ofMillis(durationMs));

        if (statusCode >= 500) {
            backendErrors.increment();
        }

        meterRegistry.counter("proxy.requests.total",
            "status", String.valueOf(statusCode),
            "backend", backend
        ).increment();
    }
}

---

Conclusion: Reverse Proxies as Infrastructure

A reverse proxy is far more than "something that forwards requests." It's a foundational architectural pattern that enables:

• Scalability: Transparently add servers without changing clients
• Reliability: Remove failed servers automatically
• Security: Hide backend servers, enforce authentication
• Performance: Cache, compress, optimize connections
• Observability: Central point to log and monitor all traffic

Companies at scale can't live without reverse proxies. Whether you choose Nginx for simplicity, HAProxy for advanced features, Envoy for cloud-native deployments, or build your own in Java for specific needs, the principles remain constant.

Key takeaways:

1. Reverse proxies sit between clients and backends
2. They handle: routing, load balancing, SSL, caching, health checks
3. Load balancing algorithms: round-robin, least connections, consistent hashing
4. Production-grade requires: health checking, circuit breaking, connection pooling, monitoring
5. Nginx for 90% of use cases; Envoy for complex cloud deployments

Master reverse proxies, and you master a cornerstone of modern backend architecture.

---

References

• Nginx Documentation: https://nginx.org/en/docs/
• HAProxy Configuration Manual: http://www.haproxy.org/
• Envoy Documentation: https://www.envoyproxy.io/docs/
• System Design Interview by Alex Xu
• Designing Data-Intensive Applications by Martin Kleppmann