Every post in this series so far has been about getting traffic to the right network and the right host — routing between subnets, DNS resolving a domain to an IP, TCP getting bytes there reliably. This post is about the next question, the one that only matters once you have more than one server: traffic has arrived at your infrastructure, so which specific machine actually handles it? That's the job of a reverse proxy, and load balancing is the most common reason to run one.
Forward Proxy vs. Reverse Proxy — Get This Straight First
These two terms get confused constantly, and the confusion is annoying enough that it's worth settling before anything else.
A forward proxy sits in front of clients and makes requests on their behalf — it hides the client from the server. This is the classic corporate-network setup: your laptop's traffic goes through a company proxy before reaching the internet, and the destination website sees the proxy's IP, not yours. VPNs and privacy tools work on a similar principle.
A reverse proxy sits in front of servers and receives requests on their behalf — it hides the server(s) from the client. When you hit api.example.com, you have no idea (and no need to know) whether that request lands on one server or gets distributed across fifty — the reverse proxy is the only thing the client ever actually talks to directly.
The mental shortcut: a forward proxy protects and represents the client. A reverse proxy protects and represents the server. Nearly everything in this post is about reverse proxies.
What a Reverse Proxy Actually Does
Strip away the load-balancing use case for a second, because a reverse proxy is useful even with a single backend server behind it:
- Hides your backend infrastructure. Clients only ever see the proxy's address, never the actual servers doing the work — which means you can change, scale, or replace backend servers without anything client-facing needing to know.
- Terminates TLS. Instead of every backend server managing its own certificates, the reverse proxy handles the TLS handshake once, decrypts the request, and can forward it to backend servers over plain (but internal, trusted) HTTP. This is called SSL/TLS termination, and it's one of the most common reasons a reverse proxy exists even with just one backend.
- Caching and compression. A reverse proxy can cache responses and serve repeat requests without ever bothering the backend, and it can compress responses before they leave the network.
- A single point for security controls. Rate limiting, request filtering, and WAF (Web Application Firewall) rules are far easier to apply once, at the proxy, than to replicate across every backend server individually.
Once you have more than one backend server, a reverse proxy gains an obvious additional job: deciding which server gets each request. That's load balancing.
Layer 4 vs. Layer 7 Load Balancing
This distinction maps directly back to the OSI post, and it's the single most important thing to understand about load balancers — because it determines what the load balancer is actually capable of seeing and deciding on.
Layer 4 (Transport) load balancing operates at the TCP/UDP level. It looks at IP addresses and ports — nothing higher — and forwards packets to a backend based purely on that information. It doesn't know or care whether the traffic is HTTP, gRPC, or a raw database connection; it's just distributing connections. This makes it extremely fast, since there's minimal processing per packet, but it also means it can't make any decision based on the actual content of the request — it can't route /api/users differently from /api/orders, because at Layer 4 it has never looked inside the packet far enough to see a URL path at all. AWS's Network Load Balancer (NLB) is a Layer 4 load balancer.
Layer 7 (Application) load balancing operates at the HTTP level and above — it actually reads the request: the URL path, the headers, the hostname, cookies, even the request body if configured to. This unlocks routing decisions Layer 4 simply cannot make: send /api/* to one backend service and /static/* to another, route based on the Host header to support multiple domains on one load balancer, or route based on a cookie to keep a specific user pinned to a specific backend. The cost is more processing overhead per request, since the proxy has to actually parse the application-layer protocol rather than just forwarding packets. AWS's Application Load Balancer (ALB), Nginx, and HAProxy in HTTP mode all operate here.
The practical rule: if you need to route purely on IP/port, or you're load balancing something that isn't HTTP at all (a raw TCP database connection, for instance), use Layer 4. If you need routing decisions based on the actual content of a request — which is true for the overwhelming majority of modern web and API traffic — you need Layer 7.
How a Load Balancer Actually Picks a Backend
Once traffic reaches the load balancer, it needs an algorithm to decide which backend server gets each request. The common ones:
- Round robin — requests are distributed to backends in simple rotating order: server A, then B, then C, then back to A. Simple, and reasonable when all backends are roughly equal in capacity.
- Least connections — send the next request to whichever backend currently has the fewest active connections. This handles uneven request durations far better than round robin, since a backend stuck processing a few slow requests won't keep getting piled on with new ones.
- IP hash — the client's IP address is hashed to consistently determine which backend it's sent to. The same client reliably lands on the same backend every time, which is a simple (though inflexible) way to get session consistency without needing shared session storage.
- Weighted variants of any of the above — useful when backend servers aren't identical in capacity, letting you send proportionally more traffic to a beefier server.
Health Checks: The Thing That Makes Load Balancing Actually Safe
An algorithm for picking a backend is useless if the load balancer doesn't know whether that backend is actually healthy. Health checks are periodic probes — typically a simple HTTP request to a dedicated endpoint like /health, or just a TCP connection attempt — that the load balancer sends to each backend on a regular interval.
If a backend fails enough consecutive health checks, the load balancer marks it unhealthy and stops sending it traffic entirely, until it starts passing checks again. This is what actually makes horizontal scaling and rolling deployments survivable in practice: you can take a server down for a deploy, let its health check start failing, watch traffic automatically redirect to the remaining healthy servers, bring it back up, and let it rejoin the pool once it's healthy again — all without a single dropped request reaching a server that couldn't handle it.
Sticky Sessions
Some applications store session state (like a shopping cart, or a logged-in user's data) in memory on whichever specific server handled their first request, rather than in a shared external store. If a load balancer then sends that same user's next request to a different backend, that server has no idea who they are.
Sticky sessions (also called session affinity) solve this by consistently routing a given client to the same backend for the duration of their session — usually via a cookie the load balancer sets, or via the IP hash method mentioned above. It's a legitimate short-term fix, but it's worth knowing the better long-term answer: externalize session state into something like Redis that every backend can read from, so any backend can serve any request and sticky sessions become unnecessary. Sticky sessions also quietly work against the entire point of load balancing — if one backend accumulates a disproportionate number of "sticky" clients, you lose the even distribution you built the load balancer to provide in the first place.
Where This Shows Up in the Stack You're Actually Building
This isn't abstract — it's the exact mechanism behind things you'll configure directly the moment you're deploying anything real:
- An AWS ALB or NLB sitting in front of an EC2 Auto Scaling Group or an ECS service — Layer 7 or Layer 4 respectively, doing exactly what's described above.
- Nginx or HAProxy, self-managed, doing the same job in front of a set of application servers you're running yourself — extremely common in smaller or self-hosted setups.
- A Kubernetes Service of type
LoadBalancer, and Ingress in front of it — Ingress is, in effect, a Layer 7 reverse proxy and load balancer purpose-built for routing traffic to the correct backend Service based on hostname and path, exactly as described above, just running inside the cluster instead of in front of it.
Every one of those is the same underlying idea wearing a different name: sit in front of a pool of backends, hide them from the client, distribute requests intelligently, and stop sending traffic to anything that's stopped responding correctly.
Tying It Back
DNS gets a client to your infrastructure's IP. TCP or UDP gets the connection there reliably (or as fast as possible). And then a reverse proxy or load balancer is the thing that decides, in the last few milliseconds before a request actually gets handled, exactly which server does the work — invisibly to the client the whole time. It's the layer that turns "one server" into "as many servers as you need," without the client ever needing to know the difference.
Top comments (0)