Layer 4 vs. Layer 7 Load Balancing: 12 Million Packets a Second and the Cost of Reading Every Request

Layer 4 vs. Layer 7 Load Balancing: 12 Million Packets a Second and the Cost of Reading Every Request

Google has used a software load balancer called Maglev since 2008. According to the paper Google published in 2016, a single Maglev machine can saturate a 10 Gbps link even with small packets. In one benchmark, a Maglev instance on an 8-core CPU topped out around 12 million packets per second, and Maglev bypasses the Linux kernel network stack, which would have capped it below 4 million. Part of why it's so fast is what it doesn't do. It never reads your web request. It looks at which connection a packet belongs to, picks a backend using consistent hashing, and tracks the connection so later packets follow the same path.

Now consider the opposite end of the spectrum. On July 2, 2019, Cloudflare pushed a single new web application firewall rule that inspected HTTP traffic for attack patterns. The rule contained a regular expression that backtracked badly, and CPU hit 100% on every core handling HTTP and HTTPS traffic across the network. Visitors saw 502 errors for 27 minutes. Cloudflare said at the time that it wasn't an attack, and the cause was a software deploy. The story isn't that inspecting requests is bad. It's that reading the content of every request costs real CPU, and doing it badly at scale hurts.

Those two stories are the whole Layer 4 versus Layer 7 debate in miniature. Both kinds of load balancer spread traffic across servers. One makes decisions from addresses and ports at enormous speed. The other reads the request itself and makes smarter, costlier decisions. This guide explains what each one actually does, where the real numbers come from, and how to decide, which in many real systems means using both.

If the transport layer is new to you, start with our TCP vs. UDP guide, and our Networking Basics and System Design courses cover the fundamentals this comparison builds on.

This post originally appeared on the Ciphemic Academia blog.

The Short Version

• Layer 4 (transport) load balancing chooses a backend using connection information: IP addresses, ports, and protocol (TCP or UDP). It doesn't look inside the traffic, so it's fast and works with almost any protocol.
• Layer 7 (application) load balancing terminates the connection, reads the request (for HTTP, that means the path, host, headers, and cookies), and routes based on what it finds. It's slower per request but far smarter.

If you want one default: for web apps and APIs, start with a Layer 7 load balancer, because routing by path or host and terminating TLS in one place is what most teams need. Reach for Layer 4 when the traffic isn't HTTP, when you need static IPs or extreme throughput, or as a front tier in front of a Layer 7 fleet.

What Both Are Actually Trying to Solve

Before the differences, the shared job:

• Spreading load so no single server is overwhelmed
• Removing unhealthy servers from rotation using health checks, so users don't hit dead backends
• Scaling horizontally by letting you add or remove servers behind one stable address
• Avoiding a single point of failure by running multiple balancer instances, as Maglev does with a pool of ordinary servers instead of one special box
• Draining gracefully so servers can be taken out of rotation without cutting off requests in flight

The difference is how much of the traffic the balancer is willing to understand before it decides. That choice drives speed, features, cost, and failure modes.

Layer 4 Load Balancing

What it actually is: a balancer that decides where to send a connection using only transport-level information. The key inputs are the source and destination IP address, the source and destination port, and the protocol, often called the five-tuple. It forwards packets or connections to a backend and doesn't care whether the payload is HTTP, a database protocol, MQTT, or game traffic. Some Layer 4 designs even let backends reply directly to the client, skipping the balancer on the way back.

A small example (HAProxy in TCP mode, balancing a PostgreSQL read pool):

frontend db_in
    mode tcp
    bind :5432
    default_backend postgres_pool

backend postgres_pool
    mode tcp
    balance leastconn
    server pg1 10.0.1.11:5432 check
    server pg2 10.0.1.12:5432 check

Where it shines:

• Raw speed: with no payload to parse, Layer 4 balancers like Maglev can handle packet rates in the millions per second per machine
• Protocol independence: anything over TCP or UDP works, including databases, message brokers, game servers, and custom protocols
• Static IPs and client IP preservation: AWS's Network Load Balancer, a Layer 4 service, offers a static IP per availability zone and passes the client's source IP through to the backend, which matters for firewall allow-lists
• Simplicity: fewer features means fewer ways to misconfigure it

Where it struggles:

• It can't see the request: you can't route /api to one pool and /images to another, because the balancer never reads the path
• Long-lived connections can skew load: with HTTP/2 or gRPC, one connection can carry many requests, and a Layer 4 balancer pins that whole connection to a single backend, so traffic can end up uneven
• Shallow health checks: many Layer 4 checks only confirm a port accepts connections, which doesn't prove your application is healthy, though some products also support HTTP health checks
• Limited observability: you can count connections and bytes, but you can't see per-URL latency or status codes

Who this suits: non-HTTP services, extreme-throughput front tiers, anything needing static IPs or source IP preservation, and databases or brokers behind a stable address.

Layer 7 Load Balancing

What it actually is: a balancer that terminates the client's connection, parses the application protocol, and then forwards the request to a backend over its own connection. Because it understands HTTP, it can route by path, host, header, cookie, or method, and it can modify requests and responses on the way through.

A small example (NGINX routing by path):

upstream api_pool { server 10.0.2.11:8080; server 10.0.2.12:8080; }
upstream web_pool { server 10.0.3.11:3000; server 10.0.3.12:3000; }

server {
    listen 443 ssl;
    server_name example.com;
    # ssl_certificate and ssl_certificate_key lines omitted for brevity

    location /api/ { proxy_pass http://api_pool; }
    location /     { proxy_pass http://web_pool; }
}

Where it shines:

• Content-based routing: path, host, and header rules let one entry point serve many services, which is how teams split a monolith into services one route at a time
• TLS termination in one place: certificates live on the balancer instead of on every backend
• Per-request balancing: it can spread individual requests across backends even when clients reuse one long-lived connection, which fixes the HTTP/2 and gRPC skew Layer 4 can suffer from
• Resilience features: retries, timeouts, and circuit breaking
• Security and traffic control: web application firewalls, authentication, and rate limiting all live naturally here
• Rich visibility: per-route latency, status codes, and error rates, the kind of data our logs vs. metrics vs. traces guide shows how to use
• Safer releases: weighted routing makes canary and blue-green deployments straightforward

Where it struggles:

• Higher cost per request: parsing, inspecting, and rewriting traffic uses CPU and memory, and Cloudflare's 2019 outage shows what heavy inspection can do when a rule goes wrong
• More latency than Layer 4: an extra hop with extra work, usually small but real
• A bigger configuration surface: more rules means more ways to break production, and the Cloudflare rule was deployed to every location at once
• It holds your secrets: terminating TLS means the balancer sees plaintext traffic and holds your certificates, so it needs strong protection
• Protocol-focused: it's strongest for HTTP and similar protocols, and less useful for arbitrary TCP or UDP

Who this suits: web applications, APIs, and microservices that need routing, TLS termination, security controls, and visibility.

Using Both Together

Large systems rarely pick just one. A common shape puts a Layer 4 tier at the front for static IPs and huge connection volume, and a Layer 7 tier behind it for smart routing:

Clients → [L4: spread connections fast] → [L7: read requests, route by path/host] → services

AWS supports this directly, since a Network Load Balancer can forward traffic to an Application Load Balancer. In Kubernetes, the same split shows up as a Service of type LoadBalancer, which typically provisions a Layer 4 cloud balancer, and an Ingress or Gateway, which handles Layer 7 HTTP routing. Tools like HAProxy, NGINX, and Envoy can run in either mode, so the layer is often a configuration choice, not a product choice.

Side-by-Side Comparison

	Layer 4	Layer 7
Decides using	IP, port, protocol	Path, host, headers, cookies, and more
Reads the payload	No	Yes
Protocols	Any TCP or UDP	Mainly HTTP and similar
Speed per request	Fastest	Slower, more work per request
TLS	Passthrough, or terminated in some products	Usually terminated here
Routing flexibility	Low	High
Long-lived connections	Pinned to one backend	Can balance per request
Health checks	Often connection-level	Application-level (status codes, paths)
Visibility	Connections and bytes	Per-route latency and errors
Typical examples	AWS NLB, Maglev, HAProxy TCP mode	AWS ALB, NGINX, Envoy, HAProxy HTTP mode
Characteristic failure	Uneven load, shallow health checks	CPU exhaustion, bad rules breaking everything
Best for	Throughput, non-HTTP, static IPs	Web apps, APIs, microservices

How This Fits a Career Path

• Cloud and DevOps engineers: choosing and configuring load balancers is routine, and the L4-versus-L7 choice comes up in almost every architecture review
• Backend engineers: you'll configure timeouts, health endpoints, and forwarded headers so your service behaves behind a balancer
• SRE and platform roles: you'll operate these tiers, watch their metrics, and plan safe rollouts of balancer configuration
• Microservices work: path and host routing at Layer 7 is how traffic reaches each service; our monolith vs. microservices post covers the architecture behind it
• System design interviews: "where do the load balancers go, and which layer?" is a standard question, and a strong answer names the trade-off: speed and protocol freedom at Layer 4, routing and visibility at Layer 7

How to Choose Without Overthinking It

1. Start from the traffic. Is it HTTP(S) and do you need routing by path or host, TLS termination, or a web application firewall? That points to Layer 7.
2. Is it something else? Databases, message brokers, game servers, and custom protocols point to Layer 4. So do static IP requirements and source IP preservation.
3. Watch out for long-lived connections. If you run gRPC or HTTP/2 behind a Layer 4 balancer, check whether traffic is spreading evenly, or use a Layer 7 balancer that balances per request.
4. Combine them at scale. Layer 4 in front for volume, Layer 7 behind for logic, is a well-worn pattern.
5. Treat Layer 7 rules like code. Review them, test them against realistic traffic, set CPU limits where you can, and roll changes out gradually instead of everywhere at once.
6. Prefer managed services unless you need custom behavior. Cloud providers run both types, so compare their features, limits, and current pricing before building your own.

A note on honesty: the Maglev figures come from Google's 2016 paper and a benchmark on its own hardware, so they show what software Layer 4 balancing can do, not what you'll measure on a typical cloud instance. The Cloudflare incident was caused by a firewall rule inspecting HTTP traffic, so it illustrates the cost of Layer 7 processing in general, not a bug in load balancing as such. Modern Layer 7 proxies are heavily optimized, and "Layer 4 is faster" is a rule of thumb with exceptions. Feature descriptions of AWS services also change over time, so check the current documentation.

Common Mistakes When Learning Load Balancing

• Thinking a load balancer just does round robin. Algorithms, health checks, connection handling, and timeouts all matter more than the rotation order.
• Using Layer 7 for everything. It adds cost and complexity you don't need for plain TCP traffic or a simple front tier.
• Using Layer 4 for HTTP microservices that need path routing. You end up bolting routing logic into the services themselves.
• Port-only health checks. A server can accept connections while the application behind it is broken. Check a real health endpoint where you can.
• Ignoring long-lived connection skew. HTTP/2 and gRPC can leave one backend overloaded behind a Layer 4 balancer.
• Losing the client IP. Layer 7 balancers usually forward it in a header such as X-Forwarded-For, while Layer 4 balancers may preserve the source address, so make sure your application reads the right one.
• Mismatched timeouts. If the balancer's idle timeout is shorter or longer than your backend's, you'll see mysterious 502 and 504 errors.
• Pushing risky rule changes everywhere at once. Roll out Layer 7 rule changes gradually, with a fast way to roll back.
• Using sticky sessions to hide state problems. If your app only works when users stick to one server, fix the state handling.

Frequently Asked Questions

Which is faster, Layer 4 or Layer 7?

Layer 4 is faster per request because it does less work, and software balancers like Maglev show how far that can go. Whether the difference matters depends on your traffic, since many applications are limited by their backends long before the balancer.

Is AWS ALB Layer 7 and NLB Layer 4?

Yes. The Application Load Balancer works at Layer 7 for HTTP and HTTPS with path and host routing, and the Network Load Balancer works at Layer 4 for TCP, UDP, and TLS with static IPs. Check current AWS documentation for exact features and limits.

Can one tool do both layers?

Often, yes. HAProxy, NGINX, and Envoy can all balance at either layer, depending on how you configure them.

Does a Layer 7 load balancer terminate TLS?

Usually, which is how it can read the request. Some setups re-encrypt traffic to the backends, and some Layer 4 balancers pass TLS through untouched.

Do I need to memorize the OSI layers?

Not all seven. For load balancing, what matters is the difference between deciding from connection details (transport layer) and deciding from request contents (application layer).

How do interviewers evaluate this topic?

Usually by asking you to draw a system and place the load balancers. Naming the layer for each tier and justifying it, such as Layer 4 in front for volume and static IPs and Layer 7 behind for routing, signals real understanding.

Choose the Depth of Understanding on Purpose

Layer 4 trades understanding for speed and simplicity. Layer 7 trades speed for understanding and control. Neither is better in general, and plenty of production systems use both. Explore the Networking Basics and System Design courses to build the skills to place each one deliberately in your own architecture.