AWS & SRE Field Manual (Part 2): Application (ALB) vs. Network (NLB) Load Balancing Architecture

1. TL;DR & Problem Statement

• Definition: A fully managed, highly available traffic distribution service that automatically routes incoming application and network traffic across multiple targets (Amazon EC2 instances, EKS Pods, AWS Lambda functions, and IP addresses) across multiple Availability Zones.
• Problem Solved: Eliminates single-point-of-failure bottlenecks, handles automated health checks, offloads cryptographic TLS termination, and provides scalable ingress routing mechanisms for cloud-native microservices and high-throughput TCP/UDP streams.
• Category: Networking & Content Delivery / Ingress Architecture

---

2. Core Architecture & Key Components

                                Incoming Client Traffic
                                           │
           ┌───────────────────────────────┴───────────────────────────────┐
           ▼                                                               ▼
┌─────────────────────────────────────────┐             ┌─────────────────────────────────────────┐
│ Layer 7: Application Load Balancer (ALB)│             │  Layer 4: Network Load Balancer (NLB)   │
├─────────────────────────────────────────┤             ├─────────────────────────────────────────┤
│ • Inspects HTTP/HTTPS Headers & Paths   │             │ • Operates strictly at TCP/UDP/TLS      │
│ • Path-Based & Host-Based Routing       │             │ • Non-Inspecting Packet Forwarding      │
│ • SSL Termination & Sticky Sessions     │             │ • Ultra-Low Latency (Sub-millisecond)   │
└────────────────────┬────────────────────┘             └────────────────────┬────────────────────┘
                     │                                                       │
        HTTP / HTTPS Ingress                                    Raw TCP/UDP Stream
                     │                                                       │
        ┌────────────┴────────────┐                                          │
        ▼                         ▼                                          ▼
┌──────────────┐          ┌──────────────┐                        ┌────────────────────┐
│ API Pods     │          │ Frontend Pods│                        │ Ingress Controller │
│ (/api/*)     │          │ (/static/*)  │                        │ / Kafka / Games    │
└──────────────┘          └──────────────┘                        └────────────────────┘

2.1. Application Load Balancer (ALB — OSI Layer 7)

• Deep Packet Inspection: Evaluates application-layer headers, hostnames, HTTP methods, query parameters, and URL paths.
• Content-Based Routing: Routes traffic intelligently across multiple Target Groups (e.g., /api/* to backend microservices, /static/* to object caches or static web pods).
• Modern Protocol Support: Natively supports HTTP/2, gRPC, and WebSockets alongside automated HTTP-to-HTTPS redirect rules.
• Sticky Sessions (Cookie Affinity): Binds subsequent requests from a specific client to the same backend target instance/pod using encrypted cookies when state is not externalized.

2.2. Network Load Balancer (NLB — OSI Layer 4)

• Raw Transport Passthrough: Operates at the transport layer, routing raw TCP, UDP, and TLS connections based exclusively on IP addresses and ports without payload inspection.
• Ultra-Low Latency: Delivers sub-millisecond connection handling directly in the data path.
• Instantaneous Burst Capacity: Capable of handling millions of requests per second (RPS) and absorbing sudden, massive traffic spikes without requiring manual pre-warming tickets.
• Static & Elastic IPs: Provides one static public IP per Availability Zone, making it ideal for client firewalls requiring strict IP whitelisting.

---

3. Deep Dive Engineering & Architectural Comparison

Feature / Metric	Application Load Balancer (ALB)	Network Load Balancer (NLB)
OSI Layer	Layer 7 (Application)	Layer 4 (Transport)
Protocols	HTTP, HTTPS, HTTP/2, gRPC, WebSockets	TCP, UDP, TLS
Routing Decisions	URL path, Host header, Query params, HTTP method	Source/Destination IP and Port
Latency Profile	Milliseconds (2-10 ms)	Sub-millisecond (< 1 ms)
Traffic Spikes	Scales elastically via DNS over minutes	Instantaneous line-rate scaling
Static IP Support	Dynamic DNS resolution (CNAME required)	Static Elastic IP per Availability Zone
Client IP Identification	Injects X-Forwarded-For HTTP header	Client IP Preservation / Proxy Protocol v2
Primary Use Cases	Microservices routing, REST/gRPC APIs, Web apps	High-throughput streaming, Kafka, Game servers, K8s Ingress Entry

---

4. Advanced Integrations & Ingress Mechanics

AWS Load Balancer Controller (Instance Mode vs. IP Mode)

• Instance Mode (target-type: instance): Routes traffic to EC2 host NodePorts. Requires an internal network hop through kube-proxy and iptables/IPVS, adding latency and SNAT overhead.
• IP Mode (target-type: ip): Leverages the AWS VPC CNI to route ingress traffic directly from the ALB/NLB to the individual Kubernetes Pod IP. Bypasses kube-proxy entirely, reducing latency and packet manipulation.

Preserving Real Client IP

• ALB: Appends the originating client IP to the X-Forwarded-For and X-Forwarded-Proto request headers.
• NLB: Because Layer 4 cannot modify HTTP headers, real client IPs are preserved via native Client IP Preservation or by enabling Proxy Protocol v2 (prepends a binary connection header).

Deregistration Delay (Connection Draining)

• Prevents inflight HTTP requests from dropping (502 Bad Gateway) when a backend instance or Kubernetes pod is marked for termination.
• The load balancer stops sending new connections to the deregistering target and waits for active transactions to complete within deregistration_delay.timeout_seconds (default: 300s, typically tuned to 15–30s in Kubernetes).

Cross-Zone Load Balancing

• Enabled by default on ALB; optional on NLB.
• Evenly distributes traffic across all registered targets in all enabled Availability Zones regardless of which AZ received the initial network packet, eliminating uneven target group saturation.

---

5. Practical Notes & Configuration Snippets

Kubernetes Ingress Manifest (ALB in IP Mode with SSL Redirect)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  namespace: production
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789012:certificate/abc-123
    alb.ingress.kubernetes.io/target-group-attributes: deregistration_delay.timeout_seconds=20
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /v1
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 8080

Terraform: Network Load Balancer with Cross-Zone Load Balancing

resource "aws_lb" "network_lb" {
  name                             = "prod-streaming-nlb"
  internal                         = false
  load_balancer_type               = "network"
  subnets                          = var.public_subnet_ids
  enable_cross_zone_load_balancing = true
  enable_deletion_protection       = true
}

resource "aws_lb_target_group" "nlb_tg" {
  name        = "tcp-stream-tg"
  port        = 9092
  protocol    = "TCP"
  vpc_id      = var.vpc_id
  target_type = "ip"

  proxy_protocol_v2 = true # Enables Proxy Protocol v2 for client IP pass-through

  health_check {
    protocol            = "TCP"
    port                = "9092"
    interval            = 10
    healthy_threshold   = 3
    unhealthy_threshold = 3
  }
}

---

6. Gotchas & Common Pitfalls

• Subnet Capacity & IP Exhaustion: Each ALB node placed in a public subnet dynamically scales and consumes multiple private IP addresses within that subnet. If your public subnet CIDR is too narrow (e.g., /28), ALB scale-out events will fail under heavy traffic.
• Misconfigured Security Group on NLB: By default, NLBs historically did not have associated Security Groups; firewalling occurred strictly at the backend instance level. While security groups on NLBs are now supported, failing to allow client ingress on the backend node security group when using client IP preservation will cause silent packet drops.
• Target Group Health Check Latency: If health check intervals and thresholds are set too conservatively (e.g., 30s interval, 5 unhealthy thresholds), dead or hung pods will receive live user traffic for over 2 minutes before the load balancer drops them from the target rotation.

---

7. Production Best Practices

• AWS WAF Integration on ALB: Always attach an AWS WAF (Web Application Firewall) WebACL to public-facing ALBs to block common OWASP Top 10 vulnerabilities, SQL injection, cross-site scripting (XSS), and rate-limit abusive IP addresses at the cloud edge.
• Access Logs to Amazon S3: Enable access logging on both ALB and NLB. Load balancers stream structured access logs (client IP, request processing latency, TLS ciphers, backend response codes) directly to an S3 bucket with lifecycle rules for compliance and security forensics.
• Match Ingress Controller Pod Disruption Budgets (PDB): Ensure the deregistration_delay on the AWS Load Balancer matches the terminationGracePeriodSeconds and preStop sleep hook of your Kubernetes pods to guarantee zero-downtime rolling deployments.