DEV Community

Cover image for Compute & Load Balancing
Rajesh
Rajesh

Posted on

Compute & Load Balancing

#aws #devops #cloud #beginners

Why do we need Load Balancing & Managed Compute?

A single server has hard limits: when CPU or memory runs out, your website crashes. If that single server reboots or fails, your entire business goes offline.

Elastic Load Balancing (ELB) and scalable EC2 instances solve this by distributing user traffic across multiple healthy servers in different Availability Zones, so if one server dies, your users never even notice.

1. Scaling Up vs. Scaling Out

Think of running a busy kitchen:

  • Scale Up (Vertical): Buying a bigger stove. You replace your small server with a massive 64-CPU machine. It's easy, but it has a physical ceiling and still represents a single point of failure.
  • Scale Out (Horizontal): Hiring more cooks with more stoves. You run five 2-CPU servers behind a load balancer. When one breaks, the other four keep cooking. You can add more anytime without restarting. 
Vertical Scaling (Scale Up)       Horizontal Scaling (Scale Out)
     [ Bigger Server ]                   [ Load Balancer ]
                                          /      |      \
                                     [App 1]  [App 2]  [App 3]
Enter fullscreen mode Exit fullscreen mode

2. Load Balancing Algorithms

How does a load balancer decide which backend server gets the next request?

  • Round Robin: Takes turns in order (1, 2, 3, 1, 2, 3...). Perfect when requests are short and servers are identical.
  • Weighted Round Robin: Sends more requests to beefier servers (e.g., 50% to App 1, 30% to App 2, 20% to App 3). Great for canary deployments.
  • Least Connections: Forwards the next request to whichever server currently has the fewest active connections. Best for long-running jobs (like video processing or WebSockets).
  • Source / IP Hash: Hashes the client's IP address to map them consistently to the same server without needing cookies.

3. Health Checks: Catching Dead Servers

A load balancer must never send traffic to a dead server.

  • How it works: The load balancer sends a background check (e.g., GET /health) every few seconds.
  • Thresholds: If a server fails 3 times in a row (unhealthy threshold), it is pulled out of rotation. Once it passes 2 checks in a row (healthy threshold), it gets traffic again.
  • Failsafe: If all servers are marked unhealthy, the load balancer "fails open" and routes traffic to all of them as a last resort instead of dropping 100% of requests.

4. The 3 AWS Elastic Load Balancers (ALB, NLB, GWLB)

Application Load Balancer Component Architecture
(Reference: Official AWS Application Load Balancer Architecture)

AWS provides three modern managed load balancers:

  1. Application Load Balancer (ALB) — Layer 7:
    • Reads HTTP and HTTPS paths, headers, hostnames, and cookies.
    • Can route /api/* to your API servers and /images/* to an S3/storage target.
    • Supports TLS termination, redirects (HTTP to HTTPS), and fixed 403/404 responses.
  2. Network Load Balancer (NLB) — Layer 4:
    • Ultra-fast, handles millions of requests per second at ultra-low latency.
    • Routes raw TCP and UDP traffic (gaming, IoT feeds, databases).
    • Provides a static IP address or Elastic IP per Availability Zone.
  3. Gateway Load Balancer (GWLB) — Layer 3:
    • Routes raw IP packets to a fleet of third-party security firewalls or inspection appliances using the GENEVE protocol.

5. Listeners, Rules & Target Groups

Inside an Application Load Balancer:

  • Listener: Listens on a specific port and protocol (e.g., HTTP:80 or HTTPS:443).
  • Rules: If-then conditions evaluated in priority order. For example: IF path is /api/* THEN forward to tg-api.
  • Target Group: A logical pool of backend servers with their own health check rules. Target types can be EC2 Instance IDs, IP addresses, or AWS Lambda functions. 
Client Request
      │
      ▼
 [ ALB Listener: 443 ]
      ├── Rule 1: path /api/*  ──► [ Target Group: API ]  ──► (EC2 in AZ-a, AZ-b)
      └── Default Rule         ──► [ Target Group: Web ]  ──► (EC2 in AZ-a, AZ-b)
Enter fullscreen mode Exit fullscreen mode

6. Cross-Zone Load Balancing & Connection Draining

Cross-Zone Load Balancing Enabled
(Reference: Official AWS Cross-Zone Load Balancing Guide)

  • Cross-Zone Balancing: If AZ-a has 2 servers and AZ-b has 8 servers, cross-zone balancing ensures every server across both AZs receives an equal 10% of the traffic. On ALB, it is enabled by default.
  • Connection Draining (Deregistration Delay): When removing an instance for updates or deployment, AWS stops sending new requests but allows in-flight requests to finish gracefully (default 300 seconds, customizable down to 30s).
  • Sticky Sessions: Uses an AWSALB cookie to tie a visitor to one specific server. (Best practice: keep apps stateless using Redis so you don't need sticky sessions).

7. EC2 Networking: Public vs. Private IPs & Elastic IPs

Every EC2 instance in a VPC gets assigned IP addresses:

  • Private IP: Permanent internal IP (e.g., 10.0.1.25) that stays with the instance for its entire life.
  • Public IP (Auto-assigned): Random public IP from Amazon's pool. Warning: If you stop and restart the instance, this IP changes!
  • Elastic IP (EIP): A fixed, permanent public IPv4 address that you own until you release it. You can re-associate it between standby servers in seconds.

8. Connecting to EC2 Without Open SSH Keys

Leaving port 22 open to 0.0.0.0/0 invites continuous bot attacks. AWS gives us three ways to connect:

  1. Traditional SSH Key Pairs: Connect using ssh -i key.pem ec2-user@<ip>. Requires port 22 open.
  2. EC2 Instance Connect: Pushes a temporary, one-time public key valid for only 60 seconds. No private key files to manage.
  3. AWS Systems Manager (SSM) Session Manager (Recommended): Opens an interactive browser terminal using the outbound SSM agent. Requires ZERO open inbound ports in your Security Group!

9. Amazon EBS (Elastic Block Store) & Instance Store

Where does EC2 store its files?

  • Amazon EBS: Detachable virtual disks (like a USB drive for cloud).
    • Lives in one specific Availability Zone.
    • Survives instance stops and reboots.
    • Types: gp3 (General Purpose SSD - default), io2 (Provisioned IOPS for high-speed databases), and st1/sc1 (cold HDDs).
  • EBS Snapshots: Point-in-time, incremental backups stored securely in S3. You can restore a snapshot into a new EBS volume in a different AZ or copy it to another AWS Region.
  • Instance Store (Ephemeral): Physical NVMe disks built directly into the host server. Blazing fast IOPS, but data is permanently wiped if the instance stops or terminates. Perfect for caches and scratch data.

Hands-On Practice & Console Walkthrough

Here is the step-by-step lab to launch two EC2 servers in different AZs and put an Application Load Balancer in front.

Step 1: Launch EC2 Web Instance

I launched an EC2 instance (t3.micro in eu-north-1b) and pasted this user-data script so Nginx installs and starts automatically on boot:

#!/bin/bash
dnf install -y nginx
echo "<h1>Hello from $(hostname -f)</h1>" > /usr/share/nginx/html/index.html
systemctl enable --now nginx
Enter fullscreen mode Exit fullscreen mode

EC2 Instance Running in AWS
Fig 1: EC2 instance running in my AWS account.

Step 2: Configure Security Groups (ec2-web-sg)

I opened port 80 (HTTP) so the Load Balancer can talk to Nginx, and left outbound open for updates.

Security Group Configuration
Fig 2: Security group configuration (ec2-web-sg).

Step 3: Create Target Group (ec2-users)

I created a target group pointing to port 80 with / as the health check path and registered the instance.

Target Group Configuration
Fig 3: Target group ec2-users created for routing traffic.

Step 4: Create the Application Load Balancer (alb)

I created an internet-facing ALB named alb, enabled both eu-north-1a and eu-north-1b subnets, and pointed the port 80 listener to ec2-users.

Application Load Balancer Details
Fig 4: Application Load Balancer active with its public DNS address.

Step 5: Local HAProxy & Failover Test

Before testing with multiple cloud instances, I ran a basic local test with 3 apps (app1, app2, app3) to see how round-robin and health checks work in real-time when a server crashes:

python lb_demo.py
Enter fullscreen mode Exit fullscreen mode

Here is my actual terminal output:

![HAProxy / Python Load Balancer Terminal Run](

Fig 5: My terminal showing round-robin in Demo 1, and traffic instantly dropping app3 in Demo 2 when it failed.

Step 6: Clean Up

Don't forget to delete resources when finished so you don't get billed:

  • Deleted the ALB (alb)
  • Deleted the target group (ec2-users)
  • Terminated the EC2 instance

Top comments (0)