DEV Community

Olusanya Olajide Emmanuel
Olusanya Olajide Emmanuel

Posted on

Building a Virtual Private Cloud (VPC) from Scratch on Linux

#networking #linux #tutorial #architecture

Introduction

Ever wondered how cloud providers like AWS, GCP, or Azure implement Virtual Private Clouds (VPCs) under the hood? In this comprehensive guide, we'll recreate VPC fundamentals entirely on Linux using native networking primitives like network namespaces, veth pairs, bridges, and iptables.

By the end of this tutorial, you'll have built a fully functional mini-VPC environment supporting:

  • Multiple isolated subnets
  • Inter-subnet routing
  • NAT gateway for internet access
  • VPC peering
  • Firewall rules (Security Groups)

Prerequisites

  • Linux machine (Ubuntu 20.04+ or similar)
  • Root/sudo access
  • Basic understanding of networking concepts (IP addresses, routing, NAT)
  • Python 3.6+ or Bash

Architecture Overview 

┌─────────────────────────────────────────────────────────┐
│                         Host OS                         │
│                                                         │
│  ┌──────────────────────────────────────────────────┐  │
│  │                    VPC 1                         │  │
│  │                                                  │  │
│  │  ┌──────────────┐         ┌──────────────┐     │  │
│  │  │   Public     │         │   Private    │     │  │
│  │  │   Subnet     │◄───────►│   Subnet     │     │  │
│  │  │ (Namespace)  │         │ (Namespace)  │     │  │
│  │  │ 10.0.1.0/24  │         │ 10.0.2.0/24  │     │  │
│  │  └──────┬───────┘         └──────┬───────┘     │  │
│  │         │                        │             │  │
│  │         └────────┬───────────────┘             │  │
│  │                  │                             │  │
│  │           ┌──────▼──────┐                      │  │
│  │           │   Bridge    │                      │  │
│  │           │ (VPC Router)│                      │  │
│  │           └──────┬──────┘                      │  │
│  └──────────────────┼──────────────────────────────┘  │
│                     │                                 │
│                     │ NAT                             │
│                     ▼                                 │
│              ┌─────────────┐                          │
│              │ eth0 (WAN)  │                          │
│              └─────────────┘                          │
└─────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Key Components

  1. Linux Bridge: Acts as a virtual switch/router for the VPC
  2. Network Namespaces: Isolated network environments representing subnets
  3. veth Pairs: Virtual ethernet cables connecting namespaces to the bridge
  4. iptables: Implements NAT and firewall rules
  5. Routing Tables: Controls packet flow between subnets

Implementation

Step 1: Set Up the Project

Create a project directory and the main CLI tool:

mkdir vpc-project
cd vpc-project
Enter fullscreen mode Exit fullscreen mode

Create vpcctl.py with the implementation (see the complete code in the GitHub repository).

Make it executable:

chmod +x vpcctl.py
Enter fullscreen mode Exit fullscreen mode

Step 2: Understanding the Core Commands

Our CLI tool provides these commands:

# Create a VPC
sudo ./vpcctl.py create-vpc <name> <cidr> [--interface eth0]

# Add subnet to VPC
sudo ./vpcctl.py add-subnet <vpc-name> <subnet-name> <cidr> [--type public|private]

# Peer two VPCs
sudo ./vpcctl.py peer <vpc1> <vpc2>

# Apply firewall rules
sudo ./vpcctl.py apply-firewall <vpc> <subnet> <policy.json>

# List all VPCs
sudo ./vpcctl.py list

# Delete VPC
sudo ./vpcctl.py delete-vpc <name>
Enter fullscreen mode Exit fullscreen mode

Step 3: Create Your First VPC

Let's create a VPC with CIDR 10.0.0.0/16:

sudo ./vpcctl.py create-vpc vpc1 10.0.0.0/16 --interface eth0
Enter fullscreen mode Exit fullscreen mode

What happens behind the scenes:

  1. Creates a Linux bridge named br-vpc1
  2. Assigns the first IP from the CIDR (10.0.0.1) to the bridge
  3. Enables IP forwarding on the system
  4. Saves VPC state to ~/.vpcctl/vpcs.json

Verify the bridge was created:

ip link show br-vpc1
Enter fullscreen mode Exit fullscreen mode

Step 4: Add Subnets

Add a public subnet (with internet access):

sudo ./vpcctl.py add-subnet vpc1 public 10.0.1.0/24 --type public
Enter fullscreen mode Exit fullscreen mode

Add a private subnet (isolated from internet):

sudo ./vpcctl.py add-subnet vpc1 private 10.0.2.0/24 --type private
Enter fullscreen mode Exit fullscreen mode

What happens:

  1. Creates network namespace (e.g., vpc1-public)
  2. Creates veth pair connecting namespace to bridge
  3. Assigns IP address to namespace interface
  4. Configures default route through bridge
  5. For public subnets: sets up NAT rules using iptables

List the created namespaces:

ip netns list
Enter fullscreen mode Exit fullscreen mode

Step 5: Deploy Test Applications

Let's deploy simple HTTP servers to test connectivity:

In public subnet:

# Start HTTP server in public subnet namespace
sudo ip netns exec vpc1-public python3 -m http.server 8080 &
Enter fullscreen mode Exit fullscreen mode

In private subnet:

# Start HTTP server in private subnet namespace
sudo ip netns exec vpc1-private python3 -m http.server 8081 &
Enter fullscreen mode Exit fullscreen mode

Step 6: Test Connectivity

Test 1: Inter-subnet communication within VPC

# From private subnet, ping public subnet
sudo ip netns exec vpc1-private ping -c 3 10.0.1.1
Enter fullscreen mode Exit fullscreen mode

✅ Expected: Success - subnets in same VPC can communicate

Test 2: Internet access from public subnet

# Public subnet should reach internet via NAT
sudo ip netns exec vpc1-public ping -c 3 8.8.8.8
Enter fullscreen mode Exit fullscreen mode

✅ Expected: Success - NAT gateway allows outbound traffic

Test 3: Internet access from private subnet

# Private subnet should NOT have internet access
sudo ip netns exec vpc1-private ping -c 3 8.8.8.8
Enter fullscreen mode Exit fullscreen mode

❌ Expected: Failure or timeout - private subnet is isolated

Test 4: HTTP access

# Access HTTP server in public subnet
sudo ip netns exec vpc1-private curl http://10.0.1.1:8080
Enter fullscreen mode Exit fullscreen mode

✅ Expected: Success - receives HTTP response

Step 7: Implement VPC Isolation

Create a second VPC:

sudo ./vpcctl.py create-vpc vpc2 172.16.0.0/16
sudo ./vpcctl.py add-subnet vpc2 web 172.16.1.0/24 --type public
Enter fullscreen mode Exit fullscreen mode

Test isolation:

# Try to ping vpc2 from vpc1
sudo ip netns exec vpc1-public ping -c 3 172.16.1.1
Enter fullscreen mode Exit fullscreen mode

❌ Expected: Failure - VPCs are isolated by default

Step 8: Enable VPC Peering

Allow controlled communication between VPCs:

sudo ./vpcctl.py peer vpc1 vpc2
Enter fullscreen mode Exit fullscreen mode

What happens:

  1. Creates veth pair between the two bridges
  2. Adds static routes for cross-VPC traffic

Test cross-VPC communication:

# Now vpc1 can reach vpc2
sudo ip netns exec vpc1-public ping -c 3 172.16.1.1
Enter fullscreen mode Exit fullscreen mode

✅ Expected: Success - peering enables cross-VPC routing

Step 9: Apply Firewall Rules

Create a security policy file firewall-policy.json:

{
  "subnet": "10.0.1.0/24",
  "ingress": [
    {"port": 8080, "protocol": "tcp", "action": "allow"},
    {"port": 22, "protocol": "tcp", "action": "deny"},
    {"port": 443, "protocol": "tcp", "action": "allow"}
  ]
}
Enter fullscreen mode Exit fullscreen mode

Apply the policy:

sudo ./vpcctl.py apply-firewall vpc1 public firewall-policy.json
Enter fullscreen mode Exit fullscreen mode

Test the rules:

# HTTP on port 8080 should work
sudo ip netns exec vpc1-private curl http://10.0.1.1:8080

# SSH on port 22 should be blocked
sudo ip netns exec vpc1-private nc -zv 10.0.1.1 22
Enter fullscreen mode Exit fullscreen mode

Step 10: Monitoring and Debugging

View routing table in namespace:

sudo ip netns exec vpc1-public ip route
Enter fullscreen mode Exit fullscreen mode

View iptables rules:

sudo iptables -t nat -L -n -v
sudo ip netns exec vpc1-public iptables -L -n -v
Enter fullscreen mode Exit fullscreen mode

Check bridge connections:

bridge link show br-vpc1
Enter fullscreen mode Exit fullscreen mode

Monitor network traffic:

# Capture traffic on bridge
sudo tcpdump -i br-vpc1 -n

# Capture traffic in namespace
sudo ip netns exec vpc1-public tcpdump -i veth-ns-public -n
Enter fullscreen mode Exit fullscreen mode

Step 11: Cleanup

Remove all resources:

# Delete individual VPC
sudo ./vpcctl.py delete-vpc vpc1

# Or delete all VPCs
sudo ./vpcctl.py delete-vpc vpc1
sudo ./vpcctl.py delete-vpc vpc2
Enter fullscreen mode Exit fullscreen mode

The delete operation automatically removes:

  • All network namespaces
  • veth pairs
  • Bridge interfaces
  • iptables NAT rules
  • Routing table entries

Troubleshooting

Issue: Cannot ping between subnets

Solution: Check that IP forwarding is enabled:

sysctl net.ipv4.ip_forward
# Should return: net.ipv4.ip_forward = 1
Enter fullscreen mode Exit fullscreen mode

Issue: No internet access from public subnet

Solution:

  1. Verify the interface name: ip link show
  2. Update --interface parameter to match your actual internet interface
  3. Check NAT rules: sudo iptables -t nat -L -n -v

Issue: "Operation not permitted" errors

Solution: All commands must run with sudo or as root user

Issue: Namespace already exists

Solution: Clean up existing namespaces:

sudo ip netns del vpc1-public
# Or delete the entire VPC
sudo ./vpcctl.py delete-vpc vpc1
Enter fullscreen mode Exit fullscreen mode

Advanced Topics

Custom NAT Rules

For more complex NAT scenarios:

# Port forwarding from host to namespace
sudo iptables -t nat -A PREROUTING -p tcp --dport 8080 \
  -j DNAT --to-destination 10.0.1.1:80
Enter fullscreen mode Exit fullscreen mode

VPC Traffic Shaping

Limit bandwidth on veth interface:

sudo tc qdisc add dev veth-vpc1-public root tbf \
  rate 1mbit burst 32kbit latency 400ms
Enter fullscreen mode Exit fullscreen mode

Multi-tenancy

Create isolated VPCs for different users/projects using unique naming prefixes.

Real-World Use Cases

  1. Development Environment: Create isolated networks for testing microservices
  2. CI/CD Pipelines: Spin up temporary network environments for integration tests
  3. Learning Platform: Understand cloud networking without cloud costs
  4. Network Security Testing: Practice firewall configurations and network segmentation

Key Takeaways

  • VPCs are built on standard Linux networking primitives
  • Network namespaces provide complete network isolation
  • Bridges act as virtual switches/routers
  • iptables implements NAT and security policies
  • VPC peering requires explicit routing configuration

Conclusion

You've successfully built a fully functional VPC implementation on Linux! This project demonstrates the core concepts behind cloud networking and gives you hands-on experience with Linux networking tools.

The same principles apply to production cloud environments, just at a much larger scale with additional features like:

  • High availability and redundancy
  • Load balancing
  • VPN connectivity
  • Flow logs and monitoring
  • Service mesh integration

Resources

GitHub Repository

Full code available at: [Your GitHub Repo URL]

Have questions or suggestions? Feel free to open an issue on GitHub!

Top comments (0)