DEV Community

Cover image for Building Your Own Virtual Private Cloud (VPC) on Linux: A Complete Beginner's Guide
Mart Young
Mart Young

Posted on

Building Your Own Virtual Private Cloud (VPC) on Linux: A Complete Beginner's Guide

#linux #beginners #networking #tutorial

Introduction

Have you ever wondered how cloud providers like AWS, Google Cloud, or Azure create isolated virtual networks? How do they ensure that your resources are completely isolated from others while still allowing controlled communication?

In this comprehensive guide, we'll build our own Virtual Private Cloud (VPC) from scratch on Linux using nothing but native Linux networking tools. By the end of this journey, you'll understand the fundamental building blocks of network virtualization and have a working VPC implementation that you can use for learning, testing, or even production workloads.

What You'll Learn

  • How to create isolated virtual networks using Linux network namespaces
  • How to connect networks using Linux bridges and virtual Ethernet pairs
  • How to implement routing between subnets
  • How to enable internet access using NAT (Network Address Translation)
  • How to enforce network isolation between different VPCs
  • How to connect VPCs using peering
  • How to implement firewall rules (Security Groups)
  • How to automate VPC management using vpcctl CLI tool

Our Approach: Learn by Doing, Then Automate

This guide follows a two-part approach:

  1. Manual Implementation (Parts 1-10): We'll build VPCs manually using Linux commands. This helps you understand exactly what's happening under the hood.

  2. Automation with vpcctl (Part 11): Once you understand the fundamentals, we'll introduce vpcctl, a CLI tool that automates all the manual steps. This is what you'll use in practice!

What is vpcctl?

vpcctl is a command-line tool written in Python that automates VPC creation and management. Instead of running dozens of commands manually, you can create a VPC with a single command:

# Manual way (what we'll learn first)
sudo ip netns add my-ns
sudo ip link add veth0 type veth peer name veth1
# ... many more commands ...

# Automated way (using vpcctl)
sudo ./vpcctl create --name myvpc --cidr 10.0.0.0/16
sudo ./vpcctl add-subnet --vpc myvpc --name public --cidr 10.0.1.0/24 --type public
Enter fullscreen mode Exit fullscreen mode

Don't worry - we'll get to vpcctl after you understand the fundamentals!

Prerequisites

Before we begin, you'll need:

  • A Linux system (Ubuntu 20.04+, Debian, or Pop!_OS recommended)
  • Root or sudo access
  • Basic familiarity with the command line
  • Git (to clone the repository)
  • Python 3.6 or higher (for vpcctl)
  • Curiosity and patience! 🚀

Tools We'll Use

All the tools we'll use are built into Linux:

  • ip - Modern Linux networking tool
  • brctl - Bridge control utility
  • iptables - Firewall and NAT tool
  • ping - Network connectivity testing
  • curl - HTTP client for testing web servers
  • vpcctl - Our automation tool (we'll get this from the repository)

Let's get started!

Part 1: Understanding the Building Blocks

What is a VPC?

A Virtual Private Cloud (VPC) is essentially a virtual network that you can use to isolate your resources. Think of it like creating a private room in a shared building - you have your own space with controlled access points.

In our implementation:

  • VPC = A virtual network with its own IP address range
  • Subnet = A smaller network within the VPC (like a room within the private space)
  • Bridge = A virtual router that connects subnets
  • Namespace = An isolated network environment (like a separate room)

The Linux Primitives

Before we dive in, let's understand the Linux networking primitives we'll be using:

  1. Network Namespaces: Isolated network environments - think of them as separate computers on the same physical machine
  2. veth Pairs: Virtual Ethernet cables - they come in pairs, like a cable with two ends
  3. Linux Bridges: Virtual switches that connect multiple networks
  4. Routing Tables: Maps that tell packets where to go
  5. iptables: Firewall and NAT rules

Don't worry if this sounds complex - we'll learn by doing!

Part 2: Setting Up Your Environment

Step 1: Clone the Repository

First, let's get the project code which includes the vpcctl tool and example scripts:

# Clone the repository
git clone https://github.com/Donkross360/vpc-project.git
cd vpc-project

# Alternatively, if you prefer to download:
# wget https://github.com/Donkross360/vpc-project/archive/main.zip
# unzip main.zip
# cd vpc-project-main
Enter fullscreen mode Exit fullscreen mode

The repository contains:

  • vpcctl - The CLI tool for automating VPC management (Python script)
  • examples/ - Example scripts and a demo web application
  • README.md - Complete documentation
  • Makefile - Automation for common tasks
  • cleanup.sh - Script to clean up all VPC resources
  • policies/ - Example firewall policy files

Step 2: Install Required Tools

Now, let's make sure we have all the necessary tools installed:

# Update package list
sudo apt-get update

# Install required tools
sudo apt-get install -y \
    iproute2 \
    bridge-utils \
    iptables \
    python3 \
    curl \
    git

# Make vpcctl executable
chmod +x vpcctl
Enter fullscreen mode Exit fullscreen mode

Step 3: Quick Start with vpcctl (Optional)

Want to see vpcctl in action right away? You can skip ahead to Part 11, or follow along manually to understand how it works. For now, here's a quick preview:

# Create a VPC with one command
sudo ./vpcctl create --name demo-vpc --cidr 10.0.0.0/16

# Add a public subnet
sudo ./vpcctl add-subnet --vpc demo-vpc --name public --cidr 10.0.1.0/24 --type public

# List all VPCs
sudo ./vpcctl list

# View VPC details
sudo ./vpcctl show demo-vpc
Enter fullscreen mode Exit fullscreen mode

But wait! Before we automate, let's understand what's happening under the hood by building manually first.

Step 4: Enable IP Forwarding

IP forwarding allows Linux to act as a router, forwarding packets between networks. This is essential for our VPC to work:

# Enable IP forwarding temporarily
sudo sysctl -w net.ipv4.ip_forward=1

# Make it permanent (so it survives reboots)
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
Enter fullscreen mode Exit fullscreen mode

What this does: This tells the Linux kernel to forward IP packets between network interfaces, which is necessary for routing traffic between our virtual networks.

Step 5: Identify Your Internet Interface

We need to know which network interface connects your machine to the internet. This will be used for NAT (Network Address Translation) later:

# Find your default internet interface
DEFAULT_INTERFACE=$(ip route show default | awk '/default/ {print $5}' | head -1)
echo "Internet interface: $DEFAULT_INTERFACE"

# Verify the interface exists and is UP
ip link show $DEFAULT_INTERFACE
Enter fullscreen mode Exit fullscreen mode

Expected output: You should see something like enp0s25 or eth0 or wlan0. This is your connection to the outside world.

Note: Modern Linux systems use "predictable naming" like enp0s25 instead of the traditional eth0. This is normal and expected!

Part 3: Creating Your First Network Namespace

Understanding Namespaces

A network namespace is like a separate computer with its own network stack. It has its own:

  • Network interfaces
  • IP addresses
  • Routing tables
  • Firewall rules

Let's create our first namespace to see how it works:

# Create a network namespace
sudo ip netns add test-ns-1

# List all namespaces
sudo ip netns list

# Verify the namespace was created
sudo ip netns exec test-ns-1 ip link show
Enter fullscreen mode Exit fullscreen mode

What you'll see: The namespace only has a lo (loopback) interface, which is like a network interface that points back to itself. It's completely isolated from your host system!

Bringing Up the Loopback Interface

Even though the namespace has a loopback interface, it's not active by default. Let's bring it up:

# Bring up the loopback interface in the namespace
sudo ip netns exec test-ns-1 ip link set lo up

# Verify it's up
sudo ip netns exec test-ns-1 ip link show lo
Enter fullscreen mode Exit fullscreen mode

Success indicator: You should see state UNKNOWN or state UP instead of state DOWN.

Part 4: Creating Virtual Ethernet Pairs

What are veth Pairs?

A veth (virtual Ethernet) pair is like a virtual network cable with two ends. Whatever you send into one end comes out the other end. We use these to connect namespaces to the host or to bridges.

Let's create our first veth pair:

# Create a veth pair
sudo ip link add veth-host type veth peer name veth-ns

# Verify both ends were created
ip link show type veth
Enter fullscreen mode Exit fullscreen mode

What you'll see: Two interfaces - veth-host and veth-ns. They're connected like two ends of a cable.

Moving One End to a Namespace

Now let's move one end into our namespace. This creates a connection between the host and the namespace:

# Create a namespace for testing
sudo ip netns add test-ns-2

# Move veth-ns to the namespace
sudo ip link set veth-ns netns test-ns-2

# Verify veth-ns is now in the namespace
sudo ip netns exec test-ns-2 ip link show

# Verify veth-host is still on the host
ip link show veth-host
Enter fullscreen mode Exit fullscreen mode

What happened:

  • veth-ns is now inside test-ns-2 and no longer visible on the host
  • veth-host is still on the host
  • They're still connected - data sent through one end will come out the other!

Assigning IP Addresses

Now let's give both ends IP addresses so they can communicate:

# Assign IP to host side
sudo ip addr add 10.0.1.1/24 dev veth-host
sudo ip link set veth-host up

# Assign IP to namespace side
sudo ip netns exec test-ns-2 ip addr add 10.0.1.10/24 dev veth-ns
sudo ip netns exec test-ns-2 ip link set veth-ns up

# Bring up loopback in namespace
sudo ip netns exec test-ns-2 ip link set lo up
Enter fullscreen mode Exit fullscreen mode

Understanding the IP addresses:

  • 10.0.1.1/24 means IP address 10.0.1.1 with subnet mask 255.255.255.0
  • The /24 means the first 24 bits are the network part
  • 10.0.1.0/24 is the network, 10.0.1.1 and 10.0.1.10 are hosts in that network

Testing Connectivity

Let's test if the host and namespace can communicate:

# Ping from host to namespace
ping -c 2 10.0.1.10

# Ping from namespace to host
sudo ip netns exec test-ns-2 ping -c 2 10.0.1.1
Enter fullscreen mode Exit fullscreen mode

Success! If both pings work, you've successfully created your first virtual network connection! 🎉

Cleanup

Let's clean up before moving on:

# Bring interfaces down
sudo ip link set veth-host down
sudo ip link delete veth-host

# Delete namespace (this also removes veth-ns)
sudo ip netns delete test-ns-2
sudo ip netns delete test-ns-1
Enter fullscreen mode Exit fullscreen mode

Part 5: Creating a Linux Bridge (Your First Router)

What is a Linux Bridge?

A Linux bridge is like a virtual network switch. It can connect multiple network interfaces and forward traffic between them. In our VPC, the bridge will act as a router, connecting multiple subnets.

Let's create our first bridge:

# Create a bridge
sudo ip link add br-test type bridge

# Bring it up
sudo ip link set br-test up

# Verify it was created
ip link show br-test
Enter fullscreen mode Exit fullscreen mode

Connecting a Namespace to the Bridge

Now let's connect a namespace to our bridge using a veth pair:

# Create namespace
sudo ip netns add test-ns-3

# Create veth pair
sudo ip link add veth-br-host type veth peer name veth-br-ns

# Move namespace end to namespace
sudo ip link set veth-br-ns netns test-ns-3

# Connect host end to bridge (IMPORTANT: Do this before bringing it up!)
sudo ip link set veth-br-host master br-test

# Bring host end up
sudo ip link set veth-br-host up

# Configure namespace
sudo ip netns exec test-ns-3 ip link set lo up
sudo ip netns exec test-ns-3 ip addr add 10.0.1.10/24 dev veth-br-ns
sudo ip netns exec test-ns-3 ip link set veth-br-ns up

# Assign IP to bridge (this acts as the gateway)
sudo ip addr add 10.0.1.1/24 dev br-test
Enter fullscreen mode Exit fullscreen mode

Key points:

  • The bridge IP (10.0.1.1) acts as the gateway for the namespace
  • The namespace and bridge must be in the same subnet (10.0.1.0/24)
  • We attach the veth to the bridge BEFORE bringing it up

Adding a Default Route

The namespace needs to know where to send packets that aren't in its local network. Let's add a default route:

# Add default route in namespace (points to bridge as gateway)
sudo ip netns exec test-ns-3 ip route add default via 10.0.1.1 dev veth-br-ns

# Verify the route
sudo ip netns exec test-ns-3 ip route show
Enter fullscreen mode Exit fullscreen mode

Testing Connectivity

Let's test if the namespace can reach the bridge:

# Ping bridge from namespace
sudo ip netns exec test-ns-3 ping -c 2 10.0.1.1

# Verify bridge sees the connection
brctl show br-test
Enter fullscreen mode Exit fullscreen mode

Expected output: The bridge should show veth-br-host as a connected interface, and the ping should succeed!

Connecting Multiple Namespaces

Now let's add a second namespace to the same bridge to create a small network:

# Create second namespace
sudo ip netns add test-ns-4

# Create second veth pair
sudo ip link add veth-br-host-2 type veth peer name veth-br-ns-2

# Move namespace end to namespace
sudo ip link set veth-br-ns-2 netns test-ns-4

# Connect to bridge
sudo ip link set veth-br-host-2 master br-test
sudo ip link set veth-br-host-2 up

# Configure second namespace
sudo ip netns exec test-ns-4 ip link set lo up
sudo ip netns exec test-ns-4 ip addr add 10.0.1.20/24 dev veth-br-ns-2
sudo ip netns exec test-ns-4 ip link set veth-br-ns-2 up
sudo ip netns exec test-ns-4 ip route add default via 10.0.1.1 dev veth-br-ns-2

# Test connectivity between namespaces
sudo ip netns exec test-ns-3 ping -c 2 10.0.1.20
sudo ip netns exec test-ns-4 ping -c 2 10.0.1.10
Enter fullscreen mode Exit fullscreen mode

Success! Both namespaces can now communicate with each other through the bridge! This is the foundation of our VPC.

Part 6: Building Your First VPC

Understanding VPC Structure

A VPC consists of:

  1. A bridge that acts as the VPC router
  2. Multiple subnets (network namespaces)
  3. veth pairs connecting subnets to the bridge
  4. Routing tables directing traffic
  5. NAT rules for internet access (for public subnets)

Let's build a complete VPC with public and private subnets!

Step 1: Create the VPC Bridge 

# Create VPC bridge
sudo ip link add br-vpc1 type bridge
sudo ip link set br-vpc1 up

# Assign bridge IP addresses for each subnet
# The bridge needs an IP in each subnet to act as the gateway
sudo ip addr add 10.0.1.1/24 dev br-vpc1  # Gateway for public subnet
sudo ip addr add 10.0.2.1/24 dev br-vpc1  # Gateway for private subnet
sudo ip addr add 10.0.0.1/16 dev br-vpc1  # VPC router IP (optional)
Enter fullscreen mode Exit fullscreen mode

Why multiple IPs? The bridge needs to be in the same subnet as each namespace it serves as a gateway for. This allows the namespaces to reach the gateway using ARP (Address Resolution Protocol).

Step 2: Create Public Subnet 

# Create public subnet namespace
sudo ip netns add ns-vpc1-public

# Create veth pair for public subnet
# Note: Interface names must be 15 characters or less (Linux limit)
sudo ip link add veth-vpc1-pub-h type veth peer name veth-vpc1-pub-n

# Move namespace end to namespace
sudo ip link set veth-vpc1-pub-n netns ns-vpc1-public

# Connect host end to bridge
sudo ip link set veth-vpc1-pub-h master br-vpc1
sudo ip link set veth-vpc1-pub-h up

# Configure public subnet
sudo ip netns exec ns-vpc1-public ip link set lo up
sudo ip netns exec ns-vpc1-public ip addr add 10.0.1.10/24 dev veth-vpc1-pub-n
sudo ip netns exec ns-vpc1-public ip link set veth-vpc1-pub-n up

# Add routes
sudo ip netns exec ns-vpc1-public ip route add default via 10.0.1.1 dev veth-vpc1-pub-n
sudo ip netns exec ns-vpc1-public ip route add 10.0.2.0/24 via 10.0.1.1 dev veth-vpc1-pub-n
Enter fullscreen mode Exit fullscreen mode

Step 3: Create Private Subnet 

# Create private subnet namespace
sudo ip netns add ns-vpc1-private

# Create veth pair for private subnet
sudo ip link add veth-vpc1-prv-h type veth peer name veth-vpc1-prv-n

# Move namespace end to namespace
sudo ip link set veth-vpc1-prv-n netns ns-vpc1-private

# Connect host end to bridge
sudo ip link set veth-vpc1-prv-h master br-vpc1
sudo ip link set veth-vpc1-prv-h up

# Configure private subnet
sudo ip netns exec ns-vpc1-private ip link set lo up
sudo ip netns exec ns-vpc1-private ip addr add 10.0.2.10/24 dev veth-vpc1-prv-n
sudo ip netns exec ns-vpc1-private ip link set veth-vpc1-prv-n up

# Add routes
sudo ip netns exec ns-vpc1-private ip route add default via 10.0.2.1 dev veth-vpc1-prv-n
sudo ip netns exec ns-vpc1-private ip route add 10.0.1.0/24 via 10.0.2.1 dev veth-vpc1-prv-n
Enter fullscreen mode Exit fullscreen mode

Step 4: Enable Proxy ARP

Proxy ARP allows the bridge to respond to ARP requests for IP addresses in different subnets. This is essential for inter-subnet routing:

# Enable proxy ARP on bridge and veth interfaces
sudo sysctl -w net.ipv4.conf.br-vpc1.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc1-pub-h.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc1-prv-h.proxy_arp=1
Enter fullscreen mode Exit fullscreen mode

What is Proxy ARP? Normally, a device only responds to ARP requests for IPs on its own interface. Proxy ARP allows the bridge to respond for IPs in connected subnets, enabling routing.

Step 5: Test Inter-Subnet Communication 

# Test: public to private
sudo ip netns exec ns-vpc1-public ping -c 2 10.0.2.10

# Test: private to public
sudo ip netns exec ns-vpc1-private ping -c 2 10.0.1.10
Enter fullscreen mode Exit fullscreen mode

Success! Your subnets can now communicate with each other! 🎉

Part 7: Enabling Internet Access with NAT

Understanding NAT

Network Address Translation (NAT) allows private IP addresses to access the internet by translating them to a public IP address. This is how your home router works!

Step 1: Enable NAT for Public Subnet 

# Get your internet interface (we identified this earlier)
DEFAULT_INTERFACE=$(ip route show default | awk '/default/ {print $5}' | head -1)

# Add NAT rule for public subnet
sudo iptables -t nat -A POSTROUTING -s 10.0.1.0/24 -o $DEFAULT_INTERFACE -j MASQUERADE

# Verify the rule
sudo iptables -t nat -L POSTROUTING -n -v
Enter fullscreen mode Exit fullscreen mode

What this does:

  • -s 10.0.1.0/24 - Source is the public subnet
  • -o $DEFAULT_INTERFACE - Outgoing interface is your internet connection
  • -j MASQUERADE - Masquerade (NAT) the traffic

Step 2: Configure DNS

Namespaces need DNS to resolve domain names:

# Create DNS configuration for namespace
sudo mkdir -p /etc/netns/ns-vpc1-public
echo "nameserver 8.8.8.8" | sudo tee /etc/netns/ns-vpc1-public/resolv.conf
echo "nameserver 8.8.4.4" | sudo tee -a /etc/netns/ns-vpc1-public/resolv.conf
Enter fullscreen mode Exit fullscreen mode

Step 3: Test Internet Access 

# Test: ping Google's DNS
sudo ip netns exec ns-vpc1-public ping -c 2 8.8.8.8

# Test: ping Google by domain name
sudo ip netns exec ns-vpc1-public ping -c 2 google.com
Enter fullscreen mode Exit fullscreen mode

Success! Your public subnet can now access the internet! 🌐

Step 4: Verify Private Subnet Isolation 

# Private subnet should NOT be able to access internet (no NAT)
sudo ip netns exec ns-vpc1-private ping -c 2 8.8.8.8
Enter fullscreen mode Exit fullscreen mode

Expected result: The ping should fail or timeout. This is correct - private subnets don't have NAT, so they can't access the internet directly.

Part 8: Creating Multiple VPCs and Enforcing Isolation

Creating a Second VPC

Let's create a second VPC to demonstrate isolation:

# Create second VPC bridge
sudo ip link add br-vpc2 type bridge
sudo ip link set br-vpc2 up

# Assign bridge IPs
sudo ip addr add 172.16.1.1/24 dev br-vpc2  # Gateway for public subnet
sudo ip addr add 172.16.2.1/24 dev br-vpc2  # Gateway for private subnet
sudo ip addr add 172.16.0.1/16 dev br-vpc2  # VPC router IP

# Create public subnet for VPC2
sudo ip netns add ns-vpc2-public
sudo ip link add veth-vpc2-pub-h type veth peer name veth-vpc2-pub-n
sudo ip link set veth-vpc2-pub-n netns ns-vpc2-public
sudo ip link set veth-vpc2-pub-h master br-vpc2
sudo ip link set veth-vpc2-pub-h up
sudo ip netns exec ns-vpc2-public ip link set lo up
sudo ip netns exec ns-vpc2-public ip addr add 172.16.1.10/24 dev veth-vpc2-pub-n
sudo ip netns exec ns-vpc2-public ip link set veth-vpc2-pub-n up
sudo ip netns exec ns-vpc2-public ip route add default via 172.16.1.1 dev veth-vpc2-pub-n
sudo ip netns exec ns-vpc2-public ip route add 172.16.2.0/24 via 172.16.1.1 dev veth-vpc2-pub-n

# Create private subnet for VPC2
sudo ip netns add ns-vpc2-private
sudo ip link add veth-vpc2-prv-h type veth peer name veth-vpc2-prv-n
sudo ip link set veth-vpc2-prv-n netns ns-vpc2-private
sudo ip link set veth-vpc2-prv-h master br-vpc2
sudo ip link set veth-vpc2-prv-h up
sudo ip netns exec ns-vpc2-private ip link set lo up
sudo ip netns exec ns-vpc2-private ip addr add 172.16.2.10/24 dev veth-vpc2-prv-n
sudo ip netns exec ns-vpc2-private ip link set veth-vpc2-prv-n up
sudo ip netns exec ns-vpc2-private ip route add default via 172.16.2.1 dev veth-vpc2-prv-n
sudo ip netns exec ns-vpc2-private ip route add 172.16.1.0/24 via 172.16.2.1 dev veth-vpc2-prv-n

# Enable proxy ARP
sudo sysctl -w net.ipv4.conf.br-vpc2.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc2-pub-h.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc2-prv-h.proxy_arp=1

# Add NAT for VPC2 public subnet
DEFAULT_INTERFACE=$(ip route show default | awk '/default/ {print $5}' | head -1)
sudo iptables -t nat -A POSTROUTING -s 172.16.1.0/24 -o $DEFAULT_INTERFACE -j MASQUERADE

# Configure DNS
sudo mkdir -p /etc/netns/ns-vpc2-public
echo "nameserver 8.8.8.8" | sudo tee /etc/netns/ns-vpc2-public/resolv.conf
echo "nameserver 8.8.4.4" | sudo tee -a /etc/netns/ns-vpc2-public/resolv.conf
Enter fullscreen mode Exit fullscreen mode

Enforcing VPC Isolation

By default, Linux might route traffic between VPCs. We need to explicitly block this:

# Block traffic between VPC1 and VPC2
sudo iptables -A FORWARD -s 10.0.0.0/16 -d 172.16.0.0/16 -j DROP
sudo iptables -A FORWARD -s 172.16.0.0/16 -d 10.0.0.0/16 -j DROP

# Verify isolation
sudo ip netns exec ns-vpc1-public ping -c 2 172.16.1.10
Enter fullscreen mode Exit fullscreen mode

Expected result: The ping should fail! VPCs are now isolated from each other. 🔒

Part 9: VPC Peering (Connecting VPCs)

What is VPC Peering?

VPC peering allows you to connect two VPCs so they can communicate with each other. This is useful when you want controlled communication between different environments.

Step 1: Create Peering Connection 

# Create veth pair for peering
sudo ip link add veth-peer-1 type veth peer name veth-peer-2

# Connect one end to VPC1 bridge
sudo ip link set veth-peer-1 master br-vpc1
sudo ip link set veth-peer-1 up
# Use a shared /30 subnet for peering (192.168.255.0/30)
sudo ip addr add 192.168.255.1/30 dev veth-peer-1

# Connect other end to VPC2 bridge
sudo ip link set veth-peer-2 master br-vpc2
sudo ip link set veth-peer-2 up
# Same subnet - this is the peer IP
sudo ip addr add 192.168.255.2/30 dev veth-peer-2
Enter fullscreen mode Exit fullscreen mode

Why a /30 subnet? A /30 subnet provides exactly 2 usable IPs (.1 and .2), perfect for a point-to-point link between two VPCs.

Step 2: Add Routes 

# Add route on host to reach VPC2 through peering link
sudo ip route add 172.16.0.0/16 via 192.168.255.2 dev veth-peer-1

# Add route on host to reach VPC1 through peering link
sudo ip route add 10.0.0.0/16 via 192.168.255.1 dev veth-peer-2

# Remove isolation rules (peering should allow communication)
sudo iptables -D FORWARD -s 10.0.0.0/16 -d 172.16.0.0/16 -j DROP 2>/dev/null || true
sudo iptables -D FORWARD -s 172.16.0.0/16 -d 10.0.0.0/16 -j DROP 2>/dev/null || true

# Add routes in namespaces to reach the peer VPC
# In VPC1 namespaces, add route to VPC2
sudo ip netns exec ns-vpc1-public ip route add 172.16.0.0/16 via 10.0.1.1 dev veth-vpc1-pub-n
sudo ip netns exec ns-vpc1-private ip route add 172.16.0.0/16 via 10.0.2.1 dev veth-vpc1-prv-n

# In VPC2 namespaces, add route to VPC1
sudo ip netns exec ns-vpc2-public ip route add 10.0.0.0/16 via 172.16.1.1 dev veth-vpc2-pub-n
sudo ip netns exec ns-vpc2-private ip route add 10.0.0.0/16 via 172.16.2.1 dev veth-vpc2-prv-n
Enter fullscreen mode Exit fullscreen mode

Step 3: Test Peering 

# Test VPC1 to VPC2 connectivity
sudo ip netns exec ns-vpc1-public ping -c 2 172.16.1.10

# Test VPC2 to VPC1 connectivity
sudo ip netns exec ns-vpc2-public ping -c 2 10.0.1.10
Enter fullscreen mode Exit fullscreen mode

Success! VPCs can now communicate through peering! 🌉

Part 10: Implementing Firewall Rules (Security Groups)

What are Security Groups?

Security Groups are firewall rules that control inbound and outbound traffic. In our implementation, we'll use iptables in each namespace to enforce these rules.

Step 1: Create a Firewall Policy

Create a JSON file with your firewall rules:

cat > firewall-policy.json <<EOF
{
  "subnet": "10.0.1.0/24",
  "ingress": [
    {"port": 80, "protocol": "tcp", "action": "allow"},
    {"port": 443, "protocol": "tcp", "action": "allow"},
    {"port": 22, "protocol": "tcp", "action": "deny"}
  ],
  "egress": [
    {"port": 0, "protocol": "all", "action": "allow"}
  ]
}
EOF
Enter fullscreen mode Exit fullscreen mode

Step 2: Apply Firewall Rules 

# Allow HTTP (port 80)
sudo ip netns exec ns-vpc1-public iptables -A INPUT -p tcp --dport 80 -j ACCEPT

# Allow HTTPS (port 443)
sudo ip netns exec ns-vpc1-public iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Deny SSH (port 22)
sudo ip netns exec ns-vpc1-public iptables -A INPUT -p tcp --dport 22 -j DROP

# Allow established connections
sudo ip netns exec ns-vpc1-public iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow loopback
sudo ip netns exec ns-vpc1-public iptables -A INPUT -i lo -j ACCEPT

# Set default policy to DROP
sudo ip netns exec ns-vpc1-public iptables -P INPUT DROP
Enter fullscreen mode Exit fullscreen mode

Step 3: Deploy a Web Server 

# Deploy a simple HTTP server in the public subnet
sudo ip netns exec ns-vpc1-public python3 -m http.server 80 &

# Test HTTP access (should work)
curl http://10.0.1.10:80

# Test SSH access (should be blocked)
nc -zv 10.0.1.10 22
Enter fullscreen mode Exit fullscreen mode

Expected results:

  • HTTP works ✅
  • SSH is blocked ❌

Part 11: Automating with vpcctl

Why Automate?

By now, you've learned how to manually create VPCs using Linux commands. You've seen how namespaces, bridges, veth pairs, routing, and NAT all work together. This knowledge is invaluable!

However, creating VPCs manually requires dozens of commands and is error-prone. That's where vpcctl comes in - it's a CLI tool that automates all the steps we've learned, making VPC management as simple as a single command.

Getting Started with vpcctl

Important: Before using vpcctl, make sure you've cloned the repository as shown in Part 2, Step 1. The vpcctl tool is included in the repository.

If you haven't done so already:

# Clone the repository
git clone https://github.com/Donkross360/vpc-project.git
cd vpc-project

# Make vpcctl executable
chmod +x vpcctl

# Verify it works
./vpcctl --help
Enter fullscreen mode Exit fullscreen mode

Understanding What vpcctl Does

vpcctl is a Python CLI tool that automates everything we did manually:

  • Creates VPCs - Sets up bridges with proper IP addresses
  • Adds Subnets - Creates namespaces, veth pairs, and configures routing
  • Enables NAT - Automatically configures NAT for public subnets
  • Manages Peering - Creates VPC peering connections
  • Applies Firewall Rules - Reads JSON policies and applies iptables rules
  • Enforces Isolation - Ensures VPCs are isolated by default
  • Maintains State - Tracks all VPCs in a JSON file

Basic Usage

Now that you understand how VPCs work manually, let's see how vpcctl simplifies everything:

# Create a VPC (replaces all the manual bridge creation)
sudo ./vpcctl create --name myvpc --cidr 10.0.0.0/16

# Add a public subnet (replaces namespace, veth, routing setup)
sudo ./vpcctl add-subnet --vpc myvpc --name public --cidr 10.0.1.0/24 --type public

# Add a private subnet
sudo ./vpcctl add-subnet --vpc myvpc --name private --cidr 10.0.2.0/24 --type private

# List all VPCs
sudo ./vpcctl list

# Show VPC details (namespace, IPs, etc.)
sudo ./vpcctl show myvpc

# Apply firewall rules from a JSON policy
sudo ./vpcctl apply-firewall --vpc myvpc --subnet public --policy policies/public-subnet.json

# Create VPC peering (connects two VPCs)
sudo ./vpcctl peer --vpc1 myvpc --vpc2 another-vpc

# Deploy an application (using helper script)
./examples/deploy-web-app.sh myvpc public 8000

# Delete a VPC (cleans up everything)
sudo ./vpcctl delete --name myvpc
Enter fullscreen mode Exit fullscreen mode

Using the Makefile (Even Easier!)

The repository includes a Makefile that makes common tasks even simpler:

# One command to set up everything and deploy a demo
make demo

# Run comprehensive tests
make test

# Clean up all VPCs
make clean

# Show all available targets
make help
Enter fullscreen mode Exit fullscreen mode

The make demo command will:

  1. Set up IP forwarding
  2. Create a demo VPC
  3. Add a public subnet
  4. Deploy a web application
  5. Show you the URL to access it

Try it out!

What vpcctl Does Under the Hood

Remember all those manual commands we ran? vpcctl does the same things automatically:

When you run sudo ./vpcctl create --name myvpc --cidr 10.0.0.0/16:

  • Creates a Linux bridge (br-myvpc)
  • Assigns an IP address to the bridge
  • Enables the bridge
  • Stores VPC information in .vpcctl/vpcs.json

When you run sudo ./vpcctl add-subnet --vpc myvpc --name public --cidr 10.0.1.0/24 --type public:

  • Creates a network namespace (ns-myvpc-public)
  • Creates a veth pair
  • Moves one end into the namespace
  • Connects the other end to the bridge
  • Assigns IP addresses
  • Configures routing tables
  • Enables proxy ARP
  • Sets up NAT (if public subnet)
  • Configures DNS

All of this happens automatically! 🎉

Benefits of Automation

  • Idempotent: Safe to run multiple times - won't create duplicates
  • State management: Tracks VPCs in JSON file (.vpcctl/vpcs.json)
  • Logging: All actions are logged to .vpcctl/vpcctl.log
  • Error handling: Validates inputs and handles errors gracefully
  • Consistent: Same results every time
  • Fast: Creates VPCs in seconds instead of minutes

Part 12: Cleanup and Best Practices

Cleaning Up

Always clean up your test resources:

# Use the cleanup script
sudo ./cleanup.sh

# Or manually delete VPCs
sudo ./vpcctl delete --name myvpc
Enter fullscreen mode Exit fullscreen mode

Best Practices

  1. Use descriptive names: Name your VPCs and subnets clearly
  2. Plan your IP ranges: Avoid overlapping CIDR blocks
  3. Test isolation: Always verify that VPCs are isolated
  4. Document your setup: Keep notes on your VPC configurations
  5. Clean up regularly: Remove unused VPCs to free resources

Common Pitfalls

  1. Interface name length: Linux limits interface names to 15 characters
  2. Subnet mismatches: Gateway IP must be in the same subnet as the namespace
  3. Missing proxy ARP: Required for inter-subnet routing
  4. DNS configuration: Don't forget to configure DNS for public subnets
  5. NAT rules: Only public subnets need NAT rules

Conclusion

Congratulations! You've successfully built your own VPC from scratch! 🎉

What You've Accomplished

  • Created isolated virtual networks using Linux namespaces
  • Connected networks using Linux bridges and veth pairs
  • Implemented routing between subnets
  • Enabled internet access using NAT
  • Enforced VPC isolation
  • Created VPC peering connections
  • Implemented firewall rules

Next Steps

  1. Experiment: Try creating more complex VPC topologies
  2. Automate: Use vpcctl to manage your VPCs
  3. Deploy applications: Run real applications in your VPCs
  4. Learn more: Explore advanced networking topics like BGP, VPNs, and load balancing

Resources

  • Repository: Clone the project from GitHub to get vpcctl and all examples
  • README.md: Complete CLI documentation and usage examples
  • Linux Networking Documentation: Official Linux networking docs
  • Makefile: Run make help to see all available automation targets

Final Thoughts

Building a VPC from scratch teaches you the fundamentals of network virtualization. The concepts you've learned here apply to cloud providers, container networking (Docker, Kubernetes), and virtualized environments.

Remember: Understanding the fundamentals makes you a better engineer. Keep learning, keep building, and keep experimenting! 🚀

Happy VPC Building! 🌐

If you found this guide helpful, please share it with others who might benefit from it!

Top comments (0)