Build a Cloud at Home
“The cloud is just someone else’s computer.”
You’ve probably heard that sentence before. Maybe a little less in 2026, but barely 10 years ago it was one of the favorite lines of the anti-cloud crowd.
What this sentence basically means is that the cloud is just a provider giving you access to a machine so you don’t have to manage that machine yourself.
That was already a pretty reductive view 10 years ago, but can we really blame users for seeing little more than a virtual machine provider and therefore not really seeing the difference between what we call a Cloud Provider and a traditional hosting provider?
From a customer’s perspective, both can indeed be summarized as “someone else’s computer.”
So, 10 years later, maybe a little late, it’s time to explain simply what the cloud actually is.
And what better way to do that than to start building one?
The First Building Blocks
So where do we start?
We can begin by breaking our cloud down into three fairly fundamental building blocks. We’ll later use these to build services, and even other building blocks for more advanced services.
Those fundamentals are:
- Logically isolated networks for each customer, with each customer representing a tenant.
- Compute, meaning the resources on which our workloads will run, whether as VMs or something else.
- Storage, which can mean disk space or object storage.
All of this is organized and orchestrated by a software layer we call the Control Plane.
“Yeah, but the cloud is hundreds, even thousands of machines. How am I supposed to build that at home?”
Don’t panic, my friend.
We’re not going to put a huge pile of computers on a network and start controlling their configuration and resource allocation remotely.
Well, you can if you want to, but we can simulate how a cloud works much more easily.
All we need is a Linux distribution and a little imagination to reproduce the concepts we want to explore.
Tenant Network Isolation
Let’s start with a simple problem.
Suppose we have two customers, Alice and Bob. Get used to Alice and Bob because they’re about to become your best friends.
They both share the same infrastructure.
How do we prevent Bob from accessing Alice’s network and vice versa?
The Network Namespace
You may have already heard some Kubernetes hipster tell you that Docker didn’t really invent anything with containers, it just used Linux Network Namespaces.
That’s not entirely wrong.
On Linux, a network namespace allows network resources used by processes to be isolated from a networking perspective.
As for Docker, we’ll get back to that another time.
In our simulation, we’re going to use each network namespace to represent a node placed inside its tenant’s network.
By node, we simply mean a resource that could represent a physical computer or a VM.
The Bridge
Another component that will be useful for simulating our cloud is the Linux bridge.
It will act as the virtual equivalent of a network switch.
We’ll use it to connect our customers’ nodes together.
Veth Pair
How do you connect a node to a network?
With a cable, obviously.
That will be the role of our Veth pair.
One side will be connected to our node, while the other side will be connected to our virtual switch.
There we go.
We now have the fundamental building blocks we need for our network.
First, let’s create our nodes, which will later be assigned to different tenants.
No need to create too many of them for now.
sudo ip netns add node10
sudo ip netns add node20
sudo ip netns add node30
sudo ip netns add node40
Then let’s create our switches.
sudo ip link add pc-alice type bridge
sudo ip link set pc-alice up
sudo ip link add pc-bob type bridge
sudo ip link set pc-bob up
Building Alice’s Network
To build Alice’s network, we need to connect Alice’s nodes to the same switch so they can communicate with each other, and only with each other.
So let’s create our first virtual cable.
sudo ip link add v-node10-h type veth peer name v-node10-n
sudo ip link set v-node10-n netns node10
sudo ip link set v-node10-h master pc-alice
sudo ip link set v-node10-h up
Then configure the subnet.
sudo ip netns exec node10 ip link set lo up
sudo ip netns exec node10 ip link set v-node10-n up
sudo ip netns exec node10 ip addr add 10.10.0.10/24 dev v-node10-n
We do exactly the same thing for the next node so we have two machines inside the same tenant.
sudo ip link add v-node20-h type veth peer name v-node20-n
sudo ip link set v-node20-n netns node20
sudo ip link set v-node20-h master pc-alice
sudo ip link set v-node20-h up
sudo ip netns exec node20 ip link set lo up
sudo ip netns exec node20 ip link set v-node20-n up
sudo ip netns exec node20 ip addr add 10.10.0.11/24 dev v-node20-n
Building Bob’s Network
One customer is nice, but we’ve never seen a Cloud Provider with only one customer.
So let’s not waste any time and do the same thing for Bob, who will own nodes 30 and 40.
sudo ip link add v-node30-h type veth peer name v-node30-n
sudo ip link set v-node30-n netns node30
sudo ip link set v-node30-h master pc-bob
sudo ip link set v-node30-h up
sudo ip netns exec node30 ip link set lo up
sudo ip netns exec node30 ip link set v-node30-n up
sudo ip netns exec node30 ip addr add 10.20.0.10/24 dev v-node30-n
sudo ip link add v-node40-h type veth peer name v-node40-n
sudo ip link set v-node40-n netns node40
sudo ip link set v-node40-h master pc-bob
sudo ip link set v-node40-h up
sudo ip netns exec node40 ip link set lo up
sudo ip netns exec node40 ip link set v-node40-n up
sudo ip netns exec node40 ip addr add 10.20.0.11/24 dev v-node40-n
The procedure is obviously identical.
And that repetition is actually important, because a large part of the challenge of cloud computing is being able to reproduce this kind of operation automatically, reliably, and at scale.
Testing Communication
Alright, now it’s time to check whether our networks actually work.
Alice’s nodes should be able to communicate with each other, and the same should be true for Bob.
But we also need to test isolation.
Our two customers must not be able to talk to each other.
Let’s ping node20 from node10, since both belong to Alice.
sudo ip netns exec node10 ping -c 3 10.10.0.11
PING 10.10.0.11 (10.10.0.11) 56(84) bytes of data.
64 bytes from 10.10.0.11: icmp_seq=1 ttl=64 time=16.0 ms
64 bytes from 10.10.0.11: icmp_seq=2 ttl=64 time=0.160 ms
64 bytes from 10.10.0.11: icmp_seq=3 ttl=64 time=0.103 ms
--- 10.10.0.11 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2006ms
rtt min/avg/max/mdev = 0.103/5.437/16.048/7.503 ms
And let’s do the same thing from node30 to node40, which both belong to Bob.
sudo ip netns exec node30 ping -c 3 10.20.0.11
PING 10.20.0.11 (10.20.0.11) 56(84) bytes of data.
64 bytes from 10.20.0.11: icmp_seq=1 ttl=64 time=0.791 ms
64 bytes from 10.20.0.11: icmp_seq=2 ttl=64 time=0.241 ms
64 bytes from 10.20.0.11: icmp_seq=3 ttl=64 time=0.260 ms
--- 10.20.0.11 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2029ms
rtt min/avg/max/mdev = 0.241/0.430/0.791/0.254 ms
On the other hand, Alice cannot communicate with Bob’s network.
sudo ip netns exec node10 ping -c 3 10.20.0.10
ping: connect: Network is unreachable
So we do indeed have a first form of isolation.
“Hey, wait a minute. You just created two networks that don’t communicate with each other. This is first-year networking stuff and you’re calling that cloud? Are you taking the piss? They can’t communicate simply because there’s no route! I do remember my networking classes, you know!”
The Route Test
Very good point.
Two distinct IP subnets need some form of routing mechanism to communicate.
So what happens if we try adding a route?
sudo ip netns exec node10 ip route add 10.20.0.0/24 dev v-node10-n
Now let’s look at the routing table.
sudo ip netns exec node10 ip route
10.10.0.0/24 dev v-node10-n proto kernel scope link src 10.10.0.10
10.20.0.0/24 dev v-node10-n scope link
There we go, we have a route.
Let’s test it.
sudo ip netns exec node10 ping -c 3 10.20.0.10
PING 10.20.0.10 (10.20.0.10) 56(84) bytes of data.
From 10.10.0.10 icmp_seq=1 Destination Host Unreachable
From 10.10.0.10 icmp_seq=2 Destination Host Unreachable
From 10.10.0.10 icmp_seq=3 Destination Host Unreachable
--- 10.20.0.10 ping statistics ---
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2049ms
pipe 3
Well, nope.
Still not working.
This route does not actually create a path to Bob.
It simply tells Alice to consider the 10.20.0.0/24 network as directly reachable through its v-node10-n interface.
This time Alice knows which interface it should try to use to send its traffic.
The problem is now lower down, at Layer 2.
Node10 looks for 10.20.0.10 on its Ethernet network by sending an ARP request, but that request remains confined to the pc-alice bridge.
There is no Layer 2 path to pc-bob.
Our two networks are therefore properly isolated in our current architecture.
So is the cloud really just first-year networking?
Yes and no.
The networking part here is fairly basic, but cloud infrastructure does not need to be complicated by design.
Quite the opposite.
It becomes more complex as we progressively add more features, so whenever something can remain simple, let’s keep it simple.
The really interesting part starts when we automate the creation and management of this infrastructure at scale: creating new tenants, deleting old ones, changing their limits, allocating and deallocating nodes, moving resources around, and so on.
Continuously managing the state of thousands, or even millions, of elements is where things really start to become interesting.
And that software layer will be the subject of Part 2, because there’s a lot to talk about.
And a lot to code.
Top comments (0)