Hello folks,
Have you ever wondered how do cloud providers build their cloud ? How do circle-ci, github actions, gitlab-ci do run your tests and builds your projects in their cloud ?
Don't wonder anymore, because in this series, i'll show to build your own mini-cloud at home.
Disclaimer:
- it will require some hardware investment,
- it will give you a glimpse of hardware architecture, but not full picture (we will not cover spine/leaf nodes architecture used in cloud datacenters)
What hardware do you need ?
- At least two raspberry pi with 4GB each (official price 55$ each),
- One MicroSD card for each Raspberry pi (i recommend taking at least a 32GB card),
- [optional] A Raspberry pi POE+ hat for each Raspberry pi (official price 20$ each)
- [optional] A POE+ ethernet switch (or the official raspberry pi power supply if you can't afford a POE switch and want to use wifi for networking)
- [Optional] A cluster case for a clean setup.
I made one based on 4 Raspberry pi's with POE+ hat and a POE+ switch. Total Budget : 400$.
If you go with only two RPI's over WIFI, it will cost you around 150$.
What it looks like?
night mode
day mode
Install Ubuntu server ARM64 on SD card
We could install the official Raspberry pi OS on each card, but unfortunately, raspberry only support 32 bit images officially (their 64 bit OS is still broken from time to time), and we want to use the full power of Raspberry pi 4 which is 64 bit. See this benchmark to compare for yourself the huge difference :
So here i'll recommend using Ubuntu Server 20.04 LTS ARM64 build provided by Raspberry pi Imager.
Select Other general purpose OS>Ubuntu> Ubuntu server 20.04 LTS
And click on the Write button.
SD Card Modification
Before putting your SD card on your raspberry pi, you first need to make a few changes to allow you to use it in Headless mode (ie: without a screen and keyboard linked to it).
With your SD card still inserted, you should see two new disks :
- bootfs
- writable
Enable SSH
Still at the root of bootfs disk, create an empty file named ssh :
touch ssh
This will enable ssh connections on your Raspberry pi.
Enable container cpu and memory isolation
Open a Terminal and go to the bootfs filesystem of your SD Card, and open the cmdline.txt file.
You should see a single line, append this at the end of the line :
cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory
This will enable you to use kubernetes (k3s to be precise) on your cluster.
Optional Enable POE+ hat driver
Open userconfig.txt and append these lines at the end of the file :
dtoverlay=rpi-poe
dtparam=poe_fan_temp0=45000
dtparam=poe_fan_temp1=55000
dtparam=poe_fan_temp2=65000
dtparam=poe_fan_temp3=75000
This will enable POE+ hat driver and fan configuration to cool down your Raspberry pi.
Rename your Rasberrys
Now go to the writable disk and edit as an admin the etc/hosts and append at the end:
127.0.1.1 <your-new-name>
For the first SD card, i suggest naming it to recognize that's it will be the master node of your cluster. Mine is called k3s-main. For the other ones, you can choose a predictable name like k3s-worker-01 ...
Also edit etc/hostname and change the current name (ubuntu) to the name you just choose.
Insert your SD card and boot
Now you can insert your SD card in your Raspberry pi's. put power on and let them boot.
Connect to each of them with ssh
Now you need to connect to each of your booted Raspberry pi's. If you have wifi and a dhcp server, you might be able to connect to them using their name with .local appended to it. For exemple :
> ssh ubuntu@k3s-main.local
When prompted for password, just use ubuntu. it will ask you to change the default password after a successfull login.
Once the password changed, it will disconnect you.
If it does not work, don't worry. First run ifconfig to get information about your network.
> ifconfig
etho0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet <YOUR.LOCAL.IP> netmask 255.255.0.0 broadcast 172.19.255.255
ether <MAC:ADDRESS> txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
This should give you your local IP. replace the last two digits by 00 and run nmap :
> nmap -sn <YOUR.LOCAL.IP.ZERO>/24
This should return your your IP, plus the one of your Raspberry pi's. you should now be able to login using ssh and the raspberry IP
> ssh ubuntu@<Rasbpberry.IP>
If you have already setup a ssh key, i recommand using it to automatically connect to your Raspberry pi's without password prompt. To do this :
> ssh-copy-id -i ~/.ssh/id_rsa.pub ubuntu@k3s-main.local
Enter password for the last time when prompted.
Update your Raspberry pi's
On each Raspberry pi's within the SSH session, run :
> sudo apt update & sudo apt upgrade
Install k3s
Main node
On your main node (k3s-main for me), run the following command :
> curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644
Once the script finished execution, you should be able to see your node running :
> kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3s-main Ready control-plane,master 1m v1.21.0
Save your cluster token
save your token to allow your workers to join the main node
> cat /var/lib/rancher/k3s/server/node-token
<YOUR-TOKEN>
Join main node on workers
On each of your worker nodes (k3s-worker-01, k3s-worker-02, k3s-worker-03 for me) join the main node by running the following command :
> curl -sfL https://get.k3s.io | K3S_URL=https://k3s-main.local:6443 K3S_TOKEN=<YOUR-TOKEN> sh -
Once the scripts finished execution, you should be able to see your nodes running (execute on main node):
> kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3s-main Ready control-plane,master 15m v1.21.0
k3s-worker-01 Ready <none> 3m v1.21.0
k3s-worker-02 Ready <none> 2m v1.21.0
k3s-worker-03 Ready <none> 1m v1.21.0
Conclusion
So we have installed a cluster on raspberry pi's. what now ?
Next time, we'll see how to install rancher on your cluster to monitor and administrate it with a beautifull UI.
Top comments (0)
Some comments have been hidden by the post's author - find out more