Setting up a Raspberry Pi Cluster

For whatever reason, in my head, running software on a raspberry pi is a lot cooler than running it on my macbook. I have one running pi-hole and a couple others just lying around, so I thought of something to use them for. I have a couple web scrapers running for whenarethefights.com on lambda and thought it would be nice to move those to a pi-cluster.

I already had 3 pis so here was my shopping list:

• Rack
• Switch
• Ethernet Cables
• microSD Cards

Installing the OS

So after I got everything I just need to flash each sd card with an OS. A list of them can be found here. I chose Ubuntu Server for no reason other than I have used it before on Digital Ocean. To flash each card I used Etcher. Its pretty straight forward and keeps me from having to use dd. I knew I wanted to run these in a headless mode so I need to place an empty file named ssh on each drive after they were done flashing. To do this you will need to re-mount the drive after Etcher ejects it; The easiest way to do that on Mac is to use the disk utility.

This is the disk utility panel

Once mounted, you can open up the terminal to place the empty file.

cd /Volumes
# find the drive with the rpi OS
cd <OS Drive>
touch ssh
cd

Unmount the drive and repeat for each drive you have

Setting up each Raspberry Pi

I like to set one pi up at a time because each pi sets its hostname as ubuntu and I don't want to arp for the ip of each one and guess. My plan was to name each pi green-pi, white-pi, etc according to whatever color ethernet cable is plugged into it. Load all the sd cards but only plug one of the pi in and wait a min. You should now be able to ssh into the first pi.

ssh ubuntu@ubuntu

# When asked for a password, enter 'ubuntu'. You will be asked to change it to a new one.

Now we need to change the hostname and restart so our dnsserver picks up the new name.

hostnamectl set-hostname green-pi
sudo reboot

If you try to ssh to ubuntu again it won't let you since the ssh keys changed, so you will need to delete ubuntu from known users. Open up ~/.ssh/known_hosts and delete the line with ubuntu on it.

vim ~/.ssh/known_hosts
ubuntu,192.168.0.88 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGRsJ...

Repeat for each pi.

Log into each Pi

So now that they are all set up, we need to log into each one so we know we can. Since the hostname was updated we should be able to use the new hostname.

ssh ubuntu@green-pi

# Enter the new password you created

Once we know that works, let's copy our keys over so we don't need to enter a password anymore.

ssh-copy-id ubuntu@green-pi

# Enter the password again. When it's successful it will log you out and tell you to login again.

ssh ubuntu@green-pi

You should be able to login without a password now.

Setup your ssh config

Less typing is always better for me so the final step is to set our ssh config so I don't have to type the username anymore. Open up ~/.ssh/config and place the following:

Host green-pi
  HostName green-pi
  User ubuntu

Host white-pi
  HostName white-pi
  User ubuntu

Host blue-pi
  HostName blue-pi
  User ubuntu

After adding that you can just ssh with the hostname and be done.

ssh green-pi

Now that the pi cluster is set up it can be used to do anything. We can do all the buzzword things like Blockchain, Kubernetes, crypto mining.

Just kidding, we're just gonna put some web scrapers on them.

Photo by Clint Patterson / Unsplash

Current word count: ~1681 / 250,000

Daily average: ~560