How I deployed portainer.io in Kubernetes Using an Ubuntu Server in an ESXi Hypervisor

ImmTB — Thu, 14 Dec 2023 17:11:12 +0000

Introduction

This is a post listing the details of how I deployed portainer.io in Kubernetes.

Why portainer.io?
Portainer.io acts as a web-based GUI for managing containers in Kubernetes. This can be more efficient for managing Kubernetes clusters with several containers.

I'll be using VMware Workstation Pro to setup a VMware ESXi server, from which I will then spawn an Ubuntu server (version 22.04.3 LTS).
Before continuing, please note that VMware hypervisors are not necessary for the deployment of a Kubernetes cluster. A dedicated Ubuntu server, an Ubuntu server within a Virtual Machine (whether that be a type-1 or type-2 hypervisor), or other solutions can be used to accomplish the same goal.

Downloading the Software and Ubuntu Disc Image

The version of VMware Workstation Pro that I am using can be downloaded at vmware.com, but VMware Workstation Player could also be used to deploy the .

The ESXi hypervisor that I will be using is vSphere 8. I will only be using the 60-day trial version of it, but it will be enough for my project today.

What is a hypervisor?
A hypervisor is a type of computer software or operating system that can be used to create and run virtual machines.

I downloaded the Ubuntu server image file from ubuntu.com. The most recent version of the Ubuntu server is 22.04.3 at the time of writing, and the .iso file containing the operating system is 2GB in size.

Step 2: Installing VMware Workstation and Creating ESXi Virtual Machine

Setup for the hypervisors are quite simple. I ran the .exe file downloaded from VMware's website to install VMware Workstation Pro. A guided installation window appeared that walked through the options for the software installation. Next, I will be creating a new virtual machine (being the ESXi server). Upon clicking the Create a New Virtual Machine, a window named New Virtual Machine Wizard appeared.

What is a virtual machine (VM)?
A virtual machine is essentially a computer running inside a computer. In this case, my computer (the host, running Windows 11) will be running a computer with an ESXi hypervisor installed on it (being the guest operating system, or guest OS). Later on, that ESXi hypervisor will be the host of the Ubuntu Server VM. This is known as "nested virtualization".

I had been prompted to choose between a typical or custom installation method, chose Typical (recommended), and clicked Next >.
Next, I am asked how guest operating system should be installed. I selected Installer disc image file (iso):, and clicked Browse... to choose the .iso file corresponding to the vSphere hypervisor, then clicked Next >.
The guest operating system will be VMware ESX, so I selected the VMware ESX option. In the Version section, I clicked the dropdown menu and made sure that VMware ESXi 7 was selected, then clicked Next >.
I named the VM "Final ESXi", since it is something unique from other virtual machines (and is the ESXi VM being used for my final here). I preferred to save the virtual machine files to a different folder on a different drive, so I clicked Browse... and selected the designated folder on the drive that I wanted to save the virtual machine files to, then clicked Next >.
Here, I specified the disk capacity (in GB). This is normally defined by the amount of free space you can spare for the VM and believe that you need for it, but I just kept the default option of 142 GB since it would be sufficient for this project.
Finally, I am presented with a screen that shows what hardware will be used with the virtual machine. I increased the CPU cores from the default to 2 cores and 2 processors per core, as well as the ram to 8GB (8192MB), then clicked Finish.

Step 3: Setting up the ESXi Hypervisor

First, I created a virtual hard drive so that the ESXi hypervisor had somewhere to store the Ubuntu VM I was about to create. I did this by selecting the ESXi hypervisor in VMware Workstation Pro (in the left panel), then clicking Edit virtual machine settings. From there, towards the bottom left of the Virtual Machine Settings window, I used the Add button to select Hard Disk in the Add Hardware Wizard. I selected SCSI for my disk type, selected Create a new virtual disk when asked what disk I wanted to use, and gave the virtual disk a maximum size of 50GB, made sure that the Allocate all disk space now was unchecked and continued with the default option of splitting the virtual disk into multiple files. Finally, I was asked to specify the disk file, to which I accepted the default Disk file and clicked Finish, then clicked OK on the Virtual Machine Settings window.

After I completed setup of the virtual hard drive, I booted up the ESXi hypervisor with the Power on this virtual machine button. The ESXi's boot process takes a while, but eventually I was greeted with a welcome screen that looks something like this:

I pressed Enter, F11 to accept the terms of use, and was prompted to select a disk to install the ESXi hypervisor to. I chose the one with 142GB of capacity and pressed Enter. If I wanted to use the other drive, I could have used the arrow keys (up and down) to select it.

I selected US Default as my keyboard layout, then press Enter to continue.
Here, I was asked for a password. My password consists of at least one symbol, capital letter, a number, and is at least 10 characters in length. After typing my password (as well as typing it in the confirm password text area), I pressed Enter to continue.
I was asked to confirm the install, as the screen warns that the disk will be partitioned. I pressed F11 to install. The installation was then in progress, and took less than a minute to complete.

Finally, the installation had completed and the screen told me to remove the installation media before rebooting. To do this, I escaped the mouse and keyboard control from the ESXi hypervisor by pressing Ctrl + Alt at the same time. Then, I right-clicked the name of the ESXi hypervisor in the left panel of VMware Workstation Pro, hovered over Removable Devices, hovered over CD/DVD (IDE), and clicked Disconnect. Returning to the ESXi hypervisor (by clicking on the area where the Installation Complete window still was), I pressed Enter, rebooting the ESXi hypervisor.

After only about 20 seconds, I am booted into the ESXi hypervisor, and can see in the yellow half of the screen the IP that I should go to in my browser in order to manage it.
Upon entering the IP address in my browser's search bar, I am shown an alert that states that the website I am going to does not have a valid SSL certificate, and is therefore insecure. Clicking Advanced, I am able to bypass the security warning and access the ESXi login page. Please note that not all browsers have the same warning when accessing pages without SSL certificates, so you may need to search how to get past the SSL warning page on your specific browser, or with your specific antivirus software while accessing an ESXi hypervisor's web console.
On the login page, I am prompted for a username and password, as seen in the screenshot below:

By default, the username is root and the password is whatever was previously defined in the setup of the ESXi hypervisor.
Finally I am in the ESXi web console, and can see data such as CPU/RAM utilization, system information, hardware, etc.

Step 4: Creating the Ubuntu Server Virtual Machine

To create the Ubuntu Server Virtual Machine, I will first need to get the Ubuntu Server .iso file into the ESXi's storage. Navigating to the storage tab in the left panel, I am shown the Datastores tab by default.

Here, I click New datastore. This opens a menu to set up the new datastore.

What is a datastore?
The datastore acts as a storage device for the Virtual Machine and its associated files.

For step one, I choose Create new VMFS datastore as my datastore creation type. For step two, I choose the 50GB drive that had been previously set up on the ESXi VM. I'll be naming the device "UbuntuServerISO" for simplicity.

On the third step, I select the default partitioning options and clicked NEXT. Step 4 is just an overview of the configuration of the datastore, so I clicked FINISH, clicking YES on the warning that stated that the virtual disk that was about to hold the datastore was about to be erased. Since it was empty (inside a VM), this was fine and the datastore was successfully created!

To upload the Ubuntu Server .iso file, I click the Datastore browser button to show the datastore browser window, then I click UbuntuServerISO to make sure that the datastore that I am about to upload to is the correct one. I click Upload in the top right of that window, which opens file explorer, from which I can select the .iso file I previously downloaded.
The .iso file took about 3 minutes, and is now visible in the Datastore browser under the UbuntuServerISO tab.

I can now close the Datastore browser tab and click on the Virtual Machines tab on the left sidebar, then click Create / Register VM to create the new virtual machine.

The New virtual machine menu appears on the screen, asking for the creation type of the VM. I click Create a new virtual machine.
The next step asks for a name and guest operating system for the VM. For the name, I use "Ubuntu Server 22.04" as it corresponds to the guest operating system name and version number that I will be using in the VM. I leave the compatibility option set to its default, ESXi 8.0 U2 virtual machine, set the Guest OS family to Linux, and set the Guest OS version option to Ubuntu Linux (64-bit).
Step 3 prompts me to select the storage type and datastore for the VM that I am about to create, so I choose the UbuntuServerISO option from the list, since it has enough space to store the files of the VM.
On step 4, I am shown a list of Virtual Hardware that I can configure, from RAM to network adapters, etc. I allocate 4GB of memory (RAM) to the VM, then head over to the CD/DVD Drive 1 category, where I select the dropdown menu to select Datastore ISO file. The Datastore browser menu shows on the screen and I am able to choose the .iso file corresponding to the Ubuntu Server by clicking the UbuntuServerISO tab on the left panel of the menu, then clicking the .iso file in the next panel to the right of the left one.

Upon clicking NEXT, I am shown all of the hardware that is going to be allocated to the VM.

Finally, the Ubuntu Server VM is created! Setting it up should be a breeze, as there are only a few steps to it.

Step 5: Setting Up the Ubuntu Server Virtual Machine

First thing's first, I'll need to start the VM by clicking the hyperlink to it on the Virtual Machines page (or under the Virtual Machines tab), then clicking the Power on button towards the top of the tab.

Now that it's started, I can click on the VM preview screen in the top-left of the window to view the Ubuntu Server VM. After less than 30 seconds, it finishes initializing. Now I am in the setup process of the Ubuntu Server.
I select English as my language (pressing Enter to select, and using the arrow keys to navigate the UI), Continue without updating when the installer update page shows (for simplicity), and select English (US) as my keyboard layout.
I select Ubuntu Server as the base for installation, keep defaults for the network connections, and since I don't have a proxy address to input, I don't configure the proxy.
The default Ubuntu Server archive mirror will do fine for this project, so I enter done, then continue when prompted for confirmation.
I keep the default of Use an entire disk on the storage configuration page, and use the arrow keys to navigate down select Done. I get a file system summary, to which I select Done once more.

I select continue when I get a warning for "destructive action", where it warns that the disk(s) selected in the VM will be formatted and the information will be erased. Since it is a VM, no harm will be done to the ESXi hypervisor or the host machine running VMware Workstation Pro. I select Continue.
Here, I enter the profile setup, where I will set up a user account for the VM. The information I input for this step can be found in the screenshot below (with the password censored, of course).

Selecting Skip for now on the Ubuntu Pro page and allowing the setup for the OpenSSH server (could come in handy later on), I select Done and am met with a Featured Server Snaps page where I can download multiple packages alongside the installation of the Ubuntu Server. Using the arrow keys to navigate the UI, I press spacebar to select docker and press Done. On the next page, when the install is complete, I select Cancel update and reboot. I unmount the installation media by clicking the Actions button in the top-right corner of the VM tab, going to Edit settings, going to the CD/DVD Drive 1 dropdown and de-selecting the Connect at power on box. I click SAVE, press Enter with the VM window in focus, and the VM boots up.

Now I can log in. I enter dockerroot as my login username, and entered my password that had previously been created in the server setup. Just like that, I'm logged into the Ubuntu Server VM!

I'll quickly run sudo apt-get update and sudo apt-get upgrade before I shut the VM down. I click the Actions button once more, hovering over Guest OS, and click Shut down. The VM window is still be open, so I click the x in the top-right corner of the VM window to hide it.
Now that I know the Ubuntu Server VM is in working condition, I'll take a snapshot of it.

Clicking the Actions button, I hover over Snapshots, click Take snapshot, enter "Clean Snapshot" for the name and "Ubuntu Server with Docker installed" for the description, and click TAKE SNAPSHOT. The snapshot only takes a few seconds to complete.

What are snapshots and why should they be used?
Snapshots are a way of saving the state of a virtual machine to be loaded back later when necessary.
Snapshots can be useful in the case that there is a misconfiguration that is made in the future that stops the virtual machine from working correctly, or if the virtual machine gets a virus. In either case, snapshots can be used to roll-back a previous state of the virtual machine that is not infected/is not misconfigured in any way. Please note that this also removes all data saved on the virtual machine after the snapshot had been taken (when loaded).

Step 6: Configuring Docker and Kubernetes

Firstly, I am going to be running the docker image hello-world to make sure that docker is working.
What is docker?
Docker is an application that can be used to containerize applications, like Portainer. Containerization is essentially a way to isolate the application/package being run, keeping all of the application files in that same container for not only simplicity, but also security and efficiency.
Kubernetes allows for management of containerized applications and clusters.

It appears that the installation of docker that I am running is working correctly, so I will install Kubernetes next.
For this next part, I'll be copy-pasting commands, so I'll want to open SSH in a terminal. I'll use PowerShell and enter the command ssh dockerroot@IP, where IP is the IP address of the Ubuntu Server VM. To get this information, I use the ip a command and locate the ens34 connection, and find where it says inet. Next to this, I can find the IP address of the Ubuntu Server VM.
After running the command in PowerShell, I am able to get into the Ubuntu Server VM (after accepting the SSH fingerprint).

What is SSH?
Secure Shell (or SSH for short) allows different systems to communicate securely through a terminal/CLI. With SSH, it is possible to send commands to a remote system without having physical access to it. In the case that I am using it in right now, it isn't 100% necessary, as I could copy the commands letter-for-letter inside the Ubuntu Server VM window, but SSH makes it a lot more convenient to copy/paste commands.

Now I'm going to start installing KUBECTL with the commands below. The first command will download KUBECTL, the second command will make it executable, the third command with move it to /usr/local/bin/kubectl, and the last command will check the version of KUBECTL to make sure it is installed.

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"

chmod +x ./kubectl

sudo mv ./kubectl /usr/local/bin/kubectl

kubectl version --client

Next, I'm going to install K3D, which will allow me to containerize any clusters that I create with docker. I completed the installation with the following command:

curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash

Step 7: Running Portainer

I'll be using this command to create the Portainer cluster:

sudo k3d cluster create portainer --api-port 6443 --servers 1 --agents 1 -p "30700-30800:30700-30800@server:0"

Now to create the namespace for Portainer and download the .yaml config file for Portainer, I'll use the commands below:

sudo kubectl create namespace portainer

sudo kubectl apply -n portainer -f https://raw.githubusercontent.com/portainer/k8s/master/deploy/manifests/portainer/portainer.yaml

Now, when I go to the IP of the Ubuntu Server VM on my browser (on port 30777), I am shown the setup login page for Portainer. I'll leave admin as my username, make a new password for Portainer, and click Create user.

Now that everything's fully set up, I have access to the Portainer management page where I can do things like create new environments, add users, etc.

Wrap-up

If you've made it this far, congratulations! I hope this helped you learn at least a bit about containers, virtual machines, or how Ubuntu servers can be utilized for services like Docker/Kubernetes. Thanks for reading, and have a great day!

DEV Community: ImmTB