DEV Community: Aurelia Peters

Setting Up The Home Lab: Setting up Kubernetes Using Ansible

Aurelia Peters — Thu, 08 Aug 2024 22:04:12 +0000

In my previous article I went over how to set up VMs in Proxmox VE using Terraform to deploy the VMs and Cloud-Init to provision them. In this article I'll discuss using Ansible playbooks to do further provisioning of VMs.

Since I want to play with Kubernetes anyway, I'll set up a k8s cluster. It will have 2 master and 4 worker nodes. Each VM will have 4 cores, 8 GB of RAM, a 32 GB root virtual disk, and a 250 GB data virtual disk for Longhorn volumes. I'll create an ansible user via cloud-init and allow access via SSH.

For the purposes of this article, I'm going to run Ansible separately, rather than from within Terraform. As soon as I figure out how to run the two together, I'll post a new article about that. XD

Anyway, let's get started. To begin with I'll add some configuration variables to my vars.tf.

variable "k8s_worker_pve_node" {
  description = "Proxmox node(s) to target"
  type = list(string)
  sensitive = false
  default = ["thebeast"]
}

variable "k8s_master_count" {
  description = "Number of k8s masters to create"
  default = 3 # I need an odd number of masters for etcd
}

variable "k8s_worker_count" {
  description = "Number of k8s workers to create"
  default = 3
}

variable "k8s_master_cores" {
  description = "Number of CPU cores for each k8s master"
  default = 4
}

variable "k8s_master_mem" {
  description = "Memory (in KB) to assign to each k8s master"
  default = 8192
}

variable "k8s_worker_cores" {
  description = "Number of CPU cores for each k8s worker"
  default = 4
}

variable "k8s_worker_mem" {
  description = "Memory (in KB) to assign to each k8s worker"
  default = 8192
}

variable "k8s_user" {
  description = "Used by Ansible"
  default = "ansible"
}

variable "k8s_nameserver" {
  default = "192.168.1.9"
}

variable "k8s_nameserver_domain" {
  default = "scurrilous.foo"
}

variable "k8s_gateway" {
  default = "192.168.1.1"
}

variable "k8s_master_ip_addresses" {
  type = list(string)
  default = ["192.168.1.80/24", "192.168.1.81/24", "192.168.1.82/24"]
}

variable "k8s_worker_ip_addresses" {
  type = list(string)
  default = ["192.168.1.90/24", "192.168.1.91/24", "192.168.1.92/24"]
}

variable "k8s_node_root_disk_size" {
  default = "32G"
}

variable "k8s_node_data_disk_size" {
  default = "250G"
}

variable "k8s_node_disk_storage" {
  default = "containers-and-vms"
}

variable "k8s_template_name" {
  default = "ubuntu-2404-base"
}

Next I'll set up my k8s master and worker nodes in Terraform.

resource "proxmox_vm_qemu" "k8s_master" {
    count = var.k8s_master_count
    name = "k8s-master-${count.index}"
    desc = "K8S Master Node"
    ipconfig0 = "gw=${var.k8s_gateway},ip=${var.k8s_master_ip_addresses[count.index]}"
    target_node = var.k8s_pve_node
    onboot = true
    clone = var.k8s_template_name
    agent = 1
    ciuser = var.k8s_user
    memory = var.k8s_master_mem
    cores = var.k8s_master_cores
    nameserver = var.k8s_nameserver
    os_type = "cloud-init"
    cpu = "host"
    scsihw = "virtio-scsi-single"
    tags="k8s,ubuntu,k8s_master"

    # Setup the disk
    disks {
        ide {
            ide2 {
                cloudinit {
                    storage = "containers-and-vms"
                }
            }
        }
        scsi {
            scsi0 {
                disk {
                  size     = var.k8s_node_root_disk_size
                  storage  = var.k8s_node_disk_storage
                  discard  = true
                  iothread = true
                }
            }
            scsi1 {
                disk {
                  size     = var.k8s_node_data_disk_size
                  storage  = var.k8s_node_disk_storage
                  discard  = true
                  iothread = true
                }
            }
        }
    }

    network {
        model = "virtio"
        bridge = var.nic_name
          tag = -1
    }

    # Setup the ip address using cloud-init.
    boot = "order=scsi0"
    skip_ipv6 = true

    lifecycle {
      ignore_changes = [
        disks,
        target_node,
        sshkeys,
        network
      ]
    }
}

resource "proxmox_vm_qemu" "k8s_workers" {
    count = var.k8s_worker_count
    name = "k8s-worker-${count.index}"
    desc = "K8S Master Node"
    ipconfig0 = "gw=${var.k8s_gateway},ip=${var.k8s_worker_ip_addresses[count.index]}"
    target_node = var.k8s_pve_node
    onboot = true
    clone = var.k8s_template_name
    agent = 1
    ciuser = var.k8s_user
    memory = var.k8s_worker_mem
    cores = var.k8s_worker_cores
    nameserver = var.k8s_nameserver
    os_type = "cloud-init"
    cpu = "host"
    scsihw = "virtio-scsi-single"
    sshkeys = file("${path.module}/files/${var.k8s_ssh_key_file}")
    tags="k8s,ubuntu,k8s_worker"

    # Setup the disk
    disks {
        ide {
            ide2 {
                cloudinit {
                    storage = "containers-and-vms"
                }
            }
        }
        scsi {
            scsi0 {
                disk {
                  size     = var.k8s_node_root_disk_size
                  storage  = var.k8s_node_disk_storage
                  discard  = true
                  iothread = true
                }
            }
            scsi1 {
                disk {
                  size     = var.k8s_node_data_disk_size
                  storage  = var.k8s_node_disk_storage
                  discard  = true
                  iothread = true
                }
            }
        }
    }

    network {
        model = "virtio"
        bridge = var.nic_name
    tag = -1
    }

    # Setup the ip address using cloud-init.
    boot = "order=scsi0"
    skip_ipv6 = true

    lifecycle {
      ignore_changes = [
        disks,
        target_node,
        sshkeys,
        network
      ]
    }
}

A quick terraform apply and I have all of my VMs set up. Next, since I've already installed Ansible on my local machine, I'll set up Kubernetes using Kubespray following Pradeep Kumar's excellent tutorial.

First I set up an inventory file, like so:

all:
  hosts:
    k8s-master-0:
    k8s-master-1:
    k8s-master-2:
    k8s-worker-0:
    k8s-worker-1:
    k8s-worker-2:
  vars:
    ansible_user: ansible
    ansible_python_interpreter: /usr/bin/python3
  children:
    kube_control_plane:
      hosts:
        k8s-master-0:
        k8s-master-1:
        k8s-master-2:
    kube_node:
      hosts:
        k8s-worker-0:
        k8s-worker-1:
        k8s-worker-2:
    etcd:
      hosts:
        k8s-master-0:
        k8s-master-1:
        k8s-master-2:
    k8s_cluster:
      children:
        kube_control_plane:
        kube_node:
    calico_rr:
      hosts: {}

Note here that I've also added DNS entries to my local nameserver for these hosts. I could also have used IP addresses instead of the hosts. In a later revision of this configuration, I'll try setting up resource discovery via the Proxmox inventory source for Ansible, but for now I'll hardcode things.

Note also that I've set the ansible_user variable in this inventory. That's important to make sure that Ansible uses the service account that I already set up in Terraform. I've also set the location of the Ansible Python interpreter (via the ansible_python_interpreter variable) so that I don't get bombarded with warnings from Ansible about using the discovered Python interpreter.

So now that I've got my hosts deployed, it's time to set up Kubernetes. I've cloned the Kubespray GitHub repo and now I'll run the cluster.yml playbook:

ansible-playbook -i ../ansible/inventory/k8s-cluster/hosts.yml --become --become-user=root cluster.yml

After some time (I think it took a good half hour, all told) Kubernetes is installed, ready for me to deploy my applications.

So now I have a working k8s installation on my home lab, but there were several steps involved in getting it set up. It sure would be nice if I could deploy and provision everything in one fell swoop. I'll discuss that next time. I'd also like to not have to SSH into one of my master nodes in order to run kubectl.

Setting Up The Home Lab: Terraform and Cloud-Init

Aurelia Peters — Wed, 31 Jul 2024 23:10:19 +0000

In my last article I talked about getting Terraform set up on Proxmox VE. In this article I want to talk about how I got Cloud-Init set up to use with my Terraform templates.

To begin with, I needed a cloud-init base VM. While I could use the cloud image that Ubuntu provides, I found a nifty article that shows you how to roll your own base image.

NOTE: The Proxmox VE cloud-init documentation suggests adding a serial console next. I have found that not to be necessary with the Ubuntu cloud image, so I'm not going to do it.

Now that we've got the base template set up (turns out I was mistaken in my last post when I said it needed to be a VM and not a template) let's set up an actual VM. I'll have a single virtual Ethernet interface that gets its IP address via DHCP, 32 GB of virtual disk, 2048 GB of RAM, and 2 processor cores.

Note here that I've broken my Terraform config into several files to make it more manageable. As long as all of the Terraform files (i.e. the ones ending in .tf or .tfvars) are in the same directory, Terraform will process them in the same way as if they were one big file.

# provider.tf - This is where I define my providers

terraform {
  required_providers {
    proxmox = {
      source = "telmate/proxmox"
      #latest version as of 16 July 2024
      version = "3.0.1-rc3"
    }
  }
}

provider "proxmox" {
  # References our vars.tf file to plug in the api_url
  pm_api_url = "https://${var.proxmox_host}:8006/api2/json"
  # Provided in a file, secrets.tfvars containing secret terraform variables
  pm_api_token_id = var.token_id 
  # Also provided in secrets.tfvars
  pm_api_token_secret = var.token_secret
  # Defined in vars.tf
  pm_tls_insecure = var.pm_tls_insecure
  pm_log_enable = true
  # this is useful for logging what Terraform is doing
  pm_log_file   = "terraform-plugin-proxmox.log"
  pm_log_levels = {
    _default    = "debug"
    _capturelog = ""
  }
}

# cloud-init.tf - This is where I store cloud-init configuration

# Source the Cloud Init Config file. NB: This file should be located 
# in the "files" directory under the directory you have your Terraform
# files in.
data "template_file" "cloud_init_test1" { 
  template  = "${file("${path.module}/files/test1.cloud_config")}"

  vars = {
    ssh_key = file("~/.ssh/id_ed25519.pub")
    hostname = var.vm_name
    domain = "scurrilous.foo"
  }
}

# Create a local copy of the file, to transfer to Proxmox.
resource "local_file" "cloud_init_test1" {
  content   = data.template_file.cloud_init_test1.rendered
  filename  = "${path.module}/files/user_data_cloud_init_test1.cfg"
}

# Transfer the file to the Proxmox Host
resource "null_resource" "cloud_init_test1" {
  connection {
    type    = "ssh"
    user    = "root"
    private_key = file("~/.ssh/id_ed25519")
    host    = var.proxmox_host
  }

  provisioner "file" {
    source       = local_file.cloud_init_test1.filename
    destination  = "/var/lib/vz/snippets/cloud_init_test1.yml"
  }
}

# main.tf - This is where I define the VMs I want to deploy with Terraform

resource "proxmox_vm_qemu" "cloudinit-test" {
    name = var.vm_name
    desc = "Testing Terraform and cloud-init"
    depends_on = [ null_resource.cloud_init_test1 ]
    # Node name has to be the same name as within the cluster
    # this might not include the FQDN
    target_node = var.proxmox_host

    # The template name to clone this vm from
    clone = var.template_name
    # Activate QEMU agent for this VM
    agent = 1

    os_type = "cloud-init"
    cores = 2
    sockets = 1
    vcpus = 0
    cpu = "host"
    memory = 2048
    scsihw = "virtio-scsi-single"

    # Setup the disk
    disks {
        ide {
            ide2 {
                cloudinit {
                    storage = "containers-and-vms"
                }
            }
        }
        scsi {
            scsi0 {
                disk {
                  size     = "32G"
                  storage  = "containers-and-vms"
                  discard  = true
                  iothread = true
                }
            }
        }
    }

    network {
        model = "virtio"
        bridge = var.nic_name
    tag = -1
    }

    # Setup the ip address using cloud-init.
    boot = "order=scsi0"
    # Keep in mind to use the CIDR notation for the ip.
    ipconfig0 = "ip=192.168.1.80/24,gw=192.168.1.1,ip6=dhcp"
    skip_ipv6 = true

    lifecycle {
      ignore_changes = [
        ciuser,
        sshkeys,
        network
      ]
    }
    cicustom = "user=local:snippets/cloud_init_test1.yml"
}

In addition to the Terraform files, we also need the cloud-config file (cloud_init_test1.yml) that we're referencing in main.tf.

IMPORTANT If you specify a value for cicustom as I did here, the ciuser and sshkeys fields in the template definition (e.g. main.tf) are ignored in favor of whatever is in the cloud-config file, even when nothing is there. This also trumps whatever is in the base template. You must specify your SSH keys in your cloud-config file.

#cloud-config

ssh_authorized_keys:
  - <ssh public key 1>
  - <ssh public key 2>

runcmd:
  - apt-get update
  - apt-get install -y nginx

write_files:
  - content: |
      #!/bin/bash
      echo "ZOMBIES RULE BELGIUM?"
    path: /usr/local/bin/my-script
    permissions: '0755'

scripts-user:
  - /usr/local/bin/my-script

So you can see here that you can run arbitrary commands at first boot with runcmd, and you can also run a custom Bash script with scripts-user and write-files. (See this writeup from SaturnCloud for more information).

You might notice that the Terraform template definition is pretty close in structure to the one I used in my last article. That's intentional - I set up the last one with cloud-init, but didn't do much with it. This one actually provisions the VM with cloud-init. You can also use Ansible playbooks to provision a VM, and I might talk about that in a future post, but in my next post I'm going to talk about doing something actually useful in my home infrastructure and setting up Plex.

Once again, we execute terraform plan. The plan looks good, so we apply it with terraform apply, wait a couple of minutes, and boom! We've got ourselves a VM with both cloud-init and QEMU Guest Agent. Pretty cool! Next time I'll show you how to use Ansible playbooks to provision your VMs

Setting up the home lab: Terraform

Aurelia Peters — Wed, 17 Jul 2024 16:11:28 +0000

When I worked for CBS, I discovered Terraform, which is tool that allows you do define infrastructure as code ("IaC"). I just recently purchased a home lab server (the details of how I have that set up will be discussed in a future article) and installed Proxmox VE on it. I'd like to improve my DevOps skills, so I thought I'd play with Terraform for my home lab. Eventually I'd like to get my entire home infrastructure represented as Terraform templates.

To get Terraform set up on my home lab server (which I call "The Beast" XD) I'll be following the excellent tutorial given by Tanner Cude. I'll be reproducing a lot of it here just in case that site goes down. I'll also update the templates Tanner provides for the current version (v3.0.1-rc3) of the Telmate proxmox provider

Note that I'll be doing the local machine portion on my desktop machine, which runs Ubuntu Linux. Tanner's instructions are for a MacBook, but things are pretty similar.

To begin with, I've installed Terraform on my desktop machine following the installation instructions provided by Hashicorp.

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

I then SSH'd into my home lab server and set up a role for the Terraform worker to assume:

pveum role add terraform-role -privs "Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Migrate VM.Monitor VM.PowerMgmt SDN.Use"

You can also do this via the web UI, but it's easier (IMO) to do via the command line.

Next, and again on my home lab server, I'll create the terraform user, associate it with the role I created above, and get an authentication token:

pveum user add terraform@pve 
pveum aclmod / -user terraform@pve -role terraform-role
pveum user token add terraform@pve terraform-token --privsep=0

And I get a response similar to the below. If you're following along, make sure to save the access token (the UUID in the last line of the response); you won't be able to retrieve it later.

┌──────────────┬──────────────────────────────────────┐
│ key          │ value                                │
╞══════════════╪══════════════════════════════════════╡
│ full-tokenid │ terraform@pve!terraform-token        │
├──────────────┼──────────────────────────────────────┤
│ info         │ {"privsep":"0"}                      │
├──────────────┼──────────────────────────────────────┤
│ value        │ xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx │
└──────────────┴──────────────────────────────────────┘

Now that the basics are set up on Proxmox, it's time to go back to my local machine.

I created a Git repo for my Terraform code called home_infra and added a .gitignore file so that secrets like the token ID and access token are only stored locally. Later on I might store them as repository secrets, but I'll have to see how that works with the Terraform command line. For now they're fine in a .tfvars file, and if I want to work on my infrastructure from another machine, I can always SSH into my desktop from there or copy the file.

Now I'm going to set up the public Terraform vars that I'll use when spinning up a VM. I'll write them to a file called vars.tf:

# TF vars for spinning up VMs

# Set your public SSH key here
variable "ssh_key" {
  default = "ssh-ed25519 <My SSH Public Key> aurelia@desktop"
}
#Establish which Proxmox host you'd like to spin a VM up on
variable "proxmox_host" {
    default = "thebeast" 
}
#Specify which template name you'd like to use
variable "template_name" {
    default = "ubuntu-2404-template2"
}
#Establish which nic you would like to utilize
variable "nic_name" {
    default = "vmbr0"
}

# I don't have VLANs set up
# #Establish the VLAN you'd like to use 
# variable "vlan_num" {
#     default = "place_vlan_number_here"
# }
#Provide the url of the host you would like the API to communicate on.
#It is safe to default to setting this as the URL for what you used
#as your `proxmox_host`, although they can be different
variable "api_url" {
    default = "https://thebeast.scurrilous.foo:8006/api2/json"
}
#Blank var for use by terraform.tfvars
variable "token_secret" {
}
#Blank var for use by terraform.tfvars
variable "token_id" {
}

Finally, I'll create main.tf to define my infrastructure, and I'll put two test VMs in just for grins. Note how this references the variables in both vars.tf and terraform.tfvars.

terraform {
  required_providers {
    proxmox = {
      source = "telmate/proxmox"
      #latest version as of 16 July 2024
      version = "3.0.1-rc3"
    }
  }
}

provider "proxmox" {
  # References our vars.tf file to plug in the api_url 
  pm_api_url = var.api_url
  # References our secrets.tfvars file to plug in our token_id
  pm_api_token_id = var.token_id
  # References our secrets.tfvars to plug in our token_secret 
  pm_api_token_secret = var.token_secret
  # Default to `true` unless you have TLS working within your pve setup 
  pm_tls_insecure = true
}

resource "proxmox_vm_qemu" "cloudinit-test1" {
    name = "terraform-test-vm"
    desc = "A test for using terraform and cloudinit"

    # Node name has to be the same name as within the cluster
    # this might not include the FQDN
    target_node = var.proxmox_host

    # The template name to clone this vm from
    clone = var.template_name

    # Activate QEMU agent for this VM
    agent = 1

    os_type = "cloud-init"
    cores = 2
    sockets = 1
    vcpus = 0
    cpu = "host"
    memory = 2048
    scsihw = "virtio-scsi-single"

    # Setup the disk
    disks {
        ide {
            ide3 {
                cloudinit {
                    storage = "local-lvm"
                }
            }
        }
        virtio {
          virtio0 {
            disk {
              size = "32G"
              storage = "containers-and-vms"
              discard = true
              iothread        = true
              # Can't emulate SSDs in virtio
            }
          }
        }
    }

    network {
      model = "virtio"
      bridge = var.nic_name
    }

    # Setup the ip address using cloud-init.
    boot = "order=virtio0"
    # Keep in mind to use the CIDR notation for the ip.
    ipconfig0 = "ip=dhcp,ip6=dhcp"
    skip_ipv6 = true
    sshkeys = var.ssh_key
}
resource "proxmox_vm_qemu" "cloudinit-test2" {
    name = "terraform-test-vm-2"
    desc = "A test for using terraform and cloudinit"

    # Node name has to be the same name as within the cluster
    # this might not include the FQDN
    target_node = var.proxmox_host

    # The template name to clone this vm from
    clone = var.template_name

    # Activate QEMU agent for this VM
    agent = 1

    os_type = "cloud-init"
    cores = 2
    sockets = 1
    vcpus = 0
    cpu = "host"
    memory = 2048
    scsihw = "virtio-scsi-single"

    # Setup the disk
    disks {
        ide {
            ide3 {
                cloudinit {
                    storage = "local-lvm"
                }
            }
        }
        virtio {
          virtio0 {
            disk {
              size = "32G"
              storage = "containers-and-vms"
              discard = true
              iothread        = true
              # Can't emulate SSDs in virtio
            }
          }
        }
    }

    network {
      model = "virtio"
      bridge = var.nic_name
    }

    # Setup the ip address using cloud-init.
    boot = "order=virtio0"
    # Keep in mind to use the CIDR notation for the ip.
    ipconfig0 = "ip=dhcp,ip6=dhcp"
    skip_ipv6 = true

    sshkeys = var.ssh_key
}

Now that I've got my main Terraform template, I can initialize Terraform and see if this works. Go-go terraform init!

aurelia@desktop:~/work/home_infra$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding telmate/proxmox versions matching "3.0.1-rc3"...
- Installing telmate/proxmox v3.0.1-rc3...
- Installed telmate/proxmox v3.0.1-rc3 (self-signed, key ID A9EBBE091B35AFCE)
Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Next we'll run terraform plan to see if we've got our template right:

aurelia@desktop:~/work/home_infra$ terraform plan -out plan.out
proxmox_vm_qemu.cloudinit-test2: Refreshing state... [id=thebeast/qemu/100]
proxmox_vm_qemu.cloudinit-test1: Refreshing state... [id=thebeast/qemu/101]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # proxmox_vm_qemu.cloudinit-test1 will be created
  + resource "proxmox_vm_qemu" "cloudinit-test1" {
      + additional_wait        = 5
      + agent                  = 1
      + automatic_reboot       = true
      + balloon                = 0
      + bios                   = "seabios"
      + boot                   = "order=virtio0"
      + bootdisk               = (known after apply)
      + clone                  = "ubuntu-2404-template2"
      + clone_wait             = 10
      + cores                  = 2
      + cpu                    = "host"
      + default_ipv4_address   = (known after apply)
      + default_ipv6_address   = (known after apply)
      + define_connection_info = true
      + desc                   = "A test for using terraform and cloudinit"
      + force_create           = false
      + full_clone             = true
      + hotplug                = "network,disk,usb"
      + id                     = (known after apply)
      + ipconfig0              = "ip=dhcp,ip6=dhcp"
      + kvm                    = true
      + linked_vmid            = (known after apply)
      + memory                 = 2048
      + name                   = "terraform-test-vm"
      + nameserver             = (known after apply)
      + onboot                 = false
      + os_type                = "cloud-init"
      + protection             = false
      + reboot_required        = (known after apply)
      + scsihw                 = "virtio-scsi-single"
      + searchdomain           = (known after apply)
      + skip_ipv4              = false
      + skip_ipv6              = true
      + sockets                = 1
      + ssh_host               = (known after apply)
      + ssh_port               = (known after apply)
      + sshkeys                = "ssh-ed25519 <My SSH Public Key> aurelia@desktop"
      + tablet                 = true
      + tags                   = (known after apply)
      + target_node            = "thebeast"
      + unused_disk            = (known after apply)
      + vcpus                  = 0
      + vm_state               = "running"
      + vmid                   = (known after apply)

      + disks {
          + ide {
              + ide3 {
                  + cloudinit {
                      + storage = "local-lvm"
                    }
                }
            }
          + virtio {
              + virtio0 {
                  + disk {
                      + backup               = true
                      + discard              = true
                      + format               = "raw"
                      + id                   = (known after apply)
                      + iops_r_burst         = 0
                      + iops_r_burst_length  = 0
                      + iops_r_concurrent    = 0
                      + iops_wr_burst        = 0
                      + iops_wr_burst_length = 0
                      + iops_wr_concurrent   = 0
                      + iothread             = true
                      + linked_disk_id       = (known after apply)
                      + mbps_r_burst         = 0
                      + mbps_r_concurrent    = 0
                      + mbps_wr_burst        = 0
                      + mbps_wr_concurrent   = 0
                      + size                 = "32G"
                      + storage              = "containers-and-vms"
                    }
                }
            }
        }

      + network {
          + bridge    = "vmbr0"
          + firewall  = false
          + link_down = false
          + macaddr   = (known after apply)
          + model     = "virtio"
          + queues    = (known after apply)
          + rate      = (known after apply)
          + tag       = -1
        }

      + smbios (known after apply)
    }

  # proxmox_vm_qemu.cloudinit-test2 will be created
  + resource "proxmox_vm_qemu" "cloudinit-test2" {
      + additional_wait        = 5
      + agent                  = 1
      + automatic_reboot       = true
      + balloon                = 0
      + bios                   = "seabios"
      + boot                   = "order=virtio0"
      + bootdisk               = (known after apply)
      + clone                  = "ubuntu-2404-template2"
      + clone_wait             = 10
      + cores                  = 2
      + cpu                    = "host"
      + default_ipv4_address   = (known after apply)
      + default_ipv6_address   = (known after apply)
      + define_connection_info = true
      + desc                   = "A test for using terraform and cloudinit"
      + force_create           = false
      + full_clone             = true
      + hotplug                = "network,disk,usb"
      + id                     = (known after apply)
      + ipconfig0              = "ip=dhcp,ip6=dhcp"
      + kvm                    = true
      + linked_vmid            = (known after apply)
      + memory                 = 2048
      + name                   = "terraform-test-vm-2"
      + nameserver             = (known after apply)
      + onboot                 = false
      + os_type                = "cloud-init"
      + protection             = false
      + reboot_required        = (known after apply)
      + scsihw                 = "virtio-scsi-single"
      + searchdomain           = (known after apply)
      + skip_ipv4              = false
      + skip_ipv6              = true
      + sockets                = 1
      + ssh_host               = (known after apply)
      + ssh_port               = (known after apply)
      + sshkeys                = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJDjuXJsm20610XQGaGgsagEupVlfzYMorJXrNo1u7Gx took@oscar"
      + tablet                 = true
      + tags                   = (known after apply)
      + target_node            = "thebeast"
      + unused_disk            = (known after apply)
      + vcpus                  = 0
      + vm_state               = "running"
      + vmid                   = (known after apply)

      + disks {
          + ide {
              + ide3 {
                  + cloudinit {
                      + storage = "local-lvm"
                    }
                }
            }
          + virtio {
              + virtio0 {
                  + disk {
                      + backup               = true
                      + discard              = true
                      + format               = "raw"
                      + id                   = (known after apply)
                      + iops_r_burst         = 0
                      + iops_r_burst_length  = 0
                      + iops_r_concurrent    = 0
                      + iops_wr_burst        = 0
                      + iops_wr_burst_length = 0
                      + iops_wr_concurrent   = 0
                      + iothread             = true
                      + linked_disk_id       = (known after apply)
                      + mbps_r_burst         = 0
                      + mbps_r_concurrent    = 0
                      + mbps_wr_burst        = 0
                      + mbps_wr_concurrent   = 0
                      + size                 = "32G"
                      + storage              = "containers-and-vms"
                    }
                }
            }
        }

      + network {
          + bridge    = "vmbr0"
          + firewall  = false
          + link_down = false
          + macaddr   = (known after apply)
          + model     = "virtio"
          + queues    = (known after apply)
          + rate      = (known after apply)
          + tag       = -1
        }

      + smbios (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.

The plan looks good. Let's apply it!

aurelia@desktop:~/work/home_infra$ terraform apply plan.out
proxmox_vm_qemu.cloudinit-test1: Creating...
proxmox_vm_qemu.cloudinit-test2: Creating...
proxmox_vm_qemu.cloudinit-test1: Still creating... [10s elapsed]
proxmox_vm_qemu.cloudinit-test2: Still creating... [10s elapsed]
proxmox_vm_qemu.cloudinit-test1: Still creating... [20s elapsed]
proxmox_vm_qemu.cloudinit-test2: Still creating... [20s elapsed]
proxmox_vm_qemu.cloudinit-test1: Still creating... [30s elapsed]
proxmox_vm_qemu.cloudinit-test2: Still creating... [30s elapsed]
proxmox_vm_qemu.cloudinit-test1: Still creating... [40s elapsed]
proxmox_vm_qemu.cloudinit-test2: Still creating... [40s elapsed]
proxmox_vm_qemu.cloudinit-test1: Still creating... [50s elapsed]
proxmox_vm_qemu.cloudinit-test2: Still creating... [50s elapsed]
proxmox_vm_qemu.cloudinit-test2: Creation complete after 51s [id=thebeast/qemu/101]
proxmox_vm_qemu.cloudinit-test1: Creation complete after 55s [id=thebeast/qemu/100]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

And sure enough, if I go into Proxmox, I can see both VMs:

Some notes on the process: I had to set up the template VM (ubuntu-2404-template2) before I could clone any VMs from it. And this VM needs to be, with regard to Proxmox, a full VM and not a (Proxmox) template. Also, make sure the VMs you clone from this template VM via Terraform use the same block driver (i.e. scsi, ide, or virtio) for the boot drive as the template VM. I initially created a template VM with a SCSI boot drive and tried to create a clone in Terraform using Virtio, and the cloned VM would just endlessly reboot with the message "No valid boot device" in the console. This may be my inexperience, however, and as always, YMMV.

And there you have it! In my next article I'll show you how to provision your VMs using Cloud-Init

Getting started with Kubernetes: Introduction

Aurelia Peters — Thu, 30 May 2024 02:00:18 +0000

I've been hearing a lot about Kubernetes of late, so I figured I ought to learn it.

To begin with, I don't have a home lab as such, so I'll run everything through minikube. I may switch things up later to get a more "realistic" k8s install, but minikube will be fine for a start. I'm also starting with the tutorials provided by the Kubernetes developers.

I've set up a GitHub repository where you can follow along with my progress.

As the series goes on, feel free to leave any questions or comments you might have.

The NgRx Component Store is neat!

Aurelia Peters — Mon, 18 Dec 2023 21:10:41 +0000

See this project on Stackblitz

Lately I've been messing around with the NgRx Component Store, which is a sort of stripped down version of their Store software. The central idea is that your Angular application (or in my case, component) has a central data store class that manages the application / component state. What is "state" in this context? It's all the data that your application / component works with. The larger application that I'm currently working on uses worker processes to handle certain long-running jobs, so in my case, the state of my component is the execution status of those jobs, as reported by an HTTP backend.

A job status record has the following fields:

worker - the name of the worker that is recording this status entry. For example, if you had a worker that was transcoding a video stream, you might call it "transcode"
job - The name of the job the worker is executing. Using the previous example of transcoding a video stream, you might call it "home-movie-1994-11-17.mpg".
jobType - This used to give more information about the job. Using the previous example, you might use "mpg_to_mp4"
jobStatus - This is used to record the status of the job. For the previous example, you might could put a completion percentage here. You could also use something like "Received", "In progress", "Complete" for jobs that aren't easily broken down into percent complete.
error - This is an optional field, only populated when something's gone wrong with the job. It will contain a (hopefully useful) error message.

To display these job status records, let's put them in a table.
Before I learned about NgRx, I would have gone about writing this component by first writing a service class that mediated the interaction with the backend. I then would have generated a component that interacted with the service class and fed data to the HTML template. If I wanted to be able to filter my results, I would have added form fields to the HTML template and added listeners on each form field's onChange event. The component class would be fairly large in terms of lines of code with all these listeners and whatnot, and I'd have to do some jiggery-pokery with RxJs to get everything plumbed up properly. I would also have had to manage the Subscriptions for each form field to make sure they all got unsubscribed whenever the component was destroyed. And that's all stuff I've done before, but it's kind of a lot.

With NgRx, however, this all becomes much easier. I create the service class as before, but then I create a component store. This component store is going to manage all of the data for my component. If I want the data laid out a certain way (in my case, I wanted a unique list of worker names, job IDs, job types, and job statuses), the code to do that will be in the component store.

The most important thing in this class is the definition of WorkerJobStatusState:

export interface WorkerJobStatusState {
  statuses: WorkerJobStatus[];
  worker: string[];
  job: string[];
  jobType: string[];
  jobStatus: string[];
}

This encapsulates the entire state of the component at any given time. Notice how it doesn't just have the list of status records. The values of each of the filters are part of the state as well.

The real beauty of a component store, I think, comes in the selectors, which are pure functions providing a custom view of the component state. They can be composed together, as shown in the selectedStatuses$ selector, below:

readonly selectedStatuses$ = this.select(
    this.statuses$,
    this.selectedWorker$,
    this.selectedJob$,
    this.selectedJobType$,
    this.selectedJobStatus$,
    (
      statuses,
      selectedWorkers,
      selectedJobs,
      selectedJobTypes,
      selectedJobStatuses
    ) =>
      statuses.filter(
        (status) =>
          (selectedWorkers.length
            ? selectedWorkers.includes(status.worker)
            : true) &&
          (selectedJobs.length ? selectedJobs.includes(status.job) : true) &&
          (status.jobType && selectedJobTypes.length
            ? selectedJobTypes.includes(status.jobType)
            : true) &&
          (selectedJobStatuses.length
            ? selectedJobStatuses.includes(status.jobStatus)
            : true)
      )
  );

20 lines of code handles everything around the status records. We take in the list of statuses provided by the backend service and the values of the filters and we return a list of matching statuses.

Now let's look at the component class for the status list. Notice how the only external class it exchanges data with is the store, and notice how it doesn't do any massaging / modification / filtering of the data the store provides. All of that is handled by the store.

We need to connect the filter form fields to the store, and we do that with these updater methods, which are methods provided by the component store allowing the caller to update the state.

    this._workerJobStatusStore.workersSelected(
      this.selectedWorker.valueChanges
    );
    this._workerJobStatusStore.jobsSelected(this.selectedJob.valueChanges);
    this._workerJobStatusStore.jobTypesSelected(
      this.selectedJobType.valueChanges
    );
    this._workerJobStatusStore.jobStatusesSelected(
      this.selectedJobStatus.valueChanges
    );

Note that all we have to do in the way of plumbing is to pass the valueChanges Observable from each of the filtering form fields to the updater functions. NgRx will handle everything else for us, including unsubscribing when the component that instantiated the store is destroyed.

Now, if you look at the component code, you will notice that it doesn't do anything to connect the component store to the backend service. The component has no idea there even is a backend service, and that's intentional. The worker job status list component is intended to be purely presentational. The data store is provided by dependency injection, but how, you may ask, does the store get connected to the backend service? That gets handled in the container component:

The reason I use a container component here is so that I have a single module that provides the data store and populates the state. If I want to add a status detail component later, one that maybe would provide the logs for each job, I could add another presentational component to do that, but the data store would still be provided by the container.

OK, so I've added a lot of extra abstraction to what could have been a fairly simple component. Why? What good will this do me? I'll tell you:

Testability - by abstracting the data interaction into the data store class, I can test those methods without having to worry about setting up the component in my test harness or mocking the backend service
Maintainability - By centralizing common functionality into a single class, I don't have to (for example) populate the data store for every component.
Extensibility - It's easy to add a new component that uses the same data store.

And I think it makes the code more readable. The job status list component only handles displaying the list. The container component only handles populating the data store. The data store class only handles viewing / modifying the state. Sure, this was a fairly simple example, but for more complex components, this could be invaluable.

So what do you think? Have you worked with component store / NgRx at all? What were your experiences like?