DEV Community: Lennard John

Fixing Proxmox Terraform Deletes with curl + jq

Lennard John — Tue, 07 Apr 2026 20:53:52 +0000

When I started automating my homelab using Terraform + Proxmox, everything worked great… until it didn’t.

The Problem

In a push-based deployment (via GitHub Actions), I hit a blocker:

The Proxmox Terraform provider cannot delete a running VM/container.

That means:

Terraform tries to destroy a resource
Proxmox rejects it
Pipeline fails ❌

So the fix is simple in theory:

✅ Check if the VM is running → stop it → then delete it

But Terraform doesn’t handle this natively — so I built a small workaround using curl and jq.

What You Need

Before anything, install:

curl (for API calls) → https://curl.se/windows/
jq (for parsing JSON) → https://jqlang.org/download/

Proxmox API Access

You’ll need a Proxmox API token.

Docs:

Set your environment variables (Windows example):

setx TF_VAR_proxmox_api_url "https://YOUR-IP:8006/api2/json" setx TF_VAR_proxmox_api_token_id "user@pam!token" setx TF_VAR_proxmox_api_token_secret "your-secret"

Quick Env Variable Ref (Windows)

Temporary:

set MY_VAR=HelloWorld echo %MY_VAR%

Persistent:

setx MY_VAR "HelloWorld"

System-wide:

setx /m MY_VAR "HelloWorld"

The Fix (Core Idea)

We query the VM status using the Proxmox API:

curl -sk -H "Authorization: PVEAPIToken=USER!TOKEN=SECRET" \ https://IP:8006/api2/json/nodes/NODE/qemu/VMID/status/current

Then extract the state with jq:

jq -r '.data.status'

Putting It Together

Here’s the logic:

STATUS=$(curl -sk -H "$PVE_AUTH" \
"${PVE_API}/nodes/${PVE_NODE}/qemu/${VMID}/status/current" \
| jq -r '.data.status')

if [ "$STATUS" = "running" ]; then
  echo "Stopping VM $VMID..."
  curl -sk -X POST -H "$PVE_AUTH" \
  "${PVE_API}/nodes/${PVE_NODE}/qemu/${VMID}/status/stop"
fi

Now Terraform can safely destroy the VM after this runs.

I Built a Production Platform… Just to Write a Blog

Lennard John — Tue, 31 Mar 2026 09:24:21 +0000

I just wanted to write a blog.

That’s it.

No scaling requirements. No users. No revenue.
Just a place to write.

So naturally, I built a 3-node Kubernetes cluster on bare metal, automated everything with Terraform and Ansible, added GitOps, monitoring, alerting, and exposed it to the internet using a zero-trust Cloudflare tunnel.

Completely normal.

🤔 Why Over-Engineer Something So Simple?

Because the goal wasn’t the blog.

The goal was to learn how real platforms are built.

Not tutorials. Not “hello world” deployments.
But the actual systems that sit behind production environments.

So I treated this like a real platform:

Fully automated
Reproducible
Observable
Secure by design
⚙️ The One Command That Does Everything

Everything starts with:

docker compose up

That single command:

Provisions infrastructure (Terraform → Proxmox)
Configures the cluster (Ansible → Kubernetes)
Deploys platform services (Ingress, monitoring, GitOps)
Deploys applications (WordPress + MariaDB)
Sets up public access (Cloudflare Tunnel)

About 25 minutes later… the entire platform is live.

No manual steps. No SSH after bootstrap.

🏗️ How It’s Actually Built

At a high level:

Git → CI/CD → Terraform → VMs → Ansible → Kubernetes → Argo CD → Apps
Infrastructure (Terraform)
Creates 3 VMs on Proxmox (1 control plane, 2 workers)
Configures Cloudflare tunnel + DNS
Configuration (Ansible)
Bootstraps Kubernetes using kubeadm
Installs core services (Ingress, storage, networking)
Deploys monitoring and applications
GitOps (Argo CD)
Watches the kubernetes/ directory
Automatically syncs changes to the cluster

After the initial build, I don’t SSH into anything anymore.

Everything is managed through Git.

🔁 GitOps in Action

This is my favourite part.

I change a file in Git → push → and Argo CD updates the cluster automatically.

No kubectl. No manual deploys.

Just Git.

That shift — from “run commands” to “declare desired state” — completely changes how you think about systems.

🔐 Zero Trust (No Open Ports)

Nothing in my home network is exposed directly.

Instead, everything goes through a Cloudflare Tunnel:

User → Cloudflare Edge → Tunnel → Kubernetes Ingress → App
No port forwarding
No public IP exposure
HTTPS handled at the edge

It’s simple, secure, and surprisingly powerful.

📊 Observability (Because Things WILL Break)

I added:

Prometheus (metrics)
Grafana (dashboards)
AlertManager (email alerts)

If:

a pod crashes
a node goes down
memory spikes

I get notified.

Because a platform you can’t observe… is a platform you don’t understand.

💥 What Went Wrong (Important Part)

This didn’t work the first time.

Some of the issues I hit:

Terraform not returning VM IPs (guest agent problems)
Ansible running before infrastructure was ready
Kubernetes probes killing WordPress before it stabilised
Infinite HTTPS redirect loops behind Cloudflare
PVC-related deployment deadlocks

Most of these came down to one thing:

I assumed things would be ready… when they weren’t.

🧠 Biggest Lesson

Stop designing systems that depend on timing.

Start designing systems that converge.

Instead of:

“wait 30 seconds and hope”

Do:

retry until ready
check actual state
design for failure

That shift alone made everything more reliable.

🔁 Reproducibility (This Is the Real Win)

The entire platform can be:

Destroyed
Rebuilt
Moved

Using the same codebase.

That means:

No snowflake servers
No hidden config
No “it only works on my machine”
🎯 What This Project Actually Demonstrates

This isn’t about WordPress.

It demonstrates:

Infrastructure as Code (Terraform)
Configuration as Code (Ansible)
GitOps (Argo CD)
Zero Trust Networking (Cloudflare)
Observability-first design

In other words:

How modern platforms are actually built.

🚀 What’s Next

I’m planning to:

Move this to a Talos-based cluster
Add multi-environment support (dev/staging/prod)
Implement proper secrets management (SOPS or Vault)
Explore multi-cluster deployments
👋 Final Thoughts

Yes — this is overkill for a blog.

But that’s kind of the point.

If you can build something like this for a simple project…
you can build it for something that actually matters.

If you want to check it out:

🌐 https://lennardjohn.org
💻 https://github.com/Lennardj/homelab-blog

I’m Lennard — currently transitioning into DevOps / Platform Engineering.

If you’re on the same journey, feel free to connect 👇

LPIC-1 Lesson 101.1 cheat sheet

Lennard John — Thu, 16 Jan 2025 08:06:31 +0000

I know this might be easy for most, but as i've learnt something new while reviewing this, i am sure someone else will.

Lesson 101.1 summarized

Bios vs UEFI
- Bios (Basic Input Output system)
- UEFI (Unified extensible firmware interface)
- Bios is older, UEFI is newer and improved
Kernel module, similar to drivers on windows


 shows all devices currently connected to the PCI (Peripheral Component Interconnect) bus.
    common commands


```bash
lspci # list all available pci device 

lspci -s \[hex_address\] -v # show(verbose) more detail about a specific device based on Hex address given

lspci -s \[hex_address\] -k # verify which kernel module is in use based on Hex address given


 Lists USB (Universal Serial Bus) devices currently connected to the machine
    common command


```bash
lsusb # list usb devices 

lsusb -v -d \[ID\] # detailed output of a specific device based on ID

lsusb -t # show curent usb device mapping as a tree

lsusb -s bus:dev # verify which device is using that module e.g lsusb -s 01:02


 extracts detail information about hardware

-

 ```lsblk```

 list block device

-

 ```kmod```

 preferable way to interact with kernel module is also be call using these:

    -

 ```modprobe```

 used to add and remove module from the linux kernel 
    -

 ```lsmod```

 shows the status of module in the linux kernel 
    -

 ```rmmod```

 simple program to remove a module from linus kernel
    -

 ```insmod```

 simple program to insert a module to the linux kernel
    -

 ```modinfo```

 shows info about a kernel module
    -

 ```depmod```

 generate modules.deb and map files

- Example


```bash
sudo modprobe module_name # load a module
sudo modprobe -r module_name # remove a module
sudo rmmod module_name # remove a module
sudo rmmod -f module_name # force remove a module
sudo insmod /path/to.module.ko # load a module into kernel but doesn't handle dependencies 
modinfo module_name # display detail information about a kernel module
modinfo -p module_name # list specific module parameter
depmod -a kernel_version # generate for a specific module version

kernel module important files - /etc/modules add module here to load at boot (depreciated), can also - /etc/modprobe.d/ use instead of /etc/module, add module config file here to load at boot - Kernel module parameter: can be change while kernel is at boot - /etc/modprobe.conf used to customized module parameter to make them persistent - /etc/modprobe.d/ add individual files with the extension .conf to customize module parameters - /etc/modprobe.d/blacklist.conf add module name here to block the loading of the module - /etc/modprobe.d/<module_name>.conf preferred method for blocking a module that will contain settings specific only to the given kernel module.

pseudo-filesystems: only exist while the system is running, not intended for conventional file storage.
- /proc contains files with information regarding running processes and hardware resources. Common /proc files
  - /proc/cpuinfo: Lists detailed information about the CPU(s) found by the operating system.
  - /proc/interrupts: A list of numbers of the interrupts per IO device for each CPU.
  - /proc/ioports: Lists currently registered Input/Output port regions in use.
  - /proc/dma:Lists the registered DMA (direct memory access) channels in use.
- sysfs pseudo-filesystem which provides an interface to kernel data structures, he files under sysfs provide information about devices, kernel modules, filesystems, and other kernel components, which are all mounted in /sys
  - /sys have similar roles to those in /proc. However, the /sys directory has the specific purpose of storing device information and kernel data related to hardware
- u*dev subsystem*: Removable devices are handled by the udev subsystem, which creates the corresponding devices in /dev
  - /dev/ every file inside /dev is associated with a system device, particularly storage devices
  - /etc/udev/rules.d/ as new devices are detected, udev searches here for a matching rule about what to do
    - /dev/hdc cd/dvd are represented by these
    - /dev/fd0 represent floppy disk
    - /dev/sda represents IDE, SSD, and USB
    - /dev/mmcblk0p1 represents SD cards
    - /dev/nvme0n1p1 represents nvme
    - Linux kernel version 2.4, most storage devices are identified as if they were SCSI devices, regardless of their hardware type.
Linux Distro
- debian-based uses dpkg and use apt (Advanced Packaging Tool) as package manager - RPM-based uses rpm and use yum (Yellowdog Updater Modified) or DNF as package manager - debian-based common commands

sudo dpkg -i \[package.deb\] # install

sudo dpkg -r \[package\] # Remove

sudo dpkg -P \[package\] # purge: remove config file

sudo dpkg -c \[package.deb\] # list content

dpkg -s \[package\] # check status

sudo dpkg -R -i \[directory\] # install from directory

dpkg -I \[package.deb\] # show package info

sudo dpkg --unpack \[package.deb\] # unpack contents

 dpkg -l # list installed package

RPM-based common commands

sudo rpm -ivh \[package.rpm\] # insall 

sudo rpm -e \[package\] # remove

sudo rpm -Uvh \[package.rpm\] # upgrade

rpm -Vp \[package.rpm\] (specific) or rpm -Va (all) # verify

# query

rpm -qdf \[package\] # Documentation 

rpm -qi \[package\] # installed info

rpm -qip \[package.rpm\] # package info from online