Deploying Ansible AWX on Kubernetes Using Helm

In this blog, I'm going to walk you through how I deployed Ansible AWX on a Kubernetes cluster using Helm. This was one of the most hands-on projects I've worked on — it involves setting up the K8s cluster from scratch, installing the container runtime, deploying Helm, and finally getting AWX up and running. I also hit a few real errors along the way, so I'll share exactly how I fixed them.
It was one of those projects that looks simple on paper but turns into a chain of small problems once you actually start doing it.
Here's the overall flow I followed:
The goal was pretty straightforward:

• Set up a Kubernetes cluster on Ubuntu 22.04
• Install containerd
• Get Helm working
• Deploy AWX using the AWX Operator
• Fix whatever broke along the way (this part took the longest 😅)

Setting Up the Kubernetes Cluster on Ubuntu 22.04
Before anything else, I want to quickly explain the two types of nodes in a K8s cluster since this matters for the setup:
Master Node — manages the control plane, API calls, pods, services, and everything else in the cluster.
Worker Node — runs the actual containers. Pods can spread across multiple worker nodes for better resource management.
Step 1 — Update and Upgrade (All Nodes)
I started by logging in as root and running updates on all nodes:

hostnamectl set-hostname siri-awx-01 # run on master node
hostnamectl set-hostname siri-awx-02 # run on worker node
exec bash

apt update
apt upgrade

Step 2 — Disable Swap
swapoff -a
sed -i '/ swap / s/^(.)$/#\1/g' /etc/fstab
**Step 3 — Load Kernel Modules*
tee /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

Step 4 — Configure Kernel Parameters
tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sysctl --system

Installing Containerd Runtime (All Nodes)
apt install -y curl gnupg2 software-properties-common apt-transport-https ca-certificates

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/docker.gpg

add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

apt update
apt install -y containerd.io

containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

systemctl restart containerd
systemctl enable containerd

Installing Kubernetes Components (All Nodes)
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Initialising the Cluster (Master Node — siri-awx-01 Only)

systemctl enable kubelet && systemctl start kubelet
kubeadm init

$:mkdir -p $HOME/.kube
$:cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$:chown $(id -u):$(id -g) $HOME/.kube/config

Tip: If you forget to save the kubeadm join output, regenerate it with:
bashkubeadm token create --print-join-command

$kubectl get nodes

Join Worker Node (siri-awx-02)
Run this on siri-awx-02 (use your actual token from kubeadm init output):

kubeadm join 192.168.237.144:6443 --token uhkvxh.mjp6aiyhy18vaxh0 \
--discovery-token-ca-cert-hash sha256:c0200907cc68957eea6d9cc4ad314282fb58ac92bceef2416e8212040441f130

Install Calico Network Plugin (siri-awx-01 Only)

curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml -O
kubectl apply -f calico.yaml
kubectl get nodes
kubectl get pods -n kube-system

Installing Helm

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

Deploying Ansible AWX Using Helm

helm install ansible-awx-operator awx-operator/awx-operator -n awx --create-namespace
kubectl get pods -n awx

Setting Up Storage — StorageClass, PV and PVC
local-storage-class.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: local-storage
namespace: awx
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

kubectl create -f local-storage-class.yaml
kubectl get sc -n awx

pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: postgres-pv
namespace: awx
spec:
capacity:
storage: 10Gi
volumeMode: Filesystem
accessModes:

• ReadWriteOnce persistentVolumeReclaimPolicy: Delete storageClassName: local-storage local: path: /mnt/storage # Mount point should available on worker node nodeAffinity: required: nodeSelectorTerms:
  • matchExpressions:
    • key: kubernetes.io/hostname operator: In values:
      • siri-awx-02 # worker node hostname

pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgres-13-ansible-awx-postgres-13-0
namespace: awx
spec:
storageClassName: local-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi

kubectl apply -f pv.yaml
kubectl create -f pvc.yaml
kubectl get pv,pvc -n awx

Deploying AWX

kubectl create -f ansible-awx.yaml

kubectl logs -f deployments/awx-operator-controller-manager -c awx-manager -n awx

Accessing the AWX Web Interface

kubectl expose deployment ansible-awx-web --name ansible-awx-web-svc --type NodePort -n awx
kubectl get svc ansible-awx-web-svc -n awx

Get admin password

kubectl get secret ansible-awx-admin-password -o jsonpath="{.data.password}" -n awx | base64 --decode ; echo

AWX is now accessible at http://: — in my case port 32418. Log in with username admin and the decoded password.

Errors I Hit and How I Fixed Them
Error 1 — Node Taint Issue (Pods Stuck in Pending)

Warning FailedScheduling 2m37s default-scheduler

0/1 nodes are available: 1 node(s) had untolerated taint
{node.kubernetes.io/not-ready: }

First, check the actual taint on your node:
kubectl describe node siri-awx-01 | grep Taints

Taints: node.kubernetes.io/not-ready:NoSchedule

Remove it

kubectl taint nodes siri-awx-01 node.kubernetes.io/not-ready:NoSchedule-

node/siri-awx-01 untainted

After removing the correct taint, pods started coming up properly.

Error 2 — ImagePullSecret Error
FailedToRetrieveImagePullSecret Unable to retrieve some image pull secrets
(redhat-operators-pull-secret); attempting to pull the image may not succeed.

Debug with:
bashkubectl logs awx-operator-controller-manager-6586fccbdd-xtdvj -n awx
kubectl get events -n awx

Fix — edit the deployment and comment out the imagePullSecrets section:

kubectl edit deployment awx-operator-controller-manager -n awx
Kubernetes automatically picks up the changes. No manual restart needed.
Accessing the AWX Web Interface
kubectl expose deployment ansible-awx-web --name ansible-awx-web-svc --type NodePort -n awx
kubectl get svc ansible-awx-web-svc -n awx

Get admin password

kubectl get secret ansible-awx-admin-password -o jsonpath="{.data.password}" -n awx | base64 --decode ; echo

AWX is now accessible at http://: — in my case port 32418. Log in with username admin and the decoded password.

Final Verification
Once logged in, I ran a demo playbook to make sure everything was wired up correctly. Seeing that first successful job run in the AWX UI after all of this setup was genuinely satisfying.

This wasn’t a smooth “follow steps → done” kind of project.

It was more like:

set up cluster → something breaks
fix networking → something else breaks
fix storage → something else breaks again
finally get AWX running → feels like victory

But I did learn a lot about how Kubernetes actually behaves when things aren’t ideal.

If you’re doing this yourself and hit weird issues, you’re probably not alone—most of the fixes I found were pieced together from random threads and trial-and-error.

Still worth it though.

• happy to help! — Sireesha