DEV Community: Sireesharaju Kamparaju

Setting Up Email Notifications in AWX via SMTP and Postfix — My DevOps Journey By Sireesha

Sireesharaju Kamparaju — Wed, 17 Jun 2026 14:30:23 +0000

After writing about Ansible Vault, I wanted to close the loop on something I'd been putting off — actually knowing when my patching workflow fails instead of finding out the next morning. So in my homelab I set up a Postfix mail server to act as my SMTP relay and connected it to AWX for email notifications.
In this blog I'll walk through exactly how I did it — setting up Postfix, configuring the AWX notification template, and getting the firewall side sorted with help from our network team since that's not something I manage directly.
In this blog I'll cover:

Setting up Postfix as my SMTP relay
Creating the email notification template in AWX
Getting the firewall opened for SMTP traffic
Attaching the notification to my patching workflow
Testing it end to end

Let's get into it.

Why I Choose with Postfix
I wanted something self-hosted rather than relying on Gmail or an external SMTP provider for my homelab. Postfix is the standard mail transfer agent on Linux and it's lightweight enough to run on a small VM just for relaying AWX notifications. It also gave me a proper feel for how a real internal SMTP relay works — which is exactly what most companies use in production.

Step 1 — Installing Postfix
I set up a small Ubuntu VM dedicated to acting as my mail relay and installed Postfix on it:

sudo apt update
sudo apt install -y postfix

During installation it asks you to choose a configuration type. I selected:

Internet Site

And set the System mail name to something like:

mail.homelab.local

Step 2 — Configuring Postfix as a Relay
I edited the main Postfix config file to allow my AWX server to relay through it:

sudo vi /etc/postfix/main.cf

The key settings I made sure were set:

myhostname = mail.homelab.local
mydestination = $myhostname, localhost.localdomain, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 <ansible-IP>/24
inet_interfaces = all

The important line is mynetworks — I added my homelab subnet /24 so Postfix trusts and relays mail coming from any of my homelab servers, including AWX, without needing authentication.
Restart Postfix to apply the changes:

sudo systemctl restart postfix
sudo systemctl enable postfix

Step 3 — Testing Postfix Locally First
Before even touching AWX, I wanted to confirm Postfix itself was working. I installed mailutils to send a quick test email from the command line:

sudo apt install -y mailutils
echo "Test email body" | mail -s "Postfix Test" your@email.com

Check the mail log to see what happened:

sudo tail -f /var/log/mail.log

If you see status=sent in the log, Postfix is relaying correctly and you're ready to connect AWX to it.

Step 4 — Getting the Firewall Opened
This is the part I couldn't do myself — our network team manages the firewall between network segments in our homelab/internal setup. I raised a request with them to allow:

AWX server subnet → Postfix server (10.215.77.X) → Port 25 TCP

They confirmed it was opened on their side. If you're managing your own firewall on the Postfix VM itself, the equivalent UFW rule would be:

sudo ufw allow from 10.215.77.0/24 to any port 25 proto tcp

And on the AWX side, if there's an outbound rule needed on the Kubernetes worker nodes:

sudo ufw allow out to 10.215.77.X port 25 proto tcp

Tip: Always test raw port connectivity before testing inside AWX. Saves you from chasing the wrong problem.

nc -zv 10.215.77.X 25

If that connects, you're clear to move on to the AWX side.

Step 5 — Creating the Notification Template in AWX

AWX UI → Administration → Notifications
Click Add

Click Save

Step 6 — Attaching the Notification to My Patching Workflow

AWX UI → Templates → open Patching Workflow → Notifications tab
Under Failure, toggle on Patching Workflow Email Alerts
Click Save

I only enabled it for Failure — I don't need an email every time the workflow succeeds, only when something goes wrong.

Step 7 — Testing the Notification
AWX has a built-in test button so I didn't need to wait for an actual failure to check it worked:

Go back to Administration → Notifications
Open Patching Workflow Email Alerts
Click Test

I checked my inbox and the test email came through within a few seconds. I also double checked the Postfix mail log on the relay server at the same time:

sudo tail -f /var/log/mail.log

Seeing the AWX server's IP show up in the log with status=sent confirmed the whole chain was working end to end — AWX → Postfix → my inbox.

Errors I Hit Along the Way

Error 1 — Connection Timed Out from AWX

SMTPConnectError: Connection to <ansoble-IP>timed out
This was the firewall not being opened yet. Once the network team confirmed the rule was in place between the AWX subnet and the Postfix server, this cleared immediately.

Error 2 — Relay Access Denied

SMTPRecipientsRefused: 554 5.7.1 Relay access denied

This happened because I forgot to add my homelab subnet to mynetworks in Postfix's main.cf. Postfix was rejecting the AWX server because it didn't recognise it as a trusted network. Fix:

sudo vi /etc/postfix/main.cf
# add your subnet to mynetworks
sudo systemctl restart postfix

Error 3 — Mail Stuck in the Queue

I noticed test emails weren't arriving and checked the Postfix queue:

mailq

There were a few stuck messages. This was because my Postfix VM didn't have a working DNS resolver set up properly, so it couldn't resolve where to deliver outbound mail. Since this was all internal traffic destined for a local mailbox, I made sure the destination mail server was reachable and resolvable, then flushed the queue:

sudo postqueue -f

Error 4 — Postfix Service Not Starting After Config Change

postfix/postfix-script: fatal: the Postfix mail system is not running
A typo in main.cf caused this. Always check the config syntax before restarting:

sudo postfix check

This pointed me straight to the line with the typo.

What I Learned from This

Test each layer separately. Postfix locally first, then raw firewall connectivity, then AWX notification test, then the actual workflow trigger. Trying to debug all of it at once just leads to confusion about where the actual problem is.

mynetworks is the equivalent of an allow-list for SMTP relaying. If your AWX server's subnet isn't in there, Postfix will reject it outright with a relay denied error — no amount of AWX configuration will fix that.

Firewall changes between teams take coordination. Since I don't manage the network firewall myself, I had to raise the request and wait for confirmation. Good reminder that in real environments, automation setups often depend on more than just your own server configs.

The Postfix mail log is your best friend. Whenever something didn't work as expected, tail -f /var/log/mail.log told me exactly what was happening — accepted, deferred, bounced or sent.

What's Next
Now that I have working email alerts on failure, the next thing I want to try is adding a Slack notification alongside email — so I get alerted in both places when the patching workflow fails at 2AM on a Sunday.

Drop your questions in the comments — happy to help if you're setting up something similar in your own homelab!
— Sireesha

Ansible Vault — Managing Secrets Securely in Ansible and AWX — My DevOps Journey By Sireesha

Sireesharaju Kamparaju — Thu, 04 Jun 2026 15:59:38 +0000

One thing I kept putting off when I started with Ansible was properly securing my secrets. Passwords, API keys, tokens — they were just sitting in plain text inside my vars files. I knew it was wrong but it worked and I kept moving forward.
Then I started thinking about what happens when this goes into GitLab. Anyone with access to the repo can see every password. That's when I properly sat down and learned Ansible Vault — and honestly I wish I'd done it from day one.
In this blog I'll walk through how I use Ansible Vault from the command line and then how I handle it properly inside AWX so scheduled jobs and workflow templates can still run without someone manually typing a password every time.

What is Ansible Vault?
Ansible Vault is a built-in feature that lets you encrypt sensitive data — passwords, keys, tokens — so they can be safely stored in your playbooks and pushed to GitLab without exposing anything.
The beauty of it is that it works right inside your existing YAML files. Your playbook structure doesn't change — Vault just encrypts the values that need protecting.

What I Was Doing Before — The Wrong Way
This is what my vars file looked like before Vault:

# vars/main.yml
ubuntu_sudo_password: MyPassword123
db_password: SuperSecret456
api_token: abcd1234efgh5678

Pushed straight to GitLab. Anyone with repo access could read every single one of those. Not good.

Encrypting a Single Value with Ansible Vault
The first thing I learned was how to encrypt just a single variable value — not the entire file. This is cleaner because the rest of the vars file stays readable.

ansible-vault encrypt_string 'MyPassword123' --name 'ubuntu_sudo_password'

It asks you to create a vault password — this is the master password you'll use to decrypt later. Enter it twice:

New Vault password:
Confirm New Vault password:

The output looks like this:

ubuntu_sudo_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          66386439653236336462626566653337386235396138623934363161623364663834623437333132
          3865663966343437326235373961316435623365663866610a663834623437333132386566396634
          34373236353337386235396138623934363161623364663834623437333132386566396634343732
          3635333738623539613862393436316162336466383462343733313238656639363434373236353337

Copy this entire block and paste it into your vars file:

# vars/main.yml
ubuntu_sudo_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          66386439653236336462626566653337386235396138623934363161623364663834623437333132
          ...
db_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          ...

Now this file is safe to push to GitLab. Anyone who opens it sees encrypted gibberish — not the actual passwords.

Encrypting an Entire Vars File
If you have a lot of secrets, encrypting the whole file is quicker:

ansible-vault encrypt vars/secrets.yml

To edit it:

ansible-vault edit vars/secrets.yml

To decrypt it permanently (be careful with this one): (I didn't try this)

ansible-vault decrypt vars/secrets.yml

Running Playbooks with Vault from Command Line
This is how I was running playbooks before I set up AWX properly. Every time I ran a playbook that used vaulted variables, I added --ask-vault-pass:

ansible-playbook patch.yml -i inventory/hosts.yml --ask-vault-pass

It prompts to enter vault password:

Vault password:

Enter vault password and the playbook runs, decrypting the secrets on the fly. Nothing is ever stored in plain text on the server.
This works perfectly from the command line. But the moment you move to AWX — scheduled jobs, workflow templates, CI/CD — you can't have AWX stopping and waiting for someone to type a password. That's where the AWX Vault credential comes in.

The Problem with --ask-vault-pass in AWX
Imagine you've set up your patching workflow to run every Sunday night at 2AM. AWX kicks off the job automatically — but your playbook uses vaulted passwords. AWX has no way to ask you for the vault password at 2AM.
The job just fails.
That's exactly the problem I hit. I had my refresh.yml scheduled for every Monday at 5AM and it kept failing because it couldn't decrypt the vault variables. I was confused at first — it ran fine from the command line. Then I realised AWX needs the vault password stored as a credential so it can decrypt automatically at runtime.

Solving It in AWX — Adding a Vault Credential
AWX has a built-in credential type specifically for Ansible Vault. Here's how I set it up:

AWX UI → Credentials → Add ->Click Save

The vault password is stored encrypted inside AWX — no one can read it back out. It's just referenced by name.

Attaching the Vault Credential to a Job Template
Now I attach this credential to any job template that uses vaulted variables:

AWX UI → Templates → open your job template → Edit
In the Credentials field — click the search icon
You'll see your existing machine credential already there
Search for ansible-vault-cred and select it as well
A job template can have multiple credentials — one for machine access, one for vault
Click Save

Now when AWX runs this job template — whether manually, on a schedule, or as part of a workflow — it automatically decrypts the vault variables using the stored credential. No one needs to type anything.

Example — Patching Workflow with Vault
Let me show you exactly how this works end to end with my patching workflow.
My pre_patch_check.yml playbook connects to servers using credentials stored in a vaulted vars file:

# vars/secrets.yml (encrypted with ansible-vault)
ubuntu_sudo_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          ...
db_connection_string: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          ...

Myplaybook

---
- name: Pre-patch checks
  hosts: all
  vars_files:
    - vars/secrets.yml
  tasks:
    - name: Check application status
      ansible.builtin.uri:
        url: "http://{{ inventory_hostname }}/health"
        headers:
          Authorization: "Bearer {{ api_token }}"
      register: health_check

From command line I run it with:

ansible-playbook pre_patch_check.yml -i inventory/hosts.yml --ask-vault-pass

In AWX — the job template has both machine-cred and ansible-vault-cred attached. When the patching workflow kicks off at 2AM on Sunday, AWX decrypts the secrets automatically and the whole workflow runs without anyone being awake to babysit it.
That's proper production automation.

Rotating Vault Password
Every now and then you should change your vault master password — especially if someone who knew it leaves the team. Here's how to rekey all your encrypted files at once:

ansible-vault rekey vars/secrets.yml

It asks for the old password, then the new one. After rekeying, update the vault credential in AWX with the new password too — otherwise your scheduled jobs will start failing.

Errors I Hit Along the Way
Error 1 — Decryption Failed

ERROR! Decryption failed (no vault secrets would unlock any vault file/string)

Fix — I had forgotten to attach the ansible-vault-cred to the job template. Once I added it, the scheduled job ran perfectly.
Error 3 — Committed Vault Password File to GitLab
Early on I accidentally committed my .vault_pass file to GitLab. As soon as I realised:

# Remove from tracking
git rm --cached .vault_pass

# Add to gitignore
echo ".vault_pass" >> .gitignore

git commit -m "removed vault password file from tracking"
git push origin dev

And immediately changed my vault password using ansible-vault rekey. Lesson learned — always add .vault_pass to .gitignore before creating it.

My Golden Rules for Ansible Vault

After going through all of this, here's what I now follow without exception:

Never store plain text passwords in vars files — encrypt everything sensitive from day one
Always add .vault_pass to .gitignore before creating the file
Use AWX Vault credentials for any job that runs on a schedule or in a workflow
Rekey when team members leave — treat it like changing a shared password
Test vault decryption after any rekey before your next scheduled job runs

What's Next
Now that secrets are properly managed, the next blog I'm planning is AWX Notifications — setting up Slack and email alerts so when a patching workflow fails at 2AM, I know about it first thing in the morning without having to log into AWX to check.

Drop your questions in the comments — happy to help!
— Sireesha

AWX Workflow Templates & Schedules — My DevOps Journey

Sireesharaju Kamparaju — Thu, 04 Jun 2026 11:24:36 +0000

If you've been following my blog series, you know I've already set up AWX, synced GitLab projects, and run my first playbook. The next piece I wanted to explore was how AWX handles more complex automation tasks.

In this post, I'll cover two features I've started using more and more: Workflow Templates and Schedules.

Here's what I'll walk through:

What Workflow Templates are and when they're useful
How I built a patching workflow by chaining multiple job templates together
How I configured a schedule to run a playbook automatically every Monday at 5 AM

Let's get started.

Workflow Templates

When I first started using AWX, I mostly worked with standard job templates. For simple tasks, they did exactly what I needed — run a playbook against an inventory and return the results.

As I started building more realistic automation tasks, I realised most jobs weren't a single playbook run. A typical maintenance task might involve patching servers, rebooting them, running validation checks, and then notifying the team once everything is complete.

I could run each of those steps manually, but it quickly became tedious and left plenty of room for mistakes.

That's where Workflow Templates came in. They allow you to link multiple job templates together and define what should happen when a step succeeds or fails. Instead of managing each stage yourself, AWX handles the flow for you.

For me, this was the point where AWX started feeling less like a tool for launching playbooks and more like a platform for automating operational processes.

My Real-World Use Case – Patching Workflow

I didn't fully appreciate Workflow Templates until I started building a server patching process.

Originally, I was launching individual job templates one after another: run pre-checks, patch the server, reboot it, verify services, and finally send a notification. It worked, but it meant I had to monitor every stage and decide manually what to do if something failed.

Moving that process into a Workflow Template made things much easier. AWX could handle the sequence automatically and stop the workflow if a critical step failed.

The workflow looked like this:

Stop Application → Pre-Patch Check → Patch Servers → Reboot → Post-Patch Check → Start Application

Each step is a separate playbook, chained together in AWX as a workflow. If any step fails, the workflow stops right there — the next step doesn't run. That's critical. You don't want servers rebooting if the pre-patch check failed.

Step 1 — Create the Individual Job Templates First
Before building the workflow, I made sure each playbook was already in GitLab and each had its own job template in AWX. Here's what I set up:

Each of these was already tested individually before I built the workflow. That's important — don't try to debug a workflow and individual playbooks at the same time.

Step 2 — Create the Workflow Template
AWX UI → Templates → Add → Add Workflow Template

Click Save
This opens the Workflow Visualiser — a drag and drop canvas where you build the flow visually. This is where it gets satisfying.

Step 3 — Build the Workflow in the Visualiser
In the Workflow Visualiser:

Click Start → a window pops up asking you to select a node
Select Stop Application job template → click Save
Hover over the Stop Application node → click the + button
Select On Success → select Pre-Patch Check → click Save
Repeat for each step — always connecting On Success to the next playbook
Here's the full chain I built:

Start
└── Stop Application
└── [On Success] Pre-Patch Check
└── [On Success] Patch Servers
└── [On Success] Reboot Servers
└── [On Success] Post-Patch Check
└── [On Success] Start Application

The key here is On Success at every step. If patching fails, AWX stops the workflow right there — servers don't reboot, application doesn't try to start on a broken system. That's exactly the safety net you need in production.

Click Save when the workflow is built
Step 4 — Launch the Workflow
Go to Templates → find Patching Workflow
Click Launch 🚀

AWX shows you the workflow running in real time — each node lights up green as it completes or red if it fails. You can click into any node to see the full playbook output for that step.

Watching a 6-step patching workflow run end to end without touching anything is one of those moments that makes all the setup worth it.

AWX Schedules — Automating Recurring Jobs
Now the second feature — Schedules. Once you have job templates set up, you can tell AWX to run them automatically at any time or frequency without anyone manually clicking Launch.

My use case was simple but important — I have a refresh.yml playbook that needs to run every Monday at 5AM. Before schedules I had to remember to run it manually. That's not automation — that's just a reminder. Schedules fixed that completely.

Setting Up the Monday 5AM Schedule AWX UI → Templates → open the job template you want to schedule Click the Schedules tab at the top Click Add

Field Value Name Weekly Data Refresh Start Date Set today's date or next Monday Start Time 05:00 AM Time Zone Select your timezone Repeat Frequency Week Run On Monday Click Save
That's it. AWX will now automatically run refresh.yml every Monday at 5AM without anyone touching anything. You can verify it's set up correctly by checking the Next Run field — it shows you exactly when the next execution is scheduled.

Managing Schedules

A few things I find useful when managing schedules in AWX:

*Enable/Disable without deleting *— there's a toggle on each schedule. If I don't want the refresh to run during a maintenance window, I just disable it and re-enable after. No need to delete and recreate.

Multiple schedules on one template — you can add more than one schedule to the same job template. For example I could run refresh.yml every Monday at 5AM AND every first day of the month at midnight if needed.

Check the Activity Stream — after a scheduled job runs, go to Activity Stream in AWX to confirm it ran and whether it succeeded or failed. This is your audit trail for scheduled jobs.

Errors I Hit Along the Way
Error 1 — Workflow Node Not Connecting
When building the workflow, I kept clicking the + button but the connection line wasn't appearing between nodes.

Fix — make sure you hover directly over the node until you see the + appear, then click it. If you click too fast it doesn't register. Slow down and wait for the hover state.

Error 2 — Wrong Timezone on Schedule
My schedule was set up but running at the wrong time. The job was triggering at 5AM UTC instead of my local time.

Fix — when creating the schedule, always explicitly set the Time Zone field to your local timezone. Don't assume it defaults to your local time — it defaults to UTC.

Error 3 — Workflow Skipping Failure Path

When a node failed, the workflow was completing instead of stopping.

Fix — I had accidentally connected some nodes with On Either instead of On Success. Go back into the Workflow Visualiser, click each connection line and check it says Run on Success. Change any that are set to On Either or Always.

What actually changed

Before workflows, patching was basically manual SSH work stitched together step by step. It worked, but it was fragile and very dependent on the person running it.

Now it’s just one workflow execution.

Same with schedules — anything recurring used to rely on memory or reminders. Now it just runs.

Nothing revolutionary individually, but together it changes how you approach automation. You start thinking in systems instead of tasks.

This is what proper production automation looks like — reliable, auditable, and hands-off.

Drop your questions in the comments — happy to help!

— Sireesha

Running Your First Playbook from Ansible AWX — My DevOps Journey By Sireesha

Sireesharaju Kamparaju — Tue, 02 Jun 2026 09:58:23 +0000

In my previous blogs, I covered how to deploy AWX on Kubernetes using Helm and how to sync GitLab projects into AWX. Now that AWX is up and running, the next thing I wanted to do was actually run a playbook from the AWX UI — and that involves setting up a few things first.
In this blog I'll walk through everything you need to configure inside AWX before you can launch your first job:

Dashboard overview
Adding an Organisation
Adding Users
Adding Inventory and Hosts
Adding Credentials
Adding a Project
Creating a Job Template and launching it

Let's go step by step.

The AWX Dashboard
When you first log in with your admin credentials, you land on the AWX dashboard. It gives you a nice overview of everything happening on your server at a glance:

Number of hosts that have successfully run playbooks
Number of hosts that failed
Total number of inventories
Number of projects and their sync status
A graph showing playbook runs over time

On the left side you'll see four main sections — Resources, Access, Administration and Settings. Everything we need is in here.

Step 1 — Add an Organisation
The first thing I set up was an organisation. In AWX, almost everything sits under an organisation — users, credentials, projects, inventories.

On the left pane go to Access → Organisations
Click Add
Enter a Name and Description
Click Save

Simple enough — but don't skip this, everything else depends on it.

Step 2 — Add a User
Next I created a user under the organisation. AWX has three user types and it's worth understanding the difference before you set one up:
Normal User — can create, use and update templates. Good for engineers who need to run playbooks but shouldn't have full admin access.
System Auditor — can view inventory, templates and job status but cannot create or modify anything. Great for someone who just needs visibility.
System Administrator — has full privileges, same as the default admin account. Use this carefully.
To add a user:

Left pane → Access → Users
Click Add
Enter First Name, Last Name, Email, Username and Password
Select the Organisation you just created
Choose the User Type based on the access you want to give
Click Save

Step 3 — Add an Inventory
An inventory in AWX is basically your list of hosts — the machines you want to run playbooks against. You can organise them into groups like development, testing, and production.
To add an inventory:

Left pane → Resources → Inventories
Click Add → Add Inventory
Enter a Name and select your Organisation
Click Save

Adding Hosts to the Inventory
Once the inventory is saved, add your hosts to it:

Open the inventory you just created
Click the Hosts tab
Click Add Host
Enter the Hostname — this can be an IP address or a URL. For example, my host IP was 192.168.237.145
Add a Description if you want
Click Save

You can add as many hosts as you need. I added two hosts to my inventory. You can also temporarily remove a host from the inventory using the on/off toggle on the right side of each host — handy when you want to skip a machine without deleting it.

Step 4 — Add Credentials
AWX stores credentials separately from everything else — which is actually a really clean way to manage it. You create a credential once and reuse it across multiple templates.

Left pane → Resources → Credentials
Click Add
Enter a Name and Description
Select your Organisation
Select Credential Type → choose Machine (this is similar to SSH login)
Enter the Username and Password of the remote machine
If your playbook needs to run as root, fill in the Privilege Escalation Username and Password as well
Click Save

If You're Pulling Playbooks from a Git Repo
If your playbooks are coming from GitHub or GitLab, you'll need a second credential for source control:

Follow the same steps above but select GitHub Personal Access Token as the credential type
Paste your token and save

To get a GitHub personal access token:

Log into GitHub → Settings → Developer Settings → Personal Access Tokens → Generate new token
Copy it and paste it into AWX — you won't see it again after leaving that page!

Step 5 — Add a Project
A project in AWX is where you point AWX to your playbooks — either from a Git repo or a directory on the AWX server itself.

Left pane → Resources → Projects
Click Add
Enter a Name and Description
Select your Organisation
Select SCM Type:
Choose Git if your playbooks are in GitHub or GitLab
Choose Manual if the playbooks are already sitting on the AWX server in a local directory

Enter the SCM URL — this is your GitHub/GitLab repo URL
Under SCM Update Options select what makes sense for your setup — I ticked Update Revision on Launch so AWX always pulls the latest version
Click Save

Once saved, hit the Sync button. Wait for the status to turn green — that means AWX has successfully pulled your playbooks from the repo.

Step 6 — Create a Job Template and Launch It
This is the exciting part — this is where you actually wire everything together and run your playbook.

Left pane → Resources → Templates
Click Add → Add Job Template
Enter Name, Job type check or run , inventory, project, crendetails
Click Save
Click Launch 🚀

AWX will start running the job and you can watch the output in real time right from the UI. Every task, every host, every result — all visible on screen. No more tailing log files!

What the Output Looks Like
Once the job runs, AWX shows you a full breakdown:

Which tasks ran on which hosts
Green for ok, yellow for changed, red for failed
Total summary at the end showing how many tasks succeeded or failed per host

If something fails, you can click into the task and see the exact error — no guessing needed.

What I Noticed Along the Way
The credential separation is really useful. Because credentials are stored separately, I can update a password in one place and every template using that credential picks it up automatically. No hunting through playbooks.
The on/off host toggle is underrated. Being able to disable a host in the inventory without deleting it saved me a few times when I wanted to test on just one machine.
Always sync your project before creating a template. If you create a template before syncing the project, the playbook dropdown will be empty. Sync first, then create the template — saves confusion.
Check mode is your friend Before running anything on production hosts, I always create the template with Job Type: Check first. It does a dry run and shows what would change without actually changing anything.

What's Next
Now that I can run playbooks from AWX, the next things I want to cover are:

Workflows — chaining multiple playbooks together in AWX
Schedules — running playbooks automatically at set times
Notifications — getting alerts when jobs succeed or fail

If you're setting up AWX for the first time and getting stuck anywhere in these steps, drop it in the comments — happy to help!
— Sireesha

Syncing GitLab Projects to AWX — My DevOps Journey

Sireesharaju Kamparaju — Tue, 19 May 2026 13:24:55 +0000

If you've been following my blog series, you already know I've set up Ansible AWX on Kubernetes using Helm. Now the next logical step for me was — how do I actually get my playbooks into AWX without manually uploading them every time?
The answer is GitLab. In this blog I'll walk through how I installed GitLab on a VM, pushed my Ansible playbooks to it, and then synced that repo into AWX as a project — in two ways:

Using an SSH key (SCM Private Key) — this is what I actually use in production
Using a Personal Access Token over HTTP — good for quick lab setups

Let's get into it.

What We're Building Here
The flow looks like this:

Ansible Server → Push Playbooks → GitLab (self-hosted VM) → Sync → AWX Project & Templates
Once this is set up, every time I update a playbook and push to GitLab, AWX syncs and picks up the latest version automatically. That's the real power of this setup.

Part 1 — Installing GitLab on a VM
I installed GitLab on a separate Ubuntu 22.04 VM. Here's exactly what I ran:
Step 1 — Install Dependencies
sudo apt update
sudo apt install -y curl openssh-server ca-certificates tzdata perl
Step 2 — Add GitLab Repository and Install
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
sudo apt install gitlab-ce
Step 3 — Configure GitLab
sudo vi /etc/gitlab/gitlab.rb
Update the external URL to your VM IP:
rubyexternal_url 'http://10.*.*.**'
Then reconfigure:
sudo gitlab-ctl reconfigure
Step 4 — Get the Initial Root Password
sudo cat /etc/gitlab/initial_root_password
Open http://10.*.*.* in your browser → log in with root and the password above.

Tip: Change your root password immediately after first login — User Settings → Password.

Part 2. Creating a repo and pushing playbooks

Inside GitLab, I created a new blank project called:

ansibleawx

Set it to private and moved on.

On my Ansible server, I configured Git:

git config --global user.name "Sireesha"
git config --global user.email "your@email.com"

Then inside my playbook directory:

cd /etc/ansible

git init
git remote add origin http://ip/ansibleawx.git
git add .
git commit -m "Initial commit - adding ansible playbooks"
git branch -M patches
git push -u origin patches

I’m using a patches branch here because that’s what I decided to track in AWX later.

That part matters more than it looks — branch mismatches will bite you.

Part3. Connecting AWX to GitLab using SSH (production way)

This is the setup I actually use properly. It’s more secure and avoids token expiry issues.

Generate SSH key (on AWX side)
ssh-keygen -t rsa -b 4096 -C "awx@ansible"

Then grab the public key:

cat ~/.ssh/id_rsa.pub

Add it to GitLab

GitLab → User Settings → SSH Keys → paste key → save

Nothing complicated here.

Create AWX credential

In AWX:

Credentials → Add

Name: gitlab-cred
Type: Source Control
Username: root
SCM Private Key: paste private key (~/.ssh/id_rsa)

Click Save.

Create AWX project

AWX → Projects → Add

Name: AnsibleAWX
SCM Type: Git
URL: http:///ansibleawx.git
Branch: patches
Credential: gitlab-cred

Once saved, AWX syncs automatically.

When it turns green, you’re good.

That’s the moment everything starts clicking.

This is exactly the gitlab-cred credential I have set up — you can see from the screenshot it shows Credential Type: Source Control and SCM Private Key: Encrypted.

Part4. Alternative: HTTP with Personal Access Token

This is the quicker setup I used in a lab environment.

Create token in GitLab

GitLab → Settings → Access Tokens

Scope: read_repository

Copy it (you only see it once).

Add to AWX

Credentials → Add

Name: gitlab-http-cred
Username: root
Password:

Save.

Create project

Same as before, but use this credential instead.

It works fine — just not as clean or durable as SSH.

Part 5 — Creating Job Templates in AWX

Once the project sync works, playbooks automatically appear in AWX.

So I created a simple job template:

AWX → Templates → Add

Name: Linux Server Setup
Job Type: Run
Inventory: your inventory
Project: AnsibleAWX
Playbook: select from dropdown
Credentials: your machine credentials

Save and launch.

At this point, AWX is basically running whatever is in GitLab.

No file copying. No manual updates.

Errors I Hit Along the Way
Error 1 — Git Push Branch Mismatch

error: src refspec main does not match any
Fix — rename the branch to match what you set in AWX:
git branch -M patches git push -u origin patches
Error 2 — AWX Sync Failed on SSH Host Key

Host key verification failed
Fix — in your AWX Project settings under Source Control Options, tick:
✅ Disregard Host Checks
Error 3 — Playbook Dropdown Empty in Job Template

After syncing, the playbook dropdown was empty when creating a job template
.
Fix — go back to your Project → hit the Sync button (circular arrow) → wait for green → go back to template. Playbooks will appear.

Before this, updating a playbook meant manually copying files and hoping AWX had the latest version. Now my workflow is:

Edit playbook → git push to patches branch → AWX syncs → Launch job

Everything is version controlled, auditable, and consistent.

Drop your questions in the comments — happy to help!
— Sireesha

Deploying Ansible AWX on Kubernetes Using Helm - By Sireesha

Sireesharaju Kamparaju — Tue, 19 May 2026 10:51:57 +0000

In this blog, I'm going to walk you through how I deployed Ansible AWX on a Kubernetes cluster using Helm. This was one of the most hands-on projects I've worked on — it involves setting up the K8s cluster from scratch, installing the container runtime, deploying Helm, and finally getting AWX up and running. I also hit a few real errors along the way, so I'll share exactly how I fixed them.
It was one of those projects that looks simple on paper but turns into a chain of small problems once you actually start doing it.
Here's the overall flow I followed:
The goal was pretty straightforward:

Set up a Kubernetes cluster on Ubuntu 22.04
Install containerd
Get Helm working
Deploy AWX using the AWX Operator
Fix whatever broke along the way (this part took the longest 😅)

Setting Up the Kubernetes Cluster on Ubuntu 22.04
Before anything else, I want to quickly explain the two types of nodes in a K8s cluster since this matters for the setup:
Master Node — manages the control plane, API calls, pods, services, and everything else in the cluster.
Worker Node — runs the actual containers. Pods can spread across multiple worker nodes for better resource management.
Step 1 — Update and Upgrade (All Nodes)
I started by logging in as root and running updates on all nodes:

hostnamectl set-hostname siri-awx-01 # run on master node hostnamectl set-hostname siri-awx-02 # run on worker node exec bash

apt update apt upgrade

Step 2 — Disable Swap

swapoff -a sed -i '/ swap / s/^$.*$$/#\1/g' /etc/fstab
Step 3 — Load Kernel Modules

tee /etc/modules-load.d/containerd.conf <<EOF overlay br_netfilter EOF

modprobe overlay modprobe br_netfilter

Step 4 — Configure Kernel Parameters
tee /etc/sysctl.d/kubernetes.conf <<EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF

sysctl --system

Installing Containerd Runtime (All Nodes)
apt install -y curl gnupg2 software-properties-common apt-transport-https ca-certificates

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/docker.gpg

add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

apt update apt install -y containerd.io

containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

systemctl restart containerd systemctl enable containerd

Installing Kubernetes Components (All Nodes)
`sudo apt-get install -y apt-transport-https ca-certificates curl gpg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg`

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update sudo apt-get install -y kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl

Initialising the Cluster (Master Node — siri-awx-01 Only)

systemctl enable kubelet && systemctl start kubelet kubeadm init

mkdir -p $HOME/.kube cp -i /etc/kubernetes/admin.conf $HOME/.kube/config chown $(id -u):$(id -g) $HOME/.kube/config

Tip: If you forget to save the kubeadm join output, regenerate it with:
bashkubeadm token create --print-join-command

$kubectl get nodes

Join Worker Node (siri-awx-02)
Run this on siri-awx-02 (use your actual token from kubeadm init output):

kubeadm join 192.168.237.144:6443 --token uhkvxh.mjp6aiyhy18vaxh0 \ --discovery-token-ca-cert-hash sha256:c0200907cc68957eea6d9cc4ad314282fb58ac92bceef2416e8212040441f130

Install Calico Network Plugin (siri-awx-01 Only)

curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml -O kubectl apply -f calico.yaml kubectl get nodes kubectl get pods -n kube-system

Installing Helm

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 chmod 700 get_helm.sh ./get_helm.sh

Deploying Ansible AWX Using Helm

helm install ansible-awx-operator awx-operator/awx-operator -n awx --create-namespace kubectl get pods -n awx

Setting Up Storage — StorageClass, PV and PVC
local-storage-class.yaml

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local-storage namespace: awx provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumer

kubectl create -f local-storage-class.yaml
kubectl get sc -n awx

pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: postgres-pv
  namespace: awx
spec:
  capacity:
    storage: 2Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /mnt/storage # Mount point should available on worker node
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - siri-awx-02    # worker node hostname

pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-13-ansible-awx-postgres-13-0
  namespace: awx
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

kubectl apply -f pv.yaml kubectl create -f pvc.yaml kubectl get pv,pvc -n awx

Deploying AWX

`kubectl create -f ansible-awx.yaml

kubectl logs -f deployments/awx-operator-controller-manager -c awx-manager -n awx
`

Accessing the AWX Web Interface

kubectl expose deployment ansible-awx-web --name ansible-awx-web-svc --type NodePort -n awx kubectl get svc ansible-awx-web-svc -n awx

Get admin password

kubectl get secret ansible-awx-admin-password -o jsonpath="{.data.password}" -n awx | base64 --decode ; echo

AWX is now accessible at http://: — in my case port 32418. Log in with username admin and the decoded password.

Errors I Hit and How I Fixed Them
Error 1 — Node Taint Issue (Pods Stuck in Pending)

Warning FailedScheduling 2m37s default-scheduler

0/1 nodes are available: 1 node(s) had untolerated taint
{node.kubernetes.io/not-ready: }

First, check the actual taint on your node:
kubectl describe node siri-awx-01 | grep Taints

Taints: node.kubernetes.io/not-ready:NoSchedule

Remove it

kubectl taint nodes siri-awx-01 node.kubernetes.io/not-ready:NoSchedule-

node/siri-awx-01 untainted

After removing the correct taint, pods started coming up properly.

Error 2 — ImagePullSecret Error
FailedToRetrieveImagePullSecret Unable to retrieve some image pull secrets
(redhat-operators-pull-secret); attempting to pull the image may not succeed.

Debug with:
kubectl logs awx-operator-controller-manager-6586fccbdd-xtdvj -n awx kubectl get events -n awx

Fix — edit the deployment and comment out the imagePullSecrets section:

kubectl edit deployment awx-operator-controller-manager -n awx
Kubernetes automatically picks up the changes. No manual restart needed.
Accessing the AWX Web Interface

kubectl expose deployment ansible-awx-web --name ansible-awx-web-svc --type NodePort -n awx kubectl get svc ansible-awx-web-svc -n awx

Get admin password

kubectl get secret ansible-awx-admin-password -o jsonpath="{.data.password}" -n awx | base64 --decode ; echo

Final Verification
Once logged in, I ran a demo playbook to make sure everything was wired up correctly. Seeing that first successful job run in the AWX UI after all of this setup was genuinely satisfying.

This wasn’t a smooth “follow steps → done” kind of project.

It was more like:

set up cluster → something breaks
fix networking → something else breaks
fix storage → something else breaks again
finally get AWX running → feels like victory

But I did learn a lot about how Kubernetes actually behaves when things aren’t ideal.

If you’re doing this yourself and hit weird issues, you’re probably not alone—most of the fixes I found were pieced together from random threads and trial-and-error.

Still worth it though.

happy to help! — Sireesha

Automating Windows Server Setup with Ansible: My DevOps Journey (Part 2)

Sireesharaju Kamparaju — Tue, 12 May 2026 13:12:31 +0000

In my previous blog, I covered how I automated Linux server provisioning using Ansible — things like SSH hardening, reusable roles, and playbooks. That setup worked really well, so the next thing I wanted to tackle was Windows.

This post is about the Windows side of the setup — enabling WinRM, creating a reusable Windows role, and finally combining both Linux and Windows automation into a single Ansible workflow..

Windows Automation Feels Different at First
I’ll be honest — moving from Linux automation to Windows automation was a little frustrating in the beginning.

With Linux, Ansible over SSH just works. Most tutorials online assume that setup, so things feel straightforward pretty quickly. Windows was different.

The first thing I learned was that Ansible talks to Windows through WinRM instead of SSH. That meant extra configuration before I could even run my first playbook.

A few things tripped me up early on:

WinRM isn’t enabled by default on fresh Windows servers
The modules are completely different from Linux
Even simple tasks use different syntax and collections
Some errors from WinRM are vague and take time to troubleshoot

At one point I thought my firewall rules were wrong, but it turned out I had messed up the WinRM transport settings in the inventory file.

Once I got past those initial issues though, the setup became much smoother.
Note: Port 5985 should be allowed in firewall.

**First Thing — Bootstrap WinRM (Just Once)
Before Ansible can do anything on a Windows server, WinRM needs to be enabled. I ran this PowerShell script once on each new Windows machine — after that, Ansible handles everything:

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$url = "https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1"
$file = "$env:temp\ConfigureRemotingForAnsible.ps1"
(New-Object -TypeName System.Net.WebClient).DownloadFile($url, $file)
powershell.exe -ExecutionPolicy ByPass -File $file

The first successful win_ping honestly felt like progress because I’d already spent a while debugging connection issues before getting there.

Adding Windows Hosts to the Inventory
I kept windows and linux in the same inventory file because I wanted one central place to manage everything.

The structure stayed mostly the same, but the connection settings were obviously different:

all:
children:
linux_servers:
hosts:
linux-01:
ansible_host: 10.0.1.10
ansible_user: ec2-user
ansible_ssh_private_key_file: ~/.ssh/id_rsa
windows_servers:
hosts:
win-01:
ansible_host: 10.0.2.10
ansible_user: Administrator
ansible_password: "{{ vault_win_password }}"
ansible_connection: winrm
ansible_winrm_transport: ntlm
ansible_port: 5985

The Windows password is vaulted using ansible-vault — I never put credentials in plain text. That's just a habit I've built early on and I'd recommend everyone do the same.

Building the Windows Role
I kept the same role-based structure I used for Linux. Here's how the Windows role looks:

roles/
windows_setup/
├── tasks/main.yml
└── defaults/main.yml

roles/windows_setup/defaults/main.yml

name: Ensure WinRM service is running and set to auto start
ansible.windows.win_service:
name: WinRM
state: started
start_mode: auto
name: Disable unencrypted WinRM traffic
ansible.windows.win_shell: |
winrm set winrm/config/service '@{AllowUnencrypted="false"}'
name: Configure Windows Firewall to allow WinRM
ansible.windows.win_firewall_rule:
name: WinRM HTTP
localport: "{{ winrm_port }}"
action: allow
direction: in
protocol: tcp
state: present
enabled: true
name: Check for available Windows Security Updates
ansible.windows.win_updates:
category_names:
- SecurityUpdates state: searched register: update_result
name: Display available updates

ansible.builtin.debug:

msg: "{{ update_result.updates | length }} security update(s) available"

The Windows Playbook

name: Configure Windows Servers hosts: windows_servers roles:
- windows_setup

One thing I noticed here — there's no become: true like I used on Linux. Windows doesn't use sudo. The Administrator account takes care of privilege escalation directly.

Bringing It All Together — site.yml
This is the part I enjoyed the most. One playbook, one command, both Linux and Windows configured together:

import_playbook: playbooks/linux_setup.yml
import_playbook: playbooks/windows_setup.yml

And to run everything:
ansible-playbook site.yml -i inventory/hosts.yml --ask-vault-pass

That's it. Ansible runs through Linux first, then Windows — clean and consistent every single time.

ansible windows_servers -i inventory/hosts.yml -m ansible.windows.win_ping

If I get pong back, I know I'm good to go.
WinRM transport depends on your environment. I used ntlm since my servers weren't in a domain. If you're working in an Active Directory setup, kerberos is the better and more secure option.
Don't mix Linux and Windows modules. Early on I made the mistake of trying to use a Linux module on a Windows host — it fails and the error isn't always obvious. Stick to ansible.windows.* for everything Windows-related.

What Changed After Automating This
Before automation, setting up a Windows server usually meant opening RDP, clicking through configuration screens, enabling services manually, and hoping I didn’t forget something important.

Now the process is much simpler:

add the server to inventory,
run the playbook,
wait a few minutes,
done.

That consistency alone made the effort worth it.

Combined with the Linux setup from Part 1, I now have a single Ansible environment managing both Linux and Windows servers from one place. Getting both platforms working together took more troubleshooting than I expected, but finishing it felt genuinely rewarding.
Coming Up in Part 3
I'm planning to cover:

User management across Linux and Windows
Scheduling automated patching
Plugging Ansible into a CI/CD pipeline

Drop your questions or thoughts in the comments — always happy to discuss!
— Sireesha

Automating Linux & Windows Server Setup with Ansible: My DevOps Journey

Sireesharaju Kamparaju — Tue, 12 May 2026 13:01:58 +0000

I started this project because managing servers manually was getting repetitive — especially when it came to setup and configuration across both Linux and Windows machines.

So I decided to automate the whole workflow using Ansible. The goal wasn’t just to “use Ansible,” but to actually standardize how servers are configured and reduce the amount of manual SSH and RDP work I was doing every time a new system came up.

In this blog, I’ll walk through how I used Ansible playbooks and roles to automate server setup, including SSH configuration on Linux and basic provisioning tasks across Windows systems.

Why I Chose Ansible for This

**
Before I started using Ansible, server setup meant logging into each machine manually, running the same commands over and over, and hoping nothing was missed. One wrong step and the configuration was inconsistent across environments. Sound familiar?
What drew me to Ansible was its simplicity — no agents to install, no complex setup. Just SSH into Linux, WinRM into Windows, and you're managing your entire fleet from a single control node. That agentless architecture was a game changer for me.

Project Overview

The goal was straightforward: automate the initial setup of both Linux and Windows servers using a clean, reusable Ansible structure. Here's what I set out to configure:

SSH hardening and configuration on Linux servers
WinRM configuration to enable Ansible to talk to Windows servers
Reusable roles for both OS types
A master playbook to tie everything together.

Setting Up the Project Structure
The first thing I did was organize everything into roles. Roles keep your code clean, reusable, and easy to share across projects. Here's the structure I used:

ansible-server-setup/
├── inventory/
│   ├── hosts.yml
├── roles/
│   ├── linux_ssh/
│   │   ├── tasks/main.yml
│   │   ├── templates/sshd_config.j2
│   │   ├── handlers/main.yml
│   │   └── defaults/main.yml
│   ├── windows_setup/
│   │   ├── tasks/main.yml
│   │   └── defaults/main.yml
├── playbooks/
│   ├── linux_setup.yml
│   ├── windows_setup.yml
└── site.yml

Keeping Linux and Windows roles separate made everything much easier to maintain and debug independently.

The Inventory

I defined both Linux and Windows hosts in a single YAML inventory, with the right connection settings for each:

all:
  children:
    linux_servers:
      hosts:
        linux-01:
          ansible_host: 10.0.1.10
          ansible_user: ec2-user
          ansible_ssh_private_key_file: ~/.ssh/id_rsa
    windows_servers:
      hosts:
        win-01:
          ansible_host: 10.0.2.10
          ansible_user: Administrator
          ansible_password: "{{ vault_win_password }}"
          ansible_connection: winrm
          ansible_winrm_transport: ntlm
          ansible_port: 5985

Notice I vaulted the Windows password using ansible-vault — never hardcode credentials in plain text. Learned that lesson early!

SSH Hardening Role for Linux
This was the core of my Linux setup. The linux_ssh role handles SSH daemon configuration to make servers secure and consistent from day one.
roles/linux_ssh/defaults/main.yml

ssh_port: 22
permit_root_login: "no"
password_authentication: "no"
max_auth_tries: 3

roles/linux_ssh/templates/sshd_config.j2

Port {{ ssh_port }}
PermitRootLogin {{ permit_root_login }}
PasswordAuthentication {{ password_authentication }}
MaxAuthTries {{ max_auth_tries }}
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

---
- name: Deploy SSH configuration
  ansible.builtin.template:
    src: sshd_config.j2
    dest: /etc/ssh/sshd_config
    owner: root
    group: root
    mode: "0600"
  notify: Restart SSH

- name: Ensure SSH service is running and enabled
  ansible.builtin.service:
    name: sshd
    state: started
    enabled: true

roles/linux_ssh/handlers/main.yml


---
- name: Restart SSH
  ansible.builtin.service:
    name: sshd
    state: restarted

The handler ensures SSH only restarts when the config actually changes — not on every run. That small detail matters a lot in production.