DEV Community

Sireesharaju Kamparaju
Sireesharaju Kamparaju

Posted on

Patching Windows Server 2022 with Ansible & AWX — A Production Approach — My DevOps Journey

If you've been following my blog series, you know I've been building out a full Ansible automation setup. Most Windows patching blogs show you a basic win_updates task and call it done. That's fine for a lab. But when you're patching 30 production Windows servers one by one, you need something that actually tells you what's going to happen before it happens, handles reboots gracefully, validates the result, and leaves a log trail you can refer back to.

In this blog I'll walk through the production-grade Windows patching playbook I built and run through AWX at work. It covers four distinct phases:

  • Pre-check — OS build, pending reboots, disk space, WSUS detection, update scan
  • Patch — two-pass patching with automatic reboot handling
  • Post-check — OS build verification, uptime, installed hotfixes
  • Validate — build comparison before vs after, final ping test

Each phase runs as a separate AWX job template using Ansible tags — so you're in full control of what runs and when. We patch one server at a time (serial: 1) so a failure on one host never cascades to the rest of the fleet.


The Architecture — How It Runs in AWX

Before looking at the playbook itself, here's how I've wired this up in AWX:

AWX Workflow: Windows Patching
│
├── Job Template 1: Pre-Check (tags: precheck)
│ └── [On Success]
├── Job Template 2: Patch (tags: patch)
│ └── [On Success]
├── Job Template 3: Post-Check (tags: postcheck)
│ └── [On Success]
└── Job Template 4: Validate (tags: validate)
Enter fullscreen mode Exit fullscreen mode

Each job template runs the same playbook (windows_patching_with_clear_logs.yaml) but with a different tag. This gives you full visibility in AWX — you can see exactly which phase passed or failed, and stop the workflow at any point before moving to the next phase.

In our environment, all 30 servers are in the same AWX inventory. The playbook uses serial: 1 which means AWX runs through all four phases on server 1 before moving to server 2. One server at a time, fully controlled.


The Playbook Variables

At the top of the playbook I define the key variables that control the patching behaviour:

vars:
  patch_categories:
    - SecurityUpdates
    - CriticalUpdates
    - UpdateRollups
  windowslogs_dir: "windowslogs/windows-patching"
  patch_passes: 2
  upd_search_log: "C:\\Windows\\Temp\\ans_upd_search.log"
Enter fullscreen mode Exit fullscreen mode

patch_categories — the three update types I install. I deliberately exclude FeaturePacks and DefinitionUpdates to keep patching focused on security and stability.

windowslogs_dir — where log files get saved on the Ansible controller, not on the Windows servers. Every host gets its own timestamped log file so you have a full audit trail.

patch_passes — set to 2 because Windows often needs a second pass after a reboot to pick up remaining updates, especially Cumulative Updates and Servicing Stack updates.


Pre-Tasks — Setting Up Log Files

Before any tasks run, I set up timestamped log file paths on the controller:

pre_tasks:
  - name: Build run timestamp on controller (for unique filenames)
    delegate_to: localhost
    run_once: true
    set_fact:
      date_str: "{{ lookup('ansible.builtin.pipe', 'date +%Y%m%d_%H%M%S') }}"
    tags: [always]

  - name: Ensure windowslogs dir exists on controller
    delegate_to: localhost
    run_once: true
    ansible.builtin.file:
      path: "{{ windowslogs_dir }}"
      state: directory
      mode: "0755"
    tags: [always]

  - name: Build per-host log file paths (on controller)
    set_fact:
      precheck_file: "{{ windowslogs_dir }}/{{ inventory_hostname }}_precheck_{{ date_str }}.txt"
      postcheck_file: "{{ windowslogs_dir }}/{{ inventory_hostname }}_postcheck_{{ date_str }}.txt"
      updates_file:  "{{ windowslogs_dir }}/{{ inventory_hostname }}_updates_{{ date_str }}.txt"
    tags: [always]
Enter fullscreen mode Exit fullscreen mode

The tags: [always] means these tasks run regardless of which tag you pass — log paths are always set up first.


Phase 1 — Pre-Check

This is the most important phase. I never patch a server without running pre-checks first.

1. Get the Full OS Build Including UBR

The standard OS build number isn't enough. Windows uses an Update Build Revision (UBR) on top of the major build — so two servers with the same OsBuildNumber might be at different patch levels:

- name: Get OS version/build before patch (full build)
  ansible.windows.win_shell: |
    $ci = Get-ComputerInfo
    $cv = Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion'
    $ubr = $cv.UBR
    $major = $ci.OsBuildNumber
    $full = '{0}.{1}' -f $major, $ubr
    [PSCustomObject]@{
      OsVersion     = $ci.OsVersion
      OsBuildNumber = $major
      UBR           = $ubr
      FullBuild     = $full
      Caption       = $ci.OsName
    } | ConvertTo-Json -Compress | Out-String
  register: pre_os
  check_mode: no
  changed_when: false
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

I then validate this output — if it comes back as UNKNOWN, the playbook fails immediately:

- name: Validate pre_os output
  ansible.builtin.assert:
    that:
      - pre_os_version     != 'UNKNOWN'
      - pre_os_build_major != 'UNKNOWN'
      - pre_os_build_full  != 'UNKNOWN'
    fail_msg: "Failed to parse OS info on {{ inventory_hostname }}."
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

2. Check for Pending Reboots

If a server already has a pending reboot, patching it is risky. I check two registry keys:

- name: Check pending reboot before patch (PowerShell fallback)
  ansible.windows.win_shell: |
    $paths = @(
      "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending",
      "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired"
    )
    $pending = $false
    foreach ($p in $paths) { if (Test-Path $p) { $pending = $true } }
    if ($pending) { "true" } else { "false" }
  register: pre_reboot_raw
  check_mode: no
  changed_when: false
  tags: [precheck]

- name: Stop if a reboot is already pending
  ansible.builtin.fail:
    msg: "Reboot is already pending on {{ inventory_hostname }}. Reboot first, then patch."
  when: pre_reboot.reboot_pending | default(false)
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

3. Check Disk Space

- name: Check free space on system drive (bytes)
  ansible.windows.win_shell: |
    $sys = ([System.Environment]::GetEnvironmentVariable('SystemDrive'))
    Get-PSDrive -Name $sys.TrimEnd(':') |
    Select-Object Used,Free | ConvertTo-Json -Compress | Out-String
  register: pre_disk
  check_mode: no
  changed_when: false
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

4. WSUS Auto-Detection

One of the things I'm most proud of in this playbook. It auto-detects whether WSUS is configured and adjusts accordingly:

- name: Detect WSUS policy (is WSUS enforced?)
  ansible.windows.win_shell: |
    $wuKey = 'HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate'
    $auKey = 'HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU'
    $obj = [ordered]@{
      UseWUServer = $null
      WUServer    = $null
      WUStatus    = $null
    }
    if (Test-Path $auKey) {
      $obj.UseWUServer = (Get-ItemProperty $auKey -ErrorAction SilentlyContinue).UseWUServer
    }
    if (Test-Path $wuKey) {
      $p = Get-ItemProperty $wuKey -ErrorAction SilentlyContinue
      $obj.WUServer  = $p.WUServer
      $obj.WUStatus  = $p.WUStatusServer
    }
    $obj | ConvertTo-Json -Compress | Out-String
  register: wsus_raw
  check_mode: no
  changed_when: false
  tags: [precheck]

- name: Decide update server selection (WSUS vs Microsoft Update)
  set_fact:
    update_server_selection: >-
      {{
        'managed_server'
        if (wsus_use_server == 1 and (wsus_url | length > 0))
        else 'windows_update'
      }}
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

5. Async Update Scan

Before touching anything, I do a full scan without downloading or installing:

- name: Kick off update search (no download/install)
  ansible.windows.win_updates:
    state: searched
    category_names: "{{ patch_categories }}"
    server_selection: "{{ update_server_selection }}"
    log_path: "{{ upd_search_log }}"
  register: pre_search_async
  async: 7200
  poll: 0
  tags: [precheck]

- name: Wait up to 20 minutes for search to finish
  async_status:
    jid: "{{ pre_search_async.ansible_job_id }}"
  register: pre_search
  until: pre_search.finished
  retries: 40
  delay: 30
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

6. Predict Patching Complexity

- name: Predict complexity (long run / multi-reboot heuristic)
  set_fact:
    candidate_updates: "{{ pre_search.updates | default([]) }}"
    will_be_long: >-
      {{ (pre_search.updates | default([])
          | selectattr('title','search','Cumulative Update|Servicing Stack|Feature Update')
          | list | length) > 0 }}
    predicted_passes: "{{ 2 if (pre_search.updates | default([])
          | selectattr('title','search','Cumulative Update|Servicing Stack')
          | list | length) > 0 else 1 }}"
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

7. Save Pre-Check Log

- name: Save pre-check summary (controller)
  delegate_to: localhost
  ansible.builtin.copy:
    dest: "{{ precheck_file }}"
    mode: "0644"
    content: |
      Host: {{ inventory_hostname }}
      Caption: {{ pre_os_caption }}
      OS Version: {{ pre_os_version }}
      OS Build (major): {{ pre_os_build_major }}
      OS Build (full) : {{ pre_os_build_full }}
      Last Boot: {{ pre_uptime.stdout | trim }}
      Pending Reboot: {{ pre_reboot.reboot_pending | default(false) }}
      Disk (bytes): {{ pre_disk.stdout | trim }}
      Candidate Updates: {{ (candidate_updates | length) if candidate_updates is defined else 0 }}
      Predicted Long/Multi-Reboot: {{ will_be_long | default(false) }}
      Suggested Passes: {{ predicted_passes | default(1) }}
      Updates:
      {% for u in candidate_updates %}
        - {{ u.title if (u is mapping and 'title' in u) else (u|string) }}
      {% endfor %}
  tags: [precheck]
Enter fullscreen mode Exit fullscreen mode

Phase 2 — Patch

Once pre-checks pass and I've reviewed the log, I launch the patch job template in AWX. Two passes — first installs and reboots, second catches anything remaining:

- name: PATCH PASS 1 - install updates
  ansible.windows.win_updates:
    category_names: "{{ patch_categories }}"
    server_selection: "{{ update_server_selection }}"
    reboot: yes
    reboot_timeout: 3600
  register: patch_pass1
  tags: [never, patch]

- name: Wait for host to return (pass 1)
  wait_for_connection:
    delay: 20
    timeout: 1200
  when: patch_pass1.reboot_required | default(false)
  tags: [never, patch]

- name: PATCH PASS 2 - install remaining updates
  ansible.windows.win_updates:
    category_names: "{{ patch_categories }}"
    server_selection: "{{ update_server_selection }}"
    reboot: yes
    reboot_timeout: 3600
  register: patch_pass2
  when: patch_passes|int >= 2
  tags: [never, patch]

- name: Wait for host to return (pass 2)
  wait_for_connection:
    delay: 20
    timeout: 1200
  when:
    - patch_passes|int >= 2
    - patch_pass2.reboot_required | default(false)
  tags: [never, patch]
Enter fullscreen mode Exit fullscreen mode

tags: [never, patch] — the never means these tasks are skipped unless you explicitly pass --tags patch. This prevents accidental patching.


Phase 3 — Post-Check

- name: Get OS version/build postchecks (full build)
  ansible.windows.win_shell: |
    $ci = Get-ComputerInfo
    $cv = Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion'
    $ubr = $cv.UBR
    $major = $ci.OsBuildNumber
    $full = '{0}.{1}' -f $major, $ubr
    [PSCustomObject]@{
      OsVersion     = $ci.OsVersion
      OsBuildNumber = $major
      UBR           = $ubr
      FullBuild     = $full
    } | ConvertTo-Json -Compress | Out-String
  register: post_os
  tags: [never, postcheck]

- name: Get installed hotfixes postchecks
  ansible.windows.win_shell: >
    Get-HotFix | Sort-Object InstalledOn |
    Select-Object HotFixID, InstalledOn |
    Format-Table -HideTableHeaders | Out-String
  register: hotfixes
  tags: [never, postcheck]

- name: Save post-check summary (controller)
  delegate_to: localhost
  ansible.builtin.copy:
    dest: "{{ windowslogs_dir }}/{{ inventory_hostname }}_postcheck_{{ date_str }}.txt"
    mode: "0644"
    content: |
      Host: {{ inventory_hostname }}
      OS Version (post): {{ post_os_version }}
      OS Build (post, major): {{ post_os_build_major }}
      OS Build (post, full) : {{ post_os_build_full }}
      Last Boot (post): {{ post_uptime.stdout | trim }}
      Installed Updates:
      {{ hotfixes.stdout | trim }}
  tags: [never, postcheck]
Enter fullscreen mode Exit fullscreen mode

Phase 4 — Validate

- name: Summarize patch result
  ansible.builtin.debug:
    msg: |
      Patch Result for {{ inventory_hostname }}:
      - Categories: {{ patch_categories | join(', ') }}
      - Pass1: installed={{ (patch_pass1.updates | default([])) | length }},
               reboot={{ patch_pass1.reboot_required | default(false) }}
      - Pass2: installed={{ (patch_pass2.updates | default([])) | length
               if (patch_pass2 is defined) else 0 }},
               reboot={{ patch_pass2.reboot_required | default(false)
               if (patch_pass2 is defined) else false }}
      - OS Build Before (full): {{ pre_os_build_full | default('N/A') }}
      - OS Build After  (full): {{ post_os_build_full | default('N/A') }}
      - Status: {{
          'PATCH SUCCESSFUL - OS BUILD CHANGED'
          if (pre_os_build_full is defined and post_os_build_full is defined
              and pre_os_build_full != post_os_build_full)
          else 'NO BUILD CHANGE - verify installed hotfixes or CU cadence'
        }}
  tags: [never, validate]

- name: Final ping test
  ansible.windows.win_ping:
  tags: [never, validate]
Enter fullscreen mode Exit fullscreen mode

Setting This Up in AWX

Step 1 — Create Four Job Templates

Job Template Playbook Job Tags
Windows Pre-Check windows_patching_with_clear_logs.yaml precheck
Windows Patch windows_patching_with_clear_logs.yaml patch
Windows Post-Check windows_patching_with_clear_logs.yaml postcheck
Windows Validate windows_patching_with_clear_logs.yaml validate

Step 2 — Create the Workflow

  • AWX UI → TemplatesAddAdd Workflow Template
  • Name: Windows Patching Workflow
  • Click SaveWorkflow Visualiser

Chain the four job templates together using On Success:

Pre-Check → [On Success] → Patch → [On Success] → Post-Check → [On Success] → Validate

Step 3 — Add a Manual Approval Node

Between Pre-Check and Patch I add a manual Approval node so someone reviews the pre-check logs before patching starts:

  • In the Workflow Visualiser → between Pre-Check and Patch
  • Click + → select Approval
  • Name: Review Pre-Check Logs Before Patching
  • Timeout: 4 hours

This is the human checkpoint before anything touches production.


Errors I Hit in Production

Error 1 — WinRM Drops After Reboot and Never Comes Back

Fix — increase the wait_for_connection timeout:

wait_for_connection:
  delay: 20
  timeout: 1800
Enter fullscreen mode Exit fullscreen mode

Error 2 — WSUS Returning No Updates

WSUS hadn't approved the latest Cumulative Update. Fix — override at runtime in AWX by passing as an extra variable:

update_server_selection: windows_update
Enter fullscreen mode Exit fullscreen mode

Error 3 — UBR Not Changing After Patching

Check Get-HotFix output in the post-check log. If new KBs are listed even though the UBR didn't change, only smaller updates installed — worth investigating before signing off.

Error 4 — Async Update Scan Timing Out

Fix — increase async timeout and retries:

async: 7200
retries: 80
delay: 30
Enter fullscreen mode Exit fullscreen mode

What This Gives You

For 30 production Windows servers, this approach gives you:

  • A full audit trail — pre and post check logs per server per patching run
  • Safety gates — pending reboot detection, disk space checks and manual approval before patching starts
  • Intelligence — WSUS auto-detection means the same playbook works across different network segments
  • Visibility in AWX — each phase is a separate node in the workflow so you can see exactly where something failed
  • Confidence — build comparison before and after tells you definitively whether the Cumulative Update actually applied

What's Next

The next improvement I'm planning is adding email notifications to this workflow — so when patching fails on any of the 30 servers at 2AM, I get an alert immediately rather than finding out in the morning.

Drop your questions in the comments — happy to help!

— Sireesha

Top comments (0)