DEV Community

Cover image for Active Directory Health Checklist: Metrics and Automation
beefed.ai
beefed.ai

Posted on • Originally published at beefed.ai

Active Directory Health Checklist: Metrics and Automation

  • Why a healthy Active Directory prevents service-wide outages
  • Which metrics actually predict outages: what to monitor and why
  • Automated AD checks, scripts, and tools that run reliably
  • Common failure modes and surgical remediation steps
  • Maintenance cadence, reporting, and dashboard must-haves
  • Actionable Checklist: runbooks, scripts, and schedules

Active Directory is the infrastructure that quietly enforces authentication, group policy, and application identity; when its replication, DNS, or time fabric breaks, failures cascade from single-user pain to domain-wide outages. Treating AD health as a monitoring problem with measurable signals and automated remediation prevents those cascades before they become incidents.

When replication stalls, symptoms look ordinary at first — slow Group Policy, delayed password changes, intermittent application auth failures — and then suddenly you're running down why service accounts stopped authenticating and why new users aren't visible across sites. Those symptoms trace back to a small set of signals you can monitor reliably: replication age and failures, NTDS performance counters, SYSVOL health, DNS correctness, available disk I/O, and time sync.

Why a healthy Active Directory prevents service-wide outages

A Domain Controller is more than an LDAP server; it's the authoritative source for authentication, authorization, policy, and many application integrations. AD replication ensures consistency across sites, and that replication depends on several moving parts: network connectivity and routing, DNS name resolution, accurate time for Kerberos (default tolerance 5 minutes), and a healthy NTDS database. Microsoft documents these dependencies and the standard troubleshooting surface to collect when things go wrong.

Important: replication is multi-layered — a network blip, a DNS mismatch, or a time skew can each present as an authentication outage. Collect the expected telemetry (repadmin/dcdiag output, Directory Service events, and NTDS counters) before making change decisions.

Which metrics actually predict outages: what to monitor and why

Below are the practical metrics that predict escalating trouble and the operational thresholds I use on client environments as baselines. Adjust tolerances against your traffic profile and SLAs; treat these as starting guards, not immutable laws.

Metric Why it matters Baseline alert thresholds (operational guidance) How to measure
Replication failures (count) Any non-zero failure count means data divergence risk — users, groups, and policies won't converge. Alert on > 0 failure(s) for any DC; escalate if persistent > 15 minutes. Get-ADReplicationFailure, repadmin /replsummary.
Last replication age (per partner) Shows how stale a DC is compared with its partners. Intra-site: notification delay defaults are seconds; surface if > 15 minutes. Inter-site: default site-link interval is 180 minutes — surface if older than configured interval. Operational target: converge intra-site within minutes; critical inter-site changes target < 60 minutes where possible. repadmin /showrepl and Get-ADReplicationPartnerMetadata.
SYSVOL replication state Group Policy and logon scripts live here; broken SYSVOL means GPOs won't apply. Any SYSVOL not shared or DFSR errors → high severity. dfsrmig /getmigrationstate, DFSR event logs.
NTDS / LDAP latency counters Long request latency indicates overloaded DC or expensive LDAP searches that slow everything. NTDS\Request Latency trending upward; NTDS\Estimated Queue Delay > 0 is a risk; investigate if Request Latency > 100ms sustained. Use Event ID 1644 analysis for expensive queries. Get-Counter '\DirectoryServices(NTDS)\*', Event ID 1644 parsing.
Disk I/O latency for NTDS volume NTDS performance is disk-bound; bad storage kills replication and auth performance. SSD: < 3ms read; 7,200 rpm: 9–12.5ms read. Generate alerts if reads/writes exceed safe range for your disk type. \LogicalDisk(<NTDS>)\Avg Disk sec/Read, capacity planning guidance.
CPU / Memory / Page faults Sustained CPU > 80% or extreme paging impairs responsiveness. Alert on sustained CPU > 80% for > 5 minutes; memory pressure causing paging is high severity. Perf counters \Processor(_Total)\% Processor Time, \Memory\% Committed Bytes In Use.
Directory Service error events (1311, 1865, 2042, 8614, 1644) Known error IDs map to topology, connectivity, or lingering-object issues. Alert at first occurrence for 1311/1865/2042; 8614/1644 require immediate triage. Query Directory Service event log.
Tombstone lifetime and backup age Restores older than tombstone lifetime are invalid; backups must be recent enough to be usable. Ensure at least daily backups; investigate if domain partition backups older than half of the tombstone lifetime. Tombstone lifetime historically varies — check the attribute on your forest. Check tombstoneLifetime and backup dates; Microsoft docs on tombstone behavior.

Key references and behaviors are documented by Microsoft for the tools and interval mechanics: dcdiag for DC functional tests, repadmin for replication state and summaries, and the site-link interval defaults (180 minutes) and intra-site notify defaults (15 seconds / 3-second subsequent pause).

Automated AD checks, scripts, and tools that run reliably

Automation reduces mean time to detection. The fast wins are small, frequent checks that capture the five high-value signals: replication failures, last replication time, SYSVOL state, NTDS performance counters, and critical Directory Service events. Use a dedicated management host (RSAT installed) or a runbook worker that has the Active Directory PowerShell module.

Recommended toolkit (field-proven):

  • repadmin, dcdiag — first-line diagnostics and topology checks.
  • Active Directory PowerShell module: Get-ADReplicationFailure, Get-ADReplicationPartnerMetadata.
  • Get-Counter / PerfMon for NTDS counters and disk latency.
  • Azure / Microsoft Entra Connect Health for hybrid telemetry when you run Azure AD Connect. The agent centralizes alerts into the Microsoft portal.
  • A SIEM (Splunk/Elastic) or APM that ingests Windows performance counters and event logs for long-term trend detection.

Minimal hourly check (PowerShell sample)

# Hourly-AD-QuickCheck.ps1  — run from a management host with AD module and RSAT
Import-Module ActiveDirectory -ErrorAction Stop

$timestamp = Get-Date -Format "yyyyMMdd-HHmm"
$outdir = "C:\ADHealth\Checks\$timestamp"; New-Item -Path $outdir -ItemType Directory -Force | Out-Null

# 1) Replication failures
Get-ADReplicationFailure -Scope Forest -Target * | Export-Csv -Path "$outdir\ReplicationFailures.csv" -NoTypeInformation

# 2) Replication partner metadata (last results)
Get-ADReplicationPartnerMetadata -Target * -Scope Server |
  Select-Object Server, Partner, LastReplicationAttempt, LastReplicationResult |
  Export-Csv "$outdir\ReplicationMetadata.csv" -NoTypeInformation

# 3) Repadmin summary (text)
repadmin /replsummary > "$outdir\repadmin_replsummary.txt"

# 4) Key perf counters (sample 5s * 3)
$ctr = @(
  '\NTDS\LDAP Searches/sec','\NTDS\Request Latency','\NTDS\Estimated Queue Delay',
  '\LogicalDisk(C:)\Avg. Disk sec/Read','\Processor(_Total)\% Processor Time'
)
Get-Counter -Counter $ctr -SampleInterval 5 -MaxSamples 3 | Export-CliXml "$outdir\PerfSample.xml"

# 5) Key Directory Service events
$ids = @(1311,1865,2042,8614,1644)
Get-WinEvent -FilterHashtable @{LogName='Directory Service'; ID=$ids; StartTime=(Get-Date).AddHours(-2)} |
  Export-Csv "$outdir\DS_Events.csv" -NoTypeInformation

# 6) Basic disk free check
Get-WmiObject Win32_LogicalDisk -Filter "DeviceID='C:'" |
  Select-Object DeviceID,FreeSpace,Size,@{n='FreePct';e={[math]::round(($_.FreeSpace/$_.Size)*100,1)}} |
  Export-Csv "$outdir\DiskSpace.csv" -NoTypeInformation
Enter fullscreen mode Exit fullscreen mode

This sample writes output to a timestamped folder that can be ingested by a SIEM or parsed by a separate alerting script. Schedule with Task Scheduler or your automation platform to run hourly; persist a rolling 7–14 day history for trend analysis.

When a single check shows replication errors, collect the triage artifacts immediately and attach them to the alert: dcdiag /v /c /e, repadmin /showrepl <DC>, repadmin /replsummary, event logs around the timestamps. dcdiag and repadmin are the canonical first-stop tools.

Common failure modes and surgical remediation steps

When you respond to an AD incident, work down a short, prioritized triage path — collect, isolate, fix. Below are common failures I see and the surgical steps that restore replication and service quickly.

  1. DNS resolution failures (clients/servers cannot find DCs)

    • Symptom: dcdiag DNS tests fail; clients get KDC or domain controller not found errors.
    • Quick triage: run dcdiag /test:DNS /v and nslookup -type=SRV _ldap._tcp.dc._msdcs.<domain>.
    • Surgical steps: verify DC SRV records in the authoritative DNS zone; run nltest /dsgetdc:<domain> to verify discovery; restart Netlogon to force record re-registration: net stop netlogon && net start netlogon. Re-check dcdiag.
  2. Time skew (Kerberos failures / replication blips)

    • Symptom: authentication fails, KDC errors, replication errors referencing Kerberos or time.
    • Triage: run w32tm /query /status on PDC Emulator and on problem DCs. Verify PDC emulator sync source.
    • Surgical steps: ensure PDC Emulator points to a reliable external NTP source and that all DCs use domain hierarchy for time. Correct large skews before remediating replication.
  3. SYSVOL / Group Policy not replicating (FRS/DFSR issues)

    • Symptom: GPOs not applied or NETLOGON/SYSVOL shares missing; DFSR/FRS event errors.
    • Triage: dfsrmig /getmigrationstate, inspect DFSR event logs (DFSR and File Replication Service logs).
    • Surgical steps: follow Microsoft’s SYSVOL migration/repair guides; perform non-authoritative/authoritative DFSR sync if required.
  4. Lingering objects / tombstone lifetime enforcement (Event 2042 / 8614)

    • Symptom: replication blocked with errors that mention tombstone lifetime or "too long since this machine replicated".
    • Triage: run repadmin /showrepl and repadmin /replsummary to find partners with errors; run repadmin /removelingeringobjects as appropriate.
    • Surgical steps: remove lingering objects and then temporarily allow replication with divergent partners only when safe: repadmin /regkey <hostname> +allowDivergent per Microsoft guidance; after successful inbound replication, reset with repadmin /regkey <hostname> -allowDivergent. Do the cleanup in a controlled maintenance window and document each change.
  5. USN rollback / VM snapshot restores (virtualized DCs)

    • Symptom: Event IDs 1109, 2170, or "invocationID attribute changed" after a VM revert, or unexpected RID pool invalidation.
    • Triage: inspect Directory Services/System event logs for GenerationID and invocationID messages.
    • Surgical steps: do not treat VM snapshots as AD backups; follow the Microsoft guidance for safe restore and, if a rollback occurred, perform the supported non-authoritative restore or rebuild the DC from system-state backup. Virtualized DCs require care — use backup methods that are AD-aware.
  6. NTDS database corruption or performance problems (heavy LDAP queries)

    • Symptom: high NTDS\Request Latency, Event 1644 entries for expensive LDAP searches, or database integrity errors.
    • Triage: collect the NTDS performance counters and run the Event1644 analysis script to surface expensive queries.
    • Surgical steps: identify and fix the bad queries (application-side), increase DC capacity or move workloads, and run database integrity/semantic analysis with ntdsutil in DSRM if corruption is suspected.
  7. Failed DC that must be removed (forced demotion / metadata left behind)

    • Symptom: a permanently offline DC still listed and causing topology confusion.
    • Surgical steps: remove the DC object via ADUC or Sites & Services (modern RSAT will perform metadata cleanup automatically) or use ntdsutil metadata cleanup following Microsoft cleanup procedures. Re-assess FSMO roles and transfer/seize as required.

Maintenance cadence, reporting, and dashboard must-haves

A predictable cadence shows trends before incidents. This is the practical schedule I deploy for enterprise AD environments:

  • Continuous / Real-time: alerting for replication failures, Directory Service critical events, and SYSVOL share down events. Send these to an on-call channel.
  • Hourly: run the minimal hourly quick-check script (replication failures, last replication times, key perf counters). Archive the last 24 hours of results for trend detection.
  • Daily: run dcdiag /v /c /e across all DCs, check backups, validate that at least one valid, recent system-state backup exists for each writeable DC (check backup age vs tombstone lifetime).
  • Weekly: review capacity trends (disk IO latency, NTDS request latency, CPU), top-k expensive LDAP queries, and replication convergence graphs.
  • Monthly: run a full topology and site-link review; validate FSMO placement and Global Catalog distribution; verify SYSVOL migration status if still on FRS.
  • Quarterly (or before major changes): run a rehearse of an authoritative/non-authoritative restore on a lab DC, validate DSRM password records and restore playbooks.

Dashboard must-haves (one-line): replication failures by DC, maximum replication age, NTDS request latency 95th percentile, disk I/O latency for NTDS volumes, count of Directory Service critical events, and backup freshness relative to tombstone lifetime. Tie these to SLA/priority buckets (P0: replication failure on DC hosting unique naming context; P1: SYSVOL not shared; P2: KPI performance degradation).

Azure/Microsoft tooling note: where you run hybrid identity, the Microsoft Entra Connect Health agents provide a centralized view for AD DS and the sync engine — ingest that into your portal for consolidated alerts.

Actionable Checklist: runbooks, scripts, and schedules

Concrete runbook snippets you can drop into ops playbooks.

1) Immediate replication triage (minutes)

  • Collect artifacts:
    • repadmin /replsummary
    • repadmin /showrepl <problemDC> /csv
    • dcdiag /v /c /e /s:<problemDC> > dcdiag_<dc>.txt
    • Export Directory Service event log around the failure time (Get-WinEvent).
  • Quick checks:
    • Verify DNS SRV records and Netlogon registration (nslookup -type=SRV _ldap._tcp.dc._msdcs.<domain>; nltest /dsgetdc:<domain>).
    • Check time skew (w32tm /query /status) — ensure less than 5 minutes skew for Kerberos.
  • Containment:
    • On safe, non-production breakout runs, allow divergent replication only as Microsoft documents for a short window; run repadmin /removelingeringobjects before allowing divergent replication. Revoke +allowDivergent after convergence.

2) Post-incident remediation checklist

  • Run dcdiag and repadmin across forest to ensure convergence.
  • Confirm SYSVOL health and DFSR state if GPOs were impacted.
  • Validate that backups exist and are newer than half your tombstone lifetime; document backup age.
  • If a DC is irrecoverable, follow metadata cleanup procedures and demote/rebuild the DC per Microsoft guidance.

3) Example escalation bundle command (collect everything into a folder)

# Run on management host; requires AD module and elevated privileges
$now = (Get-Date).ToString('yyyyMMdd-HHmm')
$dir = "C:\ADIncident\$now"; New-Item $dir -ItemType Directory -Force | Out-Null
repadmin /replsummary > "$dir\repadmin_replsummary.txt"
repadmin /showrepl * /csv > "$dir\repadmin_showrepl_all.csv"
dcdiag /v /c /e > "$dir\dcdiag_full.txt"
Get-WinEvent -FilterHashtable @{LogName='Directory Service'; StartTime=(Get-Date).AddDays(-1)} | Export-Clixml "$dir\DS_Events.xml"
Get-Counter '\DirectoryServices(NTDS)\*' -MaxSamples 1 | Export-CliXml "$dir\NTDS_Perf.xml"
Compress-Archive -Path "$dir\*" -DestinationPath "$dir.zip" -Force
Enter fullscreen mode Exit fullscreen mode

4) Scheduling and retention

  • Hourly quick checks (keep last 48 hours on disk, roll to SIEM).
  • Daily full diagnostics at 03:30 local (off-peak): dcdiag + backup validation (keep 30 days indexed).
  • Monthly full topology review and practice DR on an isolated lab.

Closing

Operational discipline — small, frequent, measurable checks coupled with short, scripted remediation playbooks — is the difference between a one-hour blip and a domain-wide outage. Focus your automation on the five signals that predict escalation, keep your runbooks executable (commands + logs), and enforce backup age rules relative to tombstone lifetime so restores remain safe. Deploy the checks, run the playbooks, and let the telemetry tell you when to act.

Sources:
DCDiag — Microsoft Learn - Reference for dcdiag tests, what they validate (DNS, LDAP, replication), and usage parameters.

Repadmin /showrepl — Microsoft Learn - Guidance for repadmin, showrepl, and replsummary usage for replication diagnostics.

Diagnose Active Directory replication failures — Microsoft Learn - Explains AD replication dependencies (DNS, network, time), common errors, and triage steps.

Determining the Interval — Microsoft Learn - Documentation of site-link replication interval defaults (default 180 minutes) and minimum interval constraints.

Modify the default intra-site DC replication interval — Microsoft Learn - Shows notification delays (default notify-first 15s, subsequent 3s) and repadmin /notifyopt usage.

Phantoms, tombstones, and the infrastructure master — Microsoft Learn - Describes tombstone lifetime semantics and lifecycle of deleted objects.

Capacity planning for Active Directory Domain Services — Microsoft Learn - Performance counters and recommended disk latency ranges for NTDS.

What is Microsoft Entra Connect? — Microsoft Learn - Overview of Microsoft Entra (Azure) Connect and the Entra Connect Health monitoring capabilities for on-prem identity.

Virtualized Domain Controller Troubleshooting — Microsoft Learn - Guidance about GenerationID, snapshot pitfalls, and supported restore methods for virtualized DCs.

Migrate SYSVOL replication from FRS to DFS Replication — Microsoft Learn - SYSVOL replication behavior and the dfsrmig migration procedure.

Use Event1644Reader.ps1 to analyze LDAP query performance — Microsoft Learn - How to analyze expensive LDAP queries and interpret Event ID 1644.

Active Directory Forest Recovery - Determine how to recover the forest — Microsoft Learn - Authoritative and non-authoritative restore concepts, DSRM and ntdsutil guidance.

Clean up Active Directory Domain Controller server metadata — Microsoft Learn - Procedures for metadata cleanup after forced DC removal and ntdsutil usage.

Active Directory replication Event ID 2042 — Microsoft Learn - Steps for addressing Event ID 2042 including repadmin /regkey +allowDivergent guidance.

Top comments (0)