DEV Community: Raghavendra R

# A Failed Compliance Audit in Azure DevOps: Rebuilding CI/CD with Policy as Code and Security Gates

Raghavendra R — Sun, 07 Dec 2025 13:18:13 +0000

Rebuilding Azure DevOps CI/CD for Compliance

Rebuilding Azure DevOps CI/CD for Compliance
- Introduction
Core Concepts
- Compliance in Azure DevOps: Where It Lives
- Policy as Code: Three Levels
- Security Gates in Azure DevOps
- Multi-Environment, Multi-Subscription Design
Step-by-Step Guide
- 1. Map Audit Findings to Concrete Controls
- 2. Standardize CI/CD Architecture
- 3. Implement Template-Driven CI Pipelines
- 4. Embed Policy as Code for Infrastructure
- 5. Define Environments and Security Gates
- 6. Integrate Security Scanners as Gates
- 7. Observability and Auditability
- 8. Rollout Strategy Across Teams
Architecture & Flow Diagram
Best Practices
Common Pitfalls
- 1. "Templates" That Are Optional
- 2. Over-Permissive Service Connections
- 3. Scanners That Don't Fail Builds
- 4. Manual Change Approvals Outside CI/CD
- 5. Azure Policy Not Integrated with CI
- 6. Ignoring Non-Prod Environments
- 7. No Runbooks for Gate Failures
FAQ
- 1. How does this map to AWS and GCP?
- 2. How do I add compliance without slowing delivery?
- 3. How can I scale this across dozens of teams?
- 4. How do I handle legacy applications and pipelines?
- 5. How do I integrate with ITSM and change management?
- 6. What KPIs show that CI/CD compliance is working?
- 7. How do I handle multi-region or DR scenarios?
- 8. What's the role of GitHub if we already use Azure DevOps?
Conclusion
References

Introduction

A failed compliance audit on an Azure DevOps–backed delivery stack usually exposes the same issues: ad-hoc pipelines, inconsistent checks across projects, manual approvals in emails, and no traceable mapping between controls and the CI/CD implementation.

Rebuilding CI/CD in Azure DevOps with policy as code and security gates turns your pipeline into an auditable control plane:

Compliance requirements become versioned, testable artifacts.
Every build and deployment path is governed by the same rules.
Approvals, scans, and checks are enforced centrally instead of relying on tribal knowledge.

This article focuses on:

Translating compliance controls (ISO 27001, SOC 2, PCI, etc.) into Azure DevOps pipeline constructs.
Implementing policy as code across infrastructure, application, and pipeline configuration.
Designing security and compliance gates using Azure DevOps Environments, Approvals & Checks, and integrated scanners.
Rolling out these patterns across dev/qa/stage/prod at enterprise scale.

The primary cloud context is Azure (Azure DevOps + Azure platform), with brief mappings to AWS/GCP where useful.

Core Concepts

Compliance in Azure DevOps: Where It Lives

In an Azure-centric environment, compliance controls surface in four main areas:

Source control & change management
- Azure Repos or GitHub (with Azure DevOps pipelines).
- Branch policies, PR workflows, commit history.
- Required linked work items and change records.
CI/CD pipelines
- Azure Pipelines (YAML) as the automation backbone.
- Template-based pipelines shared across teams.
- Build, test, scan, deploy, and approval flows.
Infrastructure and configuration
- Infrastructure as Code (Terraform, Bicep, ARM).
- Azure Policy for runtime governance.
- Secret management in Azure Key Vault; access via Managed Identity.
Runtime environments
- AKS, App Service, Functions, Container Apps.
- VNets, subnets, NSGs, private endpoints, Application Gateway/Front Door.
- Azure Monitor, Log Analytics, Application Insights, Defender for Cloud.

A compliant architecture ensures the same controls are applied consistently at each layer, encoded as code/config rather than manual processes.

Policy as Code: Three Levels

Policy as code in Azure DevOps typically spans three levels:

Platform & Azure resource level
- Azure Policy: Deny or audit non-compliant resources (e.g., public IPs, unencrypted disks, missing tags).
- Terraform/Bicep linters & policy engines: OPA/Conftest, Checkov, Terrascan enforcing rules before apply.
- Example mappings:
  - Azure Policy → AWS Config / SCPs, GCP Organization Policies.
  - OPA/Conftest rules are cloud-agnostic and can be reused multi-cloud.
Pipeline level
- Centralized YAML templates containing required stages and jobs:
  - SAST, SCA, container scanning.
  - Infrastructure policy checks before apply.
  - Build provenance and artifact signing (where applicable).
- Restricted patterns:
  - Projects must use approved templates.
  - Limited surface for "inline" pipeline code.
Application level
- Code quality and security standards:
  - SonarQube/SonarCloud quality gates.
  - SAST tools (e.g., GitHub Advanced Security, Snyk, Fortify, etc.).
  - Dependency scanning (SCA) and container vulnerability scanning.
- Organizational policies (minimum code coverage, no critical vulns in prod).

Security Gates in Azure DevOps

Security gates implement "stop points" in CI/CD where policy must be satisfied before progressing:

Environment-based gates
- Azure DevOps Environments (e.g., dev, qa, stage, prod).
- Approvals & Checks bound to environments:
- Manual approvers and groups (segregation of duties).
- Business Hours checks.
- External service checks (e.g., custom API for risk assessment).
- Azure Monitor alerts or service health-based checks.
Quality gates in CI
- SonarQube/SonarCloud "Quality Gate must pass" as a build gate.
- Security scanners configured to fail the build on high/critical findings.
Pre-deployment and post-deployment gates
- Pre-deployment: checks before rollout (compliance scans, change record validation).
- Post-deployment: smoke tests, health checks, synthetic monitoring.

These gates are centralized and auditable: approvers, timestamps, and outcomes are recorded in Azure DevOps and/or Azure logs for evidence.

Multi-Environment, Multi-Subscription Design

For real enterprises, environments are usually split by subscription and/or management group:

mgmt → shared services (DevOps tools, monitoring, policy assignments).
nonprod → dev/qa/stage subscriptions.
prod → production subscriptions.

Azure DevOps interacts via:

Service connections using Managed Identities or service principals.
Environment-specific variables and variable groups or Key Vault references.
Region- and environment-specific policies (e.g., stricter network rules in prod).

The same pipeline definition runs across environments, but gates and policies are tuned per environment via configuration and Azure governance.

Step-by-Step Guide

1. Map Audit Findings to Concrete Controls

Extract failed controls from the audit (e.g., "no evidence that code changes are peer-reviewed").
Map each control to an Azure DevOps / Azure implementation:

Peer review → Pull request policy requiring reviewers.
Change approvals → Environment approvals & work item linkage.
Infrastructure deviations → Azure Policy assignments and IaC validation.
Secrets management → Azure Key Vault + RBAC, no secrets in pipelines.

Build a controls-to-implementation matrix (ideally in a repo):

Control ID
Description
Azure DevOps mechanism (branch policy, pipeline template, gate, etc.)
Azure platform mechanism (Azure Policy, Key Vault, RBAC, etc.)
Evidence location (logs, dashboards, reports).

This matrix drives the rest of the implementation and becomes part of audit evidence.

2. Standardize CI/CD Architecture

Create a platform repo that hosts:

Common pipeline templates (/pipelines/templates/*.yml).
Shared scripts and tooling (/scripts/*).
Policy definitions (/policies/*), e.g., OPA/Conftest rules, Checkov configs.
Documentation for teams on how to onboard.

Example minimal folder structure:

platform-pipelines/
  pipelines/
    templates/
      ci-template.yml
      cd-template.yml
      policy-checks.yml
  policies/
    opa/
    checkov/
  scripts/
    security/
    infrastructure/
  docs/
    controls-matrix.md
    onboarding-guides.md

3. Implement Template-Driven CI Pipelines

Use YAML templates to enforce common CI controls:

# /pipelines/templates/ci-template.yml
parameters:
  - name: runTests
    type: boolean
    default: true
  - name: sonarProjectKey
    type: string
  - name: sonarProjectName
    type: string

stages:
- stage: Build
  jobs:
  - job: Build
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: NodeTool@0
      inputs:
        versionSpec: '20.x'
    - script: npm ci
      displayName: Install dependencies
    - script: npm run build
      displayName: Build

    - ${{ if parameters.runTests }}:
      - script: npm test
        displayName: Run unit tests

- stage: Static_Analysis
  dependsOn: Build
  jobs:
  - job: SAST
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: NodeTool@0
      inputs:
        versionSpec: '20.x'
    - script: npm ci
      displayName: Install dependencies
    - script: npm run lint
      displayName: Lint
    - task: SonarQubePrepare@5
      inputs:
        SonarQube: 'SonarQube-Connection'
        scannerMode: 'CLI'
        configMode: 'manual'
        cliProjectKey: ${{ parameters.sonarProjectKey }}
        cliProjectName: ${{ parameters.sonarProjectName }}
    - task: SonarQubeAnalyze@5
    - task: SonarQubePublish@5
      inputs:
        pollingTimeoutSec: '300'

Project pipelines reference the template:

# app repo: azure-pipelines.yml
trigger:
  branches:
    include:
      - main

extends:
  template: pipelines/templates/ci-template.yml@platform-pipelines
  parameters:
    runTests: true
    sonarProjectKey: 'my-app-key'
    sonarProjectName: 'My Application'

This ensures every repository:

Implements the same build + SAST structure.
Automatically uses Sonar quality gates.
Is easily updated by modifying the platform template once.

4. Embed Policy as Code for Infrastructure

Assume Terraform for Azure infrastructure:

# Example: Azure Policy assignment via Terraform
resource "azurerm_policy_assignment" "deny_public_ip" {
  name                 = "deny-public-ip"
  scope                = azurerm_resource_group.app_rg.id
  policy_definition_id = data.azurerm_policy_definition.deny_public_ip.id
  enforcement_mode     = "Default"

  display_name = "Deny Public IP Assignment"
  description  = "Policy to deny creation of public IP addresses"
}

# Using a built-in Azure Policy definition
data "azurerm_policy_definition" "deny_public_ip" {
  name = "6c112d4e-5bc7-47ae-a041-ea2d9dccd749"  # Built-in policy ID for "Not allowed resource types"
}

# Alternative: Reference by display name (less reliable)
# data "azurerm_policy_definition" "deny_public_ip" {
#   display_name = "Not allowed resource types"
# }

Add policy checks in CI before terraform apply:

# /pipelines/templates/policy-checks.yml
stages:
- stage: Policy_Checks
  jobs:
  - job: Terraform_Validate
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - script: terraform init
      displayName: Initialize Terraform
    - script: terraform validate
      displayName: Validate Terraform configuration
    - script: terraform plan -out=tfplan
      displayName: Generate Terraform plan

  - job: Policy_Scan
    dependsOn: Terraform_Validate
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - script: |
        checkov -d . --framework terraform --output cli --output junitxml --output-file-path console,results.xml
      displayName: Run Checkov policy scans
    - task: PublishTestResults@2
      condition: always()
      inputs:
        testResultsFormat: 'JUnit'
        testResultsFiles: 'results.xml'
        testRunTitle: 'Checkov Policy Scan Results'

Attach this to your infra repos:

extends:
  template: pipelines/templates/policy-checks.yml@platform-pipelines

If Checkov/OPA finds a policy violation, the pipeline fails, preventing non-compliant infra from being applied, irrespective of who runs it.

5. Define Environments and Security Gates

Create Azure DevOps Environments for:

dev
qa
stage
prod

For each environment:

Configure Approvals & Checks:
- dev: maybe no manual approvals, but require successful policy & security checks.
- qa/stage: manual approvers from QA/SRE; check for linked work item with "Ready for test/Release".
- prod: change-management approver group, CAB-like workflow, and external status checks.

Sample CD stage referencing environments:

# /pipelines/templates/cd-template.yml
stages:
- stage: Deploy_Dev
  dependsOn: [Build, Static_Analysis]
  jobs:
  - deployment: deploy_dev
    environment: 'dev'
    strategy:
      runOnce:
        deploy:
          steps:
          - script: ./scripts/deploy-dev.sh

- stage: Deploy_Prod
  dependsOn: Deploy_Dev
  condition: succeeded()
  jobs:
  - deployment: deploy_prod
    environment: 'prod'
    strategy:
      runOnce:
        deploy:
          steps:
          - script: ./scripts/deploy-prod.sh

Approvals & Checks are configured on the dev and prod environments in the Azure DevOps UI:

prod environment:
- Required approvers group (e.g., "Production Approvers").
- External service check calling a compliance API ("Is this release approved?").
- Business Hours check (no prod deploys outside allowed window).

Azure DevOps records:

Who approved.
When they approved.
What was deployed.

This becomes solid audit evidence.

6. Integrate Security Scanners as Gates

In the CI stage:

SAST and SCA:
- Run on every commit.
- Fail on high/critical severity issues.
Container scanning:
- Scan images before pushing to ACR.
- Fail pipeline if CVEs exceed defined thresholds.

Example snippet:

steps:
- task: SnykSecurityScan@1
  inputs:
    serviceConnectionEndpoint: 'Snyk-Connection'
    testType: 'code'
    severityThreshold: 'high'
    monitorWhen: 'always'
    failOnIssues: true
  displayName: Snyk SAST/SCA

- script: |
    # Install Trivy
    sudo apt-get update && sudo apt-get install -y wget apt-transport-https gnupg lsb-release
    wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
    echo "deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
    sudo apt-get update && sudo apt-get install -y trivy

    # Scan container image
    trivy image --exit-code 1 --severity HIGH,CRITICAL --format sarif --output trivy-results.sarif $(imageName)
  displayName: Container vulnerability scan with Trivy

- task: PublishTestResults@2
  condition: always()
  inputs:
    testResultsFormat: 'VSTest'
    testResultsFiles: 'trivy-results.sarif'
    testRunTitle: 'Trivy Container Security Scan'

In CD:

Ensure the pipeline uses only images from the internal ACR, already scanned and tagged as compliant.

7. Observability and Auditability

Wire CI/CD and runtime to observable sources:

Azure DevOps:
- Audit logs for approvals, permission changes, service connections.
- Pipeline run history, including stage results and logs.
Azure Monitor + Log Analytics:
- Resource changes (Activity Log, Resource Graph).
- Azure Policy compliance dashboard.
- Defender for Cloud / Security Center recommendations.

Create dashboards showing:

% of compliant resources per subscription.
Number of deployments per environment and their success/failure rates.
Mean time to remediate non-compliant resources.

8. Rollout Strategy Across Teams

Start with platform and security-critical services.
Mandate platform templates for any new project.
Migrate existing pipelines in phases:
- Phase 1: Add security scans and approvals.
- Phase 2: Move to shared templates.
- Phase 3: Decommission legacy build/release pipelines.

Use Azure DevOps Project-level governance:

Restrict pipeline creation to templates.
Limit who can modify service connections and environment checks.
Enforce minimal RBAC for service connections (least privilege).

Architecture & Flow Diagram

Best Practices

Centralize pipeline logic
- Use YAML templates stored in a dedicated platform repo.
- Avoid per-project custom scripts unless strictly necessary.
Use Azure DevOps Environments for deployments
- Treat environments as security boundaries with their own approvals/checks.
- Configure gates per environment rather than embedding manual approvals in YAML.
Enforce branch policies
- Require PRs to main/release branches.
- Require successful CI and quality gates before merging.
- Require at least two reviewers for critical repos.
Integrate policy as code early
- Validate IaC (Terraform/Bicep) with OPA/Checkov before apply.
- Use Azure Policy to enforce guardrails at runtime (e.g., deny public internet exposure).
Lock down service connections
- Use Managed Identities or tightly scoped service principals.
- Restrict who can create/edit service connections.
- Audit changes regularly.
Automate secret management
- Store secrets in Azure Key Vault.
- Use Key Vault references and Managed Identity instead of pipeline variables.
Treat scanners as gates, not optional tools
- Make SAST, SCA, and container scanning blocking steps with defined thresholds.
- Configure alerting on repeated failures.
Evidence-first mindset
- For every control, define:
- Implementation mechanism.
- Evidence location and retention time.
- Automate reports/dashboards to export evidence for auditors.
Segregation of duties
- Separate roles:
- Platform team owns templates and environments.
- App teams own business logic and configuration values.
- Security team owns policy definitions and thresholds.
Version everything
- Version policies, templates, and gating logic.
- Use tags and releases in the platform repo to track "policy versions" over time.

Common Pitfalls

1. "Templates" That Are Optional

Mistake: providing recommended templates but allowing teams to bypass them.
Impact: fragmented compliance posture; some apps fully gated, others wide open.
Detection:
- Scan repositories for azure-pipelines.yml not referencing the platform repo.
Fix:
- Enforce a project or org policy: pipelines must use approved templates.
- Restrict who can create/edit pipelines.

2. Over-Permissive Service Connections

Mistake: one "god" service principal with Owner on all subscriptions.
Impact: audit findings, lateral movement risk, potential blast radius of pipeline compromise.
Detection:
- Review Azure DevOps service connection permissions and associated Azure RBAC roles.
Fix:
- Create environment-specific identities with least privilege.
- Use Management Groups and RBAC to scope access tightly.

3. Scanners That Don't Fail Builds

Mistake: running SAST/SCA scans, but ignoring results or only warning.
Impact: critical vulnerabilities shipped to production.
Detection:
- Check for steps where scanners run but no failure condition is configured.
Fix:
- Configure exit codes or fail-on-severity thresholds.
- Treat security findings as blocking gates, not optional reports.

4. Manual Change Approvals Outside CI/CD

Mistake: approvals done in emails or ticket comments without integration to pipelines.
Impact: no traceable linkage between change and deployment; audit evidence is weak.
Detection:
- Compare prod deployments with change records; look for missing linkage.
Fix:
- Require linked work items in PRs and deployments.
- Use environment approvals and external status checks that validate change IDs.

5. Azure Policy Not Integrated with CI

Mistake: relying solely on Azure Policy to block non-compliant resources post-deployment.
Impact: pipelines fail late; engineers frustrated by mysterious denies.
Detection:
- Look at Azure Policy deny events; if most come from CI, you have a shift-left gap.
Fix:
- Mirror Azure Policy rules into IaC scanners (Checkov/OPA).
- Fail early in CI, before apply or deployment.

6. Ignoring Non-Prod Environments

Mistake: strict governance only in prod; dev/qa are "wild west".
Impact: drift, shadow IT, data leaks (dev often holds real data), inconsistent testing.
Detection:
- Compare policy compliance and network rules across non-prod vs prod.
Fix:
- Apply similar guardrails in non-prod, with slightly relaxed thresholds if needed.
- Use same CI/CD architecture and policy bundles across all environments.

7. No Runbooks for Gate Failures

Mistake: gates fail but teams don't know what to do.
Impact: slow incident response, friction, gate bypasses.
Detection:
- Survey teams; track MTTR for gate-related failures.
Fix:
- Publish runbooks for each gate:
- Why it fails.
- Where to view details.
- How to remediate or escalate.

FAQ

1. How does this map to AWS and GCP?

AWS:
- Azure DevOps pipelines ↔ CodePipeline/CodeBuild or GitHub Actions.
- Azure Policy ↔ AWS Config, SCPs.
- Azure Monitor ↔ CloudWatch/CloudTrail.
GCP:
- Azure DevOps pipelines ↔ Cloud Build/Cloud Deploy or GitHub Actions.
- Azure Policy ↔ Organization Policies.
- Azure Monitor ↔ Cloud Logging/Monitoring.

The pattern is the same: centralized templates, policy as code, and environment-level gates.

2. How do I add compliance without slowing delivery?

Make checks fast and automated in dev/qa.
Reserve manual approvals only for high-risk operations (e.g., prod deploys).
Shift heavy scanning earlier in the pipeline to catch issues before the approval step.
Continuously tune thresholds based on data (false positives, frequency of issues).

3. How can I scale this across dozens of teams?

Create a platform team that owns:
- Templates, policies, and gates.
- Documentation and onboarding.
Make templates easy to adopt:
- Good defaults, minimal required parameters.
- Clear examples and starter pipelines.

4. How do I handle legacy applications and pipelines?

Start by wrapping legacy pipelines:
- Add scanners and approvals around them.
Gradually migrate:
- Move to YAML pipelines.
- Move to shared templates.
Keep a sunset plan and timeline for legacy release pipelines.

5. How do I integrate with ITSM and change management?

Require a change record ID tied to:
- Pull requests.
- Deployment stages.
Use environment external checks to validate change state (e.g., "Approved").
Store change IDs as variables in pipeline runs for traceability.

6. What KPIs show that CI/CD compliance is working?

Deployment frequency per environment.
Change failure rate and MTTR.
Policy compliance percentage across resources.
Number of pipeline runs failing due to policy/security, and their remediation times.
Reduction in audit findings over time.

7. How do I handle multi-region or DR scenarios?

Use the same templates and policies per region.
Environment naming can encode region: prod-euw, prod-use.
Use Azure Traffic Manager/Front Door and global routing policies.
Ensure compliance controls are applied in both primary and DR regions; treat DR as production from a compliance standpoint.

8. What's the role of GitHub if we already use Azure DevOps?

Many orgs use:
- GitHub for source control, PRs, and security (e.g., Dependabot, GHAS).
- Azure DevOps pipelines for CI/CD into Azure.
The same pattern applies:
- Policy as code and gates in Azure Pipelines.
- Branch policies and code scanning in GitHub.

Conclusion

A failed compliance audit is usually a symptom of invisible, inconsistent pipeline behavior. Rebuilding Azure DevOps CI/CD with policy as code and security gates converts scattered practices into a standardized, auditable system:

Controls live in code and templates, not in ad-hoc wikis.
Every deployment path is governed by the same rules.
Evidence for auditors is generated automatically via logs, dashboards, and approvals.

Concrete next steps:

Build a controls-to-implementation matrix and align on ownership.
Stand up a platform repo with templates, policies, and tooling.
Introduce environment-based gates and scanners as blocking steps.
Gradually migrate teams to the new pattern, starting with critical systems.

Bookmark this guide, share it with your platform/DevSecOps team, and post your own pipeline templates and policy bundles in the comments so the community can learn from real-world configurations.

References

Connect With Me

If you enjoyed this walkthrough, feel free to connect with me here:

# When Azure Front Door Won't Fail Over: Lessons from a Real Multi-Region DR Drill

Raghavendra R — Sun, 07 Dec 2025 13:12:22 +0000

Azure Front Door didn't fail over during a real multi-region DR drill. Here's what went wrong, how we fixed it, and how to design reliable failover.

The Story / Background
- The architecture we thought we had
- The drill
- What actually happened
Core Concepts: How Azure Front Door Failover Really Works
- Origin groups, priorities, and routing
- Health probes and what "healthy" really means
- Active-active vs active-passive in DR context
- Data tier is not Front Door's job
- Observability for failover
Step-by-Step Guide: Designing Azure Front Door for Real Multi-Region DR
- 1. Define RTO/RPO and failure modes
- 2. Design origin groups and health probe strategy
- 3. Implement with Terraform (example)
- 4. Build DR-aware pipelines and configuration management
- 5. Implement synthetic tests and dashboards
- 6. Run regular DR drills and chaos tests
Architecture Diagram
Best Practices for Azure Front Door Multi-Region DR
Common Pitfalls (and How to Avoid Them)
FAQ
Conclusion
References

A few quarters ago we ran what we thought would be a routine multi-region DR game day on Azure. The plan was simple: simulate a primary region failure, watch Azure Front Door detect the issue, fail over to the secondary region, and go for coffee feeling smug.

Instead, Front Door stared at our "dead" region and kept happily sending it traffic. Users got timeouts. Dashboards lit up. Our DR runbooks suddenly looked very theoretical. I'll walk through what actually happened, how we debugged it, and the patterns I use now whenever I put Azure Front Door in front of multi-region workloads.

The Story / Background

The architecture we thought we had

This was a fairly typical enterprise setup:

Front door / CDN: Azure Front Door Standard/Premium with WAF
Two Azure regions:
- Region A (primary) – AKS + internal Application Gateway, Azure SQL with geo-replica
- Region B (secondary) – warm standby AKS + App Gateway, Azure SQL geo-replica
Routing mode: Active-passive (priority routing) in Front Door
Health probes: Configured at the origin group level to hit /health on each region's App Gateway
Infra-as-Code: Terraform for Front Door, AKS, App Gateway, SQL, and plumbing
Observability: Azure Monitor, Log Analytics, Application Insights, plus synthetic checks from multiple locations

On paper, this ticked all the boxes: multi-region, DR runbooks, IaC, WAF in front, and tests.

The drill

The DR playbook was:

Simulate a partial outage in Region A.
Observe Front Door marking the primary origin unhealthy.
Confirm automatic failover to Region B.
Run smoke tests and declare the drill successful.

Simulation method: we applied a network ACL on the primary App Gateway subnet to effectively blackhole traffic from Front Door, mimicking a critical failure in the app tier.

What actually happened

Front Door did not immediately fail over.
Users got intermittent timeouts and 5xxs, but traffic kept trying Region A for long enough to trigger a production-level incident if this had been real.
Our synthetic checks (which hit the Front Door endpoint) kept reporting "green" for several minutes.
Logs seemed contradictory: App Gateway showed traffic drops; Front Door metrics looked almost normal.

It took a painful hour-plus of log diving and config reviews to realize:

Our health probe path /health was still responding 200 OK from a separate "status" service that hadn't been affected by the simulated failure.
The probe interval and sample size made failover slower than our target RTO.
Some internal services were bypassing Front Door and talking directly to Region A's private endpoints, so even if Front Door had failed over, we still had partial breakage.

The short version: the app died, but the health probes didn't. And Front Door did exactly what we told it to do, not what we thought we configured.

Core Concepts: How Azure Front Door Failover Really Works

Let's unpack what matters for Azure Front Door in a multi-region DR setup.

Origin groups, priorities, and routing

In Azure Front Door Standard/Premium:

You define origin groups (backend pools).
Within a group, each origin (Region A, Region B) can have:
- A priority (for active-passive)
- A weight (for active-active / traffic split)
Front Door sends traffic to the lowest-priority healthy origin.
If that origin becomes unhealthy, it will fail over to the next priority.

The word "healthy" hides a lot of detail.

Health probes and what "healthy" really means

Health probes are where most DR drills go to die:

Probes are configured per origin group with:
- Protocol & port (HTTP/HTTPS, 80/443, etc.)
- Path (e.g., /healthz, /live, /ready)
- Interval & sample size
Front Door considers an origin healthy if it gets enough 2xx/3xx responses from the probe within the configured sample window.
It considers an origin unhealthy after enough failures/timeouts in that window.

Key gotchas:

If your probe hits a different component than your critical path (e.g., a static health page, a separate sidecar), you'll see green while users are screaming.
If the probe is too forgiving (long intervals, large sample size), failover is slower than your RTO.
If the probe path is behind aggressive caching or a CDN rule, Front Door might be probing a cached thing, not your real app.

Active-active vs active-passive in DR context

Active-passive (priority routing)
- Simpler mental model: Region A is primary, Region B is standby.
- Good when your data tier or regulatory constraints make multi-master tricky.
Active-active (latency / weighted)
- Better utilization and resilience, but more complex for stateful workloads.
- Requires careful handling for session affinity, data consistency, and rollouts.

Front Door supports both via routing rules and origin group configuration, but DR behavior and testing strategy differ.

Data tier is not Front Door's job

Front Door only handles HTTP(S) routing. Your data layer is your responsibility:

Azure SQL with active geo-replication or auto-failover groups
Cosmos DB with multi-region writes
Redis with geo-replication or region-local caches
Storage accounts with RA-GRS or dual-write patterns

If your data tier can't fail over fast enough, Front Door can swap regions all day and users will still see errors or stale data.

Observability for failover

For real DR:

Azure Monitor & Log Analytics for Front Door metrics and logs
Application Insights for dependency failures, response times, distributed tracing
Synthetic tests (multi-region) that hit the Front Door endpoint with app-level expectations
End-to-end dashboards showing:
- Front Door health vs backend health
- Per-region error rates
- Failover events and timings

Step-by-Step Guide: Designing Azure Front Door for Real Multi-Region DR

1. Define RTO/RPO and failure modes

Before YAML and Terraform, write down:

RTO – how fast must failover complete?
RPO – how much data loss can you tolerate?
Failure modes you care about:
- Region outage
- App tier outage
- Partial dependency outage (e.g., DB or cache)
- Front Door misconfig / WAF block

Agree this with product, business, and security. DR that only works for "region disappeared" but not "DB is slow" is half a solution.

2. Design origin groups and health probe strategy

For an active-passive setup:

Single origin group with two origins: app-region-a, app-region-b.
Use priority: Region A = 1, Region B = 2.
Configure probes to hit a realistic but cheap path, e.g. /readyz that:
- Checks app's critical dependencies (DB, cache, queue) at lightweight level.
- Returns non-2xx when something essential is broken.

3. Implement with Terraform (example)

Here's a simplified Terraform snippet for Azure Front Door Standard/Premium with two origins and a health probe tuned for DR:

# Resource Group
resource "azurerm_resource_group" "network" {
  name     = "rg-network-prod"
  location = "East US"
}

# Azure Front Door Profile
resource "azurerm_cdn_frontdoor_profile" "prod" {
  name                = "fd-prod-profile"
  resource_group_name = azurerm_resource_group.network.name
  sku_name            = "Standard_AzureFrontDoor"

  tags = {
    environment = "production"
    purpose     = "multi-region-dr"
  }
}

# Front Door Endpoint
resource "azurerm_cdn_frontdoor_endpoint" "prod" {
  name                     = "fd-prod-endpoint"
  cdn_frontdoor_profile_id = azurerm_cdn_frontdoor_profile.prod.id

  tags = {
    environment = "production"
  }
}

# Origin Group with Health Probes
resource "azurerm_cdn_frontdoor_origin_group" "app" {
  name                     = "og-app-multiregion"
  cdn_frontdoor_profile_id = azurerm_cdn_frontdoor_profile.prod.id

  session_affinity_enabled = false

  health_probe {
    interval_in_seconds = 15
    path                = "/readyz"
    protocol            = "Https"
    request_type        = "GET"
  }

  load_balancing {
    additional_latency_in_milliseconds = 0
    successful_samples_required        = 3
    sample_size                        = 4
  }
}

# Primary Origin (Region A)
resource "azurerm_cdn_frontdoor_origin" "app_region_a" {
  name                           = "app-region-a"
  cdn_frontdoor_origin_group_id  = azurerm_cdn_frontdoor_origin_group.app.id
  host_name                      = "app-gw-eastus.contoso.internal"
  http_port                      = 80
  https_port                     = 443
  origin_host_header             = "app.contoso.com"
  priority                       = 1
  weight                         = 1000
  enabled                        = true

  certificate_name_check_enabled = true
}

# Secondary Origin (Region B)
resource "azurerm_cdn_frontdoor_origin" "app_region_b" {
  name                           = "app-region-b"
  cdn_frontdoor_origin_group_id  = azurerm_cdn_frontdoor_origin_group.app.id
  host_name                      = "app-gw-westus.contoso.internal"
  http_port                      = 80
  https_port                     = 443
  origin_host_header             = "app.contoso.com"
  priority                       = 2
  weight                         = 1000
  enabled                        = true

  certificate_name_check_enabled = true
}

# Route to map requests to origin group
resource "azurerm_cdn_frontdoor_route" "app_route" {
  name                          = "app-route"
  cdn_frontdoor_endpoint_id     = azurerm_cdn_frontdoor_endpoint.prod.id
  cdn_frontdoor_origin_group_id = azurerm_cdn_frontdoor_origin_group.app.id
  patterns_to_match             = ["/*"]
  supported_protocols           = ["Http", "Https"]
  https_redirect_enabled        = true

  forwarding_protocol    = "HttpsOnly"
  link_to_default_domain = true
}

4. Build DR-aware pipelines and configuration management

Treat Front Door config as code (Terraform/Bicep).
Protect it with:
- Pull requests and mandatory reviews.
- Policy checks (e.g., checks that every origin has a probe).
- Automated validation in a non-prod "chaos" environment.
Build pipelines that can:
- Temporarily disable an origin (simulated outage).
- Flip priorities if you need a manual failover.

Example Azure CLI snippet to temporarily disable Region A origin:

#!/bin/bash

# Disable primary origin for DR testing
az afd origin update \
  --resource-group rg-network-prod \
  --profile-name fd-prod-profile \
  --origin-group-name og-app-multiregion \
  --origin-name app-region-a \
  --enabled-state Disabled

echo "Origin app-region-a has been disabled. Traffic should failover to app-region-b."

# Monitor failover progress
echo "Monitoring Front Door metrics for 5 minutes..."
sleep 300

# Re-enable origin after test
read -p "Re-enable primary origin? (y/n): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
    az afd origin update \
      --resource-group rg-network-prod \
      --profile-name fd-prod-profile \
      --origin-group-name og-app-multiregion \
      --origin-name app-region-a \
      --enabled-state Enabled
    echo "Origin app-region-a has been re-enabled."
fi

Use this in non-prod to safely observe Front Door's behavior.

5. Implement synthetic tests and dashboards

Create synthetic tests that:
- Hit https://app.contoso.com/healthcheck-end-to-end
- Validate response code, body, and latency
- Run from multiple Azure regions (or external providers)
Build dashboards that show, per region:
- Front Door origin health state
- App response times
- Error rates and timeouts

Ensure your on-call runbook includes how to read these graphs during a DR event.

6. Run regular DR drills and chaos tests

Treat DR like CI:

Schedule recurring game days (quarterly is a good start).
Test different failure modes: origin disabled, DB unavailable, cache down, WAF rule gone wild.
Time how long:
- Front Door takes to mark the origin unhealthy.
- Users experience degraded performance.
- The team takes to declare failover complete.

Capture and track those as SLOs for DR.

Architecture Diagram

The diagram below illustrates the multi-region Azure Front Door DR architecture discussed in this post:

Key Components:

Azure Front Door acts as the global load balancer with WAF protection
Priority-based routing with Region A as primary (Priority 1) and Region B as secondary (Priority 2)
Health probes monitor /readyz endpoints to determine origin health
Geo-replicated Azure SQL ensures data availability across regions
Azure Monitor provides comprehensive observability across all components

Traffic Flow:

Normal Operation: User requests → Front Door → Region A (Primary) → Application Gateway → AKS → Azure SQL Primary
During Failover: Health probe fails on Region A → Front Door redirects traffic → Region B (Secondary) → Application Gateway → AKS → Azure SQL Geo-Replica
Monitoring: All components send telemetry to Azure Monitor and Application Insights for real-time observability

Best Practices for Azure Front Door Multi-Region DR

Health checks must reflect real risk
- Probe something that depends on your critical services (DB, cache, queue) but is cheap to execute.
Use explicit priorities for active-passive
- Don't rely on latency routing if your DR strategy is "primary then fail over".
Align probe configuration with RTO
- Shorter intervals and smaller sample sizes mean faster failover, at the cost of more sensitivity to transient blips.
Decouple internal vs external paths
- Ensure internal clients also route via Front Door (or a consistent DR mechanism), otherwise they'll keep hitting a dead region.
Keep origin host headers consistent
- Use a single app host name to simplify config, TLS, and debugging.
Tag everything
- Use tags for env, region, dr-role, owner, criticality. Helps a lot in DR reviews and cost tracking.
Secure by default
- Use WAF, private origins (Private Link / internal App Gateway), and managed identities.
Centralize observability
- One place where SRE/DevOps can see Front Door + app + DB health across regions.
Automate DR verification
- After every significant infrastructure or Front Door change, run automated DR checks in lower environments.

Common Pitfalls (and How to Avoid Them)

1. Health probes hitting the wrong thing

Problem: Probes target a static /health that doesn't reflect real dependencies.

Impact: Front Door sees green while the app is actually broken, delaying failover or preventing it entirely.

Fix:

Implement /readyz or /healthz-deep that checks key dependencies.
Make sure it returns non-2xx when critical components are broken.

2. Probes behind caching or CDN rules

Problem: Health probe requests get cached or served by a rule path that hides backend errors.

Impact: Probes never see failures; Front Door won't fail over.

Fix:

Exclude health probe paths from caching and rewrites.
Validate with logs that probes hit the actual app.

3. Overly large sample sizes and long intervals

Problem: Probe interval = 60s, sample size = 16, successful samples required = 15.

Impact: It can take many minutes of continuous failures before Front Door marks an origin unhealthy.

Fix:

Tune probe interval and samples to align with your RTO.
In many enterprise setups, something like 15–30s intervals and small sample windows (e.g., 3 out of 4) is a better starting point.

4. Internal traffic bypassing Front Door

Problem: Internal services talk directly to App Gateway or App Service in Region A.

Impact: External users may fail over via Front Door, but internal APIs and jobs still rely on the failed region.

Fix:

Use Front Door (or an equivalent internal traffic manager) as the standard entry point for inter-service communication where DR matters.
Or implement separate internal traffic management with the same multi-region logic.

5. No DR for the data tier

Problem: App tier is multi-region, but SQL or Redis is single-region.

Impact: Failover appears successful at the HTTP layer, but the secondary region has no usable data.

Fix:

Plan data DR first: geo-replication, multi-region writes, failover groups.
Wire app config (connection strings, secrets) to automatically use the correct endpoint after failover.

6. DR tests only in staging

Problem: DR game days happen in lower environments that don't mirror prod topology, traffic patterns, or data sensitivity.

Impact: False confidence. Things that worked in staging break in production.

Fix:

Run carefully scoped DR drills in production: limited time windows, pre-announced, with a rollback plan.
Start small (e.g., partial traffic) and grow once you've built muscle.

7. No clear runbook for Front Door changes

Problem: During an incident, engineers manually poke around in the Azure Portal, toggling origins and routing rules.

Impact: Slow response, new mistakes, hard to audit.

Fix:

Document and automate incident playbooks:
- "Disable primary origin"
- "Force traffic to Region B"
- "Roll back to normal state"
Implement them as scripts or pipeline tasks, not "click here, then here".

FAQ

1. Azure Front Door vs Traffic Manager vs DNS for DR?

Front Door: Layer 7 routing, WAF, caching, modern Standard/Premium features; ideal for web/API DR.
Traffic Manager: DNS-based routing, good for non-HTTP workloads or hybrid scenarios.
DNS only: Very coarse and slow control. You generally layer Front Door or Traffic Manager on top of DNS, not instead of them.

For most modern web workloads, use Front Door as the primary DR switch and DNS as a coarse backup.

2. How do I test failover safely in production?

Start by failing a small percentage of traffic (e.g., use weighted routing in a subset environment).
Use short, well-announced windows.
Have an automated rollback (re-enable origin, revert routing).
Observe impact in real time on error budgets and SLO dashboards.

3. How should I choose health probe paths?

Use a dedicated endpoint like /readyz or /health-deep.
It should check critical dependencies in a lightweight way.
Return non-2xx when the app is not fit to serve traffic.
Exclude it from caching and WAF rules that could mask problems.

4. What's a reasonable failover time with Front Door?

It depends on your probe configuration, but many teams target:

Detection: 30–90 seconds
Failover complete: Under 2–3 minutes

If your RTO is stricter, tune probes more aggressively and mitigate false positives with solid observability and retry logic at the client layer.

5. How do I handle stateful sessions with multi-region Front Door?

Options:

Go stateless at the app layer (recommended where possible).
Use distributed caches (e.g., Redis) or centralized session stores that replicate between regions.
For active-passive, consider shorter session lifetimes + re-auth on failover.
Be careful with "sticky sessions" and ensure they don't lock users to a dead region.

6. How do I bring this pattern into a legacy environment?

Start by putting Front Door in front of your existing primary region.
Add a secondary region with a subset of services.
Use DR drills in lower environments first to refine runbooks.
Gradually move more legacy components behind consistent Front Door routing.

You don't have to go all-in on day one; even a partial DR capability is better than none.

7. How do I measure DR success?

Track:

RTO achieved vs target during drills.
RPO (data loss or replay needs).
User impact during failover (error rates, latency).
Time for engineers to execute runbooks.
Number of incidents where DR actually saved you.

Turn those into SLOs that leadership can understand.

8. How does this compare to AWS and GCP?

Rough mapping:

AWS: CloudFront + ALB/NLB + Route 53 health checks and routing policies.
GCP: External HTTP(S) Load Balancer + Cloud CDN + Cloud Armor.

Concepts are similar: health checks, multi-region backends, DR drills. The main differences are in configuration models, naming, and surrounding ecosystem.

Conclusion

In our DR drill, Azure Front Door didn't "fail over" because:

Our health probes were lying to it.
Our expectations didn't match our configuration.
Our DR practice was theoretical rather than muscle memory.

The good news: once you understand how Front Door evaluates backend health and how to align probes with real-world failure modes, it becomes a powerful tool for multi-region resilience.

If you take one thing from this story, let it be this:

Don't wait for a real outage to find out whether your DR works.

Start with a lower environment, codify Front Door and DR behavior in Terraform/Bicep, set up observability, and schedule regular game days. Every drill you run now is one less panic later.

If this resonated with you, follow along, drop your own DR stories in the comments, and share this with the person in your org who will be on call when Azure Front Door is your first line of defense.

References

Connect With Me

If you enjoyed this walkthrough, feel free to connect with me here:

The Evolving Role of a Solutions Architect

Raghavendra R — Mon, 03 Nov 2025 21:13:09 +0000

Introduction
From Traditional Architect to Cloud-Native Strategist
What a Modern Solutions Architect Actually Does
Core Responsibilities in the Cloud-Native Era
Architecture Example: Designing a Scalable Microservice System on Azure
Bridging Business Goals and Technical Execution
Essential Tools, Frameworks, and Libraries
Common Developer Questions
Conclusion

Introduction

The role of a Solutions Architect (SA) has evolved from diagramming servers in PowerPoint to designing resilient, automated, and cloud-native architectures that directly align with business objectives.

In a world where uptime, scalability, and cost optimization drive competitive advantage, architects aren’t just system designers—they’re strategic problem-solvers who understand both code and commerce.

From Traditional Architect to Cloud-Native Strategist

A decade ago, an architect might focus on VM provisioning, middleware stacks, and data center topology.

Today, architecture is about distributed systems, containers, event-driven design, and continuous delivery.

Here’s the thing: cloud-native architecture isn’t just “running things on the cloud.” It’s an entirely different mindset—centered around automation, observability, and business value.

Then vs Now

Era	Focus	Tools	Challenges
Legacy	Monolithic applications, static VMs	Load balancers, app servers	Scaling and maintenance
Cloud-Native	Microservices, serverless, IaC	Kubernetes, Terraform, CI/CD Pipeline	Complexity, cost visibility

What a Modern Solutions Architect Actually Does

Modern architects operate at the intersection of business strategy, engineering leadership, and hands-on design.

Key Activities

Define end-to-end architecture across frontend, backend, and infrastructure.
Translate business KPIs into technical designs.
Collaborate with developers on CI/CD pipelines, IaC (Infrastructure as Code), and security guardrails.
Make informed decisions about cloud services, API boundaries, and data flow.
Champion DevSecOps practices for governance and compliance.

Architects today are expected to code when needed, whether it’s writing a Terraform module, defining an API spec, or reviewing Helm charts.

Core Responsibilities in the Cloud-Native Era

1. Architectural Governance

Define standards for:

Naming conventions
Network topology
Cost tagging and monitoring
Disaster recovery strategies

2. Platform Engineering

Architects help design reusable platform components—shared CI/CD templates, Terraform modules, and observability stacks—to accelerate development.

3. Cloud Optimization

They balance cost efficiency and scalability. For example:

Choosing between Azure App Service vs AKS depending on workloads.
Designing auto-scaling rules aligned with business demand cycles.

Architecture Example: Designing a Scalable Microservice System on Azure

Let’s walk through a typical real-world scenario: a fintech platform processing thousands of payment transactions per minute.

Business Goal

Reduce transaction latency and scale automatically with demand—while minimizing operational overhead.

Cloud-Native Solution

Architecture Pattern: Event-driven microservices with message queues and autoscaling containers.

Azure Components:

Azure Kubernetes Service (AKS) for container orchestration.
Azure Service Bus for async messaging.
Azure Cosmos DB for globally distributed storage.
Azure Application Gateway for traffic routing.
Azure Monitor for metrics and alerting.

High-Level Flow

User submits a transaction via API Gateway.
The API layer publishes an event to Service Bus.
Worker pods in AKS process events and write results to Cosmos DB.
Application Gateway distributes traffic and performs health checks.

Infrastructure as Code Example (Terraform)

resource "azurerm_kubernetes_cluster" "aks_cluster" {
  name                = "fintech-aks"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = "fintech"

  default_node_pool {
    name       = "default"
    node_count = 3
    vm_size    = "Standard_B4ms"
  }

  identity {
    type = "SystemAssigned"
  }
}

Deployment Pipeline (Azure DevOps YAML)

trigger:
  branches:
    include:
      - main

stages:
- stage: Build
  jobs:
  - job: BuildApp
    steps:
      - script: npm install && npm run build

- stage: Deploy
  jobs:
  - job: DeployToAKS
    steps:
      - task: Kubernetes@1
        inputs:
          connectionType: 'Azure Resource Manager'
          azureSubscription: 'Fintech-Prod'
          namespace: 'payments'
          command: 'apply'
          useConfigurationFile: true
          configuration: 'manifests/deployment.yaml'

This approach keeps infrastructure codified, repeatable, and aligned with business SLAs.

Bridging Business Goals and Technical Execution

Solutions Architect doesn’t just say “use Kubernetes.” They explain why—for example:

To improve horizontal scalability during payment surges
To achieve high availability across zones
To reduce downtime via rolling deployments

The architect continuously measures technical outcomes against business KPIs:

Latency metrics → customer satisfaction
Cost per transaction → profitability
Deployment frequency → time-to-market

That’s how architecture translates into business success.

Essential Tools, Frameworks, and Libraries

Cloud & Infrastructure

Terraform – IaC for multi-cloud environments
Pulumi – IaC using real programming languages
Azure CLI / gcloud CLI – scripting cloud operations
Helm – Kubernetes package management

CI/CD & DevSecOps

GitHub Actions / Azure DevOps / GitLab CI
SonarQube for code quality
Trivy / Aqua / Checkov for security scanning

Observability

Prometheus + Grafana for metrics
Azure Monitor / GCP Operations Suite
OpenTelemetry for tracing

Tip:

Automate everything that can break silently—alerts, health checks, and cost thresholds are just as vital as code quality gates.

Common Questions

Q1. Do architects still code?

Yes, but selectively. Modern SAs prototype, review, and occasionally code IaC, pipelines, or reference implementations.

Q2. How does an architect differ from a DevOps engineer?

DevOps engineers focus on pipelines, automation, and reliability. Architects design the larger system blueprint that DevOps implements and scales.

Q3. What skills should anyone needs to have to move into architecture?

Strong fundamentals in distributed systems
Familiarity with cloud platforms (Azure, GCP, AWS)
Understanding CI/CD and IaC
System design and communication skills

Q4. How do architects ensure cost control in cloud environments?

By implementing policies like:

Automated shutdown of idle resources
Right-sizing instances
Using cost monitoring tools like Azure Cost Management

Conclusion

The modern Solutions Architect is part strategist, part engineer, and part translator between business and tech.

They build systems that are not just scalable—but sustainable, secure, and measurable.

If you’re a developer aiming for architecture roles, start by owning your system’s design decisions, automation, and operational visibility.

Follow me for more DevOps and cloud architecture tutorials.

DEV Community: Raghavendra R

# A Failed Compliance Audit in Azure DevOps: Rebuilding CI/CD with Policy as Code and Security Gates

Rebuilding Azure DevOps CI/CD for Compliance

Table of Contents

Introduction

Core Concepts

Compliance in Azure DevOps: Where It Lives

Policy as Code: Three Levels

Security Gates in Azure DevOps

Multi-Environment, Multi-Subscription Design

Step-by-Step Guide

1. Map Audit Findings to Concrete Controls

2. Standardize CI/CD Architecture

3. Implement Template-Driven CI Pipelines

4. Embed Policy as Code for Infrastructure

5. Define Environments and Security Gates

6. Integrate Security Scanners as Gates

7. Observability and Auditability

8. Rollout Strategy Across Teams

Architecture & Flow Diagram

Best Practices

Common Pitfalls

1. "Templates" That Are Optional

2. Over-Permissive Service Connections

3. Scanners That Don't Fail Builds

4. Manual Change Approvals Outside CI/CD

5. Azure Policy Not Integrated with CI

6. Ignoring Non-Prod Environments

7. No Runbooks for Gate Failures

FAQ

1. How does this map to AWS and GCP?

2. How do I add compliance without slowing delivery?

3. How can I scale this across dozens of teams?

4. How do I handle legacy applications and pipelines?

5. How do I integrate with ITSM and change management?

6. What KPIs show that CI/CD compliance is working?

7. How do I handle multi-region or DR scenarios?

8. What's the role of GitHub if we already use Azure DevOps?

Conclusion

References

Connect With Me

# When Azure Front Door Won't Fail Over: Lessons from a Real Multi-Region DR Drill

Table of Contents

The Story / Background

The architecture we thought we had

The drill

What actually happened

Core Concepts: How Azure Front Door Failover Really Works

Origin groups, priorities, and routing

Health probes and what "healthy" really means

Active-active vs active-passive in DR context

Data tier is not Front Door's job

Observability for failover

Step-by-Step Guide: Designing Azure Front Door for Real Multi-Region DR

1. Define RTO/RPO and failure modes

2. Design origin groups and health probe strategy

3. Implement with Terraform (example)

4. Build DR-aware pipelines and configuration management

5. Implement synthetic tests and dashboards

6. Run regular DR drills and chaos tests

Architecture Diagram

Key Components:

Traffic Flow:

Best Practices for Azure Front Door Multi-Region DR

Common Pitfalls (and How to Avoid Them)

1. Health probes hitting the wrong thing

2. Probes behind caching or CDN rules

3. Overly large sample sizes and long intervals

4. Internal traffic bypassing Front Door

5. No DR for the data tier

6. DR tests only in staging

7. No clear runbook for Front Door changes

FAQ

1. Azure Front Door vs Traffic Manager vs DNS for DR?

2. How do I test failover safely in production?

3. How should I choose health probe paths?

4. What's a reasonable failover time with Front Door?

5. How do I handle stateful sessions with multi-region Front Door?

6. How do I bring this pattern into a legacy environment?

7. How do I measure DR success?