The Anatomy of a Good Azure Pipeline

Muhammad Mustafa — Tue, 12 Aug 2025 22:34:14 +0000

Introduction
The Core Components of a Pipeline YAML
- Triggers
  - Push Trigger
  - Pull Request (PR) Trigger
  - Chaining Pipelines
- Variables: Your Configuration Hub
- The Structure: Stages, Jobs, and Steps
  - Stages
  - Jobs
  - Steps
Putting It All Together

Introduction

You've probably heard about CI/CD and how it's supposed to make your life easier. At the heart of this automation in the Azure world is the Azure Pipeline. But what exactly makes a pipeline good? It's not just about making it work; it's about making it clean, understandable, and maintainable.

At its core, a pipeline is just a YAML file that describes your entire CI/CD process.

A pipeline is one or more stages.
Stages are the major sections, these are things like "Build the app," "Run tests," and "Deploy to Pre-Prod."
A Job is a linear series of steps that run on an agent (a server). For example, within a "Build" stage, you might have separate jobs like "Build Backend" and "Build Frontend" that can run in parallel to speed things up.
Steps are the individual commands or tasks within a job. They can be simple scripts, built-in tasks (like building a .NET project), or even references to other templates.

The Core Components of a Pipeline YAML

Your azure-pipelines.yml file has a few key top-level sections that define how it behaves.

Component	Description
`trigger`	Specifies which branches will trigger the pipeline when code is pushed.
`pr`	Defines which target branches will trigger the pipeline on a pull request.
`resources.pipelines.pipeline`	Enables one pipeline to trigger another or consume its artifacts.
`variables`	Defines variables you can use throughout the pipeline.
`stages`	A collection of related jobs. The main way to structure your pipeline.
`jobs`	A simpler structure with one implicit stage. Good for basic pipelines.
`steps`	The simplest structure with one implicit job. Great for quick tasks.

Triggers:

How does a pipeline start? Usually, it's a push to a repo or the creation of a pull request.

Push Trigger

The trigger keyword controls runs based on pushes. You can get pretty specific here.

batch: true: If you push multiple commits quickly, this waits for the current run to finish and then starts one new run with all the changes, instead of queuing a run for every single commit. Super handy.
branches: Lets you include or exclude branches.
paths: Want to only trigger a build if the front-end code changes? paths is your friend. It lets you include or exclude file paths.

# This pipeline runs for pushes to master and release/* branches,
# but only when changes affect the 'apps/frontend' directory.
# Changes outside this directory (e.g. other 'apps' or 'local-setup') do not trigger the pipeline.
trigger:
  batch: true
  branches:
    include:
      - master
      - release/*
    exclude:
      - features/wip-*
  paths:
    include:
      - apps/frontend/*
    exclude:
      - README.md

Pull Request (PR) Trigger

The pr trigger is for validating changes before they get merged.

autoCancel: true: When a new commit is pushed to the PR's source branch, it automatically cancels the pipeline run that was in progress for the previous commit. Keeps your agent queue clean.
drafts: false: By default, pipelines run even for draft PRs. Set this to false to save build minutes until the PR is ready for review.

# This PR trigger runs for PRs targeting master and release/* branches.
# It won't run for draft PRs.
pr:
  autoCancel: true
  drafts: false
  branches:
    include:
      - master
      - release/*
  paths:
    include:
      - apps/frontend/*

Chaining Pipelines

Sometimes, one pipeline depends on another. An example is a "Deploy" pipeline that needs the artifacts from a "Build" pipeline. resources makes this easy.

# In your deploy-pipeline.yml

# Disable CI/PR triggers for this pipeline, we only want it to be triggered by another pipeline.
trigger: none
pr: none

resources:
  pipelines:
    # Give the resource a friendly ID, like 'build'.
    - pipeline: build
      # The name of the source pipeline in Azure DevOps.
      source: my-app-build-pipeline
      # trigger this deploy pipeline when the source pipeline completes.
      trigger:
        branches:
          include:
            - master
            - release/*

Variables: Your Configuration Hub

Variables let you store and reuse strings and secrets. You can define them at different levels (pipeline, stage, or job) or pull them from Variable Groups created in the Azure DevOps UI. Using Variable Groups is the best practice for secrets like connection strings and API keys!

variables:
  # A simple key-value pair
  - name: buildConfiguration
    value: 'Release'
  # Reference a variable group from the UI
  - group: 'My-App-Secrets' # Contains things like database passwords

The Structure: Stages, Jobs, and Steps

A multi-stage pipeline is the gold standard for any serious project.

Stages

Stages are the top-level divisions of work. A common pattern is Build -> QA Deploy -> Prod Deploy.

displayName: A friendly name that shows up in the UI.
dependsOn: Makes a stage wait for another one to complete.
condition: This lets you control if a stage runs. For example, you might only want your "Deploy" stage to run if the build stage succeeded and the change was on the master branch.
pool: Specifies which agent pool (and VM image) to use for the jobs in this stage.

Here's an example of a condition for a deploy stage:

stages:
- stage: Build
  displayName: 'Build the Application'
  jobs:
  # ... build jobs go here ...

- stage: Deploy
  displayName: 'Deploy to Dev'
  dependsOn: Build
  # This stage only runs if:
  # 1. The Build stage succeeded.
  # 2. The pipeline was NOT triggered by a pull request.
  # 3. The pipeline was either run manually OR it was triggered by a push to the master branch.
  condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'), or(eq(variables['Build.Reason'], 'Manual'), eq(variables['Build.SourceBranchName'], 'master')))
  jobs:
  # ... deploy jobs go here ...

Jobs

Inside a stage, you have one or more jobs. Jobs within a stage can run in parallel by default, which can seriously speed things up! For instance, you could build your frontend and backend code at the same time in separate jobs.

Steps

Steps are the smallest unit of work. They run sequentially inside a job. This is where you call tasks, run scripts, and get things done.

task: A pre-built task from the marketplace (e.g., DotNetCoreCLI@2, NodeTool@0).
script: A simple command-line script.
checkout: Controls how source code is downloaded.

Putting It All Together

Let's imagine we're deploying a full-stack application with a .NET backend, a SQL database (using a DACPAC project), and a React frontend.

Here’s what a good, clean pipeline for that might look like.

# azure-pipelines.yml

trigger:
- master

pr:
- master

variables:
- group: 'MyWebApp-Dev-Secrets' # Store secrets in a Variable Group in the Azure DevOps UI
- name: buildConfiguration
  value: 'Release'

stages:
# ========================================================================================
# BUILD STAGE: Compile all parts of the app and publish artifacts
# ========================================================================================
- stage: Build
  displayName: 'Build & Package'
  jobs:
  - job: BuildBackend
    displayName: 'Build .NET Backend'
    pool:
      vmImage: 'windows-latest'
    steps:
    - task: DotNetCoreCLI@2
      displayName: 'Restore, Build, Publish'
      inputs:
        command: 'publish'
        publishWebProjects: true
        arguments: '--configuration $(buildConfiguration) --output $(Build.ArtifactStagingDirectory)/backend'
        zipAfterPublish: true

    - task: PublishBuildArtifacts@1
      displayName: 'Publish Backend Artifact'
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)/backend'
        ArtifactName: 'backend'

  - job: BuildFrontend
    displayName: 'Build React Frontend'
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: NodeTool@0
      inputs:
        versionSpec: '18.x'
      displayName: 'Install Node.js'

    - script: |
        cd src/MyWebApp.Frontend
        npm install
        npm run build
      displayName: 'npm install and build'

    - task: PublishBuildArtifacts@1
      displayName: 'Publish Frontend Artifact'
      inputs:
        PathtoPublish: 'src/MyWebApp.Frontend/build'
        ArtifactName: 'frontend'

  - job: BuildDatabase
    displayName: 'Build Database Project'
    pool:
      vmImage: 'windows-latest'
    steps:
    - task: VSBuild@1
      displayName: 'Build DACPAC'
      inputs:
        solution: 'src/MyWebApp.Database/MyWebApp.Database.sqlproj'
        msbuildArgs: '/p:SqlPublishProfilePath=MyWebApp.Database.publish.xml /p:Configuration=$(buildConfiguration) /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:OutDir=$(Build.ArtifactStagingDirectory)/db'

    - task: PublishBuildArtifacts@1
      displayName: 'Publish Database Artifact'
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)/db'
        ArtifactName: 'database'

# ========================================================================================
# DEPLOY STAGE: Deploy all the artifacts to Azure
# ========================================================================================
- stage: Deploy
  displayName: 'Deploy to Production'
  dependsOn: Build
  # Only deploy on successful builds to master, not on PRs
  condition: and(succeeded(), eq(variables['Build.SourceBranchName'], 'master'))
  jobs:
  # A deployment job gives you better tracking and control for deployments
  - deployment: DeployToProduction
    displayName: 'Deploy to Azure'
    environment: 'Production'
    pool:
      vmImage: 'windows-latest'
    strategy:
      runOnce:
        deploy:
          steps:
          # Download all the artifacts from the Build stage
          - task: DownloadBuildArtifacts@1
            displayName: 'Download all artifacts'
            inputs:
              buildType: 'current'
              downloadType: 'single'
              artifactName: 'backend'
              downloadPath: '$(System.ArtifactsDirectory)'
          - task: DownloadBuildArtifacts@1
            inputs:
              buildType: 'current'
              downloadType: 'single'
              artifactName: 'frontend'
              downloadPath: '$(System.ArtifactsDirectory)'
          - task: DownloadBuildArtifacts@1
            inputs:
              buildType: 'current'
              downloadType: 'single'
              artifactName: 'database'
              downloadPath: '$(System.ArtifactsDirectory)'

          # Deploy the database
          - task: SqlAzureDacpacDeployment@1
            displayName: 'Deploy Database DACPAC'
            inputs:
              azureSubscription: 'My-Service-Connection'
              AuthenticationType: 'server'
              ServerName: '$(dbServerName)' # From Variable Group
              DatabaseName: 'MyWebAppDB'
              SqlUsername: '$(dbUser)' # From Variable Group
              SqlPassword: '$(dbPassword)' # From Variable Group
              DacpacFile: '$(System.ArtifactsDirectory)/database/MyWebApp.Database.dacpac'

          # Deploy the .NET backend to an App Service
          - task: AzureWebApp@1
            displayName: 'Deploy Backend to App Service'
            inputs:
              azureSubscription: 'My-Service-Connection'
              appType: 'webApp'
              appName: 'my-webapp-api-prod'
              package: '$(System.ArtifactsDirectory)/backend/**/*.zip'

          # Deploy the React frontend to a Static Website in a Storage Account
          - task: AzureCLI@2
            displayName: 'Deploy Frontend to Blob Storage'
            inputs:
              azureSubscription: 'My-Service-Connection'
              scriptType: 'bash'
              workingDirectory: '$(System.ArtifactsDirectory)/frontend'
              scriptLocation: 'inlineScript'
              inlineScript: |
                az storage blob upload-batch \
                  --account-name mywebappstaticstorage \
                  --source '$(System.ArtifactsDirectory)/frontend' \
                  --destination '$web' \
                  --overwrite

DEV Community: Muhammad Mustafa