Azure DevOps YAML release pipeline : Trigger when build pipeline completed

Finally, Azure DevOps yaml based CD has been GA!
Announcing General Availability of YAML CD features in Azure Pipelines

The very first question I had in my mind was "how to trigger release pipeline after build pipeline?"

If I author CI/CD in same yaml, even by using multi-stage, I don't have to think about "triggering CD", but what if I author CI and CD in separate yml and create two pipeline?

Resources in YAML pipelines

Resources is great way to trigger pipeline by types such as pipelines, builds, repositories, containers, and packages. In this article, I focus on pipeline resource.

Create build pipeline

Without further due, let's create build pipelines for test.

1. Create new DevOps project and new repository. I simply add a text file there.

2. Add build-pipeline.yml file and update yaml as below. As you can see, every steps is just dummy. This pipeline generate one artifact.

trigger:
  branches:
    include:
    - master
  paths:
    exclude:
    - build-pipeline.yml
    - release-pipeline.yml

variables:
  vmImageName: 'ubuntu-latest'

jobs:  
- job: Build
  pool:
    vmImage: $(vmImageName)
  steps:
  - script: | 
      echo 'do some unit test'
    displayName: 'unit test'
  - script: | 
      echo 'compile application'
    displayName: 'compile'
  - task: ArchiveFiles@2
    displayName: 'Archive files'
    inputs:
      rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
      includeRootFolder: false
      archiveType: zip
      archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
      replaceExistingArchive: true
  - upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
    artifact: drop

3.Create new pipeline by specifying exiting yaml and select 'build-pipeline.yaml'

4. Run and see the result.

5. It generates one artifact.

6. Rename the pipeline so that it's easy to see this is build pipeline.

Release pipeline

Next, create release pipeline which will be triggered after the build pipeline.

1. Add release-pipeline.yml file and update yaml.

# Explicitly set none for repositry trigger
trigger:
- none

resources:
  pipelines:
  - pipeline: myappbuild  # Name of the pipeline resource
    source: myapp-build-pipeline # Name of the triggering pipeline
    trigger: 
      branches:
      - master

variables:
  vmImageName: 'ubuntu-latest'

jobs:
- deployment: Deploy
  displayName: Deploy
  environment: dev
  pool: 
    vmImage: $(vmImageName)
  strategy:
    runOnce:
      deploy:
        steps:          
        - download: myappbuild
          artifact: drop  
        - task: ExtractFiles@1
          inputs:
            archiveFilePatterns: '$(PIPELINE.WORKSPACE)/myappbuild/drop/$(resources.pipeline.myappbuild.runID).zip'
            destinationFolder: '$(agent.builddirectory)'
            cleanDestinationFolder: false
        - script: |  
            cat $(agent.builddirectory)/greatcode.txt

2. Create new pipeline and select create file.

3. Save the pipeline.

4. Click run the pipeline, and select "Resources".

5. Select resource from build pipeline.

6. Then, run the pipeline and see the result.

7. Rename the pipeline if you want.

Test CI/CD

Now it's time to do CI/CD test. Simply update greatcode.text content and commit to master.

You see the build pipeline runs first, then release pipeline runs.

Important points

There are several things to note here.

release trigger

I set trigger to none explicitly so that the release pipeline won't be triggered in any change to branches.

# Explicitly set none for repositry trigger
trigger:
- none

Then I specify resources | pipelines. The pipeline name is just a name which I can reference later. source is where I specify build pipeline name.

resources:
  pipelines:
  - pipeline: myappbuild  # Name of the pipeline resource
    source: myapp-build-pipeline # Name of the triggering pipeline
    trigger: 
      branches:
      - master

Then I reference this in many places.
I use download task to explicitly download drop artifact. This is useful when build pipeline publish multiple artifacts but release pipeline doesn't need them all.

- download: myappbuild
  artifact: drop 

The downloaded artifact can be accessed with pre-defined variable.

$(PIPELINE.WORKSPACE)/myappbuild/drop

I also other pre-defined variable to identify zip file name.

$(resources.pipeline.myappbuild.runID).zip

I can simply speficy *.zip, but this is useful when I need Build.BuildId in release pipeline.

Summary

There are several things I need to know in advance to author pipelines, but once I know it, it's easy. Hope this information helps you.

Comments

[Rafael Camison]

A good description of how to use ressource. I'm not convinced that resource replaces a release pipeline. For the simple reason that you don't get revisions. Which is the crucial difference between a release and a build pipeline. Anyways, good job!

[Kenichiro Nakamura]

Yeah, I know what you mean. I hope "build" feature gives us more clear and granular capabilities to replace existing one in the future :)

[Rafael Camison]

Or at least a yaml variant of release pipelines but yes, I agree.

[Jan van Veldhuizen]

This is a great way to split build and deployment. Until today it was quite annoying to always wait for my build and tests to run when testing he deployment part of my pipelines.
Super!

Now trying to find out which trigger I need to make it run automatically after a successful build

[gautampambhar]

I learned a lot through this article. Thank you so much for publishing this.
I have one question though. in your build-pipeline.yml you have a task at last for file archive. what if I publish a build artifact instead of archiving file. Although Can you tell the difference between those two tasks. and what if I am publishing an artifact; in that case how the task will look like?

Thanks.

[Kenichiro Nakamura]

Thanks for your comment. you can publish the build artifact as you wish when you want to pass it to next stage, which may be run in different agent. Archive simply archive the directory and that's it. The reason you may want to use publish artifact is to store the result so that it can be picked up in the next stage.

As you may already know, there is no guarantee that some VM is used for different step, and even though it is used, no disk cache exists. That's whey you need to "publish" which simply store the result to Azure DevOps space (blob) and next step can download it. If you pickup the zip in next task in the same step, then it is still running in the same agent, so you don't have to publish it.

[thomasbinny]

Guys, the problem i see is, even though i write and implement the release.yaml, the CI build.yaml pipeline does not trigger the Release.yaml

trigger:
  - none
resources:
  pipelines:
  - pipeline: Test_CI
    source: Test_CI
    trigger:
      branches:
        - feature/xxxxx

Should i have to click on edit pipeline --> Triggers --> Enable Build Completion for this to work?

[Kenichiro Nakamura]

You don't have to. Did you configure Pipeline as pass the exact name to release yaml?

[Raja]

I am not able to configure widgets from Azure Devops if we create the release pipeline using yml. How to address this?

[Kenichiro Nakamura]

I personally don't use widget so I have no idea at the moment. Which widget do you try to configure?

[Christopher John Pepper]

So do these show up under pipeline->releases or is "releases" redundant now and we should do it all trough pipeline?

[Kenichiro Nakamura]

For my understanding, everything should be done in pipelines, not under releases. Releases menu is just there for backward compatibility.