Azure DevOps is a platform for running builds and deployments pipelines for your applications. It's recently rebranded to Azure DevOps, just to jump on the bandwagon of the buzzword. It's essentially every other build system, gives you a YAML declarative pipeline DSL, and a UI if you want to build your CICD pipelines by click and drag. It's no where as popular as some of the other big build servers like Jenkins or CircleCI, therefore it's lacking quite a bit in terms of documentations, tutorials and guides.
I recently helped someone deploying AWS infrastructure using Azure DevOps, weird I know. I was initially confused about the choice, but after checking out Azure DevOps's AWS plugin, I kinda understood why a small shop would choose it to be it's CICD tool. Azure DevOps follows Jenkins in that it only provides a few basic deployment modules (called tasks in Azure DevOps) out of the box, like bash scripts, npm builds and maven builds. The rest of its power comes from a rich library of plugins, such as the AWS plugin.
YAML DSL
The YAML DSL configures which jobs to run and on which kind of servers. It also exposes some pipeline configs like what to do when a job failed, do we continue or do we quit. Below is what I wrote for my project, minimal but gets the job done:
jobs:
- job: MyJob
pool:
vmImage: 'ubuntu-16.04'
displayName: My First Job
continueOnError: true
workspace:
clean: outputs
steps:
......
steps: starts a list of modules to run.
AWS Credentials
AWS Credentials can be configured via the UI (Project Settings -> Service connections -> add new AWS Service Connection). A caveat is that Service Connections only get loaded during pipeline initiation, any new connections added after the pipeline has been created won't get loaded automatically. So you'll have to delete and re-create your pipeline to use new connections. I've explained this in a Github issue that talks about this caveat. It looks like quite a few people are running into this issue, so I'm just sharing lessons learned everywhere.
Cloudformation Module
The AWS plugin comes with several pretty useful modules, like CloudFormation Update/Create, Lambda Deploy and S3 Upload. Unfortunately I don't think they published the documentation for it, so I had to look at their source code to find the docs. I've only used CloudFormation Update/Create and another module called AWS CLI. The CloudFormation module greatly saved my time because I didn't have to handle the idempotence of multiple updates after the initial creation, the module knows to update instead of create if the CloudFormation stack is already created.
steps:
- task: CloudFormationCreateOrUpdateStack@1
inputs:
awsCredentials: 'aws_tokens'
regionName: 'us-east-2'
stackName: 'IAMRoleStack'
templateFile: templates/iam_role.json
capabilityIAM: 'true'
capabilityNamedIAM: 'true'
Here is what I used to deploy my IAM role, awsCredentials, regionName are required for all AWS modules, and stackName and templateFile are required for all CloudFormation modules. The last two are only specific to this module.
AWS CLI Module(or not)
The AWS CLI module gave me false hopes, turned out that you still have to install AWS CLI yourself in order to use the module LOL. I first had to install setuptools and wheel, yeah, pip dependencies doesn't even come for free. I then have to install it in my user space, and it took me a while find where the CLI got installed to because it's not added to my PATH. So I just used the AWS Shell script module, which seems to do exact what the AWS CLI module does, except easier to write.
- script: |
python -m pip install --upgrade pip==9.0.3 setuptools wheel
pip install awscli --user
displayName: 'Install tools'
- task: AWSShellScript@1
inputs:
awsCredentials: 'aws_tokens'
regionName: 'us-east-2'
scriptType: 'inline'
inlineScript: |
eval $(/home/vsts/.local/bin/aws ecr get-login --no-include-email)
Top comments (8)
Well, the last paragraph of your article makes clear you have no idea about what you're saying.
Az DevOps is used by all engineering teams in Microsoft, just to mention the Windows Universal Platform (Windows, Xbox Live and other products) dev team has ~22k developers with ~500Gb of source code, so to support this size of project, Microsoft created a Virtual Filesystem for Git (github.com/Microsoft/VFSForGit/blo...).
Despite this one, there are tons of other use cases, even the Az DevOps team which generates thousands of the builds daily as well as office team which generates 7Tb of pdb files stored in Az DevOps for production debuging purposes.
Sorry but if you don't know how to use Az DevOps, it is not a problem of the product, but yours.
Sincerely, Luciano.
I don't care if Microsoft is dog-fooding their own product, as an outsider and someone who's used and built similar products, Azure DevOps has given me a bad user experience. It does't matter if the employees are using the product (they probably don't have choice anyway), at the end of the day they have to sell it and I definitely won't buy it. And I'm not a Windows user and I don't develop any projects on Windows, so I don't speak for that. From what you're saying, it seems that dog-fooding and Windows are the only two things Az DevOps is good for.
Chastina, I agree with Luciano. We use Azure DevOps to deploy Java artifacts from AWS S3 to Amazon Linux EC2 instances because it has better release functionality than AWS Code Pipeline. I would agree with Luciano that you don't have a good grasp of the features.
Maybe I spoke too soon. No I haven't used AWS Code Pipeline or used Azure DevOps the way you used it. I'm speaking for my personal experience with it and so far it sucked. I built better CICD solutions myself on bare Jenkins. I apologize if what I said offended any of the Azure DevOps users out there that had pleasant experiences with it. I'm just speaking for my experience and I don't need to have to have used all of the CICD products out there to be able to then say what I felt.
"The last paragraph of your article makes clear you have no idea about what you're saying.". If you'd be gracious enough, it would be helpful if you could point out a better way to do what Chastina is doing. DEV encourages individuals to share their experiences, and that means sharing when we're new to a given technology, and we might not have an expert understanding.
Personally, I have 0 experience with Azure DevOps so I can't really comment, but it seems like you're insinuating there's some way to avoid manually installing dependencies in the build steps when using modules? If possible, let us know the better way! :)
MS Team Services/Visual Studio Online was a popular product inside MS dev community... but nobody knew it outside that niche (yes, it is a niche, nothing wrong about that).
Once it has been rebranded to Azure DevOps (and divided into meaningful separated components) its popularity has raised over the radar of a much broader audience. But you cannot compare the documentation available for it with the resources associated with products like Jenkins.
It is a very nice managed alternative, and I'm sure it is going to be very popular once it becomes possible to define release pipelines with code (not only build pipelines) and more plugins become available.
That said, discouraging new users that are actually creating good resources for Azure DevOps community using rudeness is really against the best interest of the company you clearly love and the team behind Azure DevOps.
If it was easy wouldn't be challenging and you wouldn't write this article ;).
Thanks a lot :)
Great read, thanks for the info Chastina.