Learn how YOU can add CI/CD to your app

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

CI/CD. You might have heard the terms more than once. You might even use it every day. Regardless of which, we will explain what it is and why it is an invaluable tool in every developer's toolbox. So stay tuned.

TLDR; This article will explain what CI/CD is. We will also show you how to set up a simple form of CD for your app using Azure. This article is somewhat meaty but it does take you all the way from source code to setting up deployment and teaching you how to do A/B testing and blue/green deployment with deployment slots.

This is a series of articles.

• Part 1, Deploying our GitHub repo, we are here
• Part 2 - Azure DevOps, we will learn to work with pipelines, build pipelines, release pipelines and learn how we can configure YAML files to help us, - to be written

Finally, we will learn about how we can work with deployment slots for blue/green deploys and A/B testing.

It should be said that there are two approaches we could use to set up CD in Azure. There's however only one way that you can get both CI and CD and that is using Azure DevOps.

In this article, we will show to achieve CD using a simpler approach. If you are looking to achieve both CI/CD then I'm afraid you have to wait until part 2 of this series.

But don't worry, even if this approach is simpler, and can't achieve as much as Azure DevOps, it can still provide a lot of value to you as a developer.

References

• Setting up Continuous Deployment This docs page describes both how to set up Continuous deployment via AppService but also how to do via Azure DevOps
• Download Node.js The app we are about to deploy will use Node.js. If you don't have that on your machine you can easily install it.
• Free Azure account For this, we will use Azure. If you don't have it, it's quite easy to sign up.
• Azure DevOps overview
• Deployment slots

What is CI/CD and why do I need it?

CI/CD stands for Continuous Integration, CI and Continuous Deployment, CD.

Sounds great, but explain, please?

CI is about integrating changes from different developers in the team into a mainline, usually a master branch, as early as possible, prefera bly several times a day.

There are two things with integration that needs addressing when it comes to CI:

1. The definition of the term
2. The goal

Definition

Let's address the first point, the term itself. Different developers work on different parts of the code. You want their changes to reach the master as soon as possible. If it takes too long it might result in time spent on merging and resolving merge conflicts.

Goal

The main goal of CI is that everyone's changes hit the master as soon as possible. As a secondary goal, you also want a working code. Noone benefits from people merging in broken code. As part of this process, we want automated testing to happen, and even code reviews is another thing we can use to ensure that what we actually merge in, is of good enough quality to be merged in.

You can read more about it here:

https://codeship.com/continuous-integration-essentials
https://dev.to/quii/gamifying-continuous-integration-o1d

Make sense?

Great I like quality in what I do. What about Continuous Deployment? Don't tell me it means I deploy as soon as I push code to the repo? That sounds dangerous.

Actually. In the past we used to deploy with a few months in between. We had large teams of QA testing every nook and cranny and weeks later they would sign off on everything and every release would be a long ceremony of passing scripts from person to person like an Olympic torch

What's wrong with that? Sounds responsible, QA teams, well tested, etc. Sure I wish I could deploy more often, but safety first.

Yea well, you live in 2020. That means we look upon things differently. We should set up our software and processes in a way so that we can build all the needed components at the press of a button and you should get a working piece of software at the end of that, an artifact.

Sounds like a dream. But you know we got different teams working on different components and well they have their Backlog, priorities, strategies. Did I mention we have OPS people too and OPS and DEV well... Not sure it can be achieved?

Well, that's the thing, CI is relatively easy, adding tests to your software and running that every time you push code is achievable for most of us. Continuous Deployment, CD, is a more difficult topic because the problem is usually not technical but it's more about processes and people talking to one another and using the tools to achieve it.

Ok, but let's say my DEV department manages to organize themselves in a way so all component teams talk to each other. Come up with version strategies that we can rely on and so forth? Can I have CD, then?

Possibly, but it is a continuous work ensuring that not only component teams talks to each other but also that DEV and OPS team talks to each other and collaborates. Cause that's what it's about at the end of the day, people, processes and tools.

You said you would show me how to do this right?

Yes, correct. We chose to use Azure as our tool of choice. Hopefully, the principles and patterns behind what I'm about to show are generic enough so you easily can translate that to whatever system and tool you prefer.

Two approaches

When dealing with CI/CD on Azure it's easy to think of it as there being two different approaches or paths we can take to add CI/CD to our code project.

• The simpler approach, in this approach I will describe how to connect your repository to Azure. It will then deploy every time you push to a branch. Additionally, I will describe things like Deployment Slots and what to use that for. This article will cover this approach.
• The more advanced approach, in this approach we will connect our repo to an Azure DevOps project and we will be setting up build pipelines and release pipelines so that you can really control every step of the way. We will use this approach in a follow-up article.

Demo

As we wrote in the previous section we will show how to set up CI/CD using the simpler approach. That means we will start with a GitHub repo. Before we come that far let's build something. An app, a Node.js app with Express. This will become a REST API that we can interact with through HTTP and a deployment URL.

Creating our project

For this, you will need Node.js installed. Here's a link to the install page:

https://nodejs.org/en/download/

Let's start at our computer. Find yourself a directory and type the following:

npm init -y

This will initiate a Node.js project using smart defaults. Next, create an application file, app.js:

touch app.js

Let's add the following code to app.js:

// app.js

const express = require('express')
const app = express()
const port = process.env.PORT || 3000

app.get('/', (req, res) => res.send('Hello World!'))

app.listen(port, () => console.log(`Example app listening on ${port} port!`))

After that install our web library express using this command:

npm i express

This will install it in a directory called node_modules.

Add Git to it

Now let's create a Git repo. To initialize it type:

git init

Create a .gitignore file as well with:

touch .gitignore

Add the following content to .gitignore:

node_modules
package-lock.json

The above will ensure we don't version control files and directories that we don't need.

Ok, go to GitHub and create yourself a repo. Because we haven't pushed to it yet it should list something like this as helper info:

echo "# name of app" >> README.md
git init
git add README.md
git commit -m "first commit"
git remote add origin https://github.com/<your user>/<your app>.git
git push -u origin master

Because we've already done most steps we just type ( remember to switch out the user name and repo name for your data):

git remote add origin https://github.com/<your user>/<your app>.git

Before we can push code to our new GitHub repo we need to add our first commit. Type the following:

git add .
git push -m "first commit"

Now, let's push our code to the repo:

git push -u origin master

Create a Web App in Azure

Great. Now we have our code pushed to a GitHub repo. Time to add CI/CD to it. If you don't have an Azure account, go sign up for one with this link:

Free Azure account

Ok, let's log in to the Azure portal.

portal.azure.com

We will do two things:

1. Provision, we will create a resource for our app to live in. We will select the template Web app. This will give us a dedicated environment where our app can live. Depending on what choices we make it will install some libraries for us so that our app can run smoothly. The point is we are just asked for some options and it takes care of the rest. This is a platform as a service, nothing to manage.
2. Connect our repo, once we've created our web resource we are ready to connect our resource with a GitHub repo. We will then take the help of something called App Service. App Service is a service on Azure that will both deploy and run the web app for us. It can do a lot more things for us like dealing with scaling, security and more. For the purposes of this article though it helps us to host our Web App.

Provision our Web Resource

Once logged in we want to create a Web App. Before we have pushed our code to it, it will just be an empty shell.

At your top left in the portal you will find a button looking like so:

Click that button and now enter Web App in the search field. Click Create and you will be taken to a page looking like this:

1. Subscription, select your subscription you want to use
2. Resource group, this is a logical bucket. This is where you want to place all Azure resources that go together like a Database, WebApp, storage account and more. Choose whether to create a new one or use an existing one.
3. Name, this needs to be unique as it will be part of a global URL that anyone can reach. The full URL will be <name>.azurewebsites.net.
4. Publish, the choices are Code or Docker Container. We will go with Code this time, but we will show how to use the Docker Container option in another article.
5. Runtime stack, this is where we can choose between different coding environments like Node.js, ASP.NET Core, Python and so on. What this means is the machine our Web App will deployed to, will have these libs installed that corresponds to your option. Let's choose Node.js 12 LTS.
6. Operating system, let's go with Linux for now. We could easily have gone with Windows as well.
7. Region, Select what region is closest to you
8. App Service Plan, select default

Now, hit Review and Create and on the final step that follows click Create.

Connect our repo

This will take a minute or so but once provisioned you should have something that looks like so:

We've selected Deployment Center from the left menu and if we look to the right we have a headline Continuous Deployment. If we scroll a bit we will see all the options for that headline:

As you can see there are four major options to choose from where our code comes from. We will choose the GitHub option.

Next, we will be asked for build provider. We can choose between App Service build service and Azure Pipelines. We will go with the first option:

Next, we need to configure. We do that by selecting

• Organization, the org we belong to at GitHub
• Repository, this is the repo we just created
• Branch, now this is an interesting one. When we first created our repo we just have the master branch. But as our repo grows we will have tons of branches on it, possibly and we can use that when doing Blue-Green deploys and A/B testing. For now, select master.

Once all of this is filled in you will come to a summary page. Click the Finish button.

What follows, as seen above, is a page showing our app running and history of commits. We can learn more of its state by clicking the icon under Logs so let's do that:

Ok, above we see some logs from the system and the last entry is telling us Deployment successful.

Great do we have an app up and running now? :)

Let's see. Click Overview in the left menu and enter address under the headline URL and it shows drumroll this might take a few seconds the first time this is done as it needs to install some libs, continued drumroll ;)

How bout now? :)

Not quite, a few more seconds and there it is:

Narrator - it's a white page

What's wrong? :/

Can you guess what the problem is?

No, that's why I'm asking, what's wrong?

You have a Node app and a Node app needs a what to run?

Some kind of start command?

B I N G O and BINGO was his name oh.

So I need to add a starter instruction to my package.json

Yes. In your scripts section add:

"start": "node app.js"

Now what?

Now, we need to commit that to the repo and push it to GitHub. Thanks to the way we set things up Azure will pick up on it and redeploy it and we should get a working app. So do the following:

1. Add the code change above to package.json
2. git add .
3. git commit -m "adding this change cause the author of the article tricked me"
4. git push

Great it works, I won't forget you tricked me though.

CI

CI stands for Continuous integration and means that we integrate code to a shared repo as soon as we can. Additionally, we want to run additional automated tests as soon as we have changed our code. We run these tests to ensure that the component we are working in still works and possibly that it's still able to work with other components.

So how do we add CI to this?

I know I know. I can install Jest, write a test, add test to scripts in package.json and it fails deployment if the test fails?

Yea, NO, sorry. We need Azure DevOps for that. Also, you would need to tell a YAML file that you want to run those tests, it's not enough to just create some tests and hope Azure DevOps will pick up on it. That's all described in part two though.

So the next article? :)

A disappointment, that's what you are BOO

Sorry :)

There's gotta be more than CD right?

Yes, there are, deployment slots :)

and they work?

They work. Let's talk about them next.

Deployment slots

So what are those?

Imagine you can deploy to different slots but under the same URL.

Why would I want that?

Well, imagine you want to control traffic to your app so that 50% end up on one of the slots and 50% on the other slot. See what we can do with this?

Hmm I think so, for testing out new versions or experiments then I can send half to one slot and half to the other?

Precisely! :)

Creating slots

So, click Deployment slots in the left menu and it should look like this:

As you can see above we have one slot only, PRODUCTION.

Now let's think a bit. What do we want the other slot to be?

Depends on what we want to use it for. How bout we try to run an experiment?

Yea ok. So let's run an experiment and place the experiment on a feature branch.

So this means we need to:

1. Create a branch in git
2. Do our changes
3. Push branch to GitHub
4. Create a slot that looks almost the same as the production branch but we want it to deploy from our new branch

Create a branch

git checkout -b feature/new-experiment

Do our changes

ok, let's recap our app code. It currently looks like this:

// app.js

const express = require('express')
const app = express()
const port = process.env.PORT || 3000

const products = [
  {
    id: 1,
    name: "Star Wars"
  }
];

app.get('/', (req, res) => res.send('Hello World!'))

app.listen(port, () => console.log(`Example app listening on ${port} port!`))

let's for the sake of it change it so it has the additional route /products. The code should now look like this:

// app.js

const express = require('express')
const app = express()
const port = process.env.PORT || 3000

const products = [
  {
    id: 1,
    name: "Star Wars"
  }
];

app.get('/', (req, res) => res.send('Hello World!'))

app.get('/products', (req, res) => products)

app.listen(port, () => console.log(`Example app listening on ${port} port!`))

Push changes to GitHub

Ok, let's commit this:

git add .
git commit -m "adding new route /products"

and push it to our repo:

git push

Ok then, we are have pushed this branch to GitHub, but because our CD set up is listening to our master branch - nothing happens to our deployment. It's time to change that by creating a new slot.

Create a slot

Let's head back to our portal and our Web service resource. Select Deployment slots in the left menu. Next, click Add slot in the top menu, as indicated below:

Now, clone our existing slot for production, cause it contains most of what we want.

We need to change one detail though, namely what branch it looks at for changes.

1 Select our branch again by clicking it in the list. This should take you to a new page.

1. Select Deployment center from our left menu.
2. Click Github and Continue.
3. Click App Service Build Service and then Continue.

Now fill out the same Organization as our production slot. The same Repository as production slot and lastly change the Branch to our feature branch:

Now save this new slot. This should start to build this immediately.

Control traffic

Now that we have two deployment slots we can decide how to control traffic to our site. We do so by changing the percentage text box next to our slot.

Because we are running an experiment, we want x number of users to be sent to the production URL and y % to be sent to our feature branch. Exactly how you measure success in your experiment is up to you. However, let's talk about how that can look so we understand A/B tests a little better. A/B has a mission to get a question answered. Usually, that means we have questions like, is this design better than that design. Better is usually defined as does the user interact with a certain piece of content by input or by clicking something. At this point, you either change parts of an existing page or swap it out altogether.

Another type of A/B could also be to see what the user thinks of a change to logic, for example - if we were to change the discount percentage on a site, as an experiment, would the user still buy that item?

As you can see deployment slots can really help us by

1. Different content can be deployed to different slots
2. Traffic control helps us send a certain percentage of users to a certain experiment.

Blue/Green deploy - swap slots

Let's look at another case for deployment slots. Namely zero-downtime deploy. What does zero-downtime mean? It means we have updated our site somehow and we want to deploy the latest version of it. We want to do so in a responsible manner so the user doesn't perceive that our site is down e.g zero downtime and deployment slots can do just that.

What do we mean with responsible? Well, Continuous deployment doesn't just mean that we deploy things often but it also means that we have the tools to correct any errors fast. Being able to correct errors really fast makes us confident enough to dare to deploy often. So how do we correct errors? The answer is something called blue green deploy. This means that we have two buckets or slots. In one bucket we have our software that runs in production, let's call that bucket PROD. In the other bucket we have the software we want to release, let's call that CANARY. We want to employ the following strategy:

1. Migrate users, Slowly send users to bucket CANARY
2. Monitor our app and error logs for any errors.
3. IF there are errors - send CANARY users back to PROD by setting CANARY percentage to 0%
   • Fix errors and start from step 1 again
4. ELSE, there are no errors, gradually increase the number of CANARY users. At some point, you might feel confident enough about the CANARY release and choose the CANARY to be the new prod. What you can do now is to select swap, this will make CANARY the new PROD.

Deploying often and with confidence - isn't that cool? :)

Summary

Let's summarize our learnings. This was about learning to add Continuous deployment to our app. To do so we needed to

1. Create an app,
2. Push the app to a GitHub repo.
3. Create a Web App resource in Azure.
4. Connect the repo with our Web App resource

Additionally, we learned how to use a concept called deployment slots for A/B testing but also for blue/green deployment.

It should be said though that this approach is a nice one if you are testing things out a bit and you have a small project of 1-2 devs. The reasoning behind this is that it's somewhat limited. If you need Continuous Integration, CI you probably also want a concept such as gates and pipelines. Azure DevOps supports all those features that this approach is missing and coincidentally that's the topic of the next article in this series.

Comments

[Chris James]

Continuous Integration means that as soon as we push code to a repo we run some suitable tests to ensure that our code still works.

Not really

Gamifying Continuous Integration

Chris James ・ Nov 28 '19 ・ 6 min read

#ci #agile #devops

[Chris Noring]

thanks for pointing that out Chris. I've updated the text and added a link to your article :)

[Chris James]

Nice changes. As you say there are various tools that facilitate safe CI (eg run tests etc) but it's important we remember what the principle of CI is and why it's important

[Chris Noring]

agree :)

[Victorio Berra]

Great writeup! Highly detailed. Question for you in the DevOps/Pipelines/Releases area, how hard is it to deploy a WebJob to an existing AAS?

[Chris Noring]

hi Victorio, thanks. About your question, is this what you are looking for? docs.microsoft.com/en-us/azure/app...

[Victorio Berra]

Yes but thats through VS not Azure Pipelines Releases. Lets say you have Pipelines Pipeline run dotnet publish on your core worker service (dotnet new worker), and you publish that artifact. So you have a nice publish directory ready to deploy thanks to CI. Now you want to go into your Azure DevOps project, you click the PipeLines button with the blue rocket, and you click "Releases".

From there, you see "No release pipelines found", so you click the blue "New pipeline" button,and you are greeted with a giant list of templates. So you try and search the templates for "WebJob", up pops "Azure App Service Deployment". So you think, well WebJobs are inside AAS, so, this must have an option to deploy a webjob.

After you go through the fun of figuring out how to handle "connections" from DevOps to Azure, this is where the confusion comes in. What do I have to do in the "Deploy Azure App Service" job to be able to deploy my web job.

Side note: The naming of all this stuff is really weird. A pipeline can release, but is not itself a release. Releases have jobs that can release. So you have to be super specific when you discuss pipelines. In Atlassian Bamboo, you have Builds and Deployments. Perfectly separate and obvious naming. Links directly with the concept of CI and CD being completely separate things.

[Chris Noring]

this might be more towards your questioning, pekkahuuskonen.com/2019/01/3-ways-... yea I agree with the naming.. I think of pipeline as describing a flow of things you want to happen sequentially. Let me know if this link doesn't work for you and I'll try to chase that up interally at MS

[Victorio Berra]

I did a deep dive and published my findings. dev.to/victorioberra/gotchas-when-...

In summary, in the pipeline.yaml get the publish settings just right, and then just use the Deploy Azure App Service Pipeline Deployment task as normal.

[Shaiju T]

Added to reading list 😄, FYI, Azure portal UI gets updated once in 2 year I guess, in that case you should update all the screen shots. 👍