DEV Community: Behnam Shomali

AzureDevOps, Elixir, Docker, CI/CD, and the others — Part 2: Release and deploy.

Behnam Shomali — Fri, 22 Nov 2019 13:01:15 +0000

In the previous post (part1) I tried to use docker-compose doing build and test in AzureDevOps. The app is a hello world web site written in elixir and phoenix.

Goal

The goal is to create a continuous deployment pipeline to deploy our web app to an app service in azure by using releases in elixir and a docker image.

My todo list is to create and config these services:

Azure Container Registry
Azure Postgres SQL Server
Build & Push pipeline in AzureDevOps
App service in Azure
Deploy pipeline in AzureDevOps

Azure

First thing first, we need a container registry. It will be used to store our Docker images. Let’s make one in the Azure portal.

Don’t forget to set the Admin user to Enable.

Where to put the database? hmmm! since we don’t use docker-compose here then we need to create a Postgres Server:

The next step is to create a database. To be able to connect to your database, your IP has to be added to the Allowed list. Do it in the Connection Security section of your Postgres Server page on Azure. Also, enable Allow access to Azure services, we need it later to be able to connect from our web app. An easy way to connect is by using pgAdmin. Also, you can use psql in bash and connect with:

psql --host=**practicing**.postgres.database.azure.com --port=5432 --username=**practiceadmin@practicing** --dbname=postgres

And run the create database command:

CREATE DATABASE practicingdb;

Since the app service needs to be created based on an image, and wedon’t have it now, I’ll do it later.

‌Build & Push

Let’s back to our beloved AzureDevOps. I am going to deploy on Azure, then I need to add Azure Container Registry service connection in Project Setting.

Also, I need a Dockerfile to build our production image. This is a multistage build docker file that uses the new Release feature of ELixir 1.9. It results in a very smaller image, something like 35 MB vs 1.2 GB if we use Release.

Create a new pipeline and in the Configure step, select Docker. In the Review step, go to the Docker task and hit Settings. Make sure that your config is like this:

Hit Save and run. When it got finished, check Repositories in the Container Registry. You should see a new one there:

Ok. It’s time to back to Azure Portal and create a new App Service.

Then go to Review + create and hit Create button.

Environment Variables

Before going further, there is something in the app that should be adjusted before going to deploy it. Let’s take a look at prod.exs at the config folder. It has a reference to prod.secret.exs which contains the connection string and secret_key_base. Also, this file is excluded by .gitignore. It is not a good idea to put this file in the source control. Instead, we can feed this data by Environment Variables. First, replace the last import_config line of prod.exs by this:

config :hello_world_ci, HelloWorldCiWeb.Endpoint,  
    secret_key_base: System.get_env(“SECRET_BASE_KEY”)
# Configure your database
config :hello_world_ci, HelloWorldCi.Repo, 
    username: System.get_env("DB_USERNAME"), 
    password: System.get_env("DB_PASSWORD"), 
    database: System.get_env("DB_NAME"), 
    hostname: System.get_env("DB_HOST"), 
    pool_size: 15

Go to your App Service page and find Configuration in Settings. Add the above keys and needed values. Database related values can be found in the Postgres page in the portal. Don’t forget to hit the save button.

(Please read this section carefully. There are some tricks for using elixir releases). Everything looks good but if you deploy your web app now you would realize that the app is unable to read the environment variables! No kidding! The reason is when the release build is used, it evaluates env variables for the prod config file at build time and never calls them again at run time. There is a good solution for it. You can add a file in the config folder named releases.exs and it will be used in the release build and will be able to read environment variables at runtime. Remove config items, like DB connection string and secret key from prod.exs and move them to releases.exs. Then you will be able to use System.fetch_env! as normal.

If you didn’t commit and push your last changes, do it and let the pipeline build it. When finished, click on Release. Here we go for the release pipeline.

From the right pan, select Azure App Service deployment and Apply:

Next, go on Tasks tab, Select Stage 1 and in the right panel click on Unlink all.

Click on Deploy Azure App Service in the left and fill the right panel as shown in this (some values may be different in your account):

After clicking on the Save button, hit the Create Release.

Fill it like above and hit Create.

In this step, I am going to change a file and then see the result on the website. Go to lib>hello_world_ci_web>templates>page>index.html.eex and add this line somewhere to be seen later:

<p>MIX_ENV: <b><%= System.get_env(“MIX_ENV”) %></b></p>

Commit and push it and wait until the Build pipeline finishes its work. Immediately, the Release pipeline should start.

Wait until it finishes its job.

Wait a bit and refresh the website:

Awesome! That’s it. It may seem complicated but trust me, it’s easier than it seems. What about the next part? In the next post, I will try to deploy the app on Kubernetes.

AzureDevOps, Elixir, Docker, CI/CD, and the others - Part 1: docker-compose and testing.

Behnam Shomali — Fri, 08 Nov 2019 13:49:47 +0000

The Problem

Elixir is a new player in the market, and the tooling is not as comprehensive as the others like C# and Java. When I started learning Elixir, I had some questions like:

Where to host it?
How to build and test it in a CI style?
How to deploy it on Azure?

The immediate answer was to have a VM on Azure, but that's not an affordable solution. Fortunately, nowadays the main problem is solved by using docker for development and deployment. In this post, I will share a simple solution to using AzureDevOps for the CI part of the development and in a follow-up article, I will cover the CD part. If you want to check out the code, here is the repo: https://github.com/benyblack/elixir-phoenix-ci.

One of the most popular approaches to website creation in Elixir is by using Phoenix. To set it up using this gist. The credentials for the Postgres server exist in dev.exs in the config folder. If your credentials are different, you can change them in the script Have a look at the final result in the following:

Navigate to http://localhost:4000, and you will see something like this:

Now that we have a working code, then we can think about the build and test part. To begin, let's try to do it locally with Docker.

First, we need a docker file:

Build and run it by these commands:

docker build -t benyblack/helloworldci .
docker run --rm -p 4000:4000 benyblack/helloworldci

Navigate to http://localhost:4000, and you should get see a working web site, albeit one with lots of connection errors in the console. It's time to use docker-compose magic. The config file for docker-compose is rather simple:

Furthermore, we need to change the database hostname in test.exs. Change the hostname to database. Then run this command to build and run our tests:

docker-compose up --abort-on-container-exit

There we have it! You should see the result at the end of the console messages:

Continuous Integration

That's it! It's now ready to be used in AzureDevOps. We need to connect our GitHub and DockerHub account to the project in the AzureDevOps project settings section. Go to the Project Settings at the bottom left side of the page and then select Service connections and hit "+ New service connection". From the list select Github. It is necessary to give access to our code on Github.

Do the same for DockerHub to connect your public repositories to the project, except select Docker Registery this time. We'll use it later for using docker-compose.

The next step is to create a Pipeline. Go to Pipelines from the main menu. Select Github from the list.

Hit Next and select Existing Azure pipelines YAML file.

A popup should appear from the right. Select the config file we have created.

Then, you can adjust the config by hitting the setting link or editing it manually. If your DockerHub connection name is not the same as mine, you'll have to change it.

Finally, hit the Run button at the top left of the page.

You should see something like this:

At long last, we have to have some green lights. However, there is one more step; publishing the test result. The problem is that AzureDevOps can't parse the test result from the mix test. The solution is to apply a Formatter for the test result. In this case, I chose to use JUnitFormatter. It exports the result to an XML file. Using the library is pretty straightforward. I appended this section in my test.exs in the config folder:

# Configure publishing test result
config :junit_formatter,
report_file: “report_file_test.xml”,
report_dir: “./”,
print_report_file: true,
prepend_project_name?: true

Moreover, we need to add a test publishing step to our pipeline. Go to the build menu and select Edit to edit the pipelines. From the right panel, select Publish Test Result and set the testResultsFiles to '**/hello_world_ci-report_file_test.xml'. Click Add. It should look like this:

Save it and run it again. You should see something like this:

Go to the Tests tab. Your Tests tab should be the same as mine:

Voilà! That's it! Pretty simple. In an upcoming post, I am going to do the CD part.