DEV Community

loading...
Cover image for Setting up automated testing and deployment to Gigalixir for a Phoenix Application using GitLab.

Setting up automated testing and deployment to Gigalixir for a Phoenix Application using GitLab.

Edwin Mak
An aspiring artist using code as another medium for creation. I am currently a software developer in an EdTech company focused on improving communication in the workplace. I work heavily in Ruby!
・7 min read

Introduction

I published this article earlier on medium here and felt that the dev.to community might gain some value from this article. Therefore, I've decided to port the article to here :). Hope it helps more people.

In this article, I go over the few easy steps you can take to improving the foundation of your old or new Phoenix (1.4) application using Continuous Integration (CI) and Continuous Deployment (CD) tooling made easily accessible by GitLab. These improvements include automated testing using mix and automated deployment to Gigalixir. I hope at the end of this article/tutorial, that you will feel empowered to utilize these awesome tools in your next project!

Why GitLab & Gigalixir?

GitLab provides an easy way to get version control features & CI/CD without having to rely on different services. A common combo that I have had to exercise was using GitHub for version control and a service like Travis or CircleCI to handle my automated processes. I believe using a single service can ultimately make things easier and not require developers to juggle services.

Gigalixir provides ample support for Phoenix (Elixir) projects. Traditionally I would have started with Heroku, however many of the benefits of Elixir & Phoenix are not readily supported by them. A big point of concern was that Heroku does not support clustering (nodes talking to each other) out of the box. Clustering is one of the killer features that Elixir brings to the table. Jesse Shieh, the founder of Gigalixir, describes in this article the benefits of clustering.

Requirements

This article assumes that you have already installed Elixir & Phoenix and have the capability to start a new project via the command:

mix phx.new

If you have not yet installed Elixir and Phoenix yet, you can get up and running following this installation guide here.

Setting Up Your GitLab Project

If you haven’t already created a project, you can sign up for a GitLab account here and create a blank project here. After you have created your project, click on the Clone dropdown button, and copy the SSH (or HTTP) address of your GitLab project.

Image Of GitLab Project URL

Create your project (Skip this step if you have an existing project)

mix phx.new toyapp

And in your project root directory add the remote and push your code up to GitLab:

cd toyapp/
git remote add origin git@gitlab.com:edwinthinks/toyapp.git
git push origin master

You should see your files now populated in your GitLab project page.

Add Automated Testing

Now we are going to tell GitLab to run tests in a job on pushes of code changes to master, tags, and merge requests (Pull Requests).

First, we need to slightly modify the settings in config/test.exs to utilize the correct database when running the job. In order to do that, you can modify the database configuration to this:

Configure your database

config :toyapp, Toyapp.Repo,
  username: "postgres",
  password: "postgres",
  database: "toyapp_test",
  hostname: System.get_env("POSTGRES_HOST") || "localhost",
  pool: Ecto.Adapters.SQL.Sandbox

The reason we do this is because the job wont have a Postgres database available on localhost as you would while developing on your local environment. In this way, we can control the host used by defining the environment variable POSTGRES_HOST.

Next, we need to create the file with the instructions to tell GitLab when, what, and which jobs to execute. In your project directory, create a gitlab-ci.yml file and paste this in:

stages:
  - test
image: bitwalker/alpine-elixir-phoenix:latest
test:
  stage: test
  services:
    - postgres:latest
  variables:
    POSTGRES_HOST: postgres
    POSTGRES_USER: postgres # must match config/test.exs
    POSTGRES_PASSWORD: postgres # must match config/test.exs
    MIX_ENV: "test"
  script:
    - mix deps.get --only test
    - mix ecto.create
    - mix ecto.migrate
    - mix test
  only:
    - master
    - merge_requests

The instructions above tells GitLab to start a “test” job whenever changes occur on the “master” branch or any merge requests. This test job tells GitLab to start a Postgres database service configured via the environment variables defined in under variables. As you can see POSTGRES_HOST is defined and thus will direct our phoenix tests in the job to use the correct database.

Lets push up the changes:

git add .gitlab-ci.yml config/test.exs
git commit -m "Add gitlab test job"
git push origin master

Now we can check up on the newly spawned pipeline process in your GitLab project’s CI/CD dashboard. You can access this page from the click on the CI/CD button displayed on the sidebar on the left. In your dashboard you should now see the status and progress of your running jobs.

GitLab Img

Click onto the Pipeline ID (in the image is #44166697) to access the pipeline details and the status of the jobs that comprise the pipeline.

GitLab Img

Great! Now your application should be automatically running your tests anytime you push code changes on master or on any merge requests.

Deploying To Gigalixir

Before we can setup automated deployment, we must first be able to manually deploy our application to Gigalixir. (You can find official documentation here)

First download and install the Command Line Interface (CLI) provided by Gigalixir. If you don’t already have an account you can register via the CLI by executing this command:

$ gigalixir signup

Once you’ve finished signing up your account, login through the CLI by executing the command and say ‘yes’ when asked to save your API key in the ~./netrc file when prompted:

gigalixir login

Next we need to create the application Gigalixir and ask for a Postgres database. In your project’s root directory execute:

gigalixir create -n <your_app_name>
gigalixir pg:create --free

Lets now prepare our application be deployable (You can get more information here). First open up your config/prod.exs file and remove the line:

import_config "prod.secret.exs"

Modify the url option to include your correct hostname for your application. Add the secret_key_base to use the auto generated environment variable provided by Gigalixir and set server to true. Your application endpoint configuration something like this:

config :toyapp, ToyappWeb.Endpoint,
  http: [:inet6, port: System.get_env("PORT") || 4000],
  url: [host: "<your_app_name>.gigalixirapp.com", port: 80],
  cache_static_manifest: "priv/static/cache_manifest.json",
  secret_key_base: Map.fetch!(System.get_env(), "SECRET_KEY_BASE"),
  server: true

Afterwards we must tell the application to utilize the Postgres Database provided by Gigalixir by adding the database configuration with the DATABASE_URL. Add these contents to your config/prod.exs.

config :toyapp, Toyapp.Repo,
  adapter: Ecto.Adapters.Postgres,
  url: System.get_env("DATABASE_URL"),
  ssl: true,
  pool_size: 2

Next you’ll need to tell inform the build system to use the correct version of Elixir & Erlang to install for your phoenix application. Without setting specifying the version you may run into an issue in deployment of mismatched versions. Create a file called elixir_buildpack.config in your project parent directory with these contents:

elixir_version=1.7.4
erlang_version=21.0

If you are using Phoenix 1.4 or higher, you may have to configure the build to use webpack. You can do this by adding a compile file in your parent directory with these contents:

npm run deploy
cd $phoenix_dir
mix "${phoenix_ex}.digest"

Our application should now be ready for deployment to Gigalixir! Lets test that out by committing our changes and manually deploying.

git add config/prod.exs compile elixir_buildpack.config
git commit -m "Make deployable to Gigalixir"
git push gigalixir master

Once deployment has finished, you can check that your application was deployed successfully by going to “https://.gigalixirapp.com/” in your browser or executing gigalixir open in your terminal. If all seems well we can now move onto setting up GitLab to deploy automatically after our tests pass.

Add Automated Deploying

In your GitLab project CI/CD settings, add an environment variable named GIGALIXIR_REMOTE_URL that tells your where to upload (push) new code to. You can generate it the information in your ~/.netrc file under machine api.gigalixir.com. Note, your API key is listed as your password and the email must encode the “@” symbol to a “%40”. For example if your application name was foobar and your ~/.netrc file includes this:

machine api.gigalixir.com
 login test@example.com
 password 323b12-71a5-432f-8579-0b8g8cb60b15

Your GIGALIXIR_REMOTE_URL would be this:

https://test%40example.com:323b12-71a5-432f-8579-0b8g8cb60b155@git.gigalixir.com/foobar.git
Navigate to your GitLab project’s CI/CD setting. You can access this page from the settings option on the left hand sidebar within your project’s primary page. Within the Environment Variables option lets add GIGALIXIR_REMOTE_URL variable.

Setting up Environment Variables

In your .gitlab-ci.yml file, lets add the deployment instructions to inform GitLab how to deploy to Gigalixir. In addition to the deploy instructions, lets ensure that we check that our tests pass prior to deploying by defining pipeline stages. Your configuration should file should look something like this:

stages:
  - test
  - deploy # Add this line!
image: bitwalker/alpine-elixir-phoenix:latest
test:
  stage: test # Add this line!
  services:
    - postgres:latest
  variables:
    POSTGRES_HOST: postgres
    POSTGRES_USER: postgres
    POSTGRES_PASSWORD: postgres
    MIX_ENV: "test"
  script:
    - mix deps.get --only test
    - mix ecto.create
    - mix ecto.migrate
    - mix test
  only:
    - master
    - tags
    - merge_requests
deploy: # Add this 'deploy' stage description
  stage: deploy
  script:
    - git remote add gigalixir $GIGALIXIR_REMOTE_URL
    - git push -f gigalixir HEAD:refs/heads/master
  only:
    - master

Notably these deployment instructions do not automatically run migrations. If you want to have each deployment run a migration (I do this) you can simply add Procfile to your project root directory with this:

web: mix ecto.migrate && elixir --name $MY_NODE_NAME --cookie $MY_COOKIE -S mix phx.server

Now lets commit our code and push it up to gitlab:

git add .gitlab-ci.yml
git commit -m 'Add automated deployment to Gigalixir'
git push origin master

You should now see in your GitLab project’s pipeline page a new pipeline instance with your “Test” and “Deploy” running in order:

Running Pipes

You should see something like this soon after:

Running Pipes 2

Success! Your CI/CD pipeline is setup. Now your application should be running your tests and then deploy to Gigalixir in the event of changes made to your master branch.

Conclusion

Today it is fairly easy to setup a simple CI/CD pipeline for any project including ones born from Phoenix as this article aimed to illustrate. I hope that readers of this article are empowered to utilizing these fantastic tools for their next project.

Discussion (0)