Setting up tests in GitLab CI for Django project with Docker Engine
Konstantin Pavlovsky Feb 16 '17
In this article I will describe how we set up GitLab CI to run tests for Django project.
But first a couple of words about what tools we were using.
What we had before GitLab – Atlassian
For a few years we were using Atlassian stack for our development process. We had JIRA, Confluence, Stash(now it is called BitBucket Server) and Bamboo.
At first we were happy with that setup, because all apps had good integration with each other.
We could see all commits related to currently open issues, and we could see what issues and commits were included in build in Bamboo.
It looked wonderful at first, but after some time we noticed that:
- JIRA began consuming all our time even with Agile Board
- keeping this stack up-to-date was a huge pain. I'd spend whole weekends updating and repairing all four apps, because they’d changed requirements for Mysql settings, and logs in Atlassian products are buried deep in various directories
- Every new version of Atlassian apps was introducing numerous bugs and interface changes (which with obvious lack of QA was causing even more bugs)
Plus we only had a 16GB RAM server for all this tools.
So instead of doing our job, we were spending all our time dealing with JIRA.
New developers who were joining our team were frustrated with the JIRA interface.
At the end of last year we thought: “That’s enough, we need to replace this enterprise-focused monster with something easier and productive tool.”
At first, we started looking for a replacement for task tracking functionality. We tried many apps and services for that, but all of them, while having some strong features, also had issues or lack of functionality which prevented us from being productive.
Then I tried GitLab and suddenly we not only found the solution for convenient task tracking tool, but we found how to replace the whole Atlassian stack.
What is also is amazing we get everything for free! (Except worker, but i have unused linux box, which now serves as worker).
GitLab has a clean and simple interface, instead of the enterprise-crazy workflows of Jira, issues have labels, there is CI pipelines, Wiki, markdown for issue descriptions, comments and wiki articles, and many other great things.
We have switched to GitLab.com
GitLab projects solved all our problems — no more maintenance, no pain with productivity-killing interface, all in one solution.
So gitlab project provides git repository hosting, issue tracker, wiki, CI/CD management system. But what we need is CI runner. Runner does actual job, executing building, testing and deployment jobs.
As I already said, I had unused linux box, which now is used for runner.
Runner installation for ubuntu described here:
We used docker for running our builds.
Migration from Bamboo to GitLab CI/CD
In Bamboo build and deploy plans are setup from GUI.
.gitlab-ci.yml must be created in root of git repository.
Before I provide example of this file, I must point out that we use Postgres as a database. Thanks to docker we can require runner to run it as service with credentials from this file.
image: kpavlovsky/backoffice:latest services: - postgres:latest stages: - test - deploy variables: POSTGRES_DB: dev POSTGRES_USER: dev POSTGRES_PASSWORD:dev all_tests: stage: test script: - bash ./tests_gitlab.sh
Line by line:
image: kpavlovsky/backoffice:latest is docker image we use to run container.
This image is based on python:3-onbuild. We moved all long-running pip install and apt-get tasks there. By doing this we achieved two things: 1) each build runs faster, because it doesn’t involve packages installation 2) we do not abuse apt and pip repositories by downloading packages 10s or 100s time per day 3) we decrease time of each build so we get results of tests much faster (who wants to wait for 15 minutes after each
git push ?).
In services docker images for services are listed. Here we have only postgres.
In variables section sets up postgres database and its credentials. Hostname for postgres will be ‘postgres’.
The ordering of elements in
stages defines the ordering of builds' execution:
- Builds of the same stage are run in parallel.
- Builds of the next stage are run after the jobs from the previous stage complete successfully.
Job in ‘test’ stage goes first. Jobs in ‘deploy’ stage goes after ‘test’ stage.
all_tests is a job our of pipeline, belonging to ‘test’ stage.
script hold all commands that will be issued. We have only one command here — to run tests.
#!/bin/bash coverage run --source="app1,app2,app3" manage.py test --noinput -- testrunner="xmlrunner.extra.djangotestrunner.XMLTestRunner" app1.tests app2.tests app3.tests coverage report --skip-covered
Dockerfile for that separate
kpavlovsky/backoffice:latestimage looks this way:
FROM python:3-onbuild ENV PYTHONUNBUFFERED 1 ENV PYTHONDONTWRITEBYTECODE 0 ENV DJANGO_SETTINGS_MODULE project.settings.docker RUN apt-get update && apt-get install -y --force-yes mc nano htop python python-pip netcat gettext && rm -rf /var/lib/apt/lists/* RUN mkdir /code WORKDIR /code COPY requirements.txt /code/ RUN pip install --upgrade pip RUN pip install -r requirements.txt CMD ["bash"]
So now if we push code to repo and tests pass, then we see notification in slack and email that pipeline is successful or that pipeline is failed if tests fail.
Now that we migrated the build part, we need to deploy our project in case of successful build on ‘dev’ branch to staging.
‘Stage’ environment is another linux box, without docker, just supervisor and gunicorn.
Deployment process involves ssh-ing to remote box from runner, activating virtualenv, git pulling and running django management commands.
First step is add job in ‘deploy’ stage in our
deploy_stage: stage: deploy script: - bash ./deploy_stage_gitlab.sh when: on_success only: - dev</pre>
This job will only be run on ‘dev’ branch and only if ‘test’ stage is successful.
To ssh to ‘stage’ machine we need to transfer ssh keys to runner.
Storing keys in repository is bad practice.
Thanks to gitlab ‘Variables’ we can transfer keys via environment variables, write them to files and then issue fabric command to execute required tasks on ‘stage’ box.
First we need to generate ssh key without passphrase. For this purpose, use
Public key must be put in
~/.ssh/authorized_keys on stage server.
Then we put contents of public and private keys to Variables.
After adding variables with keys, Variables screen in gitlab project looks similar to this:
deploy_stage_gitlab.sh looks this way:
#!/usr/bin/env bash mkdir ~/.ssh/ echo $STAGE_PRIVATE_KEY > ~/.ssh/id_rsa echo $STAGE_PUBLIC_KEY > ~/.ssh/id_rsa.pub fab upd_dev
Quick note: use fabric3 in python3.5 environment!
fabric function logs in to remote server, git pull everything, runs migrations and uses supervisorctl to restart process group for this project.
.gitlab_ci.yml we have ability to change pipeline, so it reflects changes in our code. That was harder to achieve with Bamboo and its build/deploy settings in GUI. Of course, we could let some bash-script to do all tasks and it would be changed from commit to commit. In this case it is impossible to add/remove/change stages, etc.
Also Bamboo’s guy build/deploy setup requires much more time to set up, I didn’t find a way to clone it from project to project, which is very easy to do with
From now, we don’t need a separate large server for Atlassian tools, we don’t spend weekends updating them. And the best part: we can focus on doing real jobs, developing and delivering OUR apps, instead of wasting time on Atlassian’s.
If you have any ideas about gitlab CI, ideas how to improve described workflow – post comments, i will be very happy to hear and discuss them!