Testing your app with docker-compose on CircleCI

This article was originally posted on my blog.

As a dev, you use docker as a dev environment. You're quite happy that you finally ended mastering it (or at least make it work \o/). Since you're a great developer, you know the importance of automated testing, and you wrote a bunch of specs or features to test. You made everything work locally thanks to a few docker-compose commands. Maybe you're thinking now that it could be awesome to even automate the process of launching those tests, by delegating this process to a CI, that will listen to your Github events, and send you back a green or red dot on your Pull Request's page.

Since you used CircleCI a few years ago, since they claim it's easy to use with docker and since they provide a free plan, you think you're gonna make it standing on your head. Well, let me tell you that it can be quite a pain in the a** if you don't have the right instructions, and the documentation is quite awful when it comes to using it with docker-compose. Fortunately, it can be very easy when you get the right options to use in the config.

Let's dive into it!

After having created your CircleCI account and connected it to your repository, you will have a sample config file given by CircleCI. It may look like this (hum...):

version: 2
jobs:
  build:
    working_directory: ~/mern-starter
    # The primary container is an instance of the first image listed. The job's commands run in this container.
    docker:
      - image: circleci/node:4.8.2-jessie
    # The secondary container is an instance of the second listed image which is run in a common network where ports exposed on the primary container are available on localhost.
      - image: mongo:3.4.4-jessie
    steps:
      - checkout
      - run:
          name: Update npm
          command: 'sudo npm install -g npm@latest'
      - restore_cache:
          key: dependency-cache-{{ checksum "package.json" }}
      - run:
          name: Install npm wee
          command: npm install
      - save_cache:
          key: dependency-cache-{{ checksum "package.json" }}
          paths:
            - node_modules
  test:
    docker:
      - image: circleci/node:4.8.2-jessie
      - image: mongo:3.4.4-jessie
    steps:
      - checkout
      - run:
          name: Test
          command: npm test
      - run:
          name: Generate code coverage
          command: './node_modules/.bin/nyc report --reporter=text-lcov'
      - store_artifacts:
          path: test-results.xml
          prefix: tests
      - store_artifacts:
          path: coverage
          prefix: coverage

workflows:
  version: 2
  build_and_test:
    jobs:
      - build
      - test:
          requires:
            - build
          filters:
            branches:
              only: master

Well, that's quite intimidating. At least there is the docker keyword, so you think you're going in the right direction. Let me tell me you're wrong.

Actually, this docker keyword means your CircleCI build will be launched in a docker container instead of a Linux VM. After reading the whole documentation, you realize that you have to use a Linux VM to be able to use docker-compose with volumes (link: https://circleci.com/docs/2.0/executor-types/ ).

So let's create a new config file, step by step.

version: 2 # CircleCI version
jobs:
  build:
    machine: true # Use a Linux VM instead of docker environment
    working_directory: ~/repo # Default working directory, where your project will be cloned

So we have a very basic install. Nice. Let's add a checkout of the repository.

  build:
    # ..
    steps:
        - checkout
        # Maybe you need to add some config file specific to circle…
        - run: cp .circleci/.some-parameter-file .some-parameter-file

Docker-compose is by default installed on the Linux VM provided by CircleCI. We can directly build and up our containers.

  build:
    # ..
    steps:
        # - …
        - run: docker-compose up -d

Now you can install the dependencies of your project. Here we have node (yarn) and PHP (composer) dependencies.

    steps:
        # - …
        - run: docker-compose exec php composer install -n --prefer-dist
        - run: docker-compose run --rm front yarn install
        - run: docker-compose run --rm front yarn build

Now we can run our tests!

        # run tests!

    steps:
        # - …
        - run: docker-compose exec php vendor/bin/phpspec run -vvv
        - run: docker-compose exec php ./vendor/bin/behat -v

Tada! An extra bonus? Let's say you do some End2End testing and you take some screenshots when a test failed… Maybe you want to be able to access the screenshot (or log file) from the CircleCI interface? You can add a directory as an artifact. This will be then available in the artifacts link above your tests.

    steps:
        # …

        # This line is to be able to access failing screenshots in the
        # artifacts section in CircleCI
        - store_artifacts:
            path: ~/repo/features/fail-screenshots

So cool! Now you have a CI running with docker-compose, using your dev environment. Pretty swag.
Now, some of you may encounter permissions problems because of the users. This can be because CircleCI uses a user with id 1001 and a group with id 1002. Sometimes in can conflict with your local permissions (your local user being 1000 in general). If so, you can specify the user that will launch the docker command.

    steps:
        # run tests!

        # Circleci uses a user with id 1001 and a group with id 1002
        # We use this user instead of the default one in our dockerfile (1000)
        # to avoid permissions issues
        - run: docker-compose exec --user=1001:1002 php vendor/bin/phpspec run -vvv

        # We use the root user here to change logs and cache permissions
        - run: docker-compose exec --user=root php chmod -R 777 var/logs
        - run: docker-compose exec --user=root php chmod -R 777 var/cache

I hope this article helped you to make this small green dot appear on your PRs :).

(Using Gitlab? See here)

Image credits: "Bricks in circle" by Alessandro Mancini