Jenkins: Docker Compose deployment from Ansible with ECR authentication

In addition to the AWS: create an Elastic Container Registry and Jenkins deploy job post – the next part, where we will create a new Jenkins job to deploy a Docker Compose file to run our Docker image.

Docker Compose file will be updated by an Ansible role called docker-deploy, which will set the desired TAG, will copy this file to the host and will start a systemd-service.

To make this deployment works – need to create an authentification for the Docker daemon on the host – let’s use amazon-ecr-credential-helper for this.

In this post we will review:

• Amazon-ecr-credential-helper
• Ansible deploy role
• Jenkins job

Amazon-ecr-credential-helper

First, let’s run it manually.

As there is no existing package dor Debian OS – create a bash-script which will trigger amazon-ecr-credential-helper via docker:

#!/usr/bin/env bash
SECRET=$(docker run --rm -e METHOD=$1 -e REGISTRY=$(cat -) -v $HOME/.aws/credentials:/root/.aws/credentials pottava/amazon-ecr-credential-helper)
echo $SECRET | grep Secret

Set executable permissions:

root@bttrm-dev-console:/home/admin# chmod +x /usr/bin/docker-credential-ecr-login

Create an AWS profile so AWS CLI will create the ~/.aws/credentials file which will be used by the amazon-ecr-credential-helper:

root@bttrm-dev-console:/home/admin# aws configure
AWS Access Key ID [None]: AKI***6EZ
AWS Secret Access Key [None]: PpN***GNr
Default region name [None]: us-east-2
Default output format [None]: json

Update the ~/.docker/config.json config:

{
    "credHelpers": {
        "534***85.dkr.ecr.us-east-2.amazonaws.com": "ecr-login"
    }
}

Here we set authentification for the 534***85.dkr.ecr.us-east-2.amazonaws.com repository by using the ecr-login authorizator.

Pull the image:

root@bttrm-dev-console:/home/admin# docker pull pottava/amazon-ecr-credential-helper
Using default tag: latest
latest: Pulling from pottava/amazon-ecr-credential-helper
...
Status: Downloaded newer image for pottava/amazon-ecr-credential-helper:latest

And try to run some docker pull for this ECR repository:

root@bttrm-dev-console:/home/admin# docker pull 534***385.dkr.ecr.us-east-2.amazonaws.com/bttrm-receipt-consumer
Using default tag: latest
latest: Pulling from bttrm-receipt-consumer
Digest: sha256:3bb7cebd34d7642b10fe44ad8ab375e5fd772fc82b4f6fa997c59833445fdef5
Status: Image is up to date for 534***385.dkr.ecr.us-east-2.amazonaws.com/bttrm-receipt-consumer:latest

Okay – it works.

Now, we need to do the same with Ansible.

Ansible deploy role

Create the Docker config template – roles/deploy-docker/templates/config.json.j2:

{
    "credHelpers": {
        "{{ bttrm_queue_consumer_repo }}": "ecr-login"
    }
}

AWS CLI authentication file – roles/deploy-docker/templates/aws-credentials.j2:

[default]
aws_access_key_id = {{ bttrm_queue_consumer_login }}
aws_secret_access_key = {{ bttrm_queue_consumer_pass }}

Add variables in the group_vars/mobilebackend-dev.yml:

...
bttrm_queue_consumer_repo: "534***385.dkr.ecr.us-east-2.amazonaws.com"
bttrm_queue_consumer_image: "bttrm-receipt-consumer"
bttrm_queue_consumer_login: "AKI***6EZ"
bttrm_queue_consumer_pass: "PpN***GNr"
...

Create the bash-script template – roles/deploy-docker/templates/docker-credential-ecr-login-sh.j2:

#!/usr/bin/env bash
SECRET=$(docker run --rm -e METHOD=$1 -e REGISTRY=$(cat -) -v $HOME/.aws/credentials:/root/.aws/credentials pottava/amazon-ecr-credential-helper)
echo $SECRET | grep Secret

And a template for the Docker Compose file which will start the container – roles/deploy-docker/templates/bttrm-queue-consumer-compose.yml.j2:

version: '3.5'

services:

  receipts:
    container_name: bttrm-queue-consumer-{{ backend_project_name }}
    image: {{ bttrm_queue_consumer_repo}}/{{ bttrm_queue_consumer_image}}:{{ bttrm_queue_consumer_tag }}
    environment:
      - APPLICATION_ENV={{ env }}
      ...

in thus Compose file Ansible will set the desired TAG taken from the Jenkins job’s parameters with the default value “latest“:

Create a systemd-unit file for the service – roles/deploy-docker/templates/bttrm-queue-consumer-systemd.yml.j2:

[Unit]
Description=bttrm-queue-consumer client
Requires=docker.service
After=docker.service
[Service]
Restart=always
WorkingDirectory={{ bttrm_queue_consumer_home }}
# Compose up
ExecStart=/usr/local/bin/docker-compose -f bttrm-queue-consumer-compose.yml up
# Compose down, remove containers and volumes
ExecStop=/usr/local/bin/docker-compose -f bttrm-queue-consumer-compose.yml down -v
[Install]
WantedBy=multi-user.target

In the roles/deploy-docker/tasks/main.yml add files copy and service start tasks:

- name: "Copy Docker config"
  template:
    src: "templates/config.json.j2"
    dest: "/root/.docker/config.json"
  when: "'console' in inventory_hostname"
- name: "Copy AWS CLI credentials config"
  template:
    src: "templates/aws-credentials.j2"
    dest: "/root/.aws/credentials"
  when: "'console' in inventory_hostname"

- name: "Copy Docker ECR credentials script"
  template:
    src: "templates/docker-credential-ecr-login-sh.j2"
    dest: "/usr/bin/docker-credential-ecr-login"
  when: "'console' in inventory_hostname"    

- name: "Set Docker ECR credentials script executable"
  file: 
    path: "/usr/bin/docker-credential-ecr-login"
    mode: 0755
  when: "'console' in inventory_hostname"
- name: "Create bttrm-queue-consumer home"
  file:
    path: "{{ bttrm_queue_consumer_home }}"
    state: directory
    mode: 0775
    recurse: yes
  when: "'console' in inventory_hostname"
- name: "Copy bttrm-queue-consumer Compose file"
  template:
    src: "templates/bttrm-queue-consumer-compose.yml.j2"
    dest: "{{ bttrm_queue_consumer_home }}/bttrm-queue-consumer-compose.yml"
  when: "'console' in inventory_hostname"
- name: "Copy bttrm-queue-consumer systemd file"
  template:
    src: "templates/bttrm-queue-consumer-systemd.yml.j2"
    dest: "/etc/systemd/system/bttrm-queue-consumer.service"
  when: "'console' in inventory_hostname"
- name: "Start bttrm-queue-consumer service"
  service:
    name: "bttrm-queue-consumer"
    state: restarted
    enabled: yes
    daemon_reload: yes
  when: "'console' in inventory_hostname"

Jenkins job

Create a Jenkins Pipeline job with the function ansibleApply():

...
def ansibleApply(app_rsa_id='1', bastion_rsa_id='2', tags='3', limit='4', playbookFile='5', passfile_id='6', connection='7') {

    docker.image('projectname/projectname-ansible:1.1').inside('-v /var/run/docker.sock:/var/run/docker.sock') {

        stage('Ansible apply') {
            withCredentials([
                file(credentialsId: "${app_rsa_id}", variable: 'app_rsa'),
                file(credentialsId: "${passfile_id}", variable: 'passfile'),
                file(credentialsId: "${bastion_rsa_id}", variable: 'bastion_rsa')
            ]) {
                sh "ansible-playbook --private-key ${bastion_rsa} --tags ${tags} --limit=${limit} ${playbookFile} --vault-password-file ${passfile} --extra-vars \
                    \"ansible_connection=${connection} \
                    app_rsa_pem_key=${app_rsa} \
                    bastion_rsa_pem_key=${bastion_rsa}\""
               }
        }
    }
}
...

Which will be called from the Jenkinsfile:

node {
...
        // infra for CloudFormation
        // from Jenkins job's parameters
        TAGS = "${env.TAGS}"
        // limit for hosts.ini
        LIMIT = "${env.LIMIT}"

        // playbook to run, e.g. mobilebackend.yml
        PLAYBOOK = "${env.PLAYBOOK}"

        // file with ansible vault password
        // to be used in ansibleApply()'s withCredentials()
        PASSFILE_ID = "${env.PASSFILE_ID}"

        provision.ansibleRolesInstall()
        provision.ansbileSyntaxCheck("${LIMIT}", "${PLAYBOOK}")
        ...
        // ansibleApply(rsa_id='1', tags='2', limit='3', playbookFile='4', passfile_id='5', connection='6')
        provision.ansibleApply( "${APP_RSA_ID}", "${BASTION_RSA_ID}", "${TAGS}", "${LIMIT}", "${PLAYBOOK}", "${PASSFILE_ID}", "${CONN}")
    }
}
...

Here we are passing the deploy-docker tag from the Jenkins job’s parameters to the Ansible playbook:

And by this tag – Ansible will close the role to apply – tags: deploy-docker:

...
    - role: deploy-docker
      tags: deploy-docker
      backend_project_name: "{{ lookup('env','APP_PROJECT_NAME') }}"
      bttrm_queue_consumer_tag: "{{ lookup('env','BTTRM_QUEUE_CONSUMER_TAG') }}"
      when: "'backend-bastion' not in inventory_hostname"

And during that will update the Docker Compose file with the bttrm_queue_consumer_tag value.

Deploy, check:

root@bttrm-stage-console:/opt/bttrm-queue-consumer# systemctl status bttrm-queue-consumer
● bttrm-queue-consumer.service - bttrm-queue-consumer client
Loaded: loaded (/etc/systemd/system/bttrm-queue-consumer.service; enabled; vendor preset: enabled)
...
Sep 26 16:06:23 bttrm-stage-console docker-compose[21251]: bttrm-queue-consumer-projectname-me-v3 | time="2019-09-26T13:06:23Z" level=info msg="Connecting as projectname_me_v3 to stage.backend-db3-master.example.world:3306/projectname_me_v3"

Sep 26 16:06:23 bttrm-stage-console docker-compose[21251]: bttrm-queue-consumer-projectname-me-v3 | time="2019-09-26T13:06:23Z" level=info msg="Declaring Queue (itunes-receipts)"

Sep 26 16:06:23 bttrm-stage-console docker-compose[21251]: bttrm-queue-consumer-projectname-me-v3 | time="2019-09-26T13:06:23Z" level=info msg="Waiting for queue messages..."

Done.

Similar posts

• 06/18/2019 SonarQube: running tests from Jenkins Pipeline in Docker
• 03/10/2019 Prometheus: RTFM blog monitoring set up with Ansible – Grafana, Loki, and promtail