An easy way to push Docker image from Jenkins to ECR

hung5s — Thu, 03 Dec 2020 12:15:23 +0000

This is kind of my personal note so there is no introduction et'al. Long story short, you pull code from GitHub and build a Docker image. You want to push the image to ECR in order to deploy a ECS service. You want to make it quick and simple with just shell commands. How would you make it work?

Make Jenkins work with Dockers

Here assume that Jenkins is installed on an EC2 which is launched from the Amazon Linux 2 image. You will need to add the jenkins user into docker group:

sudo usermod -aG docker jenkins

Once it's done, you can build the image and tag it:

docker build -t <img name>:v_$BUILD_NUMBER --pull=true /var/lib/jenkins/workspace/<jenkins job name> \
&& docker tag <img name>:v_$BUILD_NUMBER <AWS user ID>.dkr.ecr.<region>.amazonaws.com/<ECR repo name>:v_$BUILD_NUMBER

Now comes the headache. If you try to push the image to ECR using docker push command, it will fail because there is no authentication token for jenkins to connect with ECR.

Getting the token and login

In order to get the token, we will need to run the aws ecr get-login-password (AWS CLI v2, if v1 the command is get-login). However, this only work if the AWS CLI has a credential profile for jenkins. Without it, you will get the error: Unable to locate credentials. You can configure credentials by running "aws configure".

The thing is that you don't want to run aws configure within Jenkins's build all the time. It's unnecessary and it will expose your IAM user's secrets.

What I did is SSHing to the EC2 where Jenkins is installed and run the aws configure as jenkins user:

sudo -H -u jenkins aws configure

Give AWS all the key and secret of the IAM user I've created for ECS deployment, I was able to generate the credential profile. This profile is named default and stored for jenkins user.

Given the profile, back to Jenkins job, it's ok to get the auth token and login to docker:

aws ecr get-login-password --region <region> --profile=default | docker login --username AWS --password-stdin <AWS user ID>.dkr.ecr.<region>.amazonaws.com

and push the image to ECR:

docker push <AWS user ID>.dkr.ecr.<region>.amazonaws.com/<ECR repo name>:v_$BUILD_NUMBER

Putting them all together we have one Jenkins build shell command:

docker build -t <img name>:v_$BUILD_NUMBER --pull=true /var/lib/jenkins/workspace/<jenkins job name> \
&& docker tag <img name>:v_$BUILD_NUMBER <AWS user ID>.dkr.ecr.<region>.amazonaws.com/<ECR repo name>:v_$BUILD_NUMBER \
&& aws ecr get-login-password --region <region> --profile=default | docker login --username AWS --password-stdin <AWS user ID>.dkr.ecr.<region>.amazonaws.com \
&& docker push <AWS user ID>.dkr.ecr.<region>.amazonaws.com/<ECR repo name>:v_$BUILD_NUMBER

With this shell command, Jenkins should be able to push the latest Docker image to ECR. The second shell command will deploy the code:

#!/bin/bash

REGION=<region>
SERVICE_NAME=<service name>
CLUSTER=<cluster name>
IMAGE_VERSION="v_"${BUILD_NUMBER}
TASK_FAMILY=<task name>

# Create a new task definition for this build

sed -e "s;%BUILD_NUMBER%;${BUILD_NUMBER};g" ./task-def.json > ${TASK_FAMILY}-v_${BUILD_NUMBER}.json

aws ecs register-task-definition --family ${TASK_FAMILY} --cli-input-json file://${TASK_FAMILY}-v_${BUILD_NUMBER}.json

# Update the service with the new task definition and desired count
REVISION=`aws ecs describe-task-definition --task-definition ${TASK_FAMILY} | egrep "revision" | tr "/" " " | awk '{print $2}' | sed 's/"$//'`
SERVICES=`aws ecs describe-services --services ${SERVICE_NAME} --cluster ${CLUSTER} --region ${REGION} | jq .failures[]`


#Create or update service
if [ "$SERVICES" == "" ]; then
  echo "entered existing service"
  DESIRED_COUNT=`aws ecs describe-services --services ${SERVICE_NAME} --cluster ${CLUSTER} --region ${REGION} | jq .services[].desiredCount`
  if [ ${DESIRED_COUNT} = "0" ]; then
    DESIRED_COUNT="1"
  fi
  aws ecs update-service --cluster ${CLUSTER} --region ${REGION} --service ${SERVICE_NAME} --task-definition ${TASK_FAMILY}:${REVISION} --desired-count ${DESIRED_COUNT} --deployment-configuration maximumPercent=100,minimumHealthyPercent=0
else
  echo "entered new service"
  aws ecs create-service --service-name ${SERVICE_NAME} --desired-count 1 --task-definition ${TASK_FAMILY} --cluster ${CLUSTER} --region ${REGION}
fi

Pre-requirements

An EC2 instance is launched using Amazon Linux 2 image
Jeknins is installed on the instance
An IAM user is created with EC2ContainerRegister policy attached
AWS CLI on the instance is upgraded to version 2
Your source code has the file structure similar to the below:

Source structure

\root
   + src  << source code folder
     - source files
   - Dockerfile
   - task-def.json

DEV Community: hung5s

An easy way to push Docker image from Jenkins to ECR

Make Jenkins work with Dockers

Getting the token and login

Pre-requirements

Source structure