Building a CI/CD Pipeline with Docker, Jenkins, and AWS EC2

SWAPNIL UPPIN — Mon, 23 Mar 2026 02:57:25 +0000

Introduction:

In modern DevOps workflows, Continuous Integration and Continuous Deployment (CI/CD) pipelines help teams deliver software faster and more reliably.

In this project, I built a complete CI/CD pipeline that automatically deploys a Dockerized Python application to an AWS EC2 instance using Jenkins.

The goal was to simulate a real-world deployment pipeline where code changes trigger automatic builds, tests, and deployments.

Architecture Overview:

The pipeline works as follows:

Developer → GitHub → Jenkins Pipeline → Docker Build → DockerHub → AWS EC2 → Nginx Reverse Proxy → Application Containers

Application Setup:

We will be using a simple Python Flask application that returns the message 'Hello from CI/CD auto-deploy!', along with a requirements.txt file for managing dependencies and a test_app.py file for running test cases.

app.py:

from flask import Flask, jsonify
import socket
import os

app = Flask(__name__)

@app.get("/health")
def health():
    return jsonify(status="ok")

@app.get("/")
def index():
    return jsonify(
        message="Hello from CI/CD auto-deploy!",
        hostname=socket.gethostname(),
        version=os.getenv("APP_VERSION", "dev"),

    )

if __name__ == "__main__":
    app.run(host="0.0.0.0",port=5000)

requirements.txt:

flask==3.0.3
gunicorn==22.0.0
pytest==8.2.2

test_app.py:

import sys
import os

# Add the app directory to Python path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

from app import app

def test_health():
    client = app.test_client()
    r = client.get("/health")
    assert r.status_code == 200
    assert r.json["status"] == "ok"

Dockerizing the Application:

Create a Dockerfile to containerize the application using python:3.12-slim as the base image. The application listens on port 5000, and the CMD instruction starts the Gunicorn server with 2 workers, binding it to port 5000 and serving the app object from the app module.

Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY app/requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
COPY app /app
ENV PYTHONUNBUFFERED=1
EXPOSE 5000
CMD ["gunicorn", "-w", "2", "-b", "0.0.0.0:5000", "app:app"]

Reverse Proxy Configuration:

Below is the configuration for Nginx which acts reverse proxy and Routes user traffic to the correct container. it routes the incoming traffic to either the Blue (port 8081) or Green (port 8082) container, enabling zero-downtime deployments by simply switching the proxy_pass target between the two live environments.

nginx.conf:

events{}

http{
    server{
        listen 80;

        location /
        {
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            # default (blue). deploy script will switch to 127.0.0.1:8082 for green
            proxy_pass http://127.0.0.1:8081;
        }
    }
}

scripts:

ec2_bootstrap.sh:

This is a setup script that runs on EC2 instance to install Docker and Docker Compose, and configure the necessary permissions.

#!/usr/bin/env bash
set -euo pipefail

sudo apt-get update -y
sudo apt-get install -y ca-certificates curl gnupg lsb-release

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update -y
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

sudo usermod -aG docker "$USER"

deploy_bluegreen.sh:

The deploy_bluegreen.sh script orchestrates the blue/green switch on the EC2 instance. It begins by pulling the latest Docker image and spinning up both the blue and green containers via Docker Compose. It then inspects the active Nginx proxy_pass directive to determine which slot — port 8081 (blue) or 8082 (green) — is currently serving live traffic, and targets the idle slot for the new deployment. A health check loop polls the new slot's /health endpoint up to 10 times, with a 2-second delay between attempts, before proceeding. Once the new slot is confirmed healthy, the script updates the Nginx config with a sed in-place swap and reloads Nginx — making the cutover instant and zero-downtime — then logs the active port to confirm a successful deployment.

#!/usr/bin/env bash
set -euo pipefail

# Expected env vars:
# IMAGE_FULL  (e.g. docker.io/<user>/cicd-ec2-autodeploy:build-123)
# APP_VERSION (e.g. build-123)

APP_DIR="/opt/cicd-ec2-deploy"
NGINX_CONF="${APP_DIR}/docker/nginx.conf"

mkdir -p "${APP_DIR}"
cd "${APP_DIR}"

# If first time: put repo runtime files here
# You can git clone on EC2 OR Jenkins will scp needed files. We keep it simple:
# We assume docker-compose.ec2.yml and docker/nginx.conf exist in APP_DIR.

if [[ ! -f docker-compose.ec2.yml ]]; then
    echo "ERROR: docker-compose.ec2.yml not found in ${APP_DIR}"
    exit 1
fi

echo "Pulling image: ${IMAGE_FULL}"
sudo docker pull "${IMAGE_FULL}"

# Pass vars explicitly to docker compose
sudo IMAGE_FULL="${IMAGE_FULL}" APP_VERSION="${APP_VERSION}" \
    docker compose -f docker-compose.ec2.yml up -d

# Determine currently active upstream in nginx.conf
ACTIVE_UPSTREAM="$(grep -E 'proxy_pass http://127\.0\.0\.1:808[12];' -o "${NGINX_CONF}" | tail -n 1 || true)"

if [[ "${ACTIVE_UPSTREAM} == *"8081"*" ]]; then
   NEW_PORT="8082"
   NEW_SLOT="green"
else
   NEW_PORT="8081"
   NEW_SLOT="blue"
fi

echo "Switching traffic to ${NEW_SLOT} (port ${NEW_PORT})"

# Basic health check loop in new slot
for i in {1..10}; do
  if curl -fss "http://127.0.0.1:${NEW_PORT}/health" >/dev/null; then
    echo "New slot healthy"
    break
  fi
  echo "waiting for new slot health... ${i}/10"
  sleep 2
  if [[ $i -eq 20 ]]; then
    echo "ERROR: New slot did not become healthy"
    exit 1
  fi
done

# Swap nginx upstream and reload
sudo sed -i "s#proxy_pass http://127.0.0.1:808[12];#proxy_pass http://127.0.0.1:${NEW_PORT};#g" "${NGINX_CONF}"
sudo docker exec nginx nginx -s reload

echo "Deploy complete. Live traffic now on port ${NEW_PORT} via Nginx:80"

Docker Compose Configuration:

The Docker Compose file defines three services: Nginx acts as a reverse proxy, listening on port 80 and routing traffic using a custom nginx.conf mounted as a read-only volume; app_blue runs the application container on host port 8081; and app_green runs an identical instance on host port 8082. Both the blue and green containers use the same versioned image — injected via the IMAGE_FULL environment variable — and receive an APP_VERSION variable at runtime, making it easy to verify which build is actively serving traffic at any given time.

docker-compose.ec2.yml:

services:
  nginx:
    image: nginx:1.27-alpine
    container_name: nginx
    ports:
      - "80:80"
    volumes:
      - ./docker/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - app_blue
      - app_green
    restart: always

  # Blue app (port 8081 on host)
  app_blue:
    image: ${IMAGE_FULL}
    container_name: app_blue
    environment:
      - APP_VERSION=${APP_VERSION}
    ports:
      - "8081:5000"
    restart: always

  # Green app(port 8082 on host)
  app_green:
    image: ${IMAGE_FULL}
    container_name: app_green
    environment:
      - APP_VERSION=${APP_VERSION}
    ports:
      - "8082:5000"
    restart: always

CI/CD Pipeline:

This pipeline automates the full CI/CD lifecycle and is divided into five stages: Checkout SCM pulls the latest code from source control; Test sets up a Python virtual environment and runs pytest against the application; Build Docker Image builds a versioned Docker image tagged with the Jenkins build number; Docker Push authenticates with DockerHub and pushes the built image to the registry; and finally, Deploy to EC2 (Blue/Green) securely connects to the EC2 instance over SSH, transfers the necessary configuration files — including the Docker Compose file, Nginx config, and blue/green deployment script — and executes the deployment. A post block ensures Docker is logged out and any sensitive files are cleaned up after every run, regardless of the build outcome.

Jenkinsfile:

pipeline{
    agent any
    environment{
      APP_NAME = 'cicd-ec2-deploy'
      DOCKERHUB_REPO = "${env.DOCKERHUB_REPO}"   // set in Jenkins job config
      IMAGE_TAG      = "build-${env.BUILD_NUMBER}"
      IMAGE_FULL     = "${DOCKERHUB_REPO}:${IMAGE_TAG}"
      EC2_HOST       = "${env.EC2_HOST}"         // set in Jenkins job config
      EC2_USER       = "${env.EC2_USER}"         // set in Jenkins job config (ubuntu)
    }

    stages{
        stage("checkout scm"){
            steps{
                checkout scm
            }
        }

        stage("Test"){
          steps {
                bat '''
                python -m venv .venv
                call .venv/Scripts/activate.bat
                pip install -r cicd-ec2-autodeploy/app/requirements.txt --quiet
                python -m pytest cicd-ec2-autodeploy/app -q
                '''
          }
        }

        stage("Build Docker image"){
            steps{
                bat '''
                docker build -t %IMAGE_FULL% -f cicd-ec2-autodeploy/docker/Dockerfile .
                '''

            }
        } 

        stage("docker push image"){
            steps{

                withCredentials([usernamePassword(credentialsId: 'dockerhub', usernameVariable: 'DOCKER_USER', passwordVariable: 'DOCKER_PASS')]){
                  sh '''
                    echo "${DOCKER_PASS}" | docker login -u "${DOCKER_USER}" --password-stdin
                    docker push ${IMAGE_FULL}
                  '''                  
                  }

            }
        }

        stage("Deploy to EC2 (Blue/Green)") {
    steps {
        withCredentials([file(credentialsId: 'ec2-ssh-key-file', variable: 'SSH_KEY_FILE')]) {

            // Windows Jenkins runs bat with %VAR% for local ssh/scp commands
            bat """
            icacls %SSH_KEY_FILE% /inheritance:r
            icacls %SSH_KEY_FILE% /grant:r "%USERNAME%:R"
            icacls %SSH_KEY_FILE% /remove "BUILTIN\\Users"
            icacls %SSH_KEY_FILE% /remove "Everyone"
            icacls %SSH_KEY_FILE% /grant:r "SYSTEM:F"
            icacls %SSH_KEY_FILE% /grant:r "BUILTIN\\Administrators:F"

            ssh -o StrictHostKeyChecking=no -i %SSH_KEY_FILE% ^
                %EC2_USER%@%EC2_HOST% ^
                "sudo mkdir -p /opt/%APP_NAME%/docker /opt/%APP_NAME%/scripts"

            scp -o StrictHostKeyChecking=no -i %SSH_KEY_FILE% ^
                cicd-ec2-autodeploy/docker-compose.ec2.yml ^
                %EC2_USER%@%EC2_HOST%:/tmp/docker-compose.ec2.yml

            scp -o StrictHostKeyChecking=no -i %SSH_KEY_FILE% ^
                cicd-ec2-autodeploy/docker/nginx.conf ^
                %EC2_USER%@%EC2_HOST%:/tmp/nginx.conf

            scp -o StrictHostKeyChecking=no -i %SSH_KEY_FILE% ^
                cicd-ec2-autodeploy/scripts/deploy_bluegreen.sh ^
                %EC2_USER%@%EC2_HOST%:/tmp/deploy_bluegreen.sh

            ssh -o StrictHostKeyChecking=no -i %SSH_KEY_FILE% ^
                %EC2_USER%@%EC2_HOST% ^
                "set -e && sudo apt-get install -y dos2unix -qq && sudo mv /tmp/docker-compose.ec2.yml /opt/%APP_NAME%/docker-compose.ec2.yml && sudo mv /tmp/nginx.conf /opt/%APP_NAME%/docker/nginx.conf && sudo mv /tmp/deploy_bluegreen.sh /opt/%APP_NAME%/scripts/deploy_bluegreen.sh && sudo dos2unix /opt/%APP_NAME%/scripts/deploy_bluegreen.sh && sudo chmod +x /opt/%APP_NAME%/scripts/deploy_bluegreen.sh && sudo IMAGE_FULL=%IMAGE_FULL% APP_VERSION=%IMAGE_TAG% /opt/%APP_NAME%/scripts/deploy_bluegreen.sh"
            """               
        }
    }
}               
    }

    post{
        always{
            sh "docker logout || true"
            sh "rm -rf .env || true"
        }
    }
}

step by step execution:

Step 1 — Create an EC2 Instance:

Create an EC2 instance in the AWS Console using the following configuration:

Instance Type: t2.micro (Free Tier eligible)
AMI: Ubuntu 22.04 (Free Tier eligible)
Security Group Inbound Ports: 22, 80, 8081, and 8082
Storage: 8 GB gp2 EBS volume

Step 2 — Create a Jenkins Pipeline Job:

In Jenkins, create a new job and select Pipeline as the project type.

In the Pipeline section, select Git as the SCM and enter your repository URL.

Under Branches to Build, set the Branch Specifier to */main and set the Script Path to cicd-ec2-autodeploy/Jenkinsfile.

Navigate to Configure → Build Environment → This project is parameterized and add the following as string parameters, matching the environment variables referenced in the Jenkinsfile:

Parameter         Description
DOCKERHUB_REPO    Your DockerHub repository 
EC2_HOST          Public IP or DNS of your EC2 instance
EC2_USER          SSH login user (e.g. ubuntu)

Step 3 — Generate SSH Keys on EC2 and Add to Jenkins:

SSH into your EC2 instance and generate an RSA key pair:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -N "" -m PEM

This produces two files — id_rsa (private key) and id_rsa.pub (public key). Register the public key as an authorized key on the instance:

mkdir -p ~/.ssh
cat id_rsa.pub >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 600 ~/.ssh/authorized_keys

On your Windows machine, open Notepad and paste the private key in the following format:

-----BEGIN RSA PRIVATE KEY-----
<your key content>
-----END RSA PRIVATE KEY-----

Save the file as id_rsa.pem and ensure the following:

No extra blank lines at the top or bottom
No extra spaces anywhere in the file
File encoding is UTF-8 (not UTF-8 BOM)

Now upload the key to Jenkins by navigating to Manage Jenkins → Credentials → System → Global Credentials and adding a new credential with the following settings:

Field       Value 
Kind        Secret file        
ID          ec2-ssh-key-file
File        Upload id_rsa.pem

Step 4 — Bootstrap the EC2 Instance:

Clone the repository onto your EC2 instance, or manually copy and paste the ec2-bootstrap.sh script into it, then run the script to install Docker and Docker Compose and configure the necessary permissions.
Once the installation is complete, verify that Docker is installed and running:

systemctl status docker

Step 5 — Run the Jenkins Job

Trigger the Jenkins job by selecting Build with Parameters and allow the job to run through all the stages defined in the Jenkinsfile. Monitor the Console Output to track the progress of each stage as it executes.
If any stage encounters an error, the job will mark the build as Failed and the console output will indicate exactly where the failure occurred. If all stages complete without issues, the job will mark the build as Success.

Deployment Result:

Once the Jenkins job completes successfully, the application will be deployed on the EC2 instance as two running Docker containers — app_blue on port 8081 and app_green on port 8082. Nginx listens on port 80 and routes all live traffic to one active slot at a time, switching to the idle slot on each new deployment. You can access the application either through Nginx on port 80, or directly on port 8081 or 8082 via the EC2 instance's public IP in your browser, as shown below:

Clean up:

After the completion of the project delete your EC2 instance. The purpose of deleting the EC2 instance is to release the compute resources and avoid unnecessary costs associated with running and maintaining the instance

In conclusion,This project demonstrates how a fully automated CI/CD pipeline can be built from scratch using Jenkins, Docker, and a simple Bash script — without relying on any cloud-native deployment services. Every push to the repository triggers a clean build, test, containerization, and a zero-downtime blue/green deployment to a live EC2 instance, all within a single pipeline run.

About the Repo

The full source code for this project is available on GitHub, including the Jenkinsfile, Dockerfile, Docker Compose configuration, Nginx config, blue/green deployment script, and the EC2 bootstrap script. The repository is structured to be straightforward to clone and adapt to your own use case — swap in your DockerHub credentials, point it at your EC2 instance, and the pipeline handles the rest.

🔗 GitHub: EC2-CICD-DEPLOY

Thank you for reading this article, and we hope it has provided valuable insights into deploying applications on cloud platforms.

DEV Community: SWAPNIL UPPIN

Building a CI/CD Pipeline with Docker, Jenkins, and AWS EC2