DEV Community

Python-T Point
Python-T Point

Posted on • Originally published at pythontpoint.in

⚙️ Building a Jenkins Docker CI CD pipeline tutorial made easy

#cloud #devops #kubernetes #tutorial

🚀 Counterintuitive truth — Automation

jenkins docker ci cd pipeline tutorial

A Jenkins‑Docker CI/CD pipeline orchestrates builds, tests, and deployments by running each Jenkins job inside a Docker container. Docker provides isolated PID, network, and mount namespaces, so host‑level dependencies never interfere with the build steps. The containerized agents start from a clean filesystem on every run, guaranteeing reproducible outcomes from commit to production.

📑 Table of Contents

  • 🚀 Counterintuitive truth — Automation
  • 🐳 Docker Image — Foundation
  • ⚙️ Jenkins Configuration — Pipeline
  • 🔧 Declarative vs Scripted
  • 🔧 Credentials Handling
  • 🚀 Building and Testing — Automation
  • 🔐 Registry Push
  • 📦 Deploying with Docker — Delivery
  • 🔁 Rolling Update Strategy
  • 🧹 Validation and Cleanup — Maintenance
  • 🟩 Final Thoughts
  • ❓ Frequently Asked Questions
  • How do I store Docker credentials securely in Jenkins?
  • Can I run this pipeline on a Kubernetes cluster instead of a VM?
  • What happens if a test step fails?
  • 📚 References & Further Reading

🐳 Docker Image — Foundation

A Docker image is a layered, read‑only filesystem that serves as the immutable build environment for the pipeline. Each layer is stored as a tar archive and combined at runtime via an overlay filesystem, allowing fast copy‑on‑write for the topmost writable layer.

# Dockerfile
FROM python:3.11-slim
# Install system build tools
RUN apt-get update && \ apt-get install -y -no-install-recommends gcc libffi-dev && \ rm -rf /var/lib/apt/lists/*
# Create a non‑root user
RUN useradd -m -s /bin/bash jenkins
USER jenkins
WORKDIR /app
COPY requirements.txt .
RUN pip install -no-cache-dir -r requirements.txt
COPY . .
Enter fullscreen mode Exit fullscreen mode

What this does:

  • FROM python:3.11-slim — base image with a minimal Python runtime.
  • RUN apt-get … — installs the C‑toolchain required for native Python dependencies.
  • USER jenkins — executes later steps as a non‑root user, reducing privilege exposure.
  • COPY requirements.txt — brings the exact list of Python packages into the image.
  • RUN pip install — creates a reproducible virtual environment inside the image.

Why this, not a plain host‑installed Python? The image locks the toolchain and library versions in a single artifact, eliminating “works on my machine” discrepancies across agents.

Key point: The Dockerfile defines a self‑contained build sandbox, so the Jenkins master never needs to manage language runtimes.

⚙️ Jenkins Configuration — Pipeline

A Jenkinsfile declaratively describes the CI/CD workflow that runs inside Docker containers, ensuring each stage executes in the same isolated environment.

# Jenkinsfile
pipeline { agent { docker { // Use the image built in the previous step image 'myorg/ci-image:latest' args '-u 1000:1000' // run as the jenkins user } } environment { REGISTRY = 'registry.example.com' IMAGE_NAME = 'myapp' } stages { stage('Checkout') { steps { checkout scm } } stage('Build') { steps { sh 'python -m pip install -upgrade pip' sh 'pip install -r requirements.txt' sh 'python -m compileall .' } } stage('Test') { steps { sh 'pytest -q tests/' } } stage('Publish') { steps { script { def tag = "${env.BUILD_NUMBER}" sh "docker build -t ${REGISTRY}/${IMAGE_NAME}:${tag} ." sh "docker push ${REGISTRY}/${IMAGE_NAME}:${tag}" } } } }
}
Enter fullscreen mode Exit fullscreen mode

What this does:

  • agent { docker { … } } — instructs Jenkins to start a container from the specified image for the entire pipeline.
  • environment — defines reusable variables for the registry and image name.
  • stage( 'Build') — installs dependencies and compiles byte‑code inside the container.
  • stage( 'Test') — runs the test suite; a failure aborts the pipeline.
  • stage( 'Publish') — builds a new Docker image that includes the application code and pushes it to a private registry.

Why this, not a freestyle job with ad‑hoc shell steps? Declarative pipelines are validated before execution, provide a fixed stage order, and enable Jenkins to render a visual flow. Docker isolation guarantees that each stage runs against the same dependencies. (Also read: 🚀 GitLab CI vs Jenkins for scaling startups — which one should you use?)

🔧 Declarative vs Scripted

Declarative pipelines use a static structure that Jenkins validates prior to execution. Scripted pipelines are full Groovy scripts offering greater flexibility at the cost of reduced safety. For most CI/CD workflows, the declarative form reduces configuration errors.

🔧 Credentials Handling

Jenkins stores registry credentials as Secret Text or Username/Password entries. In the pipeline they are accessed via withCredentials, which masks the values in logs and injects them only into the steps that need them.

Key point: The Jenkinsfile couples Docker execution with pipeline stages, turning the Docker image into a reproducible agent for every build step.

🚀 Building and Testing — Automation

A Jenkins agent container runs docker build and pytest as separate shell commands. Each docker build starts from the base image, adds layers for dependencies, and produces a fresh image, ensuring no residual state from prior builds can affect test results.

$ docker build -t myorg/ci-image:latest .
Sending build context to Docker daemon 12.34kB
Step 1/7: FROM python:3.11-slim --> 5e7a9c9c6c1b
Step 2/7: RUN apt-get update && apt-get install -y -no-install-recommends gcc libffi-dev && rm -rf /var/lib/apt/lists/* --> Using cache --> 1a2b3c4d5e6f
...
Successfully built 1a2b3c4d5e6f
Successfully tagged myorg/ci-image:latest



$ pytest -q tests/
..F..
=================================== FAILURES ====================================
test_example.py::test_failure ---------------------------------- Captured stdout call -------------------
Expected 1, got 0
============================== 1 failed, 4 passed in 0.12s ==============================
Enter fullscreen mode Exit fullscreen mode

Why this, not a direct docker run of the test suite? Building a fresh image each run guarantees that test failures are not caused by leftover files or altered library versions from previous builds.

🔐 Registry Push

After a successful test run, the pipeline pushes the newly built image to a private registry.

$ docker push registry.example.com/myapp:42
The push refers to repository [registry.example.com/myapp]
42: digest: sha256:abcd1234ef567890... size: 1578
Enter fullscreen mode Exit fullscreen mode

According to the Docker documentation, the digest uniquely identifies the image contents, enabling immutable deployments. (More onPythonTPoint tutorials)

Key point: Building and testing inside Docker guarantees a clean state, and the push step records an immutable image for later deployment.

📦 Deploying with Docker — Delivery

A Docker Compose file defines the runtime environment for the application. The ${BUILD_NUMBER} tag ensures the exact image produced by CI is used in production.

# docker-compose.yml
version: '3.8'
services: web: image: registry.example.com/myapp:${BUILD_NUMBER} restart: always ports: - "8080:80" environment: - ENV=production depends_on: - db db: image: postgres:15-alpine restart: always environment: POSTGRES_USER: appuser POSTGRES_PASSWORD: securepassword POSTGRES_DB: appdb volumes: - db_data:/var/lib/postgresql/data
volumes: db_data:
Enter fullscreen mode Exit fullscreen mode

What this does:

  • image: registry.example.com/myapp:${BUILD_NUMBER} — pulls the exact image built by the CI stage.
  • restart: always — guarantees container restart after a crash.
  • ports — maps host port 8080 to container port 80.
  • depends_on — ensures the database starts before the web service.

Why this, not a series of docker run commands? Compose encodes multi‑service relationships, volume persistence, and network configuration in a single, version‑controlled file.

🔁 Rolling Update Strategy

When a new image is pushed, the deployment can be refreshed with zero downtime using Docker Compose’s --scale and --no-deps flags.

$ docker-compose up -d -scale web=2
Creating network "project_default" ...
Creating container web_1 ... done
Creating container web_2 ... done
Enter fullscreen mode Exit fullscreen mode

This command launches a second instance before terminating the original, allowing existing connections to finish gracefully.

Key point: Docker Compose provides a concise, version‑controlled description of the production stack, and the CI pipeline can trigger updates automatically.

🧹 Validation and Cleanup — Maintenance

Post‑deployment validation runs a lightweight health check inside the running container; cleanup removes dangling images to keep the host tidy.

$ docker exec web curl -s http://localhost/health
{"status":"ok"}



$ docker image prune -f
Deleted Images:
untagged: myorg/ci-image@sha256:abcd...
Deleted: sha256:efgh...
Enter fullscreen mode Exit fullscreen mode

Why this, not a manual inspection? Automated health checks catch runtime regressions immediately, and docker image prune -f removes unreferenced layers, preventing disk exhaustion on Jenkins agents.

Automating every step—from source checkout to production rollout—eliminates manual drift and guarantees repeatable builds.

🟩 Final Thoughts

The pipeline described above shows how Docker can serve both as a build sandbox and as a deployment target. By anchoring each stage to a concrete image, developers gain confidence that the artifact tested in CI is identical to what runs in production. The approach scales horizontally: additional services can be added to the Compose file, and new Jenkins stages can be introduced without altering the underlying Docker image.

For teams that already use Jenkins, integrating Docker does not require a wholesale migration; it merely extends existing job definitions with container‑based agents, preserving familiar workflows while gaining reproducibility and isolation.

❓ Frequently Asked Questions

How do I store Docker credentials securely in Jenkins?

Use Jenkins Credentials Manager to create a “Username with password” entry for the registry, then reference it in the pipeline with withCredentials so the secret never appears in logs.

Can I run this pipeline on a Kubernetes cluster instead of a VM?

Yes. Replace the agent { docker { … } } block with agent { kubernetes { yamlFile 'pod.yaml' } } and supply a pod template that pulls the same Docker image.

What happens if a test step fails?

The declarative pipeline aborts the build, marks the job as failed, and skips subsequent stages, ensuring that no broken image is pushed to the registry.

💡 Want to practise this hands-on? DigitalOcean gives new accounts $200 free credit for 60 days — enough to spin up a full Linux/Docker/Kubernetes environment at no cost.

📚 Recommended reading: Best DevOps & cloud books on Amazon — from Linux fundamentals to Kubernetes in production, curated for working engineers.

📚 References & Further Reading

  • Official Jenkins documentation — pipeline syntax and Docker integration: jenkins.io
  • Docker reference documentation — building images and registry operations: docs.docker.com
  • Kubernetes official docs — using Jenkins agents on Kubernetes: kubernetes.io

Top comments (0)