Day 24: Complete Jenkins CI/CD Project

step-by-step easy to follow breakdown of the entire project.
This is assuming you already have Jenkins up, and Docker/Compose installed on the Jenkins machine (controller or agent) where the job will run.

Prerequisite (do these once)

1. Install required Jenkins plugins

• Manage Jenkins → Plugins → Available:

  • Git, GitHub, GitHub Branch Source
  • Pipeline (and “Pipeline: GitHub”)
  • Docker Pipeline (optional but handy)

• Install + Restart if asked.

1. Give Jenkins permission to use Docker

# On the Jenkins host:
sudo usermod -aG docker jenkins
sudo systemctl restart jenkins
# Log out & back in (or restart the agent) if you’ll run builds on an agent user.

Quick check: create a temporary freestyle job with one “Execute shell” step: docker version. Build should succeed.

1. Add GitHub credentials in Jenkins

• GitHub → Settings → Developer settings → Personal access tokens

  • Create token with minimal scopes: repo (and admin:repo_hook if you want Jenkins to manage hooks).

• Jenkins → Manage Jenkins → Credentials → System → Global → Add Credentials

  • Kind: Secret text
  • Secret: <your GitHub token>
  • ID: github-pat (for easy reference)

1. (Optional but recommended) Configure GitHub server in Jenkins

• Manage Jenkins → System → GitHub section → “Add GitHub Server”
• Credentials: github-pat → Test connection → should be green.

Task 1 — Fork + GitHub Integration + Webhook

A) Fork the repo

• In GitHub: open the target repo → Fork → choose your account.
• Copy your fork’s HTTPS clone URL (e.g. https://github.com/<you>/<repo>.git).

B) Create the Jenkins job (Freestyle)

1. Jenkins → New Item → Name: my-dockerized-app-freestyle → Freestyle project → OK.
2. Source Code Management → Git:

• Repository URL: your fork URL
• Credentials: Add (use your GitHub PAT if you didn’t add globally) or select existing

• Branches to build: */main (or your default)

  1. Build Triggers (choose one):

• GitHub hook trigger for GITScm polling (preferred with webhook)

• Or Poll SCM → Schedule: H/5 * * * * (fallback if you can’t expose Jenkins)

  1. (You’ll add the build steps in Task 2.)

Save.

C) Add the GitHub Webhook

• Your Jenkins must be reachable from GitHub (public URL/IP). If not, use a tunnel like ngrok/cloudflared.

• GitHub → Your fork → Settings → Webhooks → Add webhook

  • Payload URL: http://<your-jenkins-host>:8080/github-webhook/ (trailing slash is important)
  • Content type: application/json
  • Secret: optional (use only if you configure the same in Jenkins)
  • Events: Just the push event (and PRs if you want)
  • Add webhook → Recent deliveries → should show 200 on test ping.

Task 2 — Run the app with Docker Compose from Jenkins

If your repo doesn’t have a Compose file yet, add one at the repo root. Here’s a starter template you can adapt:

docker-compose.yml (example: Python web + Postgres)

version: "3.9"
services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: myapp-web
    ports:
      - "8000:8000"                # change if your app uses a different port
    env_file:
      - .env                       # optional: put app env here
    depends_on:
      - db
    restart: unless-stopped

  db:
    image: postgres:15-alpine
    container_name: myapp-db
    environment:
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: myapp
      POSTGRES_DB: myapp
    volumes:
      - db_data:/var/lib/postgresql/data
    restart: unless-stopped

volumes:
  db_data:

Dockerfile (Python/Flask—adjust to your stack)

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
# Example: gunicorn on 8000 (adjust command/port to your app)
EXPOSE 8000
CMD ["gunicorn", "-b", "0.0.0.0:8000", "app:app"]

If your app is Node/Java/Go/etc., swap the Dockerfile and ports accordingly.

Add Jenkins build steps (Freestyle → “Execute shell”)

Open your job → Configure → Build → Add build step → Execute shell
Paste this script (it works with Docker Compose v2: docker compose). If your host still uses the old v1 binary, replace docker compose with docker-compose.

set -euxo pipefail

# Ensure we’re in the repo root where docker-compose.yml lives
cd "$WORKSPACE"

# (Optional) build & run tests before deploy
# docker compose run --rm web pytest

# Stop/remove any existing stack for this project
docker compose -p myapp down -v --remove-orphans || true

# Build images and start in detached mode
docker compose -p myapp up -d --build

# (Optional) wait for health/port, simple check:
# timeout 60 bash -c 'until nc -z localhost 8000; do sleep 2; done'

echo "Containers:"
docker ps

echo "Compose services:"
docker compose -p myapp ps

Click Save.

(Optional) Add a cleanup step post-build

Add another Execute shell (either as a separate job or a post-build action in another job) if you want teardown on demand:

set -euxo pipefail
cd "$WORKSPACE"
docker compose -p myapp down -v --remove-orphans

Verify the CI/CD loop

1. Manual Build

• Open the job → Build Now → watch Console Output.
• You should see the git checkout, docker build, and compose up logs.

1. App check

• On the Jenkins host: docker ps → confirm myapp-web and myapp-db.
• Open the app in browser: http://<jenkins-host>:8000 (or your mapped port).

1. Webhook test

• Make a small commit to your fork (e.g., README change) → push.
• Jenkins job should trigger automatically and redeploy.

These are some of the errors you might encounter and how to fix them.

• Jenkins can’t run Docker:
  Add Jenkins to the docker group (usermod -aG docker jenkins), restart Jenkins, and re-run.

• Webhook 403/Timeout:
  Ensure http://<jenkins>:8080/github-webhook/ is reachable from GitHub (public URL/tunnel). Keep the trailing slash.

• Compose not found:
  Use docker compose version. If missing, install Docker Compose v2 or use docker-compose.

• Port already in use:
  Change the published port in docker-compose.yml, or stop the app using it.

• Permissions on workspace files:
  If your Jenkins runs as jenkins user, ensure the workspace has permissions for it.