🚀 Building a Real-World CI/CD Pipeline with Jenkins, Docker, and Docker Compose

Vivek Singh — Sat, 24 Jan 2026 21:46:50 +0000

From broken builds to a fully automated deployment pipeline.

As part of my DevOps learning journey, I wanted to build something practical, realistic, and close to production workflows — not just follow tutorials.

So I built a two-tier web application and automated its entire build, test, and deployment lifecycle using Jenkins, Docker, and Docker Compose.

This article walks through:
• The architecture I built
• The CI/CD pipeline design
• Real issues I faced (and fixed)
• Key DevOps learnings

🧱 Project Overview

The goal was simple but important:

Automate everything from code commit to deployment — with no manual steps.

Tech Stack
• Frontend / API: Flask (Python)
• Database: MySQL
• CI/CD: Jenkins (Pipeline as Code)
• Containerization: Docker
• Orchestration: Docker Compose
• Version Control: GitHub

🏗 Architecture Overview

The application follows a classic two-tier architecture:
• Web tier: Flask application
• Data tier: MySQL database
• Both services run in separate Docker containers
• Managed via Docker Compose

GitHub
  ↓
Jenkins Pipeline
  ↓
Docker Build
  ↓
Tests
  ↓
Docker Compose Deploy

🐳 Docker & Docker Compose Setup

Docker Compose is used to define and run multiple containers together.

Services:
• db: MySQL 8 container
• web: Flask application container

Key highlights:
• Database initialization using init.sql
• Environment variables for DB connectivity
• Service dependency (web depends on db)

📸 Image: docker-compose.yml (services definition)

🐍 Flask Application

The Flask app exposes simple endpoints:
• / → health check
• /users → fetches data from MySQL

The database connection is handled using environment variables, making the app environment-agnostic.

📸 Image: db.py (MySQL connection using env variables)

📸 Image: app.py (Flask routes and app runner)

This design ensures:
• No hardcoded credentials
• Easy containerization
• Smooth CI/CD execution

🐋 Dockerfile (Application Image)

The Flask app is containerized using a simple, optimized Dockerfile:
• Lightweight base image (python:3.9-slim)
• Dependency installation via requirements.txt
• Exposes port 5000
• Runs the app using CMD

📸 Image: Dockerfile

⚙️ Jenkins CI/CD Pipeline

This is where the real DevOps work happens.

The pipeline is written using Jenkins Declarative Pipeline and stored as a Jenkinsfile in the repository.

Pipeline Stages:
1. Checkout – Pull source code from GitHub
2. Build Image – Build Docker images using Docker Compose
3. Run Tests – Start DB container and validate application
4. Deploy – Bring up all services using Docker Compose

📸 Image: Jenkinsfile (pipeline stages)

📊 Jenkins Stage View (Pipeline Execution)

The Jenkins Stage View clearly shows:
• Which stage failed
• How long each stage took
• When the pipeline finally turned green

This helped massively during debugging.

📸 Image: Jenkins Stage View (failed → successful build)

🧨 Real Issues I Faced (and Fixed)

This project was not smooth, and that’s where the learning came from.

🔴 Issue 1: Jenkins couldn’t find Jenkinsfile
• Root cause: Case sensitivity
• jenkinsfile ≠ Jenkinsfile

✅ Fixed by forcing correct filename in Git

🔴 Issue 2: docker: not found in Jenkins

• Jenkins was running inside a container
• Docker CLI wasn’t available inside Jenkins

✅ Installed Docker CLI inside Jenkins container

🔴 Issue 3: Docker Compose v1 vs v2 mismatch

• Jenkins environment only had docker-compose
• Pipeline was using docker compose

✅ Aligned Jenkinsfile with available tooling

🔴 Issue 4: Permission denied on /var/run/docker.sock

• Jenkins user couldn’t talk to Docker daemon
• Linux socket permission issue

✅ Fixed by adjusting Docker socket permissions (for local CI)

✅ Final Result

After resolving all issues:
• ✔ Fully automated CI/CD pipeline
• ✔ Build → Test → Deploy without manual steps
• ✔ Dockerized multi-container application
• ✔ Jenkins pipeline running clean and green

📸 Image: Successful Jenkins pipeline run

📚 Key Learnings

This project taught me far more than any tutorial:
• DevOps is more about debugging systems than writing YAML
• Linux fundamentals matter a lot
• CI/CD environments behave differently than local machines
• Logs are your best debugging tool
• Small permission issues can break entire pipelines
• Automation only works when every layer is understood

🔗 Source Code

GitHub Repository:
👉 https://github.com/developedbyviv/Two-tier-Architecture-Application

I Passed the GitHub Foundations Certification: My Honest Experience

Vivek Singh — Fri, 29 Nov 2024 17:56:26 +0000

Recently, I had the opportunity to take the GitHub Foundations Certification exam, thanks to a free voucher from Wipro. After working with Git and GitHub for quite some time, I decided to put my knowledge to the test. Here's my honest take on the whole experience—from preparation to the actual exam.

📝 Exam Details

Format: 75 multiple-choice questions (60 scored + 15 pretest questions)

Duration: 120 minutes

Preparation Materials:

GitHub Certification Guide

FreeCodeCamp GitHub Course by Andrew Brown

Free Voucher Code: I used the code USUUMWDS during checkout on the GitHub exam registration site. (Not sure if it’s still valid, but worth trying!)

💡 What I Liked About the Exam

Practicality: The questions focus on real-life scenarios, making it an excellent way to refresh your knowledge of Git and GitHub.
Bite-Sized Insights: The exam covers a wide range of topics, ensuring you’re well-versed in GitHub workflows, Git commands, and collaboration practices.
Quick to Complete: I finished the exam in just 40 minutes, leaving me surprised at how manageable it was overall.

😅 What I Struggled With

Details Overload: Some questions required memorizing extensive details, like the 25 events that can trigger GitHub workflows. Realistically, this level of detail is hard to recall, even for frequent GitHub users.
Pretest Questions: It was unclear which questions were pretests and which were scored, which added an extra layer of uncertainty.

My Preparation Strategy

Certification Guide: I studied the GitHub Certification Guide, which provided a solid foundation of the key topics.
FreeCodeCamp Course: Andrew Brown’s YouTube course was a lifesaver, especially for visualizing concepts like pull requests, branching strategies, and Git commands.
Practice in Real Projects: The best preparation was applying these skills in actual projects. Real-world experience made it easier to answer scenario-based questions.

Final Thoughts

Passing this certification felt like a rewarding milestone. Even though I skipped a few questions and felt unsure about some answers, I managed to succeed! This certification isn't just about memorizing commands—it’s about understanding workflows, collaboration, and the power of GitHub in software development.

If you’re planning to take this exam, I’d recommend focusing on practical knowledge and using the resources mentioned above. And if you don’t pass on the first try, don’t stress—every attempt will make you better!

Have you taken the GitHub Foundations Certification exam? Share your experience in the comments below—I’d love to hear your thoughts!

Tags: #GitHub #Certification #Git #Career #DevOps

DEV Community: Vivek Singh