DEV Community

Aisalkyn Aidarova
Aisalkyn Aidarova

Posted on

CI/CD Pipeline Lecture

If you look at almost every DevOps job description, you will see requirements like:

  • Experience with CI/CD
  • Jenkins
  • GitHub Actions
  • GitLab CI
  • Docker
  • Kubernetes
  • AWS
  • Terraform

Why?

Because a DevOps engineer's main responsibility is to automate how software moves from a developer's computer to production where customers use it.

Without CI/CD, companies would deploy software manually. That process is slow, error-prone, and expensive.

With CI/CD, the entire deployment process can happen automatically in just a few minutes.

What Does CI/CD Mean?

CI/CD consists of two parts.

CI – Continuous Integration

Continuous Integration means developers frequently merge their code into a shared Git repository.

Every time code is pushed, the system automatically:

  • Downloads the latest code
  • Builds the application
  • Runs tests
  • Checks code quality
  • Looks for security issues

The goal is to detect problems immediately instead of finding them weeks later.

CD – Continuous Delivery / Continuous Deployment

After CI finishes successfully, CD takes over.

CD automatically prepares the application for deployment.

Depending on the company:

Continuous Delivery:

  • Deployment waits for manual approval.

Continuous Deployment:

  • Every successful build is automatically deployed to production.

Why Companies Use CI/CD

Imagine Amazon has 5,000 developers.

Each developer pushes code several times every day.

Without CI/CD:

Someone would need to:

  • Download code manually
  • Build it manually
  • Test it manually
  • Create Docker images manually
  • Upload images manually
  • Deploy manually
  • Restart servers manually

Imagine doing that hundreds of times every day.

It would be impossible.

Automation solves this problem.

Real Company Scenario

Let's imagine we work for Netflix.

The frontend team creates a new search feature.

The backend team fixes a bug.

The payment team updates billing.

The recommendation team improves suggestions.

All these developers push code to GitHub throughout the day.

Each push automatically starts a CI/CD pipeline.

Nobody has to call the DevOps engineer and ask:

"Can you deploy this?"

The pipeline handles everything automatically.

Who Does What?

Many beginners confuse the responsibilities.

Developer

The developer's responsibility is to:

  • Write application code
  • Fix bugs
  • Add new features
  • Push code to GitHub

The developer is not responsible for deploying to production in most organizations.

DevOps Engineer

The DevOps engineer creates the automation that deploys the application safely.

Typical responsibilities include:

  • Writing GitHub Actions workflows
  • Creating Jenkins pipelines
  • Building Docker images
  • Managing Kubernetes or ECS
  • Configuring AWS infrastructure
  • Monitoring applications
  • Managing deployments
  • Automating repetitive tasks

The Complete CI/CD Pipeline 

Developer

↓

Git Push

↓

GitHub Repository

↓

GitHub Actions / Jenkins

↓

Checkout Code

↓

Install Dependencies

↓

Compile / Build

↓

Run Unit Tests

↓

Run Integration Tests

↓

Static Code Analysis (SonarQube)

↓

Security Scan (Trivy)

↓

Docker Build

↓

Push Image to Amazon ECR

↓

Deploy to ECS / Kubernetes

↓

Application Starts

↓

Load Balancer

↓

Customers

↓

Monitoring & Alerts
Enter fullscreen mode Exit fullscreen mode

Every stage has a specific purpose.

Step 1 — Developer Pushes Code

The developer finishes a feature.

For example:

The login page now supports Google authentication.

The developer commits the changes.

git add .

git commit -m "Added Google Login"

git push origin main
Enter fullscreen mode Exit fullscreen mode

At this point nothing has been deployed.

GitHub simply stores the latest version of the project.

What Happens After Git Push?

GitHub constantly watches the repository.

Whenever someone pushes code, GitHub immediately checks whether a workflow exists.

It looks inside:

.github/workflows/
Enter fullscreen mode Exit fullscreen mode

Inside we might have:

deploy.yml
Enter fullscreen mode Exit fullscreen mode

or

ci.yml
Enter fullscreen mode Exit fullscreen mode

These files tell GitHub exactly what to do.

What is YAML?

A YAML file is simply a list of instructions.

Think of it like a recipe.

Instead of saying:

"Make a cake."

The recipe says:

Step 1

Mix flour.

Step 2

Add eggs.

Step 3

Bake.

CI/CD works exactly the same way.

Instead of cooking, we deploy software.

Step 2 — Runner Starts

GitHub itself cannot build your application.

It needs a computer.

This computer is called a Runner.

There are two types.

GitHub Hosted Runner

GitHub creates a temporary Linux virtual machine.

Example:

ubuntu-latest
Enter fullscreen mode Exit fullscreen mode

Advantages:

  • Easy to use
  • No maintenance
  • Automatically updated
  • Secure

Disadvantages:

  • Limited customization
  • Internet restrictions in some companies
  • Starts fresh every run

Self Hosted Runner

Many companies prefer their own servers.

Example:

AWS EC2

Azure VM

Physical server

Advantages:

  • Faster builds
  • Internal network access
  • Custom software
  • Lower cost for large companies

Step 3 — Checkout the Code

The runner is empty.

It knows nothing about our application.

First it downloads the latest version from GitHub.

actions/checkout
Enter fullscreen mode Exit fullscreen mode

Now the runner contains:

Frontend

Backend

Dockerfile

README

package.json
Enter fullscreen mode Exit fullscreen mode

Everything needed for the build is now available.

Step 4 — Install Dependencies

Applications depend on libraries.

NodeJS:

npm install
Enter fullscreen mode Exit fullscreen mode

Python:

pip install
Enter fullscreen mode Exit fullscreen mode

Java:

mvn install
Enter fullscreen mode Exit fullscreen mode

Without these packages, the application cannot run.

Step 5 — Build the Application

Now we compile the application.

React example:

npm run build
Enter fullscreen mode Exit fullscreen mode

Angular:

ng build
Enter fullscreen mode Exit fullscreen mode

Java:

mvn package
Enter fullscreen mode Exit fullscreen mode

The build process creates production-ready files.

Step 6 — Run Automated Tests

Testing is extremely important.

Imagine deploying software that immediately crashes.

Customers cannot use it.

Money is lost.

Automated tests verify:

  • Login works
  • Registration works
  • API works
  • Payments work
  • Search works

If even one critical test fails...

Pipeline immediately stops.

Nothing is deployed.

Why Stop the Pipeline?

Because broken software should never reach production.

This is one of the biggest advantages of CI/CD.

Problems are caught early.

Step 7 — SonarQube

Testing verifies functionality.

SonarQube checks code quality.

It looks for:

  • Bugs
  • Duplicate code
  • Poor coding practices
  • Hardcoded passwords
  • Memory leaks
  • SQL injection risks

Example:

password = "admin123";
Enter fullscreen mode Exit fullscreen mode

SonarQube flags this immediately.

Step 8 — Trivy Security Scan

Now we scan the Docker image.

Trivy checks:

  • Operating system vulnerabilities
  • Library vulnerabilities
  • CVEs
  • Secrets accidentally committed

Example:

Your Docker image contains an outdated OpenSSL version.

Trivy reports:

Critical Vulnerability.

The deployment stops.

Why Scan Security?

Because thousands of new vulnerabilities are discovered every year.

Deploying vulnerable software puts customer data at risk.

Step 9 — Build Docker Image

Now everything passed.

We create a Docker image.

docker build
Enter fullscreen mode Exit fullscreen mode

A Docker image contains:

  • Linux operating system
  • Application
  • Runtime
  • Libraries
  • Configuration

Everything needed to run the application anywhere.

Why Docker?

Imagine this situation.

Developer:

"It works on my laptop."

Production server:

"It doesn't work."

Why?

Different environments.

Docker solves this.

The exact same image runs everywhere.

Step 10 — Push Image to Amazon ECR

The image currently exists only on the runner.

If the runner disappears...

The image disappears too.

We store it permanently.

Amazon Elastic Container Registry (ECR)

docker push
Enter fullscreen mode Exit fullscreen mode

Now Kubernetes or ECS can download it.

Why ECR?

Think of ECR as GitHub for Docker images.

Instead of storing source code...

It stores application images.

Step 11 — Deploy

The deployment stage begins.

Depending on the company, deployment may use:

  • Amazon ECS
  • Amazon EKS
  • Kubernetes
  • Azure AKS
  • Google GKE

The platform pulls the newest image from ECR.

Old containers are replaced with new ones.

Rolling Deployment

Suppose five containers are running.

The deployment system does not stop all five.

Instead:

Stop one.

Start one.

Health check.

Repeat.

Customers continue using the application with almost no downtime.

Health Checks

After a new container starts, Kubernetes or ECS asks:

"Are you healthy?"

Example:

/health
Enter fullscreen mode Exit fullscreen mode

If the container answers correctly...

Traffic is sent to it.

If not...

It is automatically replaced.

Load Balancer

Customers never connect directly to containers.

Instead:

Users

↓

Application Load Balancer

↓

Container 1

Container 2

Container 3

Container 4
Enter fullscreen mode Exit fullscreen mode

The Load Balancer distributes requests evenly.

If one container crashes...

Traffic automatically goes to the healthy containers.

Monitoring

Deployment is complete.

Now monitoring begins.

Popular tools:

  • Amazon CloudWatch
  • Prometheus
  • Grafana
  • Datadog

We monitor:

  • CPU usage
  • Memory usage
  • Errors
  • Response time
  • Container health
  • Network traffic

If something goes wrong, alerts notify the DevOps team.

What Does a DevOps Engineer Monitor Every Day?

A DevOps engineer typically checks:

  • Did the pipeline succeed?
  • Did deployment complete successfully?
  • Are all containers healthy?
  • Is CPU usage too high?
  • Is memory usage increasing?
  • Are users reporting errors?
  • Are logs showing failures?
  • Is the application responding quickly?

Monitoring is an ongoing responsibility, not something done only during deployment.

Summary

A CI/CD pipeline automates the journey of software from a developer's code commit to a running application in production.

A typical flow is:

  1. Developer pushes code to GitHub.
  2. GitHub triggers a workflow.
  3. A runner downloads the code.
  4. Dependencies are installed.
  5. The application is built.
  6. Automated tests are executed.
  7. SonarQube analyzes code quality.
  8. Trivy scans for security vulnerabilities.
  9. A Docker image is created.
  10. The image is pushed to Amazon ECR.
  11. ECS or Kubernetes deploys the new version.
  12. A Load Balancer routes users to healthy containers.
  13. Monitoring tools watch the application and alert the DevOps team if problems occur.

This end-to-end automation is the foundation of modern DevOps and is one of the most common workflows you'll encounter in real-world engineering teams.

Top comments (0)