"Great developers write code. Great engineers automate everything around it."
One of the biggest transitions you'll make as a software engineer isn't learning another programming languageβit's learning how software gets delivered.
Many beginners think writing code is the hard part.
In reality, getting that code from your laptop to production reliably, safely, and automatically is what separates hobby projects from professional software.
That's where CI/CD comes in.
In this guide, you'll learn:
- β What CI/CD actually means
- β Build a simple FastAPI CI/CD pipeline with GitHub Actions
- β Add GitHub Secrets securely
- β Deploy automatically
- β Learn production engineering practices
- β Understand how senior DevOps and Platform engineers think
Whether you're building your first API or preparing for DevOps interviews, this guide will give you a strong foundation.
π€ What is CI/CD?
CI/CD stands for:
- CI β Continuous Integration
- CD β Continuous Delivery (or Continuous Deployment)
Think of it as an automated factory for your software.
Instead of manually doing this:
Write Code β Commit β Push β Run Tests β Build β SSH into Server β Deploy β Restart Server
CI/CD does it automatically.
π Understanding the Pipeline
Every software project follows roughly the same lifecycle:
Developer β Git Push β GitHub β GitHub Actions β Tests β Build β Deploy β Production Server
The goal is simple: Remove repetitive manual work.
π‘ Why Every Developer Should Learn CI/CD
Imagine you're working in a team of 30 developers. Without CI/CD:
- Someone forgets to run tests.
- Someone deploys the wrong version.
- Someone pushes broken code.
- Someone forgets an environment variable.
Eventually... Production breaks.
CI/CD exists to reduce these kinds of mistakes by automating repeatable checks and deployment steps.
π Our Example Project
Let's say our FastAPI project looks like this:
fastapi-backend/
βββ app/
β βββ main.py
β βββ routes.py
β βββ services.py
βββ tests/
β βββ test_api.py
βββ requirements.txt
βββ Dockerfile
βββ .github/
βββ workflows/
Everything related to GitHub Actions lives inside .github/workflows/.
π Step 1 β Create Your First GitHub Action
Create: .github/workflows/backend.yml
Basic workflow:
name: FastAPI CI
on:
push:
branches:
- main
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: |
pip install -r requirements.txt
- name: Run Tests
run: |
pytest
Every time you push to main, GitHub will:
- Create a fresh Ubuntu VM
- Clone your repository
- Install Python
- Install dependencies
- Run your tests
No manual work required.
π§ͺ Why Tests Come First
Never deploy code before testing it.
Think of tests as airport security. Passengers (your code) don't board the plane (production) until they've been checked.
A simple FastAPI test (tests/test_api.py):
from fastapi.testclient import TestClient
from app.main import app
client = TestClient(app)
def test_home():
response = client.get("/")
assert response.status_code == 200
If this test fails... Deployment stops. Exactly what we want.
π Step 2 β Adding GitHub Secrets
Never write this:
DATABASE_URL = "postgres://user:password@server"
Or:
OPENAI_API_KEY="abc123"
Those values should never be committed to Git. Instead, GitHub provides Repository Secrets.
Go to: Repository β Settings β Secrets and Variables β Actions β New Repository Secret
Example Secrets:
DATABASE_URLOPENAI_API_KEYJWT_SECRETSUPABASE_KEY
These values are encrypted and only available to your workflows.
Using Secrets in GitHub Actions
Example:
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
Now your application can access it:
import os
database = os.getenv("DATABASE_URL")
This keeps credentials out of your repository while allowing your workflow to use them securely.
π Step 3 β Deploy Automatically
Imagine your server is running Docker. After tests pass:
Developer β Push β GitHub Actions β SSH β Docker Pull β Docker Restart β Production Updated
Example deployment step:
- name: Deploy
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.SERVER_HOST }}
username: ubuntu
key: ${{ secrets.SERVER_KEY }}
script: |
cd backend
git pull
docker compose up -d --build
Now every push automatically updates your production server.
π³ Better: Deploy Docker Images
Instead of pulling Git code on the server... Build a Docker image.
Pipeline:
Push β Build Docker β Push Image β Deploy Image
Example:
docker build -t my-backend .
docker push ghcr.io/company/backend:latest
Production simply downloads the latest image. Much cleaner. Much more reliable.
π¦ Understanding Pipeline Stages
Professional CI/CD pipelines usually look like this:
Checkout β Install β Lint β Unit Tests β Integration Tests β Build β Security Scan β Docker Build β Publish β Deploy β Smoke Tests β Notify Slack
Notice how deployment is only one step. Most of the work happens before deployment.
π¦ Branch Strategies
Many companies don't deploy every push. Instead:
feature/login β Pull Request β CI β Review β Merge β Deploy
This prevents unfinished work from reaching production.
A common setup:
- Feature branches β Tests only
- develop β Deploy to staging
- main β Deploy to production
π Multiple Environments
Professional applications rarely have just one environment. Instead:
Developer β Development β Staging β Production
Each environment has:
- different databases
- different API keys
- different secrets
- different URLs
GitHub Environments let you manage secrets separately for each stage.
π Lint Before Testing
Before running tests, check code quality.
Example:
- name: Run Ruff
run: ruff check .
Or:
- name: Black
run: black --check .
A healthy pipeline often follows this order:
Lint β Format β Type Check β Tests
This catches many issues before your test suite even starts.
π§ͺ Add Type Checking
Python benefits greatly from static analysis.
Example:
- name: MyPy
run: mypy app
This helps detect type-related bugs before runtime.
π³ Build Once, Deploy Everywhere
One important production principle: Build once. Deploy many times.
Don't rebuild your application separately for staging and production.
Instead:
Build Docker Image β Push Registry β Deploy Same Image β Staging β Production
This ensures you're deploying exactly what was tested.
π Security Scanning
Modern pipelines also scan dependencies. Examples:
- Safety
- Bandit
- Trivy
- Snyk
- Dependabot
These tools help detect vulnerable packages, exposed secrets, insecure code, and outdated dependencies. Security is becoming a standard part of CI/CD.
π Monitoring After Deployment
Deployment isn't the finish line. A mature workflow continues with:
Deploy β Health Check β Logs β Metrics β Alerts
If something goes wrong, you'll know quickly. Popular tools include:
- Prometheus
- Grafana
- Sentry
- OpenTelemetry
π Moving Toward Advanced CI/CD Engineering
Once you're comfortable with the basics, start exploring:
β Matrix Builds
Test multiple Python versions:
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
β Reusable Workflows
Instead of copying YAML across repositories:
Company Workflow β Repository A, Repository B, Repository C
One workflow. Many projects.
β Self-Hosted Runners
Large companies often use their own machines instead of GitHub-hosted runners. Benefits include custom hardware, private networking, GPU support, and faster builds.
β Artifact Storage
Store build outputs for later deployment:
Tests β Build β Artifact β Deploy
Useful for traceability and rollbacks.
β Progressive Deployments
Instead of deploying to everyone at once:
10% β 25% β 50% β 100%
This reduces deployment risk. Common strategies include Blue-Green Deployments, Canary Releases, and Rolling Updates.
π‘ Tips Every CI/CD Engineer Should Know
- Fail Fast: Run inexpensive checks (linting, formatting) before expensive ones (integration tests).
- Keep Pipelines Fast: Developers shouldn't wait 30 minutes for feedback. Optimize caching and parallel jobs where possible.
- Never Store Secrets in Git: Use GitHub Secrets, Cloud Secret Managers, Vault, AWS Secrets Manager, or Azure Key Vault.
- Automate Everything: If you perform the same deployment step repeatedly, ask: "Can the pipeline do this instead?" Often, the answer is yes.
- Pipelines Are Code: Treat workflow files like production code. Review them. Test them. Version them. Improve them.
π Tools Every DevOps Engineer Should Know
| Category | Popular Tools |
|---|---|
| CI/CD | GitHub Actions, GitLab CI, Jenkins, CircleCI |
| Containers | Docker, Podman |
| Orchestration | Kubernetes, Docker Swarm |
| Infrastructure | Terraform, Pulumi |
| Configuration | Ansible |
| Monitoring | Prometheus, Grafana |
| Logging | Loki, ELK Stack |
| Security | Trivy, Snyk, Bandit |
| Secrets | Vault, AWS Secrets Manager, GitHub Secrets |
π― Learning Roadmap
If you're serious about DevOps or Platform Engineering:
Git β Linux β Docker β GitHub Actions β FastAPI β Nginx β Reverse Proxy β Terraform β Cloud (AWS/GCP/Azure) β Kubernetes β Observability β GitOps β Platform Engineering
Each skill builds on the previous one. Don't rush. Master the fundamentals first.
π Final Thoughts
CI/CD isn't just about deploying code. It's about building confidence.
When every commit is automatically tested, validated, packaged, and deployed using a repeatable process, your team spends less time worrying about releases and more time delivering features.
Start with a simple GitHub Actions workflow. Add tests. Introduce secrets. Automate deployments. Then gradually expand into security scanning, reusable workflows, infrastructure as code, and advanced deployment strategies.
Over time, you'll stop thinking of CI/CD as a toolβand start seeing it as an essential part of how modern software is built.
"The best deployment is the one nobody has to think aboutβbecause it's already automated, tested, and reliable."
Top comments (0)