DevOps is the combination of Development which builds applications and Operations who manage the deployments and various runtime environments. In order to do the development and operations of an application you need to automate every part of the process you can and that is where CI/CD pipelines come into the picture. CI/CD stands for Continuous Integration and Continuous Delivery. Continuous Integration is used to automate the verification of qualitative standards (linting, formatting, static type checking), security (vulnerability scanning) and testing of code as it is committed to a code repository. This creates code that is high quality and leads to Continuous Delivery which automates the build and release of new application versions.
https://www.eficode.com/blog/ci-cd-vulnerability-scanning-how-to-begin-your-devsecops-journey
GitHub Repository Setup
💡 The complete source code referenced in this guide is available on GitHub
https://github.com/dpills/devops-quick-start-guide
GitHub is the most popular Source Code Management system and they offer an integrated CI/CD solution GitHub Actions which we will be using to setup our CI/CD Pipeline. Other popular CI/CD tools include CircleCI, TravisCI, Gitlab CI, Jenkins, and many others.
Create a new repository in GitHub and setup a small Python FastAPI app.
💡 Refer to FastAPI Production Setup Guide 🏁⚡️🚀 for a full Python FastAPI Guide
$ poetry init
This command will guide you through creating your pyproject.toml config.
Package name [devops-quick-start-guide]:
Version [0.1.0]:
Description []:
Author [dpills, n to skip]: dpills
License []:
Compatible Python versions [^3.11]:
Would you like to define your main dependencies interactively? (yes/no) [yes] no
Would you like to define your development dependencies interactively? (yes/no) [yes] no
Generated file
[tool.poetry]
name = "devops-quick-start-guide"
version = "0.1.0"
description = ""
authors = ["dpills"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.11"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Do you confirm generation? (yes/no) [yes]
Add the production and development dependencies.
$ poetry add fastapi 'uvicorn[standard]' httpx
...
Package operations: 21 installs, 0 updates, 0 removals
$ poetry add -G dev ruff black mypy pytest coverage
...
Package operations: 11 installs, 0 updates, 0 removals
Add the linting, formatting and static type checking rules to the pyproject.toml.
📝 pyproject.toml
[tool.poetry]
name = "devops-quick-start-guide"
version = "0.1.0"
description = ""
authors = ["dpills"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.104.0"
uvicorn = { extras = ["standard"], version = "^0.23.2" }
httpx = "^0.25.0"
[tool.poetry.group.dev.dependencies]
ruff = "^0.1.2"
black = "^23.10.1"
mypy = "^1.6.1"
pytest = "^7.4.3"
coverage = "^7.3.2"
[tool.black]
line-length = 88
[tool.ruff]
select = ["E", "F", "I"]
fixable = ["ALL"]
exclude = [".git", ".mypy_cache", ".ruff_cache"]
line-length = 88
[tool.mypy]
disallow_any_generics = true
disallow_subclassing_any = true
disallow_untyped_calls = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_return_any = true
strict_equality = true
disallow_untyped_decorators = false
ignore_missing_imports = true
implicit_reexport = true
plugins = "pydantic.mypy"
[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Add an app folder with an empty __init__.py file and a main.py file with the following contents.
📝 app/main.py
import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel
app = FastAPI()
class Data(BaseModel):
item: str
@app.get("/data", response_model=Data)
async def get_data() -> Data:
"""
Get Data
"""
return Data(item="devops")
if __name__ == "__main__":
uvicorn.run(
"app.main:app",
host="0.0.0.0",
port=8000,
log_level="debug",
reload=True,
)
GitHub Actions Workflow
GitHub Actions are configured with YAML Workflow files which determine on what events should something run and what should be ran. Create the .github/workflows/ci.yml folders/file within the root of the repository and setup our first job.
Lint
📝 .github/workflows/ci.yml
name: CI
on:
push:
branches: ["main"]
tags: ["*"]
pull_request:
branches: ["main"]
jobs:
qa:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"
cache: "poetry"
- name: Install Dependencies
run: poetry install --no-root --no-interaction
- name: Formatting
run: poetry run black --check app
- name: Linting
run: poetry run ruff --output-format=github app
- name: Static Type Checking
run: poetry run mypy app
This workflow will run on commits and pull requests on the main branch and any tags pushed to the repository. It then runs the qa job which checks out the code, installs poetry in a Python 3.11 environment and installs the app dependencies. Finally, it runs the quality checks to make sure the code is formatted, passes static linting and type checking.
Git add, commit and push all of these files to your repository.
$ git add .
$ git commit -m "CI setup"
[main 74afe32] CI setup
7 files changed, 1108 insertions(+)
create mode 100644 .github/workflows/ci.yml
create mode 100644 .gitignore
create mode 100644 LICENSE
create mode 100644 app/__init__.py
create mode 100644 app/main.py
create mode 100644 poetry.lock
create mode 100644 pyproject.toml
$ git push
Enumerating objects: 13, done.
Counting objects: 100% (13/13), done.
Delta compression using up to 10 threads
Compressing objects: 100% (9/9), done.
Writing objects: 100% (12/12), 30.21 KiB | 15.10 MiB/s, done.
Total 12 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:dpills/devops-quick-start-guide.git
aa11374..74afe32 main -> main
Under the Actions tab in the GitHub repo, we can see the details of the qa job which should pass all of the checks.
Update the main.py file to intentionally break the script with an undefined_variable to make sure the job properly catches it.
📝 app/main.py
import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel
app = FastAPI()
class Data(BaseModel):
item: str
@app.get("/data", response_model=Data)
async def get_data() -> Data:
"""
Get Data
"""
undefined_variable
return Data(item="devops")
if __name__ == "__main__":
uvicorn.run(
"app.main:app",
host="0.0.0.0",
port=8000,
log_level="debug",
reload=True,
)
Push this breaking change to the repository and it should now fail the linting check.
Remove the Undefined variable so the workflow passes again.
Test
Add a test_main.py file to the app folder with a unit test so we can run a test suite and generate a coverage report.
📝 app/test_main.py
from fastapi.testclient import TestClient
from app.main import app
client = TestClient(app)
def test_get_data() -> None:
"""
Test getting data
"""
r = client.get("/data")
assert r.status_code == 200
Verify the test setup is working as expected locally.
$ poetry run coverage run --source ./app -m pytest
=========================================== test session starts ============================================
platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
rootdir: /Users/dpills/articles/devops-quick-start-guide
plugins: anyio-3.7.1
collected 1 item
app/test_main.py . [100%]
============================================ 1 passed in 0.93s =============================================
Add a new testing job under jobs in the workflow YAML file with the same repo and python setup as the qa job. Add the test coverage command and the commands to generate a coverage report.
📝 .github/workflows/ci.yml
jobs:
...
testing:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"
cache: "poetry"
- name: Install Dependencies
run: poetry install --no-root --no-interaction
- name: Test & Coverage
run: poetry run coverage run --source ./app -m pytest
- name: Coverage Report
run: |
poetry run coverage xml
poetry run coverage report -m
Push these changes to the repository and we can now see both of the jobs being ran in parallel.
💡 If you use VS Code there is a useful GitHub Actions Plugin which you can enable to view the pipelines directly in your editor.
The code coverage is printed out in the Coverage Report step but it is useful to track code coverage over time and have a repository badge which shows the current coverage percentage. There are many different code coverage and testing applications but we will use CodeCov.
Setup a CodeCov account and link your GitHub so you are able to see your repositories. Click Setup repo which will show you how to setup GitHub Actions.
This will utilize a repository secret which is used to store sensitive information such as passwords. Navigate back to your GitHub repository settings and add the CODECOV_TOKEN as a new Actions secret.
Now add the Codecov step to the testing job and use the secret with the ${{ secrets.CODECOV_TOKEN }} syntax.
📝 .github/workflows/ci.yml
jobs:
...
testing:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"
cache: "poetry"
- name: Install Dependencies
run: poetry install --no-root --no-interaction
- name: Test & Coverage
run: poetry run coverage run --source ./app -m pytest
- name: Coverage Report
run: |
poetry run coverage xml
poetry run coverage report -m
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Commit these changes and after the workflow runs we can see the coverage metrics in codecov.
The codecov settings tab has the Markdown to add to our README file to show a live coverage percentage badge.
Copy the Markdown and add it to the README.md file for your repository.
📝 README.md
# DevOps Quick Start Guide
DevOps CI/CD Quick Start Guide with GitHub Actions 🛠️🐙⚡️
[](https://codecov.io/gh/dpills/devops-quick-start-guide)
Push the changes and we can now see our Code Coverage badge on our repo!
Build
Now that the app has passed static analysis and testing it is time to work on the Continuous Delivery starting with building the application package. For this Python API we can build and package it as a container image.
💡 Refer to Containers Demystified 🐳🤔 for a Docker container guide
Add a Dockerfile to the root of the repository.
📝 Dockerfile
FROM python:3.11-slim-bookworm as requirements-stage
RUN pip install poetry
COPY ./pyproject.toml ./poetry.lock /
RUN poetry export -f requirements.txt --output requirements.txt --without-hashes --without=dev
FROM python:3.11-slim-bookworm
COPY --from=requirements-stage /requirements.txt /requirements.txt
COPY ./app /app
RUN python3 -m pip install --no-cache-dir --upgrade -r requirements.txt
EXPOSE 8000
ENTRYPOINT ["uvicorn", "--host", "0.0.0.0", "--port", "8000", "app.main:app"]
Create a new repository in DockerHub and generate a Personal Access Token.
Add your DockerHub username and personal access token as an Actions secret in GitHub.
Now we are ready to add a new build workflow job. We need to make sure the qa and testing jobs are successful before executing a new build so we can indicate this with the needs entry. Additionally, we will only want to create new builds for new application versions and we can set the version of the app using git tags such as v1.0.0 . We can specify that this job should only run when a new tag is added to the repository with the if: github.ref_type == 'tag' entry. The DockerHub secrets are used to login to DockerHub and finally the new tag name is used as the Docker image tag.
📝 .github/workflows/ci.yml
jobs:
...
build:
runs-on: ubuntu-latest
needs: # Make sure qa and testing jobs pass first
- qa
- testing
if: github.ref_type == 'tag' # Only run on new tags
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and Push Image
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64 # Multi-Arch build for AMD and ARM CPUs
push: true
tags: dpills/devops-quick-start-guide:${{ github.ref_name }} # Use git tag
Commit these changes and then push a new tag to the repository.
$ git tag v1.0.0 && git push origin -u v1.0.0
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:dpills/devops-quick-start-guide.git
* [new tag] v1.0.0 -> v1.0.0
The workflow runs for the new tag and the qa and testing steps are executed and validated prior to the build step.
After the build step completes we can check the DockerHub repository where we can see the new image tag which has been built for the AMD64 and ARM64 architectures!
Deploy with a Self-Hosted Runner
Now that the image is built we are ready to deploy the API. Since we are using containers, a great option for production deployments is Kubernetes and we can use MiniKube to deploy this on our local computer.
Refer to Kubernetes Quick Start Guide ☁️⚡️🚀 for an in-depth Kubernetes tutorial
Make sure Minikube is running.
$ minikube start --driver=docker
😄 minikube v1.31.2 on Darwin 14.0 (arm64)
✨ Using the docker driver based on user configuration
📌 Using Docker Desktop driver with root privileges
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🔥 Creating docker container (CPUs=2, Memory=4000MB) ...
🐳 Preparing Kubernetes v1.27.4 on Docker 24.0.4 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring bridge CNI (Container Networking Interface) ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
$ kubectl get all -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-5d78c9869d-8pcln 1/1 Running 0 2s
kube-system pod/etcd-minikube 1/1 Running 0 15s
kube-system pod/kube-apiserver-minikube 1/1 Running 0 15s
kube-system pod/kube-controller-manager-minikube 1/1 Running 0 15s
kube-system pod/kube-proxy-j4q8m 1/1 Running 0 3s
kube-system pod/kube-scheduler-minikube 1/1 Running 0 15s
kube-system pod/storage-provisioner 1/1 Running 0 14s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 15s
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 15s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/coredns 1/1 1 1 15s
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/coredns-5d78c9869d 1 1 1 3s
Add the Kubernetes Deployment and Service manifest files to the repository in a k8s folder.
📝 k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: devops-quick-start-guide
name: devops-quick-start-guide
spec:
replicas: 1
selector:
matchLabels:
app: devops-quick-start-guide
template:
metadata:
labels:
app: devops-quick-start-guide
spec:
containers:
- name: devops-quick-start-guide
image: dpills/devops-quick-start-guide:v1.0.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8000
protocol: TCP
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: "1"
memory: 1Gi
📝 k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: devops-quick-start-guide
name: devops-quick-start-guide-svc
spec:
ports:
- name: http
port: 8000
protocol: TCP
targetPort: 8000
selector:
app: devops-quick-start-guide
Deploy the initial version and verify that it is running.
$ kubectl apply -f k8s
deployment.apps/devops-quick-start-guide created
service/devops-quick-start-guide-svc created
$ kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
devops-quick-start-guide 1/1 1 1 2m4s devops-quick-start-guide dpills/devops-quick-start-guide:v1.0.0 app=devops-quick-start-guide
Since we are using Minikube on our local computer, it is not Internet accessible for the GitHub Hosted Runners which we have been using so far indicated with the runs-on: ubuntu-latest portion of the jobs. To address situations where you are on a closed network Github Actions allows you to use Self-Hosted Runners. Navigate to the GitHub repository setting under Actions > Runners and add a New self-hosted runner
Select your operating system, architecture, go through the setup steps and get it running.
$ mkdir actions-runner && cd actions-runner
$ curl -o actions-runner-osx-arm64-2.311.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.311.0/actions-runner-osx-arm64-2.311.0.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 98.1M 100 98.1M 0 0 20.0M 0 0:00:04 0:00:04 --:--:-- 23.5M
$ echo "fa2f107dbce709807bae014fb3121f5dbe106211b6bbe3484c41e3b30828d6b2 actions-runner-osx-arm64-2.311.0.tar.gz" | shasum -a 256 -c
actions-runner-osx-arm64-2.311.0.tar.gz: OK
$ tar xzf ./actions-runner-osx-arm64-2.311.0.tar.gz
❯ ./config.sh --url https://github.com/dpills/devops-quick-start-guide --token AGDCRGCMZWN34QIVISIO5XXXXXX
--------------------------------------------------------------------------------
| ____ _ _ _ _ _ _ _ _ |
| / ___(_) |_| | | |_ _| |__ / \ ___| |_(_) ___ _ __ ___ |
| | | _| | __| |_| | | | | '_ \ / _ \ / __| __| |/ _ \| '_ \/ __| |
| | |_| | | |_| _ | |_| | |_) | / ___ \ (__| |_| | (_) | | | \__ \ |
| \____|_|\__|_| |_|\__,_|_.__/ /_/ \_\___|\__|_|\___/|_| |_|___/ |
| |
| Self-hosted runner registration |
| |
--------------------------------------------------------------------------------
# Authentication
√ Connected to GitHub
# Runner Registration
Enter the name of the runner group to add this runner to: [press Enter for Default]
Enter the name of runner: [press Enter for dpills-mac]
This runner will have the following labels: 'self-hosted', 'macOS', 'ARM64'
Enter any additional labels (ex. label-1,label-2): [press Enter to skip]
√ Runner successfully added
√ Runner connection is good
# Runner settings
Enter name of work folder: [press Enter for _work]
√ Settings Saved.
❯ ./run.sh
√ Connected to GitHub
Current runner version: '2.311.0'
2023-10-27 13:32:16Z: Listening for Jobs
The runner should now show up in the settings with an idle status indicating that it is ready to pick up new jobs.
Finally we can update the workflow config to include the deploy step which uses the self-hosted runner, waits for the build step to finish, only triggers on new tags and updates the Kubernetes deployment image to the new container image.
📝 .github/workflows/ci.yml
name: CI
on:
push:
branches: ["main"]
tags: ["*"]
pull_request:
branches: ["main"]
jobs:
qa:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"
cache: "poetry"
- name: Install Dependencies
run: poetry install --no-root --no-interaction
- name: Formatting
run: poetry run black --check app
- name: Linting
run: poetry run ruff --output-format=github app
- name: Static Type Checking
run: poetry run mypy app
testing:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"
cache: "poetry"
- name: Install Dependencies
run: poetry install --no-root --no-interaction
- name: Test & Coverage
run: poetry run coverage run --source ./app -m pytest
- name: Coverage Report
run: |
poetry run coverage xml
poetry run coverage report -m
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
build:
runs-on: ubuntu-latest
needs:
- qa
- testing
if: github.ref_type == 'tag'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and Push Image
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: dpills/devops-quick-start-guide:${{ github.ref_name }}
deploy:
runs-on: self-hosted
needs:
- build
if: github.ref_type == 'tag'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Update Deployment
run: kubectl set image deployment/devops-quick-start-guide devops-quick-start-guide=dpills/devops-quick-start-guide:${{ github.ref_name }}
Commit these changes and push a new version tag to the repository.
$ git tag v1.0.1 && git push origin -u v1.0.1
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:dpills/devops-quick-start-guide.git
* [new tag] v1.0.1 -> v1.0.1
All of the steps are now executed in order.
We can see that the self-hosted runner has ran the job.
$ ./run.sh
√ Connected to GitHub
Current runner version: '2.311.0'
2023-10-27 14:20:42Z: Listening for Jobs
2023-10-27 14:23:50Z: Running job: deploy
2023-10-27 14:23:57Z: Job deploy completed with result: Succeeded
Verify that the new image has been updated in the Kubernetes deployment. We should now see the v1.0.1 image being used.
$ kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
devops-quick-start-guide 1/1 1 1 42m devops-quick-start-guide dpills/devops-quick-start-guide:v1.0.1 app=devops-quick-start-guide
Congrats! 🎉 You have fully automated the quality, testing, build and deployment of your application and can now use the time you are saving to deliver more amazing features to your users. 😊 You should now have the basic knowledge to continue to build out the CI/CD setup. Some additional things you may want to add is stage build/deployment steps, security scanning, additional testing, tracking, etc. I hope you have found this article useful and good luck on your next project!
Top comments (0)