Manual deployment quickly becomes unsustainable once infrastructure complexity increases.
Repeated SSH sessions, manual git pull, rebuilding containers, restarting services, and validating application health introduce operational inconsistency and deployment risk.
This walkthrough documents a reproducible CI/CD deployment pipeline where every push to GitHub automatically rebuilds and redeploys Docker Compose services on AWS EC2.
The final deployment architecture consisted of:
GitHub Repository
│
▼
GitHub Actions
│
▼
SSH into AWS EC2
│
▼
Docker Compose Orchestration
│
┌──────┼────────┐
▼ ▼ ▼
Nginx Flask PostgreSQL
│
▼
Public Endpoint (:8080)
The implementation surfaced multiple real-world infrastructure issues:
workflow detection failures
missing YAML assumptions
Docker Compose context problems
healthy containers with inaccessible public services
AWS networking restrictions
Rather than omitting failures, this guide documents the actual debugging process and resolutions.
Prerequisites
Before starting, provision the following:
Infrastructure
AWS EC2 instance (Ubuntu 22.04 LTS)
GitHub repository
Docker
Docker Compose
SSH access
Skills
Basic familiarity with:
Linux terminal
Git
Docker
GitHub Actions
Step 1 — Provision AWS EC2
Launch an Ubuntu EC2 instance.
Recommended configuration:
Setting Value
OS Ubuntu 22.04 LTS
Instance t2.micro (lab)
Storage 20GB
Security Group SSH + App Port
Validate SSH Connectivity
Before touching CI/CD, validate server accessibility.
From local terminal:
nc -zv 22
Expected result:
Connection to port 22 succeeded
Then SSH into the instance:
ssh ubuntu@
Successful connection should display Ubuntu system information.
Verify SSH Service
Once connected:
sudo systemctl status ssh
Expected output:
Active: active (running)
This confirms remote deployment automation will be possible.
Step 2 — Configure Repository Structure
Repository structure matters for GitHub Actions detection.
Final structure:
devops-lab/
│
├── .github/
│ └── workflows/
│ └── compose-rebuild-test.yml
│
└── compose-rebuild-test/
├── app/
├── nginx/
├── docker-compose.yml
└── test.txt
A common failure occurs when workflow files are placed inside nested folders.
Incorrect:
project-folder/.github/workflows
Correct:
repository-root/.github/workflows
GitHub only detects workflows at repository root.
Step 3 — Create Docker Compose Stack
The stack consisted of:
Flask application
PostgreSQL database
Nginx reverse proxy
docker-compose.yml
version: "3.9"
services:
flask:
build: ./app
container_name: flask_app
environment:
POSTGRES_HOST: postgres
POSTGRES_DB: mydatabase
POSTGRES_USER: myuser
POSTGRES_PASSWORD: mypassword
depends_on:
- postgres
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
interval: 10s
timeout: 3s
retries: 3
postgres:
image: postgres:15
container_name: postgres_db
environment:
POSTGRES_DB: mydatabase
POSTGRES_USER: myuser
POSTGRES_PASSWORD: mypassword
volumes:
- postgres_data:/var/lib/postgresql/data
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -U myuser -d mydatabase"]
interval: 10s
timeout: 5s
retries: 5
nginx:
image: nginx:latest
container_name: nginx_server
ports:
- "8080:80"
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
depends_on:
- flask
restart: unless-stopped
volumes:
postgres_data:
Why This Architecture?
Flask
Application runtime.
Responsibilities:
business logic
database communication
API handling
Health checks ensure unhealthy containers are detected:
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
PostgreSQL
Persistent storage layer.
Volume mounting ensures state persistence:
volumes:
- postgres_data:/var/lib/postgresql/data
This prevents data loss during redeployments.
Nginx
Acts as public ingress layer.
Traffic flow:
Internet
│
▼
Nginx (:8080)
│
▼
Flask (:5000)
│
▼
PostgreSQL (:5432)
Step 4 — Configure GitHub Secrets
GitHub Actions requires secure authentication to EC2.
Navigate to:
Repository Settings
→ Secrets and Variables
→ Actions
Add:
Secret Purpose
EC2_HOST EC2 Public IP
EC2_USER ubuntu
EC2_SSH_KEY private SSH key
Step 5 — Create GitHub Actions Workflow
Create:
.github/workflows/compose-rebuild-test.yml
Add:
name: Compose Rebuild Test Deploy
on:
push:
branches:
- main
paths:
- 'compose-rebuild-test/**'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v4
- name: Deploy compose-rebuild-test
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.EC2_HOST }}
username: ${{ secrets.EC2_USER }}
key: ${{ secrets.EC2_SSH_KEY }}
script_stop: true
timeout: 60s
command_timeout: 20m
script: |
echo "Connected to EC2"
cd /home/ubuntu/devops-lab
git pull origin main
cd compose-rebuild-test
docker compose down || true
docker compose up --build -d
docker ps
Deployment Behavior
Every push to:
main
that modifies:
compose-rebuild-test/**
automatically triggers:
Workflow execution
SSH into EC2
Repository sync
Compose teardown
Image rebuild
Container restart
Deployment becomes deterministic.
Step 6 — Validate Containers
After workflow execution:
SSH into EC2:
ssh ubuntu@
Check containers:
docker ps -a
Expected:
nginx_server Up
flask_app Healthy
postgres_db Healthy
Step 7 — First Major Failure
At this stage, infrastructure appeared healthy.
Containers:
✅ running
Health checks:
✅ healthy
Application:
❌ inaccessible publicly
This created misleading signals.
Local validation succeeded:
Response:
CI/CD Auto Deploy Working
Yet:
http://:8080
failed externally.
Root Cause Analysis
This was not an application issue.
This was not Docker failure.
This was not Nginx misconfiguration.
Root cause:
AWS Security Group ingress restrictions
Port 8080 was blocked.
Step 8 — Fix AWS Security Group
Navigate to:
EC2
→ Security Groups
→ Inbound Rules
Add:
Type Port Source
Custom TCP 8080 0.0.0.0/0
Save changes.
Retry:
http://:8080
Success:
CI/CD Auto Deploy Working
Deployment validated.
Common Errors Encountered
Error 1 — Missing deploy.yml
Attempt:
cat .github/workflows/deploy.yml
Error:
No such file or directory
Cause:
Incorrect workflow assumption.
Actual file:
compose-rebuild-test.yml
Error 2 — Docker Compose Failure
Attempt:
docker compose ps
Error:
no configuration file provided
Cause:
Executed outside compose directory.
Fix:
cd compose-rebuild-test
docker compose ps
Error 3 — Healthy Containers, Broken Access
Cause:
Cloud networking layer.
Fix:
Open security group ingress.
Final Deployment Flow
Final architecture:
git push
│
▼
GitHub Actions
│
▼
SSH into EC2
│
▼
git pull origin main
│
▼
docker compose down
docker compose up --build -d
│
▼
Health validation
│
▼
Public endpoint live
Key Engineering Lessons
Healthy containers do not guarantee reachable services
Application validation must include:
Container
Host
Firewall
Cloud Security Group
Public Access
Repository topology matters
GitHub workflows must exist at:
.github/workflows
Docker Compose is execution-context sensitive
Directory placement matters.
Cloud networking frequently masquerades as application failure
Infrastructure debugging should never stop at container health.
Closing Thoughts
CI/CD implementation rarely fails for a single reason.
The real complexity emerges from interactions between:
cloud networking
orchestration
repository topology
deployment automation
infrastructure permissions
Once validated, the result becomes a reproducible deployment system where every push automatically rebuilds and redeploys production-like workloads.
Top comments (0)