DEV Community: Mart Young

🧪 Selling Digital Privacy - My GDRP Toolkit Journey

Mart Young — Fri, 12 Dec 2025 09:25:24 +0000

Two months ago, I shelved my GDRP toolkit project halfway. Thoughts like "who cares?", "How accurate enough can this be", and mostly, "Who would want to buy?" was constantly what i had to deal with every step of the way.

Well, it's live on Gumroad, the idea of helping others navigate compliance and also generating a trickle of passive income is now a reality.

How I built, launched, and learned from my first difital product focusing on one of internet's most intimidating acronyms - GDRP.

Why GDRP

I built a GDRP-Compliant AWS infrastructure setup using Terraform, packaged with detailed documentation, reuseable Terraform modules, and deployment templates. Think of it as a plug-and-play toolkit for DevOps engineers, founders. or freelancers who want to launch EU-friendly cloud products without hiring a privacy lawyer.

Why GDRP? As a DevOps engineer who has worked across cloud infrastructure and compliance-heavy idustries over time, I have witnessed first hand how intimidating data privacy laws can be, especially for small teams. It's not like they don't care, they are just overwhlmed, confused or resource-strapped.
And that was my in.

What Sparked this Project

One thing is quite clear GDRP is notoriously vague, and AWS on the other hand is notoriously complex. When the two mix, they live choas in their wake.

As a DevOps engineer and cloud consultant, I kept noticing a pattern - most clients know they needed GDRP compliance, but rarely knew where or how to start. Some just copy-paste outdated checklists, others would trust their cloud setup defaults. One client even said, "We're GDRP-ready because we use AWS. Right?"

Spoiler alert: Wrong.

This was when it hit me What if i could codify compliance into reuseable infra that meets GDPR standards by design?

The Dice was cast (and Nearly Giving Up)

Research & Validation

Before writing a single line of Terraform, I asked around freelancer communities, Reddit and whatsapp groups. I posted a simple question: "Would a pre-built GDRP-friendly AWS starter kit help you or your clients?"

The responses were quite encouraging. Some even offered to pay for an early copy.

🛠️ Tools I Used

Terraform: For infrastructure-as-code module (VPCs, ECS, RDS, KMS, S3, CloudTrail, Cloudwatch, IAM - all privacy-hardened).
Notion: For organizing my checklist and writing the compliance documentation.
VS Code & Github: My dev and version control environment

⏳ Development Time

I worked this project mostly late in the evenings and during the weekends. It took me about ~35 hours over 4 weeks. Mostly spent on:

Testing and re-testing deployments.
Writing clean, expandable Terraform code.
Making sure every module met GDRP principles: data encryption, logging, auditability and region restrictions.

Bigest Challenge?

Legal confidence. Like yok already know, I'm not a lawyer, I had to make sure the toolkit aligned with technical requirements of GDRP. Not just the spirit, but also the letters. I had to review the ICO guidance, read AWS whitepapers extensively and had a privacy advisor look over the docs. (worth every penny).

Launching on Gumroad

🛒 Why?

I chose Gumroad for three reasons:

No upfront cost, perfect for testing the waters.
Simple setup, I could lauch in munites.
Digital product friendly, Gumroad's audience gets toolkits, templates, and niche SaaS assets.

💰 Pricing Strategy

I decided on \$59 for personal use and \$99 for basic comercial/agency use.
Why?

It undercut most legal templates.
The value was in saving time and avoiding legal risk a nightmare for freelancers and bootstrapped startups.

What I Learned

📈 Results

Over \$600 in sales within first 2 weeks.
X former Twitter, followered by direct links via Emails
11 purchases: mostly personal use
Most customers were freelancers, not startup founders. That flipped my assumptions
Gumroad discovery is weak. Most sales came from my own network, not their marketplace
"Compliance" sounds boring. I got better results using phrases like "data privacy-ready" and "cloud compliance toolkit".

Lessons Learned

Overdeliver with clarity. Documentation matters more than design.
People pay for simplicity. Just making AWS + GDRP less intimidating is valuable.

What's Next?

Expanding the kit to include support for other cloud providers like Azure & GCP.
Creating a micro-course on GDRP-compliant cloud design.
Exploring privacy laws beyond GDRP, like CCPA or Brazil's LGPD

I am also testing selling on Lemon Squeezy or Payhip for a better visibiity.

Final Thoughts

I did not set out to "start a business", just to solve a problem I noticed over and over again. If you are thinking about lauching your own digital product, here's my nugde: Pick a niche, validate it fast, and ship even if it's not perfect.

Want to check out the toolkit? Use this Here link
Got any questions? I'd love to hear your thoughts.

Build a Blue/Green deployment with Nginx Auto-Failover

Mart Young — Tue, 09 Dec 2025 15:54:05 +0000

Imagine you run two identical kitchens: Blue and Green. One serves customers, the other is warmed up and ready. If the active kitchen has trouble, you quietly switch orders to the standby and nobody notices. That’s Blue/Green. In this post we’ll build it ourselves, line by line, with Nginx doing the instant handoff—no prior code or prebuilt images required.

How It Fits Together (Quick Map)

Nginx: Front door. Sends traffic to the main pool, retries fast, and falls back to backup if the main one misbehaves. Logs everything as JSON.
Apps (Blue & Green): Same Node.js app, two copies. Env vars label which is which. They expose /healthz, /version, and chaos endpoints so we can test.
Dockerfile: Builds the app once; both Blue and Green use it.
docker-compose.yaml: Starts both apps, Nginx, and (if you want) the Slack watcher. Sets ports and health checks.
nginx.conf.template: Tells Nginx who’s primary, who’s backup, and to be impatient with failures.
watcher.py: Reads Nginx logs and posts to Slack when failover or high errors happen (optional, but helpful).
.env: One place to pick the active pool and set labels/alert thresholds.

What You’ll Learn

Blue/Green basics (two identical apps, one live, one ready).
How Nginx routes to a primary and instantly falls back to a backup.
Why health checks, short timeouts, and retries make failover fast.
How to add chaos endpoints to prove failover works.
How to read structured logs (and send Slack alerts) so you know which pool served traffic.
How to wire it all together with Docker Compose—no Kubernetes needed.

Prerequisites

Docker + Docker Compose.
Node.js (so we can build the tiny app locally).
(Optional) Slack webhook URL if you want alerts.
A terminal and a text editor. That’s it.

1) Create the Project from Scratch

Let’s start with nothing and build every file ourselves. Copy/paste is fine—understanding why each piece exists is the real goal.

1.1 package.json

This defines our minimal Node app and its dependencies.

cat > package.json <<'EOF'
{
  "name": "blue-green-app",
  "version": "1.0.0",
  "main": "app.js",
  "license": "MIT",
  "scripts": {
    "start": "node app.js"
  },
  "dependencies": {
    "express": "^4.18.2"
  }
}
EOF

1.2 app.js (with health + chaos endpoints)

This tiny server:

Responds to /healthz so Nginx can decide if we’re alive.
Responds to /version with headers that tell us which pool handled the request.
Has chaos endpoints so we can intentionally break one pool and watch traffic fail over.

cat > app.js <<'EOF'
const express = require('express');
const app = express();

const APP_POOL = process.env.APP_POOL || 'unknown';
const RELEASE_ID = process.env.RELEASE_ID || 'unknown';
const PORT = process.env.PORT || 3000;

let chaosMode = false;
let chaosType = 'error'; // 'error' or 'timeout'

// Add headers for tracing
app.use((req, res, next) => {
  res.setHeader('X-App-Pool', APP_POOL);
  res.setHeader('X-Release-Id', RELEASE_ID);
  next();
});

app.get('/', (req, res) => {
  res.json({
    service: 'Blue/Green Demo',
    pool: APP_POOL,
    releaseId: RELEASE_ID,
    status: chaosMode ? 'chaos' : 'healthy',
    chaosMode,
    chaosType: chaosMode ? chaosType : null,
    timestamp: new Date().toISOString(),
    endpoints: { version: '/version', health: '/healthz', chaos: '/chaos/start, /chaos/stop' }
  });
});

app.get('/healthz', (req, res) => {
  res.status(200).json({ status: 'healthy', pool: APP_POOL });
});

app.get('/version', (req, res) => {
  if (chaosMode && chaosType === 'error') return res.status(500).json({ error: 'Chaos: server error' });
  if (chaosMode && chaosType === 'timeout') return; // simulate hang
  res.json({ version: '1.0.0', pool: APP_POOL, releaseId: RELEASE_ID, timestamp: new Date().toISOString() });
});

app.post('/chaos/start', (req, res) => {
  const mode = req.query.mode || 'error';
  chaosMode = true;
  chaosType = mode;
  res.json({ message: 'Chaos started', mode, pool: APP_POOL });
});

app.post('/chaos/stop', (req, res) => {
  chaosMode = false;
  chaosType = 'error';
  res.json({ message: 'Chaos stopped', pool: APP_POOL });
});

app.listen(PORT, '0.0.0.0', () => {
  console.log(`App (${APP_POOL}) listening on ${PORT}`);
  console.log(`Release ID: ${RELEASE_ID}`);
});
EOF

1.3 Dockerfile (build the app image)

We’ll build the same image for Blue and Green; only the environment variables differ.

cat > Dockerfile <<'EOF'
FROM node:18-alpine
WORKDIR /app

# Install dependencies
COPY package*.json ./
RUN npm install --only=production

# Copy app code
COPY . .

EXPOSE 3000
CMD ["npm", "start"]
EOF

2) Nginx Config (Auto-Failover Upstreams)

Nginx is our traffic director. We template it so a single env var (ACTIVE_POOL) chooses who is primary. Create nginx.conf.template:

cat > nginx.conf.template <<'EOF'
events {
    worker_connections 1024;
}

http {
    # Structured JSON access logs
    log_format custom_json '{"time":"$time_iso8601"'
                          ',"remote_addr":"$remote_addr"'
                          ',"method":"$request_method"'
                          ',"uri":"$request_uri"'
                          ',"status":$status'
                          ',"bytes_sent":$bytes_sent'
                          ',"request_time":$request_time'
                          ',"upstream_response_time":"$upstream_response_time"'
                          ',"upstream_status":"$upstream_status"'
                          ',"upstream_addr":"$upstream_addr"'
                          ',"pool":"$sent_http_x_app_pool"'
                          ',"release":"$sent_http_x_release_id"}';

    upstream blue_pool {
        server app-blue:3000 max_fails=1 fail_timeout=3s;
        server app-green:3000 backup;
    }

    upstream green_pool {
        server app-green:3000 max_fails=1 fail_timeout=3s;
        server app-blue:3000 backup;
    }

    server {
        listen 80;
        server_name localhost;

        # Write JSON logs (shared volume)
        access_log /var/log/nginx/access.json custom_json;

        # Health check for LB
        location /healthz {
            access_log off;
            return 200 "healthy\n";
            add_header Content-Type text/plain;
        }

        location / {
            proxy_pass http://$UPSTREAM_POOL;

            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 1s;
            proxy_send_timeout 3s;
            proxy_read_timeout 3s;

            proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
            proxy_next_upstream_tries 2;
            proxy_next_upstream_timeout 10s;

            proxy_pass_request_headers on;
            proxy_hide_header X-Powered-By;
        }
    }
}
EOF

Why these settings? (plain English)

max_fails=1 fail_timeout=3s: one bad request is enough to say “try the other one” for a few seconds.
Short timeouts (1s connect, 3s send/read): don’t wait around; switch fast.
proxy_next_upstream + retries: if the main one errors or stalls, immediately try the backup within ~10s total.

What just happened? Nginx knows who’s main, who’s backup, and to give up quickly on a slow/broken main.

3) Optional Alerts: watcher.py + requirements.txt

Think of this as a friendly pager: it reads Nginx’s JSON logs and pings Slack when failover happens or errors spike. If you don’t want alerts, you can skip this section and remove the watcher service later.

requirements.txt:

cat > requirements.txt <<'EOF'
requests==2.32.3
EOF

watcher.py:

cat > watcher.py <<'EOF'
import json, os, time, requests
from collections import deque
from datetime import datetime, timezone

LOG_PATH = os.environ.get("NGINX_LOG_FILE", "/var/log/nginx/access.json")
SLACK_WEBHOOK_URL = os.environ.get("SLACK_WEBHOOK_URL", "")
SLACK_PREFIX = os.environ.get("SLACK_PREFIX", "from: @Watcher")
ACTIVE_POOL = os.environ.get("ACTIVE_POOL", "blue")
ERROR_RATE_THRESHOLD = float(os.environ.get("ERROR_RATE_THRESHOLD", "2"))
WINDOW_SIZE = int(os.environ.get("WINDOW_SIZE", "200"))
ALERT_COOLDOWN_SEC = int(os.environ.get("ALERT_COOLDOWN_SEC", "300"))
MAINTENANCE_MODE = os.environ.get("MAINTENANCE_MODE", "false").lower() == "true"

def now_iso(): return datetime.now(timezone.utc).isoformat()

def post_to_slack(text: str):
    if not SLACK_WEBHOOK_URL:
        return
    try:
        requests.post(SLACK_WEBHOOK_URL, json={"text": f"{SLACK_PREFIX} | {text}"}, timeout=5).raise_for_status()
    except Exception:
        pass

def parse(line: str):
    try:
        data = json.loads(line.strip())
        return {
            "pool": data.get("pool"),
            "release": data.get("release"),
            "status": int(data["status"]) if data.get("status") else None,
            "upstream_status": str(data.get("upstream_status") or ""),
            "upstream_addr": data.get("upstream_addr"),
        }
    except Exception:
        return None

class AlertState:
    def __init__(self):
        self.last_pool = ACTIVE_POOL
        self.window = deque(maxlen=WINDOW_SIZE)
        self.cooldowns = {}
    def cooldown_ok(self, key):
        now = time.time()
        last = self.cooldowns.get(key)
        if last is None or (now - last) >= ALERT_COOLDOWN_SEC:
            self.cooldowns[key] = now
            return True
        return False
    def error_rate_pct(self):
        if not self.window: return 0.0
        err = 0
        for evt in self.window:
            if any(s.startswith("5") for s in evt.get("upstream_status","").split(",") if s):
                err += 1
            elif evt.get("status") and 500 <= int(evt["status"]) <= 599:
                err += 1
        return (err / len(self.window)) * 100.0
    def handle(self, evt):
        self.window.append(evt)
        if MAINTENANCE_MODE:
            return
        pool = evt.get("pool")
        if pool and self.last_pool and pool != self.last_pool:
            if self.cooldown_ok(f"failover_to_{pool}"):
                post_to_slack(f"*Failover Detected*: {self.last_pool} → {pool}\n• time: {now_iso()}\n• error_rate: {self.error_rate_pct():.2f}%\n• upstream: {evt.get('upstream_addr')}")
            self.last_pool = pool
        if len(self.window) >= max(10, int(WINDOW_SIZE * 0.5)):
            rate = self.error_rate_pct()
            if rate > ERROR_RATE_THRESHOLD and self.cooldown_ok(f"error_rate_{int(round(rate))}"):
                post_to_slack(f"*High Error Rate*: {rate:.2f}% over last {len(self.window)} requests\n• time: {now_iso()}\n• active_pool: {pool or self.last_pool}")

def tail(path):
    with open(path, "r") as f:
        f.seek(0, os.SEEK_END)
        while True:
            line = f.readline()
            if not line:
                time.sleep(0.2)
                continue
            yield line

def main():
    state = AlertState()
    while not os.path.exists(LOG_PATH):
        time.sleep(0.5)
    for line in tail(LOG_PATH):
        evt = parse(line)
        if evt: state.handle(evt)

if __name__ == "__main__":
    main()
EOF

4) docker-compose.yaml (Build + Orchestrate)

Compose glues everything together: it builds the single app image, runs it twice (Blue/Green), starts Nginx, and (optionally) the Slack watcher. This is the “one file to rule them all.”

cat > docker-compose.yaml <<'EOF'
version: '3.8'

services:
  app-blue:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: blue-app
    environment:
      - APP_POOL=blue
      - RELEASE_ID=${RELEASE_ID_BLUE}
      - PORT=${PORT:-3000}
    ports:
      - "8081:3000"
    healthcheck:
      test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://127.0.0.1:3000/healthz || exit 1"]
      interval: 5s
      timeout: 3s
      retries: 3
      start_period: 10s

  app-green:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: green-app
    environment:
      - APP_POOL=green
      - RELEASE_ID=${RELEASE_ID_GREEN}
      - PORT=${PORT:-3000}
    ports:
      - "8082:3000"
    healthcheck:
      test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://127.0.0.1:3000/healthz || exit 1"]
      interval: 5s
      timeout: 3s
      retries: 3
      start_period: 10s

  nginx:
    image: nginx:alpine
    container_name: nginx-lb
    ports:
      - "8080:80"
    environment:
      - ACTIVE_POOL=${ACTIVE_POOL}
      - UPSTREAM_POOL=${ACTIVE_POOL}_pool
    volumes:
      - ./nginx.conf.template:/etc/nginx/nginx.conf.template:ro
      - nginx_logs:/var/log/nginx
    depends_on:
      - app-blue
      - app-green
    command: >
      sh -c "
        envsubst '$$UPSTREAM_POOL' < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf &&
        nginx -g 'daemon off;'
      "

  alert_watcher:
    image: python:3.11-slim
    container_name: alert-watcher
    depends_on:
      - nginx
    environment:
      - SLACK_WEBHOOK_URL=${SLACK_WEBHOOK_URL}
      - SLACK_PREFIX=${SLACK_PREFIX:-from: @Watcher}
      - ACTIVE_POOL=${ACTIVE_POOL}
      - ERROR_RATE_THRESHOLD=${ERROR_RATE_THRESHOLD:-2}
      - WINDOW_SIZE=${WINDOW_SIZE:-200}
      - ALERT_COOLDOWN_SEC=${ALERT_COOLDOWN_SEC:-300}
      - MAINTENANCE_MODE=${MAINTENANCE_MODE:-false}
      - NGINX_LOG_FILE=/var/log/nginx/access.json
    volumes:
      - nginx_logs:/var/log/nginx
      - ./watcher.py:/opt/watcher/watcher.py:ro
      - ./requirements.txt:/opt/watcher/requirements.txt:ro
    command: >
      sh -c "pip install --no-cache-dir -r /opt/watcher/requirements.txt && python /opt/watcher/watcher.py"

volumes:
  nginx_logs:
EOF

Want it ultra-minimal? Comment out/remove alert_watcher if you don’t need Slack alerts. The stack still works without it.

What just happened? We wired four pieces: one shared app image, two containers (Blue/Green) with different env vars, Nginx in front, and an optional watcher that shares Nginx logs.

5) .env (Wire It All Up)

One place for all the knobs: which pool is primary, release labels, and alert thresholds. Changing ACTIVE_POOL later lets you flip who is “live” without touching code.

cat > .env <<'EOF'
# Which pool is primary (blue or green)
ACTIVE_POOL=blue

# Release IDs (just labels for observability)
RELEASE_ID_BLUE=release-v1.0.0-blue
RELEASE_ID_GREEN=release-v1.0.0-green

# App port inside the container
PORT=3000

# Optional Slack alerts
SLACK_WEBHOOK_URL=
SLACK_PREFIX=from: @YourName
ERROR_RATE_THRESHOLD=2
WINDOW_SIZE=200
ALERT_COOLDOWN_SEC=300
MAINTENANCE_MODE=false
EOF

6) Run Everything

Bring the whole stack up. Compose will build the image once and reuse it for both Blue and Green, then start Nginx and the watcher.

docker compose up -d
docker compose ps

You should see containers for blue, green, nginx, and (optionally) alert-watcher.

7) Sanity Checks

These calls prove traffic flows and headers are set so you can tell which pool responded.

# Through Nginx (main entry)
curl http://localhost:8080/version

# Direct to Blue
curl http://localhost:8081/version

# Direct to Green
curl http://localhost:8082/version

You should see JSON with pool and releaseId. By default, Blue is
active.

8) Prove Auto-Failover (Chaos Testing)

Time to break things on purpose. We’ll poison Blue and watch Nginx slide traffic to Green without customers seeing errors.

1) Baseline (Blue active):

curl http://localhost:8080/version
# Expect X-App-Pool: blue

2) Break Blue:

curl -X POST http://localhost:8081/chaos/start?mode=error

3) Check via Nginx:

curl http://localhost:8080/version
# Expect X-App-Pool: green (failover)

4) Heal Blue:

curl -X POST http://localhost:8081/chaos/stop

5) Try timeout chaos:

curl -X POST http://localhost:8081/chaos/start?mode=timeout

6) Light load test (should stay 200s, most from active pool):

for i in {1..50}; do curl -s http://localhost:8080/version >/dev/null; done

What just happened? We proved failover under two kinds of pain: errors and timeouts. Nginx noticed, retried, and shifted traffic to keep responses healthy.

9) Switch Pools Manually

Edit .env:

ACTIVE_POOL=green

Restart:

docker compose down
docker compose up -d

Nginx will now route to Green as primary, Blue as backup.

10) Slack Alerts (If Enabled)

If you kept alert_watcher, set SLACK_WEBHOOK_URL in .env, then:

docker compose up -d

Trigger chaos on Blue:

curl -X POST http://localhost:8081/chaos/start?mode=error
for i in {1..50}; do curl -s http://localhost:8080/version >/dev/null; done
curl -X POST http://localhost:8081/chaos/stop

You should see Slack messages for failover and (if errors > threshold) high error rate. Tune thresholds in .env.

What just happened? The watcher tailed Nginx’s JSON logs, spotted failover/high-error signals, and pinged Slack so humans know immediately.

11) Cleanup

docker compose down
# Full clean (images/volumes):
docker compose down -v --rmi all

What just happened? We shut down everything, and if you ran the full clean, you also removed images and volumes for a fresh slate next time.

Troubleshooting Quick Hits

Ports busy: Free 8080/8081/8082 or change mappings in compose.
No failover: Check health endpoints (/healthz), timeouts, and chaos mode.
Headers missing: Ensure app sets X-App-Pool/X-Release-Id and Nginx passes headers.
Slack silent: Verify webhook, internet egress, and watcher logs.
Slow failover: Tighten timeouts (proxy_connect_timeout, max_fails, fail_timeout).

Why This Matters

Zero downtime: Swap versions or survive failures without users noticing.
Confidence: Chaos testing proves failover actually works.
Clarity: Structured logs + headers show exactly who served each request.
Simplicity: Docker Compose + Nginx — no Kubernetes required.

Next Steps

Add CI to build/push the app image, tag blue/green releases.
Add canary routing (gradual traffic shift) on top of blue/green.
Ship logs to ELK/Datadog, add dashboards.
Extend alerts to email/PagerDuty.

You just built Blue/Green with automatic failover, chaos testing, and optional Slack alerts — from scratch. Happy shipping! 🚀

From Zero to Production: A Complete Guide to Deploying Microservices with Terraform, Ansible and CI/CD

Mart Young — Tue, 09 Dec 2025 15:30:22 +0000

How I built a production-ready DevOps pipeline for a microservices TODO application - and how you can too, even if you're just starting out.

Introduction: Why This Matters

If you're reading this, you've probably heard terms like "DevOps," "Infrastructure as Code," and "CI/CD" thrown around, but maybe you're not entirely sure what they mean or how they fit together. That's exactly where I was when I started.

This guide isn't just about completing a task - it's about understanding the why behind each decision, learning from common mistakes, and building something you can be proud of. By the end, you'll have deployed a real application to the cloud with automated infrastructure, proper security, and a professional workflow.

What you'll build:

A microservices application with 5 different services (Vue.js, Go, Node.js, Java, Python)
Automated cloud infrastructure using Terraform
Server configuration and deployment with Ansible
CI/CD pipelines that detect when things go wrong
Multi-environment setup (dev, staging, production)
Secure HTTPS with automatic SSL certificates
A single command that deploys everything

What you'll learn:

How containerization actually works (beyond just "docker run")
Why infrastructure as code matters (and how it saves you from disasters)
How to think about security in a cloud environment
The importance of automation and what happens when you skip it

Let's dive in.

Part 1: Understanding What We're Building

Before we start writing code, let's understand what we're actually building. This isn't just a TODO app - it's a microservices architecture, which means instead of one big application, we have multiple small services that work together.

The Architecture

Think of it like a restaurant:

Frontend (Vue.js) - The dining room where customers interact
Auth API (Go) - The host who checks if you have a reservation (authentication)
Todos API (Node.js) - The waiter who takes your order (manages your todos)
Users API (Java) - The manager who knows all the customers (user management)
Log Processor (Python) - The kitchen staff who process orders (background processing)
Redis - The order board where everyone can see what's happening (message queue)

Each service runs in its own container, which is like giving each part of the restaurant its own kitchen. If the waiter (Todos API) has a problem, it doesn't crash the whole restaurant.

Why Containerization?

You might be thinking: "Why not just run everything on one server?" Great question! Here's why containers matter:

Isolation: If one service crashes, others keep running
Consistency: "It works on my machine" becomes "it works everywhere"
Scalability: Need more power? Spin up more containers
Portability: Move from AWS to Azure? Just change where you run containers

The Challenge

The real challenge isn't just getting containers to run - it's:

Making sure they can talk to each other
Securing them with HTTPS
Automating deployment so you don't manually SSH into servers
Detecting when someone changes things manually (drift detection)
Managing multiple environments without chaos

That's what makes this a real-world project.

Part 2: Setting Up Your Development Environment

Before we write any code, let's make sure you have everything you need. Don't worry if some of these are new - I'll explain what each one does.

Required Accounts

GitHub Account (Free)

This is where your code lives and where CI/CD runs
Think of it as your code's home and your automation's brain
Sign up at github.com if you don't have one

AWS Account (Free tier available)

This is where your servers will run
AWS has a free tier that's perfect for learning
You'll need a credit card, but we'll stay within free limits
Sign up at aws.amazon.com

Domain Name (Optional but recommended - ~$10-15/year)

This is your website's address (like yourname.com)
You can use services like Namecheap, GoDaddy, or Cloudflare
Why you need it: Let's Encrypt (free SSL) requires a real domain
Alternative: You can test with localhost but won't get real SSL

Installing Required Tools

Docker & Docker Compose

# On Ubuntu/Debian
sudo apt-get update
sudo apt-get install docker.io docker-compose-plugin

# Verify installation
docker --version
docker compose version

What is Docker? Think of it as a shipping container for software. Just like shipping containers standardize how goods are transported, Docker standardizes how applications run.

Terraform (Version 1.5.0 or higher)

# Download from hashicorp.com or use package manager
wget https://releases.hashicorp.com/terraform/1.5.0/terraform_1.5.0_linux_amd64.zip
unzip terraform_1.5.0_linux_amd64.zip
sudo mv terraform /usr/local/bin/

# Verify
terraform version

What is Terraform? It's like a blueprint for your cloud infrastructure. Instead of clicking buttons in AWS console (which you'll forget), you write code that describes what you want, and Terraform makes it happen.

Ansible

sudo apt-get install ansible

# Verify
ansible --version

What is Ansible? Think of it as a remote control for servers. Instead of SSHing into each server and typing commands, you write a "playbook" that tells Ansible what to do, and it does it on all your servers.

AWS CLI

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Configure with your credentials
aws configure

What is AWS CLI? It's a command-line interface to AWS. Instead of using the web console, you can control AWS from your terminal.

Setting Up AWS

This is where many beginners get stuck, so let's go through it step by step.

Step 1: Create an IAM User

Why not use your root account? Security best practice - root account has unlimited power. If it gets compromised, your entire AWS account is at risk.

Go to AWS Console → IAM → Users
Click "Create user"
Name it something like terraform-user
Important: Check "Provide user access to the AWS Management Console" if you want console access, OR just programmatic access for CLI/API
Attach policies:
- AmazonEC2FullAccess (for creating servers)
- AmazonS3FullAccess (for storing Terraform state)
- AmazonDynamoDBFullAccess (for state locking)
- AmazonSESFullAccess (for email notifications)
Save the Access Key ID and Secret Access Key - you'll need these!

Step 2: Create S3 Bucket for Terraform State

What is Terraform state? Terraform needs to remember what it created. This "memory" is stored in a state file. We put it in S3 so it's:

Backed up automatically
Accessible from anywhere
Versioned (can see history)

Go to S3 → Create bucket
Name it something like yourname-terraform-state
Important settings:
- Region: Choose one (remember which one!)
- Block Public Access: Keep all enabled (security)
- Versioning: Enable this (so you can recover if state gets corrupted)
Click Create

Step 3: Create DynamoDB Table for State Locking

Why do we need locking? Imagine two people trying to deploy at the same time. Without locking, they might both try to create the same server, causing conflicts. DynamoDB prevents this.

Go to DynamoDB → Create table
Table name: terraform-state-lock
Partition key: LockID (type: String)
Table settings: Use default
Capacity: On-demand (pay per request - perfect for this use case)
Click Create

Step 4: Create EC2 Key Pair

What is a key pair? It's like a password, but more secure. Instead of typing a password, you use a private key file to authenticate.

Go to EC2 → Key Pairs → Create key pair
Name: my-terraform-key (or whatever you prefer)
Key pair type: RSA
Private key file format: .pem
Click Create
IMPORTANT: The .pem file downloads automatically. Save it somewhere safe! You'll need it to SSH into your servers.

Step 5: Verify AWS CLI Works

# Test your credentials
aws sts get-caller-identity

# Should show your user ARN

If this works, you're all set! If not, check your aws configure settings.

Part 3: Containerizing Your Application

Now that your environment is set up, let's containerize the application. This is where the magic happens.

Understanding Dockerfiles

A Dockerfile is like a recipe. It tells Docker:

What base image to start with (like choosing an operating system)
What files to copy
What commands to run
What port to expose
What command to run when the container starts

Creating Dockerfiles for All Services

Now, you might be wondering: "How do I know what Dockerfile to create for each service?" Great question! Let me show you the pattern.

The rule of thumb: Each service folder needs its own Dockerfile. Look at your project structure:

DevOps-deployment/
├── frontend/          → Needs Dockerfile (Vue.js)
├── auth-api/          → Needs Dockerfile (Go) 
├── todos-api/         → Needs Dockerfile (Node.js)
├── users-api/         → Needs Dockerfile (Java)
└── log-message-processor/ → Needs Dockerfile (Python)

How to figure out what each service needs:

Check what language/framework it uses (look for package.json, pom.xml, requirements.txt, go.mod)
Find the entry point (usually server.js, main.go, main.py, or a compiled JAR)
Determine the port it runs on (check the code or config files)
Follow the pattern for that language

Frontend Dockerfile (Vue.js)

First, check the service:

cd frontend
ls -la
# You'll see: package.json, src/, public/
# This tells you: It's a Vue.js app that needs to be built

Vue.js apps are special - they compile to static HTML/CSS/JS files that need a web server. We use a "multi-stage build":

Build stage: Use Node.js to compile the Vue app
Runtime stage: Use nginx (lightweight web server) to serve the compiled files

# Step 1: Build the application
FROM node:18-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build

# Step 2: Serve it with nginx
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80

Breaking it down:

FROM node:18-alpine AS build - Start with Node.js for building (the AS build names this stage)
COPY package*.json ./ - Copy dependency files first (Docker caching optimization!)
RUN npm install - Install dependencies
RUN npm run build - Compile Vue.js to static files (creates dist/ folder)
FROM nginx:alpine - Start a NEW stage with nginx (much smaller image)
COPY --from=build - Copy the built files from the build stage
EXPOSE 80 - nginx serves on port 80

Why two stages? The build stage has Node.js + all build tools (~500MB). The runtime stage only has nginx + static files (~20MB). This makes the final image 25x smaller!

Auth API Dockerfile (Go)

Check the service:

cd auth-api
ls -la
# You'll see: go.mod, main.go
# This tells you: It's Go, entry point is main.go

Go is special - it compiles to a single binary file. No runtime needed! We also use multi-stage build:

# Build stage
FROM golang:1.21-alpine AS build
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o auth-api

# Runtime stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=build /app/auth-api /auth-api
EXPOSE 8081
CMD ["/auth-api"]

Breaking it down:

FROM golang:1.21-alpine AS build - Go compiler for building
RUN go mod download - Download Go dependencies
RUN go build -o auth-api - Compile to a single binary file
FROM alpine:latest - Tiny Linux (only 5MB!)
COPY --from=build /app/auth-api - Copy the compiled binary
CMD ["/auth-api"] - Run the binary

Key differences from Vue.js:

Go compiles to a single binary (no runtime needed!)
Final image is super small (~10MB vs ~500MB for Node.js)
CGO_ENABLED=0 creates a static binary (no external dependencies)

Todos API Dockerfile (Node.js)

Check the service:

cd todos-api
ls -la
# You'll see: package.json, server.js, routes.js
# This tells you: It's Node.js, entry point is server.js

Check package.json for the start command:

{
  "scripts": {
    "start": "node server.js"
  }
}

Node.js API Dockerfile (simpler than Vue.js - no build step needed):

FROM node:18-alpine
WORKDIR /app

# Copy dependency files first (Docker caching optimization)
COPY package*.json ./

# Install dependencies (production only for smaller image)
RUN npm ci --only=production

# Copy application code
COPY . .

# Expose the port (check server.js to see which port)
EXPOSE 8082

# Start the application
CMD ["node", "server.js"]

Why this pattern?

npm ci --only=production - Faster, more reliable than npm install, and skips dev dependencies
Copy package*.json first - If dependencies don't change, Docker reuses the cached layer
node:18-alpine - Lightweight Node.js image

How to test it:

# Build the image
docker build -t todos-api ./todos-api

# Run it
docker run -p 8082:8082 todos-api

# Test it
curl http://localhost:8082

Users API Dockerfile (Java Spring Boot)

Check the service:

cd users-api
ls -la
# You'll see: pom.xml, src/
# This tells you: It's Java with Maven, needs to be compiled

Java services need two stages:

Build stage - Compile the code
Runtime stage - Run the compiled JAR

# Stage 1: Build
FROM maven:3.9-eclipse-temurin-17 AS build
WORKDIR /app

# Copy Maven config first (caching optimization)
COPY pom.xml .
# Download dependencies (cached if pom.xml doesn't change)
RUN mvn dependency:go-offline

# Copy source code
COPY src ./src

# Build the application
RUN mvn clean package -DskipTests

# Stage 2: Runtime
FROM eclipse-temurin:17-jre-alpine
WORKDIR /app

# Install JAXB dependencies (needed for Java 17+)
RUN apk add --no-cache wget && \
    mkdir -p /app/lib && \
    wget -q -O /app/lib/jaxb-api.jar https://repo1.maven.org/maven2/javax/xml/bind/jaxb-api/2.3.1/jaxb-api-2.3.1.jar && \
    wget -q -O /app/lib/jaxb-runtime.jar https://repo1.maven.org/maven2/org/glassfish/jaxb/jaxb-runtime/2.3.1/jaxb-runtime-2.3.1.jar && \
    apk del wget

# Copy the built JAR from build stage
COPY --from=build /app/target/*.jar app.jar

EXPOSE 8083

# Run the Spring Boot application
ENTRYPOINT ["java", \
    "--add-opens", "java.base/java.lang=ALL-UNNAMED", \
    "--add-opens", "java.base/java.lang.reflect=ALL-UNNAMED", \
    "--add-opens", "java.base/java.util=ALL-UNNAMED", \
    "-cp", "app.jar:/app/lib/*", \
    "org.springframework.boot.loader.JarLauncher"]

Why this is complex:

Java needs compilation (Maven does this)
Spring Boot creates a "fat JAR" (includes everything)
Java 17+ removed some libraries (JAXB), so we add them back
The --add-opens flags are needed for Java 17+ module system

Don't worry if this looks complicated - Java Dockerfiles are the most complex. The pattern is always:

Build stage: Install dependencies, compile
Runtime stage: Copy compiled artifact, run it

Log Message Processor Dockerfile (Python)

Check the service:

cd log-message-processor
ls -la
# You'll see: requirements.txt, main.py
# This tells you: It's Python, entry point is main.py

Python Dockerfile:

FROM python:3.11-slim
WORKDIR /app

# Install build dependencies (needed to compile some Python packages)
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc \
    g++ \
    python3-dev \
    && rm -rf /var/lib/apt/lists/*

# Copy and install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Remove build dependencies (they're not needed at runtime)
RUN apt-get purge -y gcc g++ python3-dev && \
    apt-get autoremove -y && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Copy application code
COPY . .

# Run the application
CMD ["python", "main.py"]

Why install then remove build dependencies?

Some Python packages need to compile C extensions
We install gcc, g++ to compile them
After installation, we remove them (saves ~200MB!)
The compiled packages still work without the compilers

Simpler alternative (if no C extensions needed):

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "main.py"]

The Pattern: How to Create Any Dockerfile

Here's the mental model:

Identify the language → Check for language-specific files
- package.json → Node.js
- pom.xml or build.gradle → Java
- requirements.txt → Python
- go.mod → Go
- Cargo.toml → Rust
Find the base image → Use official images
- Node.js → node:18-alpine
- Java → eclipse-temurin:17-jre-alpine (runtime) or maven:3.9-eclipse-temurin-17 (build)
- Python → python:3.11-slim
- Go → golang:1.21-alpine (build) or alpine:latest (runtime)
Copy dependencies first → Docker caching optimization
- Copy package.json / pom.xml / requirements.txt / go.mod
- Install dependencies
- Then copy application code
Expose the port → Check the code for which port it uses
Set the command → How to start the application

Testing Each Dockerfile

Before adding to docker-compose, test each one:

# Test Frontend
cd frontend
docker build -t frontend-test .
docker run -p 8080:80 frontend-test
curl http://localhost:8080

# Test Auth API
cd ../auth-api
docker build -t auth-api-test .
docker run -p 8081:8081 auth-api-test
curl http://localhost:8081/health

# Test Todos API
cd ../todos-api
docker build -t todos-api-test .
docker run -p 8082:8082 todos-api-test
curl http://localhost:8082

# Test Users API
cd ../users-api
docker build -t users-api-test .
docker run -p 8083:8083 users-api-test
curl http://localhost:8083/health

# Test Log Processor
cd ../log-message-processor
docker build -t log-processor-test .
docker run log-processor-test
# (This might not have HTTP endpoint, check logs)

Common issues and fixes:

"Module not found" or "Package not found"
- Make sure you copied dependency files before installing
- Check that requirements.txt / package.json is in the right place
"Port already in use"
- Another container is using that port
- Use docker ps to see what's running
- Stop it with docker stop <container-id>
"Cannot connect to database" or "Connection refused"
- Services need to be in the same Docker network
- Use service names (e.g., redis) not localhost
- Wait for dependencies to be ready (use depends_on in docker-compose)
Image too large
- Use multi-stage builds (build in one stage, copy artifacts to smaller runtime stage)
- Use alpine or slim base images
- Remove build dependencies after installation

Creating docker-compose.yml

Now we need to orchestrate all these containers. That's where Docker Compose comes in.

Think of docker-compose.yml as a conductor's score - it tells all the musicians (containers) when to play, how to play together, and in what order.

Let's build it piece by piece:

Step 1: The Reverse Proxy (Traefik)

Traefik is like a smart receptionist at a hotel:

It receives all incoming requests (guests)
It looks at the URL and decides which service should handle it (which room)
It automatically gets SSL certificates from Let's Encrypt (security badges)
It handles HTTPS redirects (escorts HTTP guests to HTTPS)

services:
  # Reverse Proxy - Routes traffic to the right service
  traefik:
    image: traefik:latest
    container_name: traefik
    command:
      - "--api.insecure=true"  # Enable dashboard (for debugging)
      - "--providers.docker=true"  # Watch Docker containers
      - "--providers.docker.exposedbydefault=false"  # Only expose containers with labels
      - "--entrypoints.web.address=:80"  # HTTP entry point
      - "--entrypoints.websecure.address=:443"  # HTTPS entry point
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"  # Use HTTP challenge
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"  # Challenge on port 80
      - "--certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL:-your-email@example.com}"  # Email for cert
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"  # Where to store certs
    ports:
      - "80:80"    # HTTP
      - "443:443"  # HTTPS
      - "8080:8080"  # Traefik dashboard (for debugging)
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro  # Let Traefik see other containers
      - ./letsencrypt:/letsencrypt  # Store SSL certificates
    networks:
      - app-network
    restart: unless-stopped  # Auto-restart if it crashes

Key concepts:

ports: "80:80" means "map host port 80 to container port 80"
volumes: /var/run/docker.sock lets Traefik discover other containers automatically
networks: app-network puts Traefik on the same network as other services

Step 2: The Frontend Service

The Vue.js frontend needs to be built and served:

  # Frontend - Vue.js application
  frontend:
    build:
      context: ./frontend  # Where the Dockerfile is
      dockerfile: Dockerfile
    container_name: frontend
    environment:
      - PORT=80  # Port the app runs on inside container
      - AUTH_API_ADDRESS=http://auth-api:8081  # Use service name, not localhost!
      - TODOS_API_ADDRESS=http://todos-api:8082
    labels:
      # Tell Traefik to route traffic to this service
      - "traefik.enable=true"
      # Route requests for your domain to this service
      - "traefik.http.routers.frontend.rule=Host(`${DOMAIN:-yourdomain.com}`)"
      # Use HTTPS
      - "traefik.http.routers.frontend.entrypoints=websecure"
      # Get SSL certificate automatically
      - "traefik.http.routers.frontend.tls.certresolver=letsencrypt"
      # Frontend runs on port 80 inside container
      - "traefik.http.services.frontend.loadbalancer.server.port=80"
      # Redirect HTTP to HTTPS
      - "traefik.http.routers.frontend-redirect.rule=Host(`${DOMAIN:-yourdomain.com}`)"
      - "traefik.http.routers.frontend-redirect.entrypoints=web"
      - "traefik.http.routers.frontend-redirect.middlewares=redirect-to-https"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
    networks:
      - app-network
    depends_on:
      - auth-api
      - todos-api
      - users-api

Important points:

build: context: ./frontend tells Docker to build from the frontend folder
environment: sets variables the app can read
AUTH_API_ADDRESS=http://auth-api:8081 - Notice we use auth-api (the service name), not localhost!
depends_on: ensures these services start before frontend
Labels tell Traefik how to route traffic

Step 3: The Auth API (Go)

  # Auth API - Handles authentication
  auth-api:
    build:
      context: ./auth-api
      dockerfile: Dockerfile
    container_name: auth-api
    environment:
      - AUTH_API_PORT=8081
      - USERS_API_ADDRESS=http://users-api:8083
      - JWT_SECRET=${JWT_SECRET:-myfancysecret}  # Secret key for tokens
      - REDIS_URL=redis://redis:6379  # Redis connection
    labels:
      - "traefik.enable=true"
      # Route /api/auth requests to this service
      - "traefik.http.routers.auth.rule=Host(`${DOMAIN:-yourdomain.com}`) && PathPrefix(`/api/auth`)"
      - "traefik.http.routers.auth.entrypoints=websecure"
      - "traefik.http.routers.auth.tls.certresolver=letsencrypt"
      - "traefik.http.services.auth.loadbalancer.server.port=8081"
      # Also handle /login route (frontend calls this)
      - "traefik.http.routers.auth-login.rule=Host(`${DOMAIN:-yourdomian.com}`) && (Path(`/login`) || PathPrefix(`/login/`))"
      - "traefik.http.routers.auth-login.entrypoints=websecure"
      - "traefik.http.routers.auth-login.tls.certresolver=letsencrypt"
      - "traefik.http.routers.auth-login.service=auth"
    networks:
      - app-network
    depends_on:
      - redis  # Needs Redis for session storage

Routing explained:

PathPrefix(/api/auth) means any URL starting with /api/auth goes here
Example: https://your-domain.com/api/auth/login → auth-api
Path(/login) means exactly /login goes here

Step 4: The Todos API (Node.js)

  # Todos API - Manages todo items
  todos-api:
    build:
      context: ./todos-api
      dockerfile: Dockerfile
    container_name: todos-api
    environment:
      - PORT=8082
      - AUTH_API_URL=http://auth-api:8081  # To validate tokens
      - JWT_SECRET=${JWT_SECRET:-myfancysecret}
      - REDIS_URL=redis://redis:6379
    labels:
      - "traefik.enable=true"
      # Route /api/todos requests here
      - "traefik.http.routers.todos.rule=Host(`${DOMAIN:-yourdomain.com}`) && PathPrefix(`/api/todos`)"
      - "traefik.http.routers.todos.entrypoints=websecure"
      - "traefik.http.routers.todos.tls.certresolver=letsencrypt"
      - "traefik.http.services.todos.loadbalancer.server.port=8082"
    networks:
      - app-network
    depends_on:
      - redis
      - auth-api  # Needs auth-api to validate tokens

Step 5: The Users API (Java)

  # Users API - Manages user accounts
  users-api:
    build:
      context: ./users-api
      dockerfile: Dockerfile
    container_name: users-api
    environment:
      - SERVER_PORT=8083
      - JWT_SECRET=${JWT_SECRET:-myfancysecret}
      - REDIS_URL=redis://redis:6379
    labels:
      - "traefik.enable=true"
      # Route /api/users requests here
      - "traefik.http.routers.users.rule=Host(`${DOMAIN:-yourdomian.com}`) && PathPrefix(`/api/users`)"
      - "traefik.http.routers.users.entrypoints=websecure"
      - "traefik.http.routers.users.tls.certresolver=letsencrypt"
      - "traefik.http.services.users.loadbalancer.server.port=8083"
    networks:
      - app-network
    depends_on:
      - redis

Step 6: The Log Processor (Python)

This service doesn't need Traefik routing - it's a background worker:

  # Log Processor - Background worker that processes messages
  log-message-processor:
    build:
      context: ./log-message-processor
      dockerfile: Dockerfile
    container_name: log-message-processor
    environment:
      - REDIS_HOST=redis  # Use service name
      - REDIS_PORT=6379
      - REDIS_CHANNEL=log-messages
    networks:
      - app-network
    depends_on:
      - redis
    restart: unless-stopped  # Keep it running

Why no Traefik labels? This service doesn't serve HTTP requests - it just listens to Redis for messages.

Step 7: Supporting Services

  # Redis - Message queue and cache
  redis:
    image: redis:7-alpine  # Use pre-built image, no Dockerfile needed
    container_name: redis
    ports:
      - "6379:6379"  # Expose for debugging (optional)
    volumes:
      - redis-data:/data  # Persist data
    networks:
      - app-network
    restart: unless-stopped

  # Zipkin handler - Service for /zipkin endpoint
  zipkin-handler:
    image: nginx:alpine
    container_name: zipkin-handler
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.zipkin.rule=Host(`${DOMAIN:-yourdomain.com}`) && PathPrefix(`/zipkin`)"
      - "traefik.http.routers.zipkin.entrypoints=websecure"
      - "traefik.http.routers.zipkin.tls.certresolver=letsencrypt"
      - "traefik.http.services.zipkin.loadbalancer.server.port=80"
    networks:
      - app-network
    command: >
      sh -c "echo 'server {
        listen 80;
        location / {
          return 200 \"OK\";
          add_header Content-Type text/plain;
        }
      }' > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"

Step 8: Networks and Volumes

At the end of the file, define shared resources:

# Networks - How containers communicate
networks:
  app-network:
    driver: bridge  # Default network type

# Volumes - Persistent storage
volumes:
  redis-data:  # Named volume for Redis data

Why networks? Containers on the same network can talk to each other using service names (like auth-api instead of IP addresses).

Why volumes? Data in containers is lost when they're removed. Volumes persist data.

Complete docker-compose.yml Structure

Here's the mental model:

docker-compose.yml
├── services (all your containers)
│   ├── traefik (reverse proxy)
│   ├── frontend (Vue.js app)
│   ├── auth-api (Go service)
│   ├── todos-api (Node.js service)
│   ├── users-api (Java service)
│   ├── log-message-processor (Python worker)
│   ├── redis (database/queue)
│   └── zipkin-handler (dummy endpoint)
├── networks (how they connect)
│   └── app-network
└── volumes (persistent storage)
    └── redis-data

Understanding Traefik Labels (Deep Dive)

Labels are how you tell Traefik what to do. Let's break down a complex example:

labels:
  - "traefik.enable=true"  # Step 1: Enable Traefik for this service
  - "traefik.http.routers.auth.rule=Host(`example.com`) && PathPrefix(`/api/auth`)"  # Step 2: Define routing rule
  - "traefik.http.routers.auth.entrypoints=websecure"  # Step 3: Use HTTPS
  - "traefik.http.routers.auth.tls.certresolver=letsencrypt"  # Step 4: Get SSL cert
  - "traefik.http.services.auth.loadbalancer.server.port=8081"  # Step 5: Which port to forward to

Breaking it down:

Router = A set of rules for routing traffic
Rule = Conditions that must match (domain + path)
Entrypoint = Which port/protocol (web = HTTP, websecure = HTTPS)
Service = The actual container and port
Middleware = Transformations (redirects, rewrites, etc.)

Example flow:

User visits https://example.com/api/auth/login
Traefik receives request on port 443 (websecure entrypoint)
Traefik checks rules: "Does this match Host(example.com) && PathPrefix(/api/auth)?" → Yes!
Traefik forwards to auth service on port 8081
Auth-api container handles the request

Testing Your docker-compose.yml

Before deploying to the cloud, test locally:

Step 1: Create Environment File

Create a .env file in the root directory:

cat > .env <<EOF
DOMAIN=localhost
LETSENCRYPT_EMAIL=your-email@example.com
JWT_SECRET=test-secret-key-change-this-in-production
EOF

What's in .env?

DOMAIN - Your domain name (use localhost for local testing)
LETSENCRYPT_EMAIL - Email for SSL certificate notifications
JWT_SECRET - Secret key for JWT tokens (use a strong random string in production)

Step 2: Start All Services

# Build and start all containers in the background
docker compose up -d

# Watch all logs in real-time
docker compose logs -f

# Or watch specific service logs
docker compose logs -f frontend
docker compose logs -f auth-api

What -d means: Detached mode - runs in the background so you can use your terminal.

Step 3: Verify Everything is Running

# Check status of all containers
docker compose ps

# You should see something like:
# NAME                    STATUS              PORTS
# traefik                 Up 2 minutes        0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp
# frontend                Up 2 minutes        
# auth-api                Up 2 minutes        
# todos-api               Up 2 minutes        
# users-api               Up 2 minutes        
# log-message-processor   Up 2 minutes        
# redis                   Up 2 minutes        0.0.0.0:6379->6379/tcp

Step 4: Test Each Service

# Test frontend (should return HTML)
curl http://localhost

# Test auth API (should return "Not Found" - that's expected!)
curl http://localhost/api/auth

# Test todos API (should return "Invalid Token" - also expected!)
curl http://localhost/api/todos

# Test users API
curl http://localhost/api/users

# Check Traefik dashboard (optional)
# Open http://localhost:8080 in your browser

Expected responses:

Frontend: HTML page (login screen)
/api/auth without path: "Not Found" (correct - needs specific endpoint)
/api/todos without auth: "Invalid Token" (correct - needs authentication)
/api/users without auth: "Missing or invalid Authorization header" (correct)

Step 5: Test with Browser

Open http://localhost in your browser
You should see the login page
Try logging in (if you have test credentials)
Check browser console (F12) for any errors

Step 6: Check Logs for Errors

# View logs for a specific service
docker compose logs frontend
docker compose logs auth-api
docker compose logs traefik

# View last 100 lines
docker compose logs --tail=100 traefik

# Follow logs in real-time
docker compose logs -f traefik

What to look for:

✅ "Server started" or "Listening on port X" = Good!
❌ "Connection refused" = Service dependency not ready
❌ "Module not found" = Missing dependency in Dockerfile
❌ "Port already in use" = Another service is using that port

Step 7: Stop Everything

# Stop all containers
docker compose down

# Stop and remove volumes (clean slate)
docker compose down -v

# Stop and remove images too (full cleanup)
docker compose down --rmi all -v

Common Issues and Solutions

Issue 1: "Port 80 already in use"

Problem: Another service (like Apache, Nginx, or another Docker container) is using port 80.

Solution:

# Find what's using port 80
sudo lsof -i :80
# or
sudo netstat -tulpn | grep :80

# Stop the conflicting service
sudo systemctl stop apache2  # or nginx, or whatever it is

# Or change Traefik ports in docker-compose.yml:
ports:
  - "8080:80"   # Use 8080 instead of 80
  - "8443:443"  # Use 8443 instead of 443

Issue 2: "Build failed" or "Module not found"

Problem: Dockerfile has issues or dependencies are missing.

Solution:

# Build a specific service to see detailed errors
docker compose build frontend

# Check the Dockerfile syntax
# Make sure COPY commands are in the right order
# Make sure RUN commands install dependencies before copying code

Issue 3: "Container keeps restarting"

Problem: The application is crashing on startup.

Solution:

# Check why it's restarting
docker compose logs <service-name>

# Common causes:
# - Missing environment variables
# - Database/Redis not ready (add depends_on)
# - Port conflict
# - Missing files or dependencies

Issue 4: "Cannot connect to auth-api" or "Connection refused"

Problem: Services can't find each other.

Solution:

✅ Use service names (e.g., http://auth-api:8081), not localhost
✅ Make sure all services are on the same network (app-network)
✅ Check depends_on - services might be starting before dependencies are ready
✅ Add health checks or wait scripts if needed

Issue 5: "SSL certificate error" or "Let's Encrypt failed"

Problem: Let's Encrypt can't verify your domain.

Solution:

For local testing: Use localhost and HTTP only (remove HTTPS redirect)
For production: Make sure DNS points to your server
Make sure ports 80 and 443 are open in firewall
Check Traefik logs: docker compose logs traefik

Quick Reference: docker-compose Commands

# Start services
docker compose up              # Start and show logs
docker compose up -d           # Start in background

# Stop services
docker compose stop            # Stop but don't remove
docker compose down           # Stop and remove containers

# View logs
docker compose logs            # All services
docker compose logs -f         # Follow (live updates)
docker compose logs <service>  # Specific service

# Rebuild
docker compose build           # Build all
docker compose build <service> # Build specific service
docker compose up --build      # Build and start

# Status
docker compose ps              # Show running containers
docker compose top             # Show running processes

# Execute commands
docker compose exec <service> <command>  # Run command in container
docker compose exec frontend sh           # Get shell in frontend container

Part 4: Infrastructure as Code with Terraform

Now comes the infrastructure part. This is where many people get intimidated, but it's actually simpler than it seems.

Why Infrastructure as Code?

Imagine you're building a house. You could:

Manual approach: Tell the builder "put a window here, a door there" every time
Blueprint approach: Draw a blueprint once, builder follows it every time

Infrastructure as Code is the blueprint approach. Benefits:

Reproducible: Same code = same infrastructure, every time
Version controlled: See what changed and when
Testable: Try changes without breaking production
Documented: The code IS the documentation

Understanding Terraform Basics

Terraform uses a language called HCL (HashiCorp Configuration Language). It's designed to be human-readable.

Basic structure:

resource "aws_instance" "todo_app" {
  ami           = "ami-12345"
  instance_type = "t3.medium"
}

This says: "Create an AWS EC2 instance resource, call it 'todo_app', with these properties."

Creating Your First Terraform Configuration

Let's build it step by step:

Step 1: Provider Configuration

terraform {
  required_version = ">= 1.5.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    # We'll configure this during init
  }
}

provider "aws" {
  region = var.aws_region
}

What's happening:

required_version - Ensures everyone uses compatible Terraform
required_providers - Tells Terraform which plugins to download
backend "s3" - Where to store state (we'll configure this later)
provider "aws" - Which cloud provider to use

Step 2: Data Sources (Getting Information)

Before creating resources, we often need to look things up:

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"]  # Canonical (Ubuntu's publisher)

  filter {
    name   = "name"
    values = ["*ubuntu-jammy-22.04-amd64-server*"]
  }
}

What is an AMI? Amazon Machine Image - it's like a template for a virtual machine. This code finds the latest Ubuntu 22.04 image.

Why use data sources? AMI IDs change in different regions. Instead of hardcoding ami-12345, we let Terraform find the right one.

Step 3: Security Group (Firewall Rules)

resource "aws_security_group" "todo_app" {
  name        = "todo-app-sg-${var.environment}"
  description = "Security group for TODO application"

  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]  # Allow from anywhere
  }

  ingress {
    description = "HTTPS"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = [var.ssh_cidr]  # Only from your IP (security!)
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"  # All protocols
    cidr_blocks = ["0.0.0.0/0"]  # Allow all outbound
  }

  tags = {
    Name        = "todo-app-sg-${var.environment}"
    Environment = var.environment
  }
}

What is a security group? It's AWS's firewall. It controls what traffic can reach your server.

Breaking it down:

ingress - Incoming traffic rules
egress - Outgoing traffic rules
cidr_blocks = ["0.0.0.0/0"] - From anywhere (0.0.0.0/0 means "everywhere")
var.ssh_cidr - A variable (we'll set this to your IP for security)

Security tip: In production, restrict SSH to your IP only! Use a service like whatismyip.com to find your IP, then set ssh_cidr = "YOUR_IP/32".

Step 4: EC2 Instance (Your Server)

resource "aws_instance" "todo_app" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  key_name               = var.key_pair_name
  vpc_security_group_ids = [aws_security_group.todo_app.id]

  user_data = <<-EOF
    #!/bin/bash
    apt-get update
    apt-get install -y python3 python3-pip
  EOF

  tags = {
    Name        = "todo-app-server-${var.environment}"
    Environment = var.environment
    Project     = "hngi13-stage6"
  }

  lifecycle {
    create_before_destroy = true
  }
}

What's happening:

ami - Which OS image to use (from our data source)
instance_type - Server size (t3.medium = 2 vCPU, 4GB RAM)
key_name - Which SSH key to install
vpc_security_group_ids - Which firewall rules to apply
user_data - Script that runs when server starts (bootstrap script)
lifecycle - Terraform behavior (create new before destroying old = zero downtime)

Step 5: Variables (Making It Flexible)

variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t3.medium"
}

variable "key_pair_name" {
  description = "AWS Key Pair name"
  type        = string  # Required - no default
}

variable "environment" {
  description = "Environment name (dev, stg, prod)"
  type        = string
  default     = "dev"

  validation {
    condition     = contains(["dev", "stg", "prod"], var.environment)
    error_message = "Environment must be one of: dev, stg, prod"
  }
}

Why variables? Makes your code reusable. Same code works for dev, staging, and production - just change the variables!

Step 6: Outputs (Getting Information Back)

output "server_ip" {
  description = "Public IP of the server"
  value       = aws_instance.todo_app.public_ip
}

output "ansible_inventory_path" {
  description = "Path to generated Ansible inventory"
  value       = local_file.ansible_inventory.filename
}

What are outputs? After Terraform creates resources, you often need information about them (like the server IP). Outputs make that information available.

Environment-Specific Configuration

Create separate .tfvars files for each environment. This is crucial - you'll have three files, one for each environment:

terraform.dev.tfvars:

environment = "dev"
aws_region  = "us-east-1"
instance_type = "t3.small"  # Smaller for dev (saves money)
key_pair_name = "my-terraform-key"
ssh_key_path = "~/.ssh/my-terraform-key.pem"
ssh_cidr = "0.0.0.0/0"  # Less restrictive for dev
server_user = "ubuntu"
skip_ansible_provision = false  # Run Ansible automatically

terraform.stg.tfvars:

environment = "stg"
aws_region  = "us-east-1"
instance_type = "t3.small"  # Can be same size as dev for staging
key_pair_name = "my-terraform-key"
ssh_key_path = "~/.ssh/my-terraform-key.pem"
ssh_cidr = "0.0.0.0/0"  # Can be less restrictive than prod
server_user = "ubuntu"
skip_ansible_provision = false

terraform.prod.tfvars:

environment = "prod"
aws_region  = "us-east-1"
instance_type = "t3.medium"  # More power for production
key_pair_name = "my-terraform-key"
ssh_key_path = "~/.ssh/my-terraform-key.pem"
ssh_cidr = "YOUR_IP/32"  # Restrict SSH to your IP only!
server_user = "ubuntu"
skip_ansible_provision = false

Why three separate files? Different environments have different needs:

Dev: Smaller instance, less security (for quick testing)
Staging: Similar to dev, but closer to production setup (for pre-production testing)
Prod: Larger instance, maximum security (for real users)

File structure:

infra/terraform/
├── main.tf
├── variables.tf
├── outputs.tf
├── terraform.dev.tfvars    ← Development environment
├── terraform.stg.tfvars    ← Staging environment
└── terraform.prod.tfvars   ← Production environment

Remote State Configuration

Remember the S3 bucket we created? Now we use it. Important: Each environment needs its own state file path!

For Development:

terraform init \
  -backend-config="bucket=yourname-terraform-state" \
  -backend-config="key=terraform-state/dev/terraform.tfstate" \
  -backend-config="region=us-east-1" \
  -backend-config="dynamodb_table=terraform-state-lock" \
  -backend-config="encrypt=true"

For Staging:

terraform init \
  -backend-config="bucket=yourname-terraform-state" \
  -backend-config="key=terraform-state/stg/terraform.tfstate" \
  -backend-config="region=us-east-1" \
  -backend-config="dynamodb_table=terraform-state-lock" \
  -backend-config="encrypt=true"

For Production:

terraform init \
  -backend-config="bucket=yourname-terraform-state" \
  -backend-config="key=terraform-state/prod/terraform.tfstate" \
  -backend-config="region=us-east-1" \
  -backend-config="dynamodb_table=terraform-state-lock" \
  -backend-config="encrypt=true"

What this does:

bucket - Where to store state (same bucket for all environments)
key - File path in bucket (different per environment!)
dynamodb_table - For locking (prevents conflicts when multiple people run Terraform)
encrypt=true - Encrypt state at rest (security)

Why separate keys per environment?

terraform-state/dev/terraform.tfstate → Development infrastructure
terraform-state/stg/terraform.tfstate → Staging infrastructure
terraform-state/prod/terraform.tfstate → Production infrastructure

These are completely separate files. This means:

✅ Dev, staging, and prod infrastructure are isolated
✅ You can destroy dev without affecting staging or prod
✅ Each environment has its own state history
✅ No risk of accidentally modifying the wrong environment

Your First Terraform Run

Let's deploy to development first (always start with dev!):

# 1. Initialize (downloads providers, sets up backend)
terraform init \
  -backend-config="bucket=yourname-terraform-state" \
  -backend-config="key=terraform-state/dev/terraform.tfstate" \
  -backend-config="region=us-east-1" \
  -backend-config="dynamodb_table=terraform-state-lock" \
  -backend-config="encrypt=true"

# 2. Plan (see what will be created - SAFE, doesn't change anything)
terraform plan -var-file=terraform.dev.tfvars

# 3. Apply (actually create resources)
terraform apply -var-file=terraform.dev.tfvars

What happens:

terraform init - Downloads AWS provider, configures backend for dev environment
terraform plan - Shows you what will be created/changed/destroyed (dry run)
terraform apply - Actually creates the resources

Pro tip: Always run plan first! It's like a dry run. Review the output carefully before applying.

For other environments, repeat the same steps but:

Use the appropriate -backend-config="key=terraform-state/ENV/terraform.tfstate"
Use the matching -var-file=terraform.ENV.tfvars

Example for staging:

terraform init \
  -backend-config="bucket=yourname-terraform-state" \
  -backend-config="key=terraform-state/stg/terraform.tfstate" \
  -backend-config="region=us-east-1" \
  -backend-config="dynamodb_table=terraform-state-lock" \
  -backend-config="encrypt=true"

terraform plan -var-file=terraform.stg.tfvars
terraform apply -var-file=terraform.stg.tfvars

Understanding Drift Detection (Critical for Safety!)

What is drift? Imagine you have a blueprint for a house (Terraform code), but someone goes and changes the actual house (AWS infrastructure) without updating the blueprint. That's drift - your code and reality don't match anymore.

Real-world example:

You deploy infrastructure with Terraform ✅
Later, you manually add a tag to your EC2 instance in AWS Console 🏷️
Terraform doesn't know about this change
Next time you run Terraform, it sees the difference → DRIFT DETECTED!

Why drift is dangerous:

🔴 Security risk: Someone might have changed something maliciously
🔴 Data loss: Terraform might try to "fix" things and delete your changes
🔴 Confusion: You don't know what changed or why
🔴 Breaking changes: Manual changes might break your application

How drift detection works:

Think of it like a security guard checking your house:

Terraform Plan = Security guard walks around and notes what's different
Check Git History = Did you change the blueprint (Terraform files)?
- ✅ If yes → Expected changes (you updated the code)
- ❌ If no → DRIFT! (Someone changed infrastructure without updating code)

Alert = Security guard calls you immediately

Approval = You review and decide what to do

Action = Apply changes or investigate further

The detection logic:

# Step 1: Run terraform plan to see what's different
terraform plan -out=tfplan

# Step 2: Check if Terraform files changed in this commit
git diff HEAD~1 HEAD --name-only | grep -E "\.tf$|\.tfvars$"

# Step 3: Determine the type of change
if [ no terraform files changed ] && [ plan shows changes ]; then
  echo "🚨 DRIFT DETECTED!"
  # Send email, create GitHub issue, wait for approval
else
  echo "✅ Expected changes (code was updated)"
  # Proceed automatically
fi

Setting Up Email Notifications for Drift

When drift is detected, you need to know immediately! That's where email notifications come in.

Step 1: Verify Your Email in AWS SES

AWS SES (Simple Email Service) is like a post office for your applications. First, you need to verify your email address:

Go to AWS Console → SES (Simple Email Service)
Click "Verified identities" → "Create identity"
Choose "Email address"
Enter your email (e.g., your-email@gmail.com)
Click "Create identity"
Check your email and click the verification link

Why verify? AWS prevents spam by requiring you to verify you own the email address.

Step 2: Create the Email Notification Script

Create infra/ci-cd/scripts/email-notification.sh:

#!/bin/bash
# Email Notification Script for Terraform Drift
# Sends email alert when infrastructure drift is detected

set -e

DRIFT_SUMMARY="${1:-}"

if [ -z "$DRIFT_SUMMARY" ]; then
  echo "Error: Drift summary not provided"
  exit 1
fi

# Email configuration from environment variables
EMAIL_TO="${EMAIL_TO:-}"
EMAIL_FROM="${EMAIL_FROM:-}"
AWS_REGION="${AWS_REGION:-us-east-1}"

# GitHub Actions variables for workflow link
GITHUB_SERVER_URL="${GITHUB_SERVER_URL:-https://github.com}"
GITHUB_REPOSITORY="${GITHUB_REPOSITORY:-}"
GITHUB_RUN_ID="${GITHUB_RUN_ID:-}"

# Check if email is configured
if [ -z "$EMAIL_TO" ] || [ -z "$EMAIL_FROM" ]; then
  echo "⚠️  Email not configured. Skipping email notification."
  exit 0
fi

# Build workflow run URL if GitHub variables are available
WORKFLOW_URL=""
if [ -n "$GITHUB_REPOSITORY" ] && [ -n "$GITHUB_RUN_ID" ]; then
  WORKFLOW_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
fi

# Create email body
SUBJECT="🚨 Terraform Drift Detected - Action Required"
BODY=$(cat <<EOF
Terraform infrastructure drift has been detected.

This means infrastructure was changed OUTSIDE of Terraform (e.g., manually in AWS Console).

Please review the changes and approve the deployment in GitHub Actions.

$(if [ -n "$WORKFLOW_URL" ]; then echo "🔗 View Workflow Run: $WORKFLOW_URL"; echo ""; fi)

Drift Summary:
$DRIFT_SUMMARY

---
This is an automated message from GitHub Actions.
EOF
)

# Send email via AWS SES
echo "📧 Sending drift alert email via AWS SES..."
aws ses send-email \
  --region "$AWS_REGION" \
  --from "$EMAIL_FROM" \
  --to "$EMAIL_TO" \
  --subject "$SUBJECT" \
  --text "$BODY" \
  || echo "⚠️  Failed to send email. Check AWS credentials and SES configuration."

echo "✅ Email notification sent!"

Make it executable:

chmod +x infra/ci-cd/scripts/email-notification.sh

What this script does:

Takes the drift summary as input
Checks if email is configured
Builds a nice email message with the drift details
Sends it via AWS SES
Includes a link to the GitHub workflow run

Step 3: Add GitHub Secrets

In your GitHub repository, go to Settings → Secrets and variables → Actions, and add:

EMAIL_TO - Your email address (where to send alerts)
EMAIL_FROM - Your verified SES email (must match the one you verified in AWS)
AWS_ACCESS_KEY_ID - Your AWS access key
AWS_SECRET_ACCESS_KEY - Your AWS secret key
AWS_REGION - Your AWS region (e.g., us-east-1)

Security tip: Never commit these values to your repository! Always use GitHub Secrets.

Understanding GitHub Issue Approval

When drift is detected, the workflow creates a GitHub issue and waits for your approval. This is like a safety checkpoint.

How it works:

Drift detected → Workflow pauses
GitHub issue created → Contains:
- What changed
- Why it's drift (no code changes)
- Link to workflow run
- Plan summary
You review → Check the issue
You approve → Comment "approve" or click approve button
Workflow continues → Terraform applies the changes

Example GitHub Issue:

🚨 REAL DRIFT DETECTED - Infrastructure Changed Outside Terraform (dev)

⚠️ CRITICAL: Real Infrastructure Drift Detected

Infrastructure has been modified outside of Terraform. This is unexpected.

Environment: dev

What happened:
- Terraform code files were NOT modified
- But infrastructure plan shows changes
- This indicates manual changes or changes from another process

Action Required:
1. Review the plan below
2. Investigate what caused the drift
3. Approve if changes are intentional, or revert if unauthorized

Plan Summary:

aws_instance.todo_app will be updated in-place

~ resource "aws_instance" "todo_app" {
~ tags = {
- "ManualTag" = "test" -> null
}
}


Workflow Run:
🔗 View Workflow Run: https://github.com/yourusername/repo/actions/runs/123456

Next Steps:
- Approve to apply these changes
- Or investigate and revert unauthorized changes

How to approve:

Go to the GitHub issue (you'll get a notification)
Review the changes carefully
If changes are OK: Comment "approve" or click the approve button
If changes are suspicious: Investigate first, then approve or revert

Why this matters:

✅ Prevents accidental changes
✅ Gives you time to investigate
✅ Creates an audit trail (who approved what, when)
✅ Protects production from unauthorized changes

Testing Drift Detection

Want to test if drift detection works? Here's how:

Step 1: Deploy infrastructure normally

terraform apply -var-file=terraform.dev.tfvars

Step 2: Manually change something in AWS Console

Go to AWS Console → EC2 → Instances
Find your instance
Click "Tags" → "Manage tags"
Add a new tag: TestTag = "drift-test"
Save

Added port 8080 via AWS console

Step 3: Trigger the workflow

Go to GitHub Actions
Run the "Infrastructure Deployment" workflow
Select "dev" environment
Watch it detect drift!

Step 4: Check your email

You should receive an email alert
Check spam folder if you don't see it

Step 5: Approve in GitHub

A GitHub issue should be created
Review and approve
Watch Terraform apply the changes

Approved github issue

Destroying Infrastructure (When You Need to Start Over)

Sometimes you need to tear everything down and start fresh. Here's how to do it safely.

⚠️ WARNING: Destroying infrastructure will DELETE EVERYTHING:

Your EC2 instance
All data on the server
Security groups
Everything created by Terraform

Make sure you:

✅ Have backups if you need data
✅ Are destroying the right environment (dev, not prod!)
✅ Really want to delete everything

Method 1: Destroy via Command Line

# Step 1: Initialize Terraform (if not already done)
cd infra/terraform
terraform init \
  -backend-config="bucket=yourname-terraform-state" \
  -backend-config="key=terraform-state/dev/terraform.tfstate" \
  -backend-config="region=us-east-1" \
  -backend-config="dynamodb_table=terraform-state-lock" \
  -backend-config="encrypt=true"

# Step 2: Plan the destruction (see what will be deleted)
terraform plan -destroy -var-file=terraform.dev.tfvars

# Step 3: Review the plan carefully!
# Make sure it's only deleting what you want

# Step 4: Destroy everything
terraform destroy -var-file=terraform.dev.tfvars

What happens:

Terraform reads the state file
Plans what needs to be destroyed
Shows you the plan (review it!)
Asks for confirmation (type yes)
Deletes everything in reverse order

Method 2: Destroy via GitHub Actions

If you have a destroy workflow set up:

Go to GitHub Actions

Find "Destroy Infrastructure" workflow (if you have one)
Click "Run workflow"
Select environment (be careful - don't destroy prod!)
Confirm and run

Example destroy workflow:

name: Destroy Infrastructure

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to destroy'
        required: true
        type: choice
        options: [dev, stg, prod]

jobs:
  destroy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3

      - name: Terraform Init
        working-directory: infra/terraform
        run: |
          terraform init \
            -backend-config="bucket=${{ secrets.TERRAFORM_STATE_BUCKET }}" \
            -backend-config="key=terraform-state/${{ github.event.inputs.environment }}/terraform.tfstate" \
            -backend-config="region=${{ secrets.AWS_REGION }}" \
            -backend-config="dynamodb_table=${{ secrets.TERRAFORM_STATE_LOCK_TABLE }}" \
            -backend-config="encrypt=true"

      - name: Terraform Plan Destroy
        working-directory: infra/terraform
        run: |
          terraform plan -destroy \
            -var-file=terraform.${{ github.event.inputs.environment }}.tfvars \
            -var="key_pair_name=${{ secrets.TERRAFORM_KEY_PAIR_NAME }}" \
            -out=tfplan

      - name: Manual Approval
        uses: trstringer/manual-approval@v1
        with:
          secret: ${{ github.TOKEN }}
          approvers: ${{ github.actor }}
          minimum-approvals: 1
          issue-title: "⚠️ DESTROY Infrastructure - ${{ github.event.inputs.environment }}"
          issue-body: |
            **⚠️ WARNING: Infrastructure Destruction Requested**

            This will **DELETE ALL INFRASTRUCTURE** for environment: **${{ github.event.inputs.environment }}**

            **This action cannot be undone!**

            Review the plan and approve only if you're sure.

      - name: Terraform Destroy
        if: steps.manual-approval.outcome == 'success'
        working-directory: infra/terraform
        run: terraform apply -auto-approve tfplan

Safety features:

✅ Manual approval required (can't destroy by accident)
✅ Shows what will be destroyed
✅ Creates GitHub issue for review
✅ Environment selection (prevents destroying wrong env)

Method 3: Destroy Specific Resources

Don't want to destroy everything? You can target specific resources:

# Destroy only the EC2 instance (keep security group)
terraform destroy -target=aws_instance.todo_app -var-file=terraform.dev.tfvars

# Destroy only the security group
terraform destroy -target=aws_security_group.todo_app -var-file=terraform.dev.tfvars

When to use this:

You want to recreate just one resource
Something is broken and you want to rebuild it
You're testing changes

After Destruction

After destroying, your state file still exists in S3, but it's empty (or has no resources). You can:

Start fresh: Run terraform apply again to recreate everything
Clean up state: Delete the state file from S3 (optional)
Keep state: Leave it (Terraform will just create new resources)

Best practice: Keep the state file. It's useful for history and doesn't cost much.

Summary: The Complete Terraform Workflow

Here's the full picture of how everything works together:

┌─────────────────────────────────────────────────────────┐
│ 1. You make changes to Terraform code                   │
│    (or someone changes infrastructure manually)        │
└─────────────────┬───────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────┐
│ 2. GitHub Actions workflow runs                         │
│    - Checks out code                                    │
│    - Runs terraform plan                                │
└─────────────────┬───────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────┐
│ 3. Drift Detection                                      │
│    - Did Terraform files change?                        │
│    - Does plan show changes?                            │
│    - If no code changes + plan changes = DRIFT!         │
└─────────────────┬───────────────────────────────────────┘
                  │
        ┌─────────┴─────────┐
        │                   │
        ▼                   ▼
┌──────────────┐   ┌──────────────────┐
│ DRIFT        │   │ EXPECTED CHANGES  │
│ DETECTED     │   │ (Code updated)    │
└──────┬───────┘   └────────┬───────────┘
       │                    │
       ▼                    ▼
┌──────────────────┐   ┌──────────────────┐
│ Send Email       │   │ Apply directly   │
│ Create Issue     │   │ (No approval)    │
│ Wait for Approval│   └──────────────────┘
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ You Review       │
│ & Approve        │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Apply Changes    │
│ (terraform apply) │
└──────────────────┘

Key takeaways:

✅ Always run terraform plan first (see what will happen)
✅ Drift detection protects you from unexpected changes
✅ Email notifications keep you informed
✅ Manual approval prevents accidents
✅ Destroy carefully - it's permanent!

Part 5: Server Configuration with Ansible

Terraform created your server, but it's just a blank Ubuntu machine. Now we need to:

Install Docker
Clone your code
Start the application

That's where Ansible comes in.

Understanding Ansible

Ansible is like having a robot assistant that can:

SSH into your servers
Run commands
Install software
Copy files
Start services

Why Ansible over SSH scripts?

Idempotent: Run it multiple times safely (won't break if run twice)
Declarative: You say "what" you want, not "how" to do it
Organized: Roles and playbooks keep things organized
Reusable: Write once, use for dev/stg/prod

Ansible Playbook Structure

---
- name: Configure TODO Application Server
  hosts: all
  become: yes  # Use sudo
  gather_facts: yes  # Collect info about the server

  vars:
    app_user: ubuntu
    app_dir: /opt/todo-app

  roles:
    - role: dependencies  # Install Docker, etc.
    - role: deploy        # Deploy the application

Breaking it down:

hosts: all - Run on all servers in inventory
become: yes - Use sudo (needed for installing packages)
gather_facts - Ansible learns about the server (OS, IP, etc.)
roles - Reusable collections of tasks

Creating the Dependencies Role

This role installs everything the server needs:

roles/dependencies/tasks/main.yml:

---
- name: Update apt cache
  apt:
    update_cache: yes
    cache_valid_time: 3600

- name: Install required packages
  apt:
    name:
      - git
      - curl
      - python3
      - python3-pip
    state: present

- name: Check if Docker is already installed
  command: docker --version
  register: docker_check
  changed_when: false
  failed_when: false
  ignore_errors: yes

- name: Install Docker (only if not installed)
  apt:
    name:
      - docker-ce
      - docker-ce-cli
      - containerd.io
      - docker-compose-plugin
    state: present
  when: docker_check.rc != 0  # Only if Docker not found

- name: Add user to docker group
  user:
    name: "{{ app_user }}"
    groups: docker
    append: yes

- name: Start and enable Docker
  systemd:
    name: docker
    state: started
    enabled: yes

Key concepts:

register - Save command output to a variable
when - Conditional execution (only if condition is true)
changed_when: false - This task never "changes" anything (just checks)
state: present - Ensure package is installed (idempotent!)

Creating the Deploy Role

This role actually deploys your application:

roles/deploy/tasks/main.yml:

---
- name: Create application directory
  file:
    path: "{{ app_dir }}"
    state: directory
    owner: "{{ app_user }}"
    group: "{{ app_user }}"
    mode: '0755'

- name: Clone repository
  git:
    repo: "{{ repo_url }}"
    dest: "{{ app_dir }}"
    version: "{{ repo_branch | default('main') }}"
    update: yes
  register: git_pull_result
  changed_when: git_pull_result.changed

- name: Create .env file
  copy:
    content: |
      DOMAIN="{{ domain }}"
      LETSENCRYPT_EMAIL="{{ letsencrypt_email }}"
      JWT_SECRET="{{ jwt_secret }}"
      # ... other variables
    dest: "{{ app_dir }}/.env"
    owner: "{{ app_user }}"
    mode: '0600'
  register: env_file_result
  changed_when: env_file_result.changed

- name: Determine if rebuild is needed
  set_fact:
    needs_rebuild: "{{ git_pull_result.changed | default(false) or env_file_result.changed | default(false) }}"

- name: Build images if code/config changed
  shell: docker compose build
  args:
    chdir: "{{ app_dir }}"
  when: needs_rebuild | default(false)

- name: Start/update containers
  shell: docker compose up -d
  args:
    chdir: "{{ app_dir }}"

Making it idempotent:

Only rebuilds if code or config changed
docker compose up -d is idempotent (won't restart if nothing changed)
Safe to run multiple times

Environment-Specific Variables

Just like Terraform, Ansible needs separate configuration files for each environment. Create three files:

group_vars/dev/vars.yml:

---
domain: "dev.yourdomain.com"
letsencrypt_email: "dev-email@example.com"
jwt_secret: "dev-secret-key"
repo_url: "https://github.com/yourusername/path-to-codebase.git"
repo_branch: "dev"  # Use dev branch for development

group_vars/stg/vars.yml:

---
domain: "stg.yourdomain.com"
letsencrypt_email: "staging-email@example.com"
jwt_secret: "staging-secret-key"  # Different from dev!
repo_url: "https://github.com/yourusername/path-to-codebase.git"
repo_branch: "staging"  # Use staging branch for staging

group_vars/prod/vars.yml:

---
domain: "yourdomain.com"
letsencrypt_email: "prod-email@example.com"
jwt_secret: "super-secure-production-secret"  # Different per environment!
repo_url: "https://github.com/yourusername/path-to-codebase.git"
repo_branch: "main"  # Use main branch for production

Why three separate files? Each environment needs:

Different domains: dev.yourdomain.com, stg.yourdomain.com, yourdomain.com
Different secrets: If dev gets compromised, staging and prod are still safe
Different branches:
- dev branch → development environment (experimental features)
- staging branch → staging environment (testing before production)
- main branch → production environment (stable, tested code)

File structure:

infra/ansible/
├── playbook.yml
├── inventory/
│   ├── dev.yml
│   ├── stg.yml
│   └── prod.yml
└── group_vars/
    ├── dev/
    │   └── vars.yml      ← Development variables
    ├── stg/
    │   └── vars.yml      ← Staging variables
    └── prod/
        └── vars.yml      ← Production variables

Branch strategy explained:

Development (dev branch): Where you experiment and develop new features
Staging (staging branch): Where you test features before they go to production
Production (main branch): The stable code that real users interact with

This way, you can test changes in dev/staging without affecting production!

Generating Inventory

Terraform automatically generates the Ansible inventory:

templates/inventory.tpl:

all:
  hosts:
    todo-app-server:
      ansible_host: ${server_ip}
      ansible_user: ${server_user}
      ansible_ssh_private_key_file: ${ssh_key_path}
      ansible_ssh_common_args: '-o StrictHostKeyChecking=no'

This gets generated as ansible/inventory/dev.yml (or stg.yml, prod.yml) with the actual server IP.

Running Ansible

# From the ansible directory
cd infra/ansible

# Run the playbook
ansible-playbook -i inventory/dev.yml playbook.yml

# With verbose output (for debugging)
ansible-playbook -i inventory/dev.yml playbook.yml -vvv

What happens:

Ansible connects to your server via SSH
Runs the dependencies role (installs Docker)
Runs the deploy role (clones code, starts containers)
Your application is live!

Part 6: CI/CD with GitHub Actions

Now we automate everything. Instead of running commands manually, GitHub Actions does it for us.

Understanding CI/CD

CI (Continuous Integration): Automatically test and build when code changes
CD (Continuous Deployment): Automatically deploy when tests pass

Why CI/CD?

Consistency: Same process every time
Speed: Deploy in minutes, not hours
Safety: Automated tests catch bugs before production
History: See what was deployed when

Setting Up GitHub Secrets

Before workflows can run, they need credentials:

Go to your GitHub repo → Settings → Secrets and variables → Actions
Add these secrets:
- AWS_ACCESS_KEY_ID - From your IAM user
- AWS_SECRET_ACCESS_KEY - From your IAM user
- TERRAFORM_STATE_BUCKET - Your S3 bucket name
- TERRAFORM_STATE_LOCK_TABLE - Your DynamoDB table name
- TERRAFORM_KEY_PAIR_NAME - Your EC2 key pair name
- SSH_PRIVATE_KEY - Contents of your .pem file
- EMAIL_TO - Where to send drift alerts
- EMAIL_FROM - Your verified SES email

Infrastructure Workflow

This workflow runs when infrastructure code changes:

name: Infrastructure Deployment

on:
  push:
    paths:
      - 'infra/terraform/**'
      - 'infra/ansible/**'
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment (dev, stg, prod)'
        required: true
        type: choice
        options: [dev, stg, prod]

jobs:
  terraform-plan:
    name: Terraform Plan & Drift Detection
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3

      - name: Terraform Init
        working-directory: infra/terraform
        run: terraform init -backend-config=...

      - name: Terraform Plan
        run: terraform plan -out=tfplan

      - name: Check for Drift
        id: drift-check
        run: |
          # Detect if this is drift (infrastructure changed outside Terraform)
          # vs expected changes (Terraform code changed)

      - name: Send Drift Email
        if: steps.drift-check.outputs.change_type == 'drift'
        run: ./infra/ci-cd/scripts/email-notification.sh "$(cat drift_summary.txt)"

      - name: Manual Approval
        if: steps.drift-check.outputs.change_type == 'drift'
        uses: trstringer/manual-approval@v1

      - name: Terraform Apply
        if: steps.manual-approval.outcome == 'success'
        run: terraform apply -auto-approve tfplan

Drift Detection in CI/CD (Quick Reference)

Note: For a detailed explanation of drift detection, email setup, and GitHub approval, see the "Understanding Drift Detection" section earlier in this guide.

Quick summary:

Drift = Infrastructure changed outside Terraform
Detected automatically in CI/CD
Email sent + GitHub issue created
Manual approval required before applying

Application Deployment Workflow

Separate workflow for application code changes:

name: Application Deployment

on:
  push:
    paths:
      - 'frontend/**'
      - 'auth-api/**'
      - 'todos-api/**'
      - 'docker-compose.yml'
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment'
        type: choice
        options: [dev, stg, prod]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Get Server IP
        run: |
          # Find server by tag
          INSTANCE_ID=$(aws ec2 describe-instances ...)
          SERVER_IP=$(aws ec2 describe-instances ...)

      - name: Deploy with Ansible
        run: |
          ansible-playbook -i inventory/${ENV}.yml playbook.yml --tags deploy

Why separate workflows? Infrastructure changes are rare and need careful review. Application changes are frequent and should deploy quickly.

Infrastructure Destruction Workflow

⚠️ CRITICAL: This workflow DESTROYS EVERYTHING. Use with extreme caution!

The destroy workflow is separate from the deployment workflow for safety. It has multiple confirmation steps to prevent accidental destruction.

How the Destroy Workflow Works

Step 1: Manual Trigger Only

Only runs when you manually trigger it (no automatic triggers)
Requires you to select the environment
Requires you to type "DESTROY" to confirm

Step 2: Validation

Checks that you typed "DESTROY" correctly (case-sensitive)
Prevents typos from accidentally destroying infrastructure

Step 3: State File Handling

Tries to download state from artifacts (most recent)
Falls back to S3 remote backend if artifacts missing
Imports resources if state is completely missing

Step 4: Destroy Plan

Shows you exactly what will be destroyed
Review this carefully before proceeding

Step 5: Destruction

Deletes all resources in the correct order
Handles dependencies (e.g., detaches volumes before deleting)

Step 6: Verification

Checks that everything was destroyed
Cleans up orphaned resources
Provides a summary

The Complete Destroy Workflow

Here's what the actual workflow looks like:

name: Infrastructure Destruction

on:
  workflow_dispatch:  # Manual trigger only - safe!
    inputs:
      environment:
        description: 'Environment to destroy (dev, stg, prod)'
        required: true
        type: choice
        options: [dev, stg, prod]
        default: 'dev'
      confirm_destroy:
        description: 'Type "DESTROY" to confirm (case-sensitive)'
        required: true
        type: string

jobs:
  validate-destroy:
    name: Validate Destruction Request
    runs-on: ubuntu-latest
    steps:
      - name: Validate confirmation
        run: |
          if [ "${{ github.event.inputs.confirm_destroy }}" != "DESTROY" ]; then
            echo "❌ Invalid confirmation. You must type 'DESTROY' to proceed."
            exit 1
          fi
          echo "✅ Destruction confirmed. Proceeding..."

  terraform-destroy:
    name: Destroy Infrastructure
    needs: validate-destroy
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3

      - name: Terraform Init (with S3 backend)
        working-directory: infra/terraform
        run: |
          terraform init \
            -backend-config="bucket=${{ secrets.TERRAFORM_STATE_BUCKET }}" \
            -backend-config="key=terraform-state/${{ github.event.inputs.environment }}/terraform.tfstate" \
            -backend-config="region=${{ secrets.AWS_REGION }}" \
            -backend-config="dynamodb_table=${{ secrets.TERRAFORM_STATE_LOCK_TABLE }}" \
            -backend-config="encrypt=true"

      - name: Terraform Plan Destroy
        working-directory: infra/terraform
        run: |
          terraform plan -destroy \
            -var-file=terraform.${{ github.event.inputs.environment }}.tfvars \
            -var="key_pair_name=${{ secrets.TERRAFORM_KEY_PAIR_NAME }}" \
            -out=destroy.tfplan
          echo ""
          echo "⚠️ DESTRUCTION PLAN SUMMARY:"
          terraform show -no-color destroy.tfplan | head -100

      - name: Terraform Destroy
        working-directory: infra/terraform
        run: |
          echo "🔥 Starting infrastructure destruction..."
          terraform destroy -auto-approve \
            -var-file=terraform.${{ github.event.inputs.environment }}.tfvars \
            -var="key_pair_name=${{ secrets.TERRAFORM_KEY_PAIR_NAME }}"

      - name: Verify Destruction
        working-directory: infra/terraform
        run: |
          echo "🔍 Verifying all resources are destroyed..."
          # Check for orphaned resources and clean them up

How to Use the Destroy Workflow

Step 1: Go to GitHub Actions

Open your repository on GitHub
Click the "Actions" tab
Find "Infrastructure Destruction" in the workflow list

Step 2: Run the Workflow

Click "Run workflow" button (top right)
Select the environment you want to destroy:
- ⚠️ Be very careful - make sure you select the right one!
- Dev is usually safe to destroy
- Staging should be destroyed carefully
- NEVER destroy production unless absolutely necessary!
In the "Type DESTROY to confirm" field, type exactly: DESTROY
- Must be all caps
- Must be exactly "DESTROY" (no extra spaces)
Click "Run workflow"

Step 3: Watch It Run

The workflow will start with a validation job
- Checks that you typed "DESTROY" correctly
- If wrong, workflow fails immediately (safe!)
Then the terraform-destroy job runs:
- Initializes Terraform with the correct backend
- Creates a destroy plan (shows what will be deleted)
- Review the plan carefully!
- Destroys all resources
- Verifies everything is gone

Step 4: Review the Results

Check the workflow logs
Verify in AWS Console that resources are gone
Check that costs are now $0.00

Safety Features

The destroy workflow has multiple safety features:

Manual trigger only - Can't be triggered automatically
Confirmation required - Must type "DESTROY" exactly
Environment selection - Prevents destroying wrong environment
Plan before destroy - Shows you what will be deleted
Validation job - Double-checks confirmation before proceeding
State file handling - Works with remote state (S3)
Verification - Checks that everything was destroyed

What Gets Destroyed

When you run the destroy workflow, it deletes:

✅ EC2 instance - Your server and everything on it
✅ Security groups - Firewall rules
✅ EBS volumes - Any attached storage (if using EBS for state)
✅ All containers - Docker containers running on the instance
✅ All data - Everything on the server is permanently lost

What stays:

✅ S3 bucket - Your Terraform state bucket (not deleted)
✅ DynamoDB table - State locking table (not deleted)
✅ GitHub repository - Your code (not deleted)

When to Use the Destroy Workflow

Good reasons to destroy:

✅ You're done with the project and want to stop costs
✅ You want to start completely fresh
✅ You're testing and need to clean up
✅ You're moving to a different AWS account

Bad reasons to destroy:

❌ Just to restart services (use Ansible instead)
❌ To fix a small issue (fix the issue, don't destroy)
❌ Because something isn't working (debug first)
❌ In production without a backup plan

After Destruction

What happens:

All infrastructure is deleted
State file in S3 is updated (shows no resources)
You stop paying for AWS resources
All data is permanently lost

To recreate:

Run the Infrastructure Deployment workflow again
It will create everything from scratch
You'll need to redeploy your application

Important notes:

State file history is preserved in S3
You can see what was destroyed in the workflow logs
GitHub Actions artifacts are kept for 90 days
You can manually delete artifacts if needed

Destroy Workflow vs Manual Destroy

Use the workflow when:

✅ You want safety features (confirmation, validation)
✅ You want to destroy from anywhere (don't need local setup)
✅ You want an audit trail (GitHub Actions logs)
✅ You're working with a team (everyone can see what happened)

Use manual destroy when:

✅ You need to destroy specific resources only
✅ You're debugging and need more control
✅ You don't have GitHub Actions set up

Example: Destroying Development Environment

Let's walk through destroying a dev environment:

Go to Actions → Infrastructure Destruction
Click "Run workflow"
Select environment: dev
Type confirmation: DESTROY
Click "Run workflow"

What you'll see:

✅ validate-destroy: Validation passed
✅ terraform-destroy: 
   - Terraform Init: Success
   - Terraform Plan Destroy: Shows what will be deleted
   - Terraform Destroy: Deleting resources...
   - Verify Destruction: All resources destroyed

After completion:

Check AWS Console → EC2 → No instances
Check AWS Console → Security Groups → No groups
Check AWS Billing → Costs should be $0.00

Troubleshooting Destroy Workflow

Issue: "Invalid confirmation"

Problem: You didn't type "DESTROY" exactly
Solution: Type exactly DESTROY (all caps, no spaces)

Issue: "State file not found"

Problem: State file is missing or in wrong location
Solution: Workflow will try to import resources automatically

Issue: "Resources still exist after destroy"

Problem: Some resources might be stuck
Solution: Check the verification step - it will try to clean up orphaned resources

Issue: "Can't destroy because of dependencies"

Problem: Resources have dependencies (e.g., volume attached)
Solution: Workflow handles this automatically (detaches volumes first)

Best Practices for Destruction

Always destroy dev first - Test the workflow in dev before using in staging/prod
Review the plan - Check what will be destroyed before confirming
Backup important data - If you need any data, back it up first
Destroy during off-hours - If others are using the environment
Document why - Add a comment in the workflow run explaining why you destroyed
Verify after - Check AWS Console to confirm everything is gone
Clean up artifacts - Delete GitHub Actions artifacts if you want

Part 7: Single Command Deployment

The ultimate goal: one command that does everything.

How It Works

When you run:

terraform apply -var-file=terraform.dev.tfvars -auto-approve

Here's what happens behind the scenes:

Terraform provisions infrastructure
- Creates security group
- Launches EC2 instance
- Waits for instance to be ready
Terraform generates Ansible inventory
- Creates ansible/inventory/dev.yml with server IP
- Ready for Ansible to use
Terraform triggers Ansible (via null_resource)
- Waits for SSH to be available
- Runs Ansible playbook
- Installs Docker
- Clones repository
- Starts containers
Traefik gets SSL certificate
- Contacts Let's Encrypt
- Verifies domain ownership
- Gets certificate
- Enables HTTPS
Application is live!
- Frontend accessible at https://yourdomain.com
- APIs at https://yourdomain.com/api/*

The Magic: null_resource

resource "null_resource" "ansible_provision" {
  triggers = {
    instance_id = aws_instance.todo_app.id
  }

  provisioner "local-exec" {
    command = <<-EOT
      # Wait for SSH
      until ssh ... 'echo "ready"'; do sleep 10; done

      # Run Ansible
      cd ../ansible
      ansible-playbook -i inventory/${var.environment}.yml playbook.yml
    EOT
  }
}

What is null_resource? It's a Terraform resource that doesn't create anything in AWS. It just runs a command. Perfect for triggering Ansible!

Why the wait? EC2 instances take 30-60 seconds to boot. We wait for SSH to be ready before running Ansible.

Testing the Single Command

# Make sure you're in the terraform directory
cd infra/terraform

# Initialize (one-time setup)
terraform init -backend-config=...

# The magic command
terraform apply -var-file=terraform.dev.tfvars -auto-approve

# Watch it work!
# You'll see:
# 1. Security group created
# 2. EC2 instance launching
# 3. Waiting for SSH...
# 4. Running Ansible...
# 5. Application deployed!

Pro tip: The first run takes 5-10 minutes. Subsequent runs are faster (only changes what's needed).

Part 8: Multi-Environment Setup

Real applications need multiple environments. Here's how to set it up properly.

Why Multiple Environments?

Dev: Where you experiment (break things safely)
Staging: Mirror of production (test before going live)
Production: The real thing (users depend on it)

Environment Isolation

Each environment is completely separate:

Different EC2 instances
Different security groups
Different state files in S3
Different domains
Different secrets

Why isolation matters: If dev gets hacked, staging and prod are still safe. If you break dev, staging and prod keep running. This is why we have three separate environments!

Setting Up Per-Environment Configuration

Terraform:

terraform.dev.tfvars - Dev configuration
terraform.stg.tfvars - Staging configuration
terraform.prod.tfvars - Production configuration

Ansible:

group_vars/dev/vars.yml - Dev variables
group_vars/stg/vars.yml - Staging variables
group_vars/prod/vars.yml - Production variables

State files:

terraform-state/dev/terraform.tfstate
terraform-state/stg/terraform.tfstate
terraform-state/prod/terraform.tfstate

Deploying to Different Environments

Via GitHub Actions:

Go to Actions → Infrastructure Deployment
Click "Run workflow"
Select environment (dev/stg/prod)
Click "Run workflow"

Via command line:

# Dev
terraform apply -var-file=terraform.dev.tfvars -auto-approve

# Staging
terraform apply -var-file=terraform.stg.tfvars -auto-approve

# Production
terraform apply -var-file=terraform.prod.tfvars -auto-approve

Important: Always test in dev first! Never deploy to prod without testing.

Part 9: Common Issues and Solutions

Every project has issues. Here are the ones you'll likely encounter and how to fix them.

Issue 1: Let's Encrypt Certificate Errors

Symptom: Timeout during connect (likely firewall problem)

Causes:

DNS not pointing to your server
Security group blocking ports 80/443
Firewall on the server blocking ports

Solutions:

# 1. Verify DNS
dig yourdomain.com
# Should show your server IP

# 2. Check security group
# AWS Console → EC2 → Security Groups
# Verify ports 80 and 443 allow 0.0.0.0/0

# 3. Check server firewall
sudo ufw status
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# 4. Switch to HTTP challenge (more reliable)
# In docker-compose.yml:
- "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
- "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"

Issue 2: Terraform State Lock

Symptom: Error acquiring the state lock

Cause: Another Terraform run is in progress, or a previous run crashed

Solution:

# Check what's locking
aws dynamodb scan --table-name terraform-state-lock

# If you're sure no one else is running Terraform:
terraform force-unlock <LOCK_ID>

# Be careful! Only do this if you're certain.

Issue 3: Ansible Connection Failed

Symptom: SSH connection failed or Permission denied

Causes:

Security group doesn't allow SSH from your IP
Wrong key pair
Server not ready yet

Solutions:

# 1. Test SSH manually
ssh -i ~/.ssh/your-key.pem ubuntu@<server-ip>

# 2. Check security group
# Make sure it allows port 22 from your IP

# 3. Verify key pair name matches
# AWS Console → EC2 → Key Pairs
# Should match what's in terraform.tfvars

# 4. Wait longer (server might still be booting)
# EC2 instances take 1-2 minutes to be ready

Issue 4: Drift Detection Not Working

Symptom: Changes made manually but drift not detected

Check:

Did Terraform files change? (That's "expected", not drift)
Is state in S3? (Local state won't work properly)
Check drift detection logic in workflow

Test drift:

Manually add a tag to your EC2 instance in AWS Console
Run the infrastructure workflow
Should detect drift and send email

Issue 5: Containers Keep Restarting

Symptom: docker ps shows containers restarting constantly

Debug:

# Check logs
docker logs <container-name>

# Check all containers
docker compose logs

# Common causes:
# - Configuration error in .env
# - Port conflict
# - Missing environment variables
# - Application crash on startup

Part 10: Best Practices and Security

Now that everything works, let's make it production-ready.

Security Best Practices

Never commit secrets
- Use GitHub Secrets
- Use environment variables
- Add .env to .gitignore
Restrict SSH access
- In production, set ssh_cidr to your IP only
- Use YOUR_IP/32 format (e.g., 1.2.3.4/32)
Use different secrets per environment
- Dev JWT secret ≠ Staging JWT secret ≠ Prod JWT secret
- If dev is compromised, staging and prod are still safe
Enable MFA
- On AWS account
- On GitHub account
- Extra layer of protection
Regular updates
- Keep Docker images updated
- Keep system packages updated
- Security patches are important!

Infrastructure Best Practices

Always use remote state
- S3 + DynamoDB
- Never commit state files
- Enable versioning on S3 bucket
Separate state per environment
- Different S3 keys
- Complete isolation
- Can't accidentally affect prod from dev
Use version constraints
- In Terraform: version = "~> 5.0"
- Prevents unexpected breaking changes
Tag everything
- Makes it easy to find resources
- Helps with cost tracking
- Required for organization

Deployment Best Practices

Test in dev first
- Always deploy to dev → staging → prod (in that order!)
- Catch issues early before they reach production
- Dev is for breaking things
Review drift alerts
- Don't ignore them!
- Investigate unexpected changes
- Could be security issue
Use idempotent deployments
- Safe to run multiple times
- Ansible should be idempotent
- Terraform is idempotent by design
Monitor your infrastructure
- Set up CloudWatch alarms
- Monitor costs
- Watch for unusual activity

Cost Optimization

Right-size instances
- Dev: t3.small (saves money)
- Prod: t3.medium (enough power)
- Don't over-provision
Stop dev when not in use
- Dev doesn't need to run 24/7
- Stop instances when not testing
- Saves ~70% of costs
Clean up unused resources
- Delete old instances
- Remove unused security groups
- Regular cleanup prevents waste

Part 11: Going Further

You've built a solid foundation. Here's where to go next.

Monitoring and Observability

Add CloudWatch:

Monitor CPU, memory, disk
Set up alarms
Track costs

Add Application Monitoring:

Prometheus + Grafana
ELK stack for logs
APM tools (New Relic, Datadog)

Scaling

Horizontal Scaling:

Add load balancer
Multiple instances
Auto-scaling groups

Vertical Scaling:

Larger instance types
More CPU/RAM
Better performance

Backup and Disaster Recovery

Backup Strategy:

Database backups
State file backups (S3 versioning)
Configuration backups

Disaster Recovery:

Multi-region deployment
Automated failover
Recovery procedures

Advanced Topics

Kubernetes: Container orchestration at scale
Terraform Modules: Reusable infrastructure code
Ansible Roles: Shareable configuration
GitOps: Git as source of truth
Infrastructure Testing: Test your infrastructure code

Conclusion: What You've Accomplished

Let's take a moment to appreciate what you've built:

✅ A microservices application running in containers
✅ Automated infrastructure with Terraform
✅ Automated deployment with Ansible
✅ CI/CD pipelines that detect problems
✅ Multi-environment setup (dev/staging/prod)
✅ Secure HTTPS with automatic certificates
✅ Single-command deployment that just works
✅ Production-ready practices and security

This isn't just a tutorial project - this is real infrastructure that follows industry best practices. You can use this as a foundation for actual production applications.

Key Takeaways

Infrastructure as Code saves time and prevents mistakes
Automation is your friend - manual processes break
Security isn't optional - build it in from the start
Testing in dev/staging prevents production disasters
Documentation (this blog post!) helps you and others

Next Steps

Deploy your own project using this as a template
Experiment - break things in dev, learn from it
Share - help others learn what you've learned
Iterate - improve based on real-world experience

Resources

Thank you for reading! If this helped you, please share it with others who might benefit. And if you have questions or run into issues, don't hesitate to reach out.

Happy deploying! 🚀

This guide was written as part of the HNG Internship Stage 6 DevOps task. The complete implementation is available on GitHub.

Building Your Own Virtual Private Cloud (VPC) on Linux: A Complete Beginner's Guide

Mart Young — Mon, 10 Nov 2025 10:06:31 +0000

Introduction

Have you ever wondered how cloud providers like AWS, Google Cloud, or Azure create isolated virtual networks? How do they ensure that your resources are completely isolated from others while still allowing controlled communication?

In this comprehensive guide, we'll build our own Virtual Private Cloud (VPC) from scratch on Linux using nothing but native Linux networking tools. By the end of this journey, you'll understand the fundamental building blocks of network virtualization and have a working VPC implementation that you can use for learning, testing, or even production workloads.

What You'll Learn

How to create isolated virtual networks using Linux network namespaces
How to connect networks using Linux bridges and virtual Ethernet pairs
How to implement routing between subnets
How to enable internet access using NAT (Network Address Translation)
How to enforce network isolation between different VPCs
How to connect VPCs using peering
How to implement firewall rules (Security Groups)
How to automate VPC management using vpcctl CLI tool

Our Approach: Learn by Doing, Then Automate

This guide follows a two-part approach:

Manual Implementation (Parts 1-10): We'll build VPCs manually using Linux commands. This helps you understand exactly what's happening under the hood.
Automation with vpcctl (Part 11): Once you understand the fundamentals, we'll introduce vpcctl, a CLI tool that automates all the manual steps. This is what you'll use in practice!

What is vpcctl?

vpcctl is a command-line tool written in Python that automates VPC creation and management. Instead of running dozens of commands manually, you can create a VPC with a single command:

# Manual way (what we'll learn first)
sudo ip netns add my-ns
sudo ip link add veth0 type veth peer name veth1
# ... many more commands ...

# Automated way (using vpcctl)
sudo ./vpcctl create --name myvpc --cidr 10.0.0.0/16
sudo ./vpcctl add-subnet --vpc myvpc --name public --cidr 10.0.1.0/24 --type public

Don't worry - we'll get to vpcctl after you understand the fundamentals!

Prerequisites

Before we begin, you'll need:

A Linux system (Ubuntu 20.04+, Debian, or Pop!_OS recommended)
Root or sudo access
Basic familiarity with the command line
Git (to clone the repository)
Python 3.6 or higher (for vpcctl)
Curiosity and patience! 🚀

Tools We'll Use

All the tools we'll use are built into Linux:

ip - Modern Linux networking tool
brctl - Bridge control utility
iptables - Firewall and NAT tool
ping - Network connectivity testing
curl - HTTP client for testing web servers
vpcctl - Our automation tool (we'll get this from the repository)

Let's get started!

Part 1: Understanding the Building Blocks

What is a VPC?

A Virtual Private Cloud (VPC) is essentially a virtual network that you can use to isolate your resources. Think of it like creating a private room in a shared building - you have your own space with controlled access points.

In our implementation:

VPC = A virtual network with its own IP address range
Subnet = A smaller network within the VPC (like a room within the private space)
Bridge = A virtual router that connects subnets
Namespace = An isolated network environment (like a separate room)

The Linux Primitives

Before we dive in, let's understand the Linux networking primitives we'll be using:

Network Namespaces: Isolated network environments - think of them as separate computers on the same physical machine
veth Pairs: Virtual Ethernet cables - they come in pairs, like a cable with two ends
Linux Bridges: Virtual switches that connect multiple networks
Routing Tables: Maps that tell packets where to go
iptables: Firewall and NAT rules

Don't worry if this sounds complex - we'll learn by doing!

Part 2: Setting Up Your Environment

Step 1: Clone the Repository

First, let's get the project code which includes the vpcctl tool and example scripts:

# Clone the repository
git clone https://github.com/Donkross360/vpc-project.git
cd vpc-project

# Alternatively, if you prefer to download:
# wget https://github.com/Donkross360/vpc-project/archive/main.zip
# unzip main.zip
# cd vpc-project-main

The repository contains:

vpcctl - The CLI tool for automating VPC management (Python script)
examples/ - Example scripts and a demo web application
README.md - Complete documentation
Makefile - Automation for common tasks
cleanup.sh - Script to clean up all VPC resources
policies/ - Example firewall policy files

Step 2: Install Required Tools

Now, let's make sure we have all the necessary tools installed:

# Update package list
sudo apt-get update

# Install required tools
sudo apt-get install -y \
    iproute2 \
    bridge-utils \
    iptables \
    python3 \
    curl \
    git

# Make vpcctl executable
chmod +x vpcctl

Step 3: Quick Start with vpcctl (Optional)

Want to see vpcctl in action right away? You can skip ahead to Part 11, or follow along manually to understand how it works. For now, here's a quick preview:

# Create a VPC with one command
sudo ./vpcctl create --name demo-vpc --cidr 10.0.0.0/16

# Add a public subnet
sudo ./vpcctl add-subnet --vpc demo-vpc --name public --cidr 10.0.1.0/24 --type public

# List all VPCs
sudo ./vpcctl list

# View VPC details
sudo ./vpcctl show demo-vpc

But wait! Before we automate, let's understand what's happening under the hood by building manually first.

Step 4: Enable IP Forwarding

IP forwarding allows Linux to act as a router, forwarding packets between networks. This is essential for our VPC to work:

# Enable IP forwarding temporarily
sudo sysctl -w net.ipv4.ip_forward=1

# Make it permanent (so it survives reboots)
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf

What this does: This tells the Linux kernel to forward IP packets between network interfaces, which is necessary for routing traffic between our virtual networks.

Step 5: Identify Your Internet Interface

We need to know which network interface connects your machine to the internet. This will be used for NAT (Network Address Translation) later:

# Find your default internet interface
DEFAULT_INTERFACE=$(ip route show default | awk '/default/ {print $5}' | head -1)
echo "Internet interface: $DEFAULT_INTERFACE"

# Verify the interface exists and is UP
ip link show $DEFAULT_INTERFACE

Expected output: You should see something like enp0s25 or eth0 or wlan0. This is your connection to the outside world.

Note: Modern Linux systems use "predictable naming" like enp0s25 instead of the traditional eth0. This is normal and expected!

Part 3: Creating Your First Network Namespace

Understanding Namespaces

A network namespace is like a separate computer with its own network stack. It has its own:

Network interfaces
IP addresses
Routing tables
Firewall rules

Let's create our first namespace to see how it works:

# Create a network namespace
sudo ip netns add test-ns-1

# List all namespaces
sudo ip netns list

# Verify the namespace was created
sudo ip netns exec test-ns-1 ip link show

What you'll see: The namespace only has a lo (loopback) interface, which is like a network interface that points back to itself. It's completely isolated from your host system!

Bringing Up the Loopback Interface

Even though the namespace has a loopback interface, it's not active by default. Let's bring it up:

# Bring up the loopback interface in the namespace
sudo ip netns exec test-ns-1 ip link set lo up

# Verify it's up
sudo ip netns exec test-ns-1 ip link show lo

Success indicator: You should see state UNKNOWN or state UP instead of state DOWN.

Part 4: Creating Virtual Ethernet Pairs

What are veth Pairs?

A veth (virtual Ethernet) pair is like a virtual network cable with two ends. Whatever you send into one end comes out the other end. We use these to connect namespaces to the host or to bridges.

Let's create our first veth pair:

# Create a veth pair
sudo ip link add veth-host type veth peer name veth-ns

# Verify both ends were created
ip link show type veth

What you'll see: Two interfaces - veth-host and veth-ns. They're connected like two ends of a cable.

Moving One End to a Namespace

Now let's move one end into our namespace. This creates a connection between the host and the namespace:

# Create a namespace for testing
sudo ip netns add test-ns-2

# Move veth-ns to the namespace
sudo ip link set veth-ns netns test-ns-2

# Verify veth-ns is now in the namespace
sudo ip netns exec test-ns-2 ip link show

# Verify veth-host is still on the host
ip link show veth-host

What happened:

veth-ns is now inside test-ns-2 and no longer visible on the host
veth-host is still on the host
They're still connected - data sent through one end will come out the other!

Assigning IP Addresses

Now let's give both ends IP addresses so they can communicate:

# Assign IP to host side
sudo ip addr add 10.0.1.1/24 dev veth-host
sudo ip link set veth-host up

# Assign IP to namespace side
sudo ip netns exec test-ns-2 ip addr add 10.0.1.10/24 dev veth-ns
sudo ip netns exec test-ns-2 ip link set veth-ns up

# Bring up loopback in namespace
sudo ip netns exec test-ns-2 ip link set lo up

Understanding the IP addresses:

10.0.1.1/24 means IP address 10.0.1.1 with subnet mask 255.255.255.0
The /24 means the first 24 bits are the network part
10.0.1.0/24 is the network, 10.0.1.1 and 10.0.1.10 are hosts in that network

Testing Connectivity

Let's test if the host and namespace can communicate:

# Ping from host to namespace
ping -c 2 10.0.1.10

# Ping from namespace to host
sudo ip netns exec test-ns-2 ping -c 2 10.0.1.1

Success! If both pings work, you've successfully created your first virtual network connection! 🎉

Cleanup

Let's clean up before moving on:

# Bring interfaces down
sudo ip link set veth-host down
sudo ip link delete veth-host

# Delete namespace (this also removes veth-ns)
sudo ip netns delete test-ns-2
sudo ip netns delete test-ns-1

Part 5: Creating a Linux Bridge (Your First Router)

What is a Linux Bridge?

A Linux bridge is like a virtual network switch. It can connect multiple network interfaces and forward traffic between them. In our VPC, the bridge will act as a router, connecting multiple subnets.

Let's create our first bridge:

# Create a bridge
sudo ip link add br-test type bridge

# Bring it up
sudo ip link set br-test up

# Verify it was created
ip link show br-test

Connecting a Namespace to the Bridge

Now let's connect a namespace to our bridge using a veth pair:

# Create namespace
sudo ip netns add test-ns-3

# Create veth pair
sudo ip link add veth-br-host type veth peer name veth-br-ns

# Move namespace end to namespace
sudo ip link set veth-br-ns netns test-ns-3

# Connect host end to bridge (IMPORTANT: Do this before bringing it up!)
sudo ip link set veth-br-host master br-test

# Bring host end up
sudo ip link set veth-br-host up

# Configure namespace
sudo ip netns exec test-ns-3 ip link set lo up
sudo ip netns exec test-ns-3 ip addr add 10.0.1.10/24 dev veth-br-ns
sudo ip netns exec test-ns-3 ip link set veth-br-ns up

# Assign IP to bridge (this acts as the gateway)
sudo ip addr add 10.0.1.1/24 dev br-test

Key points:

The bridge IP (10.0.1.1) acts as the gateway for the namespace
The namespace and bridge must be in the same subnet (10.0.1.0/24)
We attach the veth to the bridge BEFORE bringing it up

Adding a Default Route

The namespace needs to know where to send packets that aren't in its local network. Let's add a default route:

# Add default route in namespace (points to bridge as gateway)
sudo ip netns exec test-ns-3 ip route add default via 10.0.1.1 dev veth-br-ns

# Verify the route
sudo ip netns exec test-ns-3 ip route show

Testing Connectivity

Let's test if the namespace can reach the bridge:

# Ping bridge from namespace
sudo ip netns exec test-ns-3 ping -c 2 10.0.1.1

# Verify bridge sees the connection
brctl show br-test

Expected output: The bridge should show veth-br-host as a connected interface, and the ping should succeed!

Connecting Multiple Namespaces

Now let's add a second namespace to the same bridge to create a small network:

# Create second namespace
sudo ip netns add test-ns-4

# Create second veth pair
sudo ip link add veth-br-host-2 type veth peer name veth-br-ns-2

# Move namespace end to namespace
sudo ip link set veth-br-ns-2 netns test-ns-4

# Connect to bridge
sudo ip link set veth-br-host-2 master br-test
sudo ip link set veth-br-host-2 up

# Configure second namespace
sudo ip netns exec test-ns-4 ip link set lo up
sudo ip netns exec test-ns-4 ip addr add 10.0.1.20/24 dev veth-br-ns-2
sudo ip netns exec test-ns-4 ip link set veth-br-ns-2 up
sudo ip netns exec test-ns-4 ip route add default via 10.0.1.1 dev veth-br-ns-2

# Test connectivity between namespaces
sudo ip netns exec test-ns-3 ping -c 2 10.0.1.20
sudo ip netns exec test-ns-4 ping -c 2 10.0.1.10

Success! Both namespaces can now communicate with each other through the bridge! This is the foundation of our VPC.

Part 6: Building Your First VPC

Understanding VPC Structure

A VPC consists of:

A bridge that acts as the VPC router
Multiple subnets (network namespaces)
veth pairs connecting subnets to the bridge
Routing tables directing traffic
NAT rules for internet access (for public subnets)

Let's build a complete VPC with public and private subnets!

Step 1: Create the VPC Bridge

# Create VPC bridge
sudo ip link add br-vpc1 type bridge
sudo ip link set br-vpc1 up

# Assign bridge IP addresses for each subnet
# The bridge needs an IP in each subnet to act as the gateway
sudo ip addr add 10.0.1.1/24 dev br-vpc1  # Gateway for public subnet
sudo ip addr add 10.0.2.1/24 dev br-vpc1  # Gateway for private subnet
sudo ip addr add 10.0.0.1/16 dev br-vpc1  # VPC router IP (optional)

Why multiple IPs? The bridge needs to be in the same subnet as each namespace it serves as a gateway for. This allows the namespaces to reach the gateway using ARP (Address Resolution Protocol).

Step 2: Create Public Subnet

# Create public subnet namespace
sudo ip netns add ns-vpc1-public

# Create veth pair for public subnet
# Note: Interface names must be 15 characters or less (Linux limit)
sudo ip link add veth-vpc1-pub-h type veth peer name veth-vpc1-pub-n

# Move namespace end to namespace
sudo ip link set veth-vpc1-pub-n netns ns-vpc1-public

# Connect host end to bridge
sudo ip link set veth-vpc1-pub-h master br-vpc1
sudo ip link set veth-vpc1-pub-h up

# Configure public subnet
sudo ip netns exec ns-vpc1-public ip link set lo up
sudo ip netns exec ns-vpc1-public ip addr add 10.0.1.10/24 dev veth-vpc1-pub-n
sudo ip netns exec ns-vpc1-public ip link set veth-vpc1-pub-n up

# Add routes
sudo ip netns exec ns-vpc1-public ip route add default via 10.0.1.1 dev veth-vpc1-pub-n
sudo ip netns exec ns-vpc1-public ip route add 10.0.2.0/24 via 10.0.1.1 dev veth-vpc1-pub-n

Step 3: Create Private Subnet

# Create private subnet namespace
sudo ip netns add ns-vpc1-private

# Create veth pair for private subnet
sudo ip link add veth-vpc1-prv-h type veth peer name veth-vpc1-prv-n

# Move namespace end to namespace
sudo ip link set veth-vpc1-prv-n netns ns-vpc1-private

# Connect host end to bridge
sudo ip link set veth-vpc1-prv-h master br-vpc1
sudo ip link set veth-vpc1-prv-h up

# Configure private subnet
sudo ip netns exec ns-vpc1-private ip link set lo up
sudo ip netns exec ns-vpc1-private ip addr add 10.0.2.10/24 dev veth-vpc1-prv-n
sudo ip netns exec ns-vpc1-private ip link set veth-vpc1-prv-n up

# Add routes
sudo ip netns exec ns-vpc1-private ip route add default via 10.0.2.1 dev veth-vpc1-prv-n
sudo ip netns exec ns-vpc1-private ip route add 10.0.1.0/24 via 10.0.2.1 dev veth-vpc1-prv-n

Step 4: Enable Proxy ARP

Proxy ARP allows the bridge to respond to ARP requests for IP addresses in different subnets. This is essential for inter-subnet routing:

# Enable proxy ARP on bridge and veth interfaces
sudo sysctl -w net.ipv4.conf.br-vpc1.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc1-pub-h.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc1-prv-h.proxy_arp=1

What is Proxy ARP? Normally, a device only responds to ARP requests for IPs on its own interface. Proxy ARP allows the bridge to respond for IPs in connected subnets, enabling routing.

Step 5: Test Inter-Subnet Communication

# Test: public to private
sudo ip netns exec ns-vpc1-public ping -c 2 10.0.2.10

# Test: private to public
sudo ip netns exec ns-vpc1-private ping -c 2 10.0.1.10

Success! Your subnets can now communicate with each other! 🎉

Part 7: Enabling Internet Access with NAT

Understanding NAT

Network Address Translation (NAT) allows private IP addresses to access the internet by translating them to a public IP address. This is how your home router works!

Step 1: Enable NAT for Public Subnet

# Get your internet interface (we identified this earlier)
DEFAULT_INTERFACE=$(ip route show default | awk '/default/ {print $5}' | head -1)

# Add NAT rule for public subnet
sudo iptables -t nat -A POSTROUTING -s 10.0.1.0/24 -o $DEFAULT_INTERFACE -j MASQUERADE

# Verify the rule
sudo iptables -t nat -L POSTROUTING -n -v

What this does:

-s 10.0.1.0/24 - Source is the public subnet
-o $DEFAULT_INTERFACE - Outgoing interface is your internet connection
-j MASQUERADE - Masquerade (NAT) the traffic

Step 2: Configure DNS

Namespaces need DNS to resolve domain names:

# Create DNS configuration for namespace
sudo mkdir -p /etc/netns/ns-vpc1-public
echo "nameserver 8.8.8.8" | sudo tee /etc/netns/ns-vpc1-public/resolv.conf
echo "nameserver 8.8.4.4" | sudo tee -a /etc/netns/ns-vpc1-public/resolv.conf

Step 3: Test Internet Access

# Test: ping Google's DNS
sudo ip netns exec ns-vpc1-public ping -c 2 8.8.8.8

# Test: ping Google by domain name
sudo ip netns exec ns-vpc1-public ping -c 2 google.com

Success! Your public subnet can now access the internet! 🌐

Step 4: Verify Private Subnet Isolation

# Private subnet should NOT be able to access internet (no NAT)
sudo ip netns exec ns-vpc1-private ping -c 2 8.8.8.8

Expected result: The ping should fail or timeout. This is correct - private subnets don't have NAT, so they can't access the internet directly.

Part 8: Creating Multiple VPCs and Enforcing Isolation

Creating a Second VPC

Let's create a second VPC to demonstrate isolation:

# Create second VPC bridge
sudo ip link add br-vpc2 type bridge
sudo ip link set br-vpc2 up

# Assign bridge IPs
sudo ip addr add 172.16.1.1/24 dev br-vpc2  # Gateway for public subnet
sudo ip addr add 172.16.2.1/24 dev br-vpc2  # Gateway for private subnet
sudo ip addr add 172.16.0.1/16 dev br-vpc2  # VPC router IP

# Create public subnet for VPC2
sudo ip netns add ns-vpc2-public
sudo ip link add veth-vpc2-pub-h type veth peer name veth-vpc2-pub-n
sudo ip link set veth-vpc2-pub-n netns ns-vpc2-public
sudo ip link set veth-vpc2-pub-h master br-vpc2
sudo ip link set veth-vpc2-pub-h up
sudo ip netns exec ns-vpc2-public ip link set lo up
sudo ip netns exec ns-vpc2-public ip addr add 172.16.1.10/24 dev veth-vpc2-pub-n
sudo ip netns exec ns-vpc2-public ip link set veth-vpc2-pub-n up
sudo ip netns exec ns-vpc2-public ip route add default via 172.16.1.1 dev veth-vpc2-pub-n
sudo ip netns exec ns-vpc2-public ip route add 172.16.2.0/24 via 172.16.1.1 dev veth-vpc2-pub-n

# Create private subnet for VPC2
sudo ip netns add ns-vpc2-private
sudo ip link add veth-vpc2-prv-h type veth peer name veth-vpc2-prv-n
sudo ip link set veth-vpc2-prv-n netns ns-vpc2-private
sudo ip link set veth-vpc2-prv-h master br-vpc2
sudo ip link set veth-vpc2-prv-h up
sudo ip netns exec ns-vpc2-private ip link set lo up
sudo ip netns exec ns-vpc2-private ip addr add 172.16.2.10/24 dev veth-vpc2-prv-n
sudo ip netns exec ns-vpc2-private ip link set veth-vpc2-prv-n up
sudo ip netns exec ns-vpc2-private ip route add default via 172.16.2.1 dev veth-vpc2-prv-n
sudo ip netns exec ns-vpc2-private ip route add 172.16.1.0/24 via 172.16.2.1 dev veth-vpc2-prv-n

# Enable proxy ARP
sudo sysctl -w net.ipv4.conf.br-vpc2.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc2-pub-h.proxy_arp=1
sudo sysctl -w net.ipv4.conf.veth-vpc2-prv-h.proxy_arp=1

# Add NAT for VPC2 public subnet
DEFAULT_INTERFACE=$(ip route show default | awk '/default/ {print $5}' | head -1)
sudo iptables -t nat -A POSTROUTING -s 172.16.1.0/24 -o $DEFAULT_INTERFACE -j MASQUERADE

# Configure DNS
sudo mkdir -p /etc/netns/ns-vpc2-public
echo "nameserver 8.8.8.8" | sudo tee /etc/netns/ns-vpc2-public/resolv.conf
echo "nameserver 8.8.4.4" | sudo tee -a /etc/netns/ns-vpc2-public/resolv.conf

Enforcing VPC Isolation

By default, Linux might route traffic between VPCs. We need to explicitly block this:

# Block traffic between VPC1 and VPC2
sudo iptables -A FORWARD -s 10.0.0.0/16 -d 172.16.0.0/16 -j DROP
sudo iptables -A FORWARD -s 172.16.0.0/16 -d 10.0.0.0/16 -j DROP

# Verify isolation
sudo ip netns exec ns-vpc1-public ping -c 2 172.16.1.10

Expected result: The ping should fail! VPCs are now isolated from each other. 🔒

Part 9: VPC Peering (Connecting VPCs)

What is VPC Peering?

VPC peering allows you to connect two VPCs so they can communicate with each other. This is useful when you want controlled communication between different environments.

Step 1: Create Peering Connection

# Create veth pair for peering
sudo ip link add veth-peer-1 type veth peer name veth-peer-2

# Connect one end to VPC1 bridge
sudo ip link set veth-peer-1 master br-vpc1
sudo ip link set veth-peer-1 up
# Use a shared /30 subnet for peering (192.168.255.0/30)
sudo ip addr add 192.168.255.1/30 dev veth-peer-1

# Connect other end to VPC2 bridge
sudo ip link set veth-peer-2 master br-vpc2
sudo ip link set veth-peer-2 up
# Same subnet - this is the peer IP
sudo ip addr add 192.168.255.2/30 dev veth-peer-2

Why a /30 subnet? A /30 subnet provides exactly 2 usable IPs (.1 and .2), perfect for a point-to-point link between two VPCs.

Step 2: Add Routes

# Add route on host to reach VPC2 through peering link
sudo ip route add 172.16.0.0/16 via 192.168.255.2 dev veth-peer-1

# Add route on host to reach VPC1 through peering link
sudo ip route add 10.0.0.0/16 via 192.168.255.1 dev veth-peer-2

# Remove isolation rules (peering should allow communication)
sudo iptables -D FORWARD -s 10.0.0.0/16 -d 172.16.0.0/16 -j DROP 2>/dev/null || true
sudo iptables -D FORWARD -s 172.16.0.0/16 -d 10.0.0.0/16 -j DROP 2>/dev/null || true

# Add routes in namespaces to reach the peer VPC
# In VPC1 namespaces, add route to VPC2
sudo ip netns exec ns-vpc1-public ip route add 172.16.0.0/16 via 10.0.1.1 dev veth-vpc1-pub-n
sudo ip netns exec ns-vpc1-private ip route add 172.16.0.0/16 via 10.0.2.1 dev veth-vpc1-prv-n

# In VPC2 namespaces, add route to VPC1
sudo ip netns exec ns-vpc2-public ip route add 10.0.0.0/16 via 172.16.1.1 dev veth-vpc2-pub-n
sudo ip netns exec ns-vpc2-private ip route add 10.0.0.0/16 via 172.16.2.1 dev veth-vpc2-prv-n

Step 3: Test Peering

# Test VPC1 to VPC2 connectivity
sudo ip netns exec ns-vpc1-public ping -c 2 172.16.1.10

# Test VPC2 to VPC1 connectivity
sudo ip netns exec ns-vpc2-public ping -c 2 10.0.1.10

Success! VPCs can now communicate through peering! 🌉

Part 10: Implementing Firewall Rules (Security Groups)

What are Security Groups?

Security Groups are firewall rules that control inbound and outbound traffic. In our implementation, we'll use iptables in each namespace to enforce these rules.

Step 1: Create a Firewall Policy

Create a JSON file with your firewall rules:

cat > firewall-policy.json <<EOF
{
  "subnet": "10.0.1.0/24",
  "ingress": [
    {"port": 80, "protocol": "tcp", "action": "allow"},
    {"port": 443, "protocol": "tcp", "action": "allow"},
    {"port": 22, "protocol": "tcp", "action": "deny"}
  ],
  "egress": [
    {"port": 0, "protocol": "all", "action": "allow"}
  ]
}
EOF

Step 2: Apply Firewall Rules

# Allow HTTP (port 80)
sudo ip netns exec ns-vpc1-public iptables -A INPUT -p tcp --dport 80 -j ACCEPT

# Allow HTTPS (port 443)
sudo ip netns exec ns-vpc1-public iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Deny SSH (port 22)
sudo ip netns exec ns-vpc1-public iptables -A INPUT -p tcp --dport 22 -j DROP

# Allow established connections
sudo ip netns exec ns-vpc1-public iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow loopback
sudo ip netns exec ns-vpc1-public iptables -A INPUT -i lo -j ACCEPT

# Set default policy to DROP
sudo ip netns exec ns-vpc1-public iptables -P INPUT DROP

Step 3: Deploy a Web Server

# Deploy a simple HTTP server in the public subnet
sudo ip netns exec ns-vpc1-public python3 -m http.server 80 &

# Test HTTP access (should work)
curl http://10.0.1.10:80

# Test SSH access (should be blocked)
nc -zv 10.0.1.10 22

Expected results:

HTTP works ✅
SSH is blocked ❌

Part 11: Automating with vpcctl

Why Automate?

By now, you've learned how to manually create VPCs using Linux commands. You've seen how namespaces, bridges, veth pairs, routing, and NAT all work together. This knowledge is invaluable!

However, creating VPCs manually requires dozens of commands and is error-prone. That's where vpcctl comes in - it's a CLI tool that automates all the steps we've learned, making VPC management as simple as a single command.

Getting Started with vpcctl

Important: Before using vpcctl, make sure you've cloned the repository as shown in Part 2, Step 1. The vpcctl tool is included in the repository.

If you haven't done so already:

# Clone the repository
git clone https://github.com/Donkross360/vpc-project.git
cd vpc-project

# Make vpcctl executable
chmod +x vpcctl

# Verify it works
./vpcctl --help

Understanding What vpcctl Does

vpcctl is a Python CLI tool that automates everything we did manually:

Creates VPCs - Sets up bridges with proper IP addresses
Adds Subnets - Creates namespaces, veth pairs, and configures routing
Enables NAT - Automatically configures NAT for public subnets
Manages Peering - Creates VPC peering connections
Applies Firewall Rules - Reads JSON policies and applies iptables rules
Enforces Isolation - Ensures VPCs are isolated by default
Maintains State - Tracks all VPCs in a JSON file

Basic Usage

Now that you understand how VPCs work manually, let's see how vpcctl simplifies everything:

# Create a VPC (replaces all the manual bridge creation)
sudo ./vpcctl create --name myvpc --cidr 10.0.0.0/16

# Add a public subnet (replaces namespace, veth, routing setup)
sudo ./vpcctl add-subnet --vpc myvpc --name public --cidr 10.0.1.0/24 --type public

# Add a private subnet
sudo ./vpcctl add-subnet --vpc myvpc --name private --cidr 10.0.2.0/24 --type private

# List all VPCs
sudo ./vpcctl list

# Show VPC details (namespace, IPs, etc.)
sudo ./vpcctl show myvpc

# Apply firewall rules from a JSON policy
sudo ./vpcctl apply-firewall --vpc myvpc --subnet public --policy policies/public-subnet.json

# Create VPC peering (connects two VPCs)
sudo ./vpcctl peer --vpc1 myvpc --vpc2 another-vpc

# Deploy an application (using helper script)
./examples/deploy-web-app.sh myvpc public 8000

# Delete a VPC (cleans up everything)
sudo ./vpcctl delete --name myvpc

Using the Makefile (Even Easier!)

The repository includes a Makefile that makes common tasks even simpler:

# One command to set up everything and deploy a demo
make demo

# Run comprehensive tests
make test

# Clean up all VPCs
make clean

# Show all available targets
make help

The make demo command will:

Set up IP forwarding
Create a demo VPC
Add a public subnet
Deploy a web application
Show you the URL to access it

Try it out!

What vpcctl Does Under the Hood

Remember all those manual commands we ran? vpcctl does the same things automatically:

When you run sudo ./vpcctl create --name myvpc --cidr 10.0.0.0/16:

Creates a Linux bridge (br-myvpc)
Assigns an IP address to the bridge
Enables the bridge
Stores VPC information in .vpcctl/vpcs.json

When you run sudo ./vpcctl add-subnet --vpc myvpc --name public --cidr 10.0.1.0/24 --type public:

Creates a network namespace (ns-myvpc-public)
Creates a veth pair
Moves one end into the namespace
Connects the other end to the bridge
Assigns IP addresses
Configures routing tables
Enables proxy ARP
Sets up NAT (if public subnet)
Configures DNS

All of this happens automatically! 🎉

Benefits of Automation

Idempotent: Safe to run multiple times - won't create duplicates
State management: Tracks VPCs in JSON file (.vpcctl/vpcs.json)
Logging: All actions are logged to .vpcctl/vpcctl.log
Error handling: Validates inputs and handles errors gracefully
Consistent: Same results every time
Fast: Creates VPCs in seconds instead of minutes

Part 12: Cleanup and Best Practices

Cleaning Up

Always clean up your test resources:

# Use the cleanup script
sudo ./cleanup.sh

# Or manually delete VPCs
sudo ./vpcctl delete --name myvpc

Best Practices

Use descriptive names: Name your VPCs and subnets clearly
Plan your IP ranges: Avoid overlapping CIDR blocks
Test isolation: Always verify that VPCs are isolated
Document your setup: Keep notes on your VPC configurations
Clean up regularly: Remove unused VPCs to free resources

Common Pitfalls

Interface name length: Linux limits interface names to 15 characters
Subnet mismatches: Gateway IP must be in the same subnet as the namespace
Missing proxy ARP: Required for inter-subnet routing
DNS configuration: Don't forget to configure DNS for public subnets
NAT rules: Only public subnets need NAT rules

Conclusion

Congratulations! You've successfully built your own VPC from scratch! 🎉

What You've Accomplished

Created isolated virtual networks using Linux namespaces
Connected networks using Linux bridges and veth pairs
Implemented routing between subnets
Enabled internet access using NAT
Enforced VPC isolation
Created VPC peering connections
Implemented firewall rules

Next Steps

Experiment: Try creating more complex VPC topologies
Automate: Use vpcctl to manage your VPCs
Deploy applications: Run real applications in your VPCs
Learn more: Explore advanced networking topics like BGP, VPNs, and load balancing

Resources

Repository: Clone the project from GitHub to get vpcctl and all examples
README.md: Complete CLI documentation and usage examples
Linux Networking Documentation: Official Linux networking docs
Makefile: Run make help to see all available automation targets

Final Thoughts

Building a VPC from scratch teaches you the fundamentals of network virtualization. The concepts you've learned here apply to cloud providers, container networking (Docker, Kubernetes), and virtualized environments.

Remember: Understanding the fundamentals makes you a better engineer. Keep learning, keep building, and keep experimenting! 🚀

Happy VPC Building! 🌐

If you found this guide helpful, please share it with others who might benefit from it!

Creating Your First Isolated Linux Container with SSH Access: A Step-by-Step Guide

Mart Young — Thu, 27 Mar 2025 22:46:01 +0000

Introduction

Have you ever wondered how tools like Docker create isolated environments? In this guide, we’ll build a simple container from scratch using Linux namespaces. You’ll create a fully isolated environment with its own network and filesystem, then access it remotely via SSH or NSENTER. - No prior container experience needed!

What You’ll Learn

How Linux namespaces work
Setting up a minimal container filesystem
Configuring isolated networking
Create a Custom cgroup
Limit CPU Usage
Startup a webserver & host a simple webpage
nsenter or SSH access with Dropbear (a lightweight SSH server) optional

Prerequisites

A Linux system (Ubuntu 24.04 used here)
sudo access
Basic terminal familiarity
Internet connection

Step 1: The Complete Script

Here’s the full script we’ll be using. Don’t worry if it looks complex—we’ll break it down line by line!

Save this as create-container.sh:

#!/usr/bin/env bash
# setup_isolated_container_ssh.sh
# This script sets up a fully isolated container environment with its own network namespace using a veth pair,
# then chroots into a minimal filesystem and starts Dropbear SSH daemon and a BusyBox HTTP server.

set -eo pipefail

# CONFIGURATION VARIABLES
CONTAINER_ROOT="/opt/isolated_container"
HOST_VETH="veth-host"
CONTAINER_VETH="veth-container"
HOST_IP="192.168.200.1/24"
CONTAINER_IP="192.168.200.2/24"
GATEWAY_IP="192.168.200.1"
SSH_PORT=22
HTTP_PORT=80

# Clean up any existing container filesystem
rm -rf "${CONTAINER_ROOT}"
mkdir -p "${CONTAINER_ROOT}"/{bin,etc/dropbear,proc,dev,www}

# Create device nodes (if not already present)
if [ ! -e "${CONTAINER_ROOT}/dev/null" ]; then
    mknod -m 666 "${CONTAINER_ROOT}/dev/null" c 1 3
fi
if [ ! -e "${CONTAINER_ROOT}/dev/urandom" ]; then
    mknod -m 666 "${CONTAINER_ROOT}/dev/urandom" c 1 9
fi

# Set up BusyBox in the container filesystem
cp /bin/busybox "${CONTAINER_ROOT}/bin/"
chmod +x "${CONTAINER_ROOT}/bin/busybox"
cd "${CONTAINER_ROOT}/bin"
ln -sf busybox sh
ln -sf busybox ls
ln -sf busybox mkdir
ln -sf busybox cat
ln -sf busybox echo
ln -sf busybox httpd
cd -

# Set up Dropbear in the container filesystem
cp /usr/sbin/dropbear "${CONTAINER_ROOT}/bin/dropbear"
chmod +x "${CONTAINER_ROOT}/bin/dropbear"
# Copy required libraries for dropbear (using ldd)
ldd /usr/sbin/dropbear | awk '/=>/ {print $3}' | while read -r lib; do
    dest_dir="${CONTAINER_ROOT}$(dirname "$lib")"
    mkdir -p "$dest_dir"
    cp "$lib" "$dest_dir/"
done

# Create minimal passwd and group files
echo "root:x:0:0:root:/:/bin/sh" > "${CONTAINER_ROOT}/etc/passwd"
echo "root:x:0:" > "${CONTAINER_ROOT}/etc/group"

# Generate SSH host key if not present
if [ ! -f "${CONTAINER_ROOT}/etc/dropbear/dropbear_rsa_host_key" ]; then
    dropbearkey -t rsa -f "${CONTAINER_ROOT}/etc/dropbear/dropbear_rsa_host_key"
fi

# Set up web content (for testing; remove if not needed)
echo "Isolated Container Web Server Ready!" > "${CONTAINER_ROOT}/www/index.html"

# Set up veth pair on the host
ip link add "$HOST_VETH" type veth peer name "$CONTAINER_VETH"
ip addr add "$HOST_IP" dev "$HOST_VETH"
ip link set "$HOST_VETH" up

# Launch an isolated container shell with unshare and capture its PID
CONTAINER_PID=$(unshare --fork --pid --mount --uts --ipc --net --user --map-root-user bash -c 'echo $$; exec sleep infinity')
echo "Container PID: $CONTAINER_PID"

# Move the container side of the veth pair into the container's network namespace
ip link set "$CONTAINER_VETH" netns "$CONTAINER_PID"

# Configure networking inside the container using nsenter
nsenter --target "$CONTAINER_PID" --net bash -c "
  ip addr add '$CONTAINER_IP' dev $CONTAINER_VETH &&
  ip link set $CONTAINER_VETH up &&
  ip link set lo up &&
  ip route add default via '$GATEWAY_IP'
"
# Chroot into the container and start SSH and HTTP services
  ip link set lo up &&
  ip route add default via '$GATEWAY_IP'
"

# Mount proc in the container
mkdir -p "${CONTAINER_ROOT}/proc"
nsenter --target "$CONTAINER_PID" --mount bash -c "mount -t proc proc ${CONTAINER_ROOT}/proc"


nsenter --target "$CONTAINER_PID" --mount chroot "${CONTAINER_ROOT}" /bin/sh -c "
nsenter --target "$CONTAINER_PID" --mount chroot "${CONTAINER_ROOT}" /bin/sh -c "
  echo 'Inside container: starting HTTP server and Dropbear SSH daemon...';
  echo 'Inside container: starting HTTP server and Dropbear SSH daemon...';
  /bin/httpd -f -p $HTTP_PORT -h /www &
  exec /bin/dropbear -F -E -p $SSH_PORT
"

# End of script

Step 2: Understanding Key Components

1. Container Filesystem

/bin: Contains BusyBox (provides basic Linux commands)
/dev: Device files like null and urandom (required for programs to work)
/etc/dropbear: Stores SSH host keys

2. Network Setup

Creates a virtual Ethernet pair (host-side and container-side)
Container gets IP 192.168.100.2, host uses 192.168.100.1
Port 2222 on host forwards to container’s SSH port (22)

Step 3: Running the Script

Make it executable:

   chmod +x create-container.sh

Execute with sudo:

   sudo ./create-container.sh

Connect via NSENTER:

   - to get the PID run : ps aux | grep sleep  
   sudo nsenter --target <PID> --mount --uts --ipc --net --pid bash

Connect via SSH:

   ssh root@localhost -p 2222

Step 4: What to Expect

Connection

You’ll see a warning about the SSH key fingerprint (this is normal). Type yes to continue.

Inside the Container

Try these commands:

ls /      # See container filesystem
ip addr   # Show container's network
ps aux    # List running processes (only container's)

Step 5. Start a Web Server

Create a file, e.g., /opt/isolated_container/start_services.sh

  #!/bin/sh
  echo "Starting HTTP server..."
  /bin/httpd -p 80 -h /www &
  echo "Starting Dropbear SSH daemon..."
  /bin/dropbear -F -E -p 22

  chmod +x /opt/isolated_container/start_services.sh

  chroot /opt/isolated_container /bin/sh -c "/start_services.sh"

Troubleshooting

SSH Connection Fails

Verify port forwarding:

  iptables -t nat -L PREROUTING

Permission Denied

Ensure you used sudo when running the script
Recreate device files if missing:

  mknod -m 666 /opt/my_container/dev/null c 1 3

How It Works: Beginner’s Glossary

Namespace: A isolated workspace for processes (like a private room)
Veth Pair: Virtual Ethernet cable connecting host and container
BusyBox: Swiss Army knife of Linux commands in a single file
Dropbear: Lightweight SSH server (uses less resources than OpenSSH)

Next Steps

Add Persistence:

Create files in /opt/my_container/www.

Customize SSH:

Add your public key to /opt/my_container/root/.ssh/authorized_keys.

** Verify cgroups v2 is Active**

   mount | grep cgroup2

Create a Custom cgroup
mkdir /sys/fs/cgroup/mycontainer

Limit CPU Usage

   echo "50000 100000" > /sys/fs/cgroup/mycontainer/cpu.max

Limit Memory Usage

   echo $((512 * 1024 * 1024)) > /sys/fs/cgroup/mycontainer/memory.max

Explore More:

Try installing additional software in the container using chroot.

Conclusion

You’ve just created a fully isolated Linux container from scratch! While this isn’t production-ready, it demonstrates core containerization concepts used by Docker and Kubernetes. Experiment with modifying the network setup or adding new services to deepen your understanding.

Happy container hacking! 🐧🔒

Hosting a Page with Nginx: A Newbie’s Journey

Mart Young — Wed, 29 Jan 2025 18:34:31 +0000

How I learned to launch a simple HTML site (and you can too!)

As someone just dipping their toes into DevOps Engineer, Kubernetes Specialists and web hosting, I wanted to conquer a classic "first project": setting up a web server. My goal? Run Nginx on Ubuntu, host a page with my name, and document every stumble and triumph. Spoiler: It worked! Here’s how,

Step 1: Preparing My Ubuntu Playground
I started by firing up a clean Ubuntu 22.04 environment. For simplicity, I used a virtual machine(ec2-Instance) on AWS Docs-here, but you could also

Use any other cloud server (Google cloud/Azure/Digital Ocean)
Or even Docker (which I’ll mention later!).

Pro Tip: If you’re using Docker, this one-liner saves time (see Docker’s getting-started guide):

docker run -it -p 80:80 --name my-nginx ubuntu:latest

Step 2: Installing Nginx – The “Aha!” Moment

Updating packages felt like stretching before a workout:

sudo apt update

Then came the magic command, as recommended in Nginx’s official installation guide:

sudo apt install nginx -y

Fun Fact: Nginx (pronounced "Engine-X") powers over 33% of websites. Now I’m one of them!

Step 3: Bringing Nginx to Life

Starting the service:

sudo systemctl start nginx

Making sure it auto-restarts on reboots (learn more about systemctl here):

sudo systemctl enable nginx

At this point, I nervously opened my browser to http://<aws public ip>... and saw the default Nginx page! Progress!

Step 4: Crafting My Digital Hello World

Time to replace the generic page with my own. I navigated to the web root (Nginx’s default directory structure documented here):

cd /var/www/html

Then edited index.html with Nano (Vim warriors, fight me later ):

sudo nano index.html

I wrote this barebones HTML (swap "[My Name]" with yours!):

<!DOCTYPE html>
<html>
<head>
    <title>Welcome to My Corner of the Internet</title>
    <style>
        body {
            display: flex;
            flex-direction: column;
            align-items: center;
            justify-content: center;
            min-height: 100vh; /* Full viewport height */
            margin: 0;
            font-family: Arial, sans-serif;
        }
        h1 {
            color: #2c3e50; /* Modern dark-blue color */
        }
        p {
            font-size: 1.2em;
        }
    </style>
</head>
<body>
    <h1>Hey there! I'm [my name].</h1>
    <h2>Welcome to DevOps Stage 0 - [slack display name]</h2>
</body>
</html>

Ctrl+O to save, Ctrl+X to exit. Simple as that!

Step 5: The Grand Reveal

I refreshed http://<AWS - public-ip> and… there it was! My name in bold, staring back like a digital monument. For extra validation, on my terminal I ran:

curl http://<AWS - public-ip>

:For Docker

curl http://localhost

The terminal echoed my HTML – pure satisfaction.

Pro Tip: When trying to access your webpage with AWS public-IP, remember to use http and not https

Bumps Along the way

Security Group Fiasco: My first attempt failed due to security group issues on AWS. I had to spend sometime reading this AWS-Docs and was able to resolve it by adding a rule to my security group
Docker Dilemma: When testing the Docker method, I forgot to mount the HTML file. Protip for Docker fans (Docker volumes explained here):

  docker run -d -p 80:80 -v $(pwd)/index.html:/usr/share/nginx/html/index.html nginx

Why This Matters

This tiny project taught me:

Servers aren’t magic – just software following instructions.
Documentation is survival gear. I kept notes like this:

   # My Cheat Sheet
   - Web root: /var/www/html
   - Test command: curl localhost
   - Logs: /var/log/nginx/error.log

Troubleshooting is 80% of the job. (But that 20% success? Worth it.)

References & Further Reading

Nginx Beginner’s Guide
Ubuntu Server Documentation
Docker’s Official Tutorials
Mozilla’s HTML Basics (for crafting your page)
Get started with Amazon EC2

Final Thoughts

In under an hour, I went from zero to hosting my own page. Whether you’re building a portfolio, a blog, or just experimenting, Nginx on Ubuntu is a sturdy foundation. Next step? Maybe add CSS, or try HTTPS with Let’s Encrypt!

Got questions or war stories? Drop them in the comments (or tweet me)! Let’s geek out.

Enjoyed this walkthrough? Share it with a friend who’s starting out their web hosting or DevOps journey!