DEV Community: anitaalicloud

Containerizing a Node.js API with Docker and Automating Deployments to Amazon ECR with GitHub Action

anitaalicloud — Fri, 10 Jul 2026 09:55:07 +0000

Introduction

In my previous article, I walked through building a lightweight Node.js REST API and deploying it live on an AWS EC2 instance with Nginx and PM2. If you haven't read that yet, I'd recommend starting there — https://dev.to/anitaalicloud/how-to-build-and-deploy-a-nodejs-rest-api-on-aws-ec2-with-pm2-cg8 — as this article picks up right where that one left off.

Here, we take the same API further. We containerize it using a multi stage Dockerfile, create a private Amazon ECR repository to store our images, and wire up a GitHub Actions CI pipeline that automatically builds, scans, and pushes a versioned Docker image every time we push to main.

By the end, every push to your repository will trigger a pipeline that produces a clean, security scanned, semantically versioned Docker image sitting in ECR with zero manual steps.

What We Are Adding

What	Why
Multi-stage Dockerfile	Containerize the API with a lean, secure production image
Amazon ECR	Private registry to store and version our Docker images
GitHub Actions CI	Automate the build, scan, and push on every push to `main`
Trivy security scan	Gate the pipeline — no vulnerable image reaches ECR
Semantic versioning	Every image build gets a unique, traceable tag

Prerequisites

The Node.js API from Article 1- https://dev.to/anitaalicloud/how-to-build-and-deploy-a-nodejs-rest-api-on-aws-ec2-with-pm2-cg8 already on GitHub
Docker installed locally
An AWS account with an IAM user (not root)
A public GitHub repository

Step 1: The Multi Stage Dockerfile

Why Multi Stage?

A standard Dockerfile builds everything in one layer, dev tools, build dependencies, and runtime all end up in the final image. A multi-stage build separates concerns: the first stage installs and builds, the second stage starts clean and copies only what is needed to run the app. The result is a smaller, more secure production image.

Create a Dockerfile in the root of your repo:

# ─── Stage 1: Build ───────────────────────────────────────────
FROM node:20-alpine AS builder

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

COPY . .

# ─── Stage 2: Production ──────────────────────────────────────
FROM node:20-alpine AS production

WORKDIR /app

# Run as a non-root user for security
RUN addgroup -S appgroup && adduser -S appuser -G appgroup

COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/index.js .
COPY --from=builder /app/package.json .

USER appuser

EXPOSE 3000

CMD ["node", "index.js"]

A few important details:

npm ci reads from package-lock.json to install exact dependency versions, making builds reproducible across environments
--from=builder copies only the files the app needs to run. Nothing else from the build stage makes it into the final image. Running as appuser instead of root reduces the attack surface if the container is ever compromised

Testing the Build Locally

docker build --target production -t personal-api:local .
docker run -p 3000:3000 personal-api:local

In a second terminal:

curl http://localhost:3000/
curl http://localhost:3000/health
curl http://localhost:3000/me

Verify the running container:

docker ps

Push the Dockerfile to GitHub:

git add Dockerfile
git commit -m "docker: add multi-stage Dockerfile"
git push origin main

Step 2: Create the Amazon ECR Repository

ECR is AWS's private container registry. This is where our pipeline will push versioned images.

Via AWS Console

Log into AWS as your IAM user (not root)
Go to ECR → Create repository
Make sure you are in the correct region — eu-north-1 (Stockholm) in my case
Name the repository personal-api, visibility: Private
Click Create repository
Copy the repository URI — you will need this shortly:

   123456789012.dkr.ecr.eu-north-1.amazonaws.com/personal-api

Via AWS CLI

aws ecr create-repository \
  --repository-name personal-api \
  --region eu-north-1

Step 3: Set Up the IAM User

Never use root AWS credentials in a CI pipeline. Create a scoped IAM user with only the permissions the pipeline needs:

Go to IAM → Users → Create user
Name it devops-user
Attach the policy: AmazonEC2ContainerRegistryFullAccess
Go to Security credentials → Create access key
Select CLI, then copy both the Access Key ID and Secret Access Key — you only see the secret key once

Step 4: Add GitHub Secrets

The pipeline uses your AWS credentials to authenticate with ECR. Store them as GitHub secrets so they are never exposed in your code.

Go to your repo → Settings → Secrets and variables → Actions → New repository secret and add:

Secret	Value
`AWS_ACCESS_KEY_ID`	Your IAM access key ID
`AWS_SECRET_ACCESS_KEY`	Your IAM secret access key
`AWS_REGION`	`eu-north-1`
`ECR_REPOSITORY_URI`	Your full ECR URI

Step 5: The GitHub Actions CI Pipeline

Create the file .github/workflows/ci.yml in your repo:

name: CI — Build, Scan & Push to ECR

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  ECR_REPOSITORY_URI: ${{ secrets.ECR_REPOSITORY_URI }}
  AWS_REGION: ${{ secrets.AWS_REGION }}

jobs:
  build-scan-push:
    name: Build, Scan & Push
    runs-on: ubuntu-latest

    steps:
      # ── Phase 1: Build ─────────────────────────────────────────
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Generate semantic version
        id: version
        run: |
          MAJOR=1
          MINOR=0
          PATCH=${{ github.run_number }}
          echo "VERSION=v${MAJOR}.${MINOR}.${PATCH}" >> $GITHUB_OUTPUT

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build Docker image
        run: |
          docker build \
            --target production \
            -t ${{ env.ECR_REPOSITORY_URI }}:${{ steps.version.outputs.VERSION }} \
            -t ${{ env.ECR_REPOSITORY_URI }}:latest \
            .

      # ── Phase 2: Scan ──────────────────────────────────────────
      - name: Scan image with Trivy
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: ${{ env.ECR_REPOSITORY_URI }}:${{ steps.version.outputs.VERSION }}
          format: table
          exit-code: 1
          severity: CRITICAL
          ignore-unfixed: true

      # ── Phase 3: Push ──────────────────────────────────────────
      - name: Push image to ECR
        run: |
          docker push ${{ env.ECR_REPOSITORY_URI }}:${{ steps.version.outputs.VERSION }}
          docker push ${{ env.ECR_REPOSITORY_URI }}:latest

Understanding the Three Pipeline Phases

Phase 1 — Build

The pipeline checks out the code, generates a semantic version tag, authenticates with AWS using the secrets you added, logs into ECR, and builds the Docker image.

The version tag follows the format v{MAJOR}.{MINOR}.{PATCH}. Major and minor are set manually. PATCH is set to github.run_number — a counter GitHub increments automatically on every pipeline run. So your images end up tagged like:

v1.0.1  ← first push
v1.0.2  ← second push
v1.0.3  ← third push

Each build also gets a latest tag pointing to the most recent successful push. This means you always have a traceable history of every image ever built, and rolling back is as simple as pulling a previous version tag from ECR.

Phase 2 — Scan

Before any image reaches ECR, Trivy scans it for known security vulnerabilities. Setting exit-code: 1 means the pipeline fails immediately if any CRITICAL vulnerabilities are found — the image is discarded and never pushed to the registry.

ignore-unfixed: true skips vulnerabilities that have no available fix yet. This keeps the pipeline practical rather than blocking builds over issues that cannot be resolved at this point in time.

Phase 3 — Push

This step only runs if the Trivy scan passes. It pushes both tags — the semantic version and latest — to ECR. After a successful run, you can verify the pushed image in AWS Console → ECR → personal-api → Images.

Triggering the Pipeline

Push the workflow file to GitHub:

git add .github/workflows/ci.yml
git commit -m "ci: add GitHub Actions CI pipeline"
git push origin main

Go to your repo → Actions tab and watch the pipeline run through all three phases.

Final Project Structure

nodejs-api-ci/
├── .github/
│   └── workflows/
│       └── ci.yml          # CI pipeline
├── Dockerfile               # Multi-stage build
├── index.js                 # Express API
├── package.json
├── package-lock.json
└── README.md

Key Takeaways

Multi-stage Dockerfiles produce smaller, cleaner images. The production stage only contains what is needed to run — nothing from the build environment leaks through.
Never use root AWS credentials in CI. A scoped IAM user with only ECR permissions limits the blast radius if credentials are ever compromised.
Never hardcode credentials. GitHub secrets keep sensitive values out of your codebase entirely — the workflow file is public, your secrets are not.
Scan before you push. Trivy as a pipeline gate means only clean images reach your registry.
Semantic versioning makes every build traceable. If something breaks in production, you know exactly which image is running and can roll back to any previous version instantly.

Live Project

📦 GitHub Repo: https://github.com/AnitaAliCloud/nodejs-api-ci

🐳 ECR Repository: devopsprod.duckdns.org

This is Part 2 of my DevOps series. Read Part 1 here: https://dev.to/anitaalicloud/how-to-build-and-deploy-a-nodejs-rest-api-on-aws-ec2-with-pm2-cg8

How to Build and Deploy a Node.js REST API on AWS EC2 with PM2

anitaalicloud — Tue, 16 Jun 2026 11:34:09 +0000

Introduction

Knowing how to get an API from your local machine to a live, publicly accessible server is one of the most practical skills in a developer or DevOps engineer's toolkit. In this guide, I'll walk you through the full process, from writing a minimal Express API to deploying it on AWS EC2 with PM2 keeping it alive persistently.

By the end, you'll have a running API accessible from any browser or HTTP client, with no manual restarts required.

Prerequisites

An AWS account (free tier works)
Node.js installed locally
Basic familiarity with the Linux command line
A GitHub account

What We're Building

A lightweight three-endpoint JSON API:

Endpoint	Response
`GET /`	`{ "message": "API is running" }`
`GET /health`	`{ "message": "healthy" }`
`GET /me`	`Name`, `email`, and `GitHub link`

All endpoints return Content-Type: application/json, HTTP 200, and respond in well under 500ms.

Step 1: Writing the API

On your local machine, initialize a new Node.js project and install Express:

mkdir personal-api
cd personal-api
npm init -y
npm install express

Create index.js:

const express = require("express");
const app = express();
const PORT = 3000;

app.get("/", (req, res) => {
  res.status(200).json({ message: "API is running" });
});

app.get("/health", (req, res) => {
  res.status(200).json({ message: "healthy" });
});

app.get("/me", (req, res) => {
  res.status(200).json({
    name: "Your Full Name",
    email: "you@example.com",
    github: "https://github.com/yourusername",
  });
});

app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

Test it locally:

node index.js
curl http://localhost:3000/
curl http://localhost:3000/health
curl http://localhost:3000/me

All three should return the expected JSON with a 200 status. ✅

Step 2: Push to GitHub

Create a public repository and push your code:

git init
git add .
git commit -m "Initial commit: personal API"
git remote add origin https://github.com/YOUR_USERNAME/personal-api.git
git push -u origin main

Step 3: Provision an AWS EC2 Instance

Launch a t2.micro (free tier eligible) instance with Ubuntu 22.04 LTS
Create or reuse a key pair for SSH
In the Security Group, open inbound traffic on:
- Port 22 (SSH)
- Port 80 (HTTP)

ssh -i your-key.pem ubuntu@YOUR_EC2_PUBLIC_IP

Step 4: Set Up the Server Environment

Install Node.js

curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs
node -v

Clone the Repo and Install Dependencies

cd ~
git clone https://github.com/YOUR_USERNAME/personal-api.git
cd personal-api
npm install

Quick sanity check:

node index.js

Step 5: Keep the App Running with PM2

If you start the app with node index.js and close your SSH session, the process dies. PM2 solves this — it's a production process manager that restarts your app on crashes and survives server reboots.

Install PM2

sudo npm install -g pm2

Start the App

pm2 start index.js --name personal-api

Enable PM2 on Boot

pm2 startup
# Run the command it outputs, then:
pm2 save

From now on, the app restarts automatically after any reboot or crash.

Verify:

pm2 list

Step 6: Configure Nginx as a Reverse Proxy

The app runs on port 3000 internally. Rather than exposing that port to the public, we use Nginx to listen on port 80 and forward requests to the app. This is the standard production pattern.

Install Nginx

sudo apt update
sudo apt install nginx -y
sudo systemctl enable nginx
sudo systemctl start nginx

Create the Site Config

sudo nano /etc/nginx/sites-available/personal-api

Paste the following (replace YOUR_EC2_PUBLIC_IP with your actual IP or domain):

server {
    listen 80;
    server_name YOUR_EC2_PUBLIC_IP;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Enable and Reload

sudo ln -s /etc/nginx/sites-available/personal-api /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

This should be your expected result.

Step 7: Test the Live Deployment

curl http://YOUR_EC2_PUBLIC_IP/
curl http://YOUR_EC2_PUBLIC_IP/health
curl http://YOUR_EC2_PUBLIC_IP/me

All three return the correct JSON over HTTP. ✅

Final Project Structure

personal-api/
├── index.js        # Express API
├── package.json
└── README.md

Key Takeaways

Never expose your app port directly. Nginx as a reverse proxy adds a layer of control and is the correct production pattern — it also makes it easy to host multiple apps on one server later.
PM2 is non-negotiable for Node.js in production. Without a process manager, one crash or reboot silently kills your service.
Always test locally first. It's much faster to debug on your machine than on a remote server.
EC2 security groups are your firewall. By default, everything is blocked — be deliberate about what you open.

Live Demo

GitHub Repo: https://github.com/AnitaAliCloud/hng-stage1-api

Containerizing and Deploying a Microservice App with Docker and CI/CD

anitaalicloud — Mon, 01 Jun 2026 17:17:41 +0000

Introduction

In this project, I worked on a microservice based application and focused on making it production ready using Docker and CI/CD pipelines.

Instead of just running the application locally, the goal was to:

Understand how multiple services interact
Package them using Docker
Orchestrate them with Docker Compose
Automate testing, security checks, and deployment using GitHub Actions

This article explains how to run, test, and deploy the project from scratch.

Project Repository

GitHub Repo: https://github.com/AnitaAliCloud/hng14-stage2-devops

System Architecture

The system consists of four main components:

Frontend (Node.js) — user interface for submitting jobs
Backend API (FastAPI) — handles job creation and status
Worker Service (Python) — processes jobs asynchronously
Redis — message broker between services

Flow Diagram:

Frontend → API → Redis Queue → Worker → Redis → API → Frontend

Prerequisites

Before running this project, ensure you have:

Docker installed → https://docs.docker.com/get-docker/
Docker Compose installed
Git installed
(Optional) Node.js and Python for local debugging

How to Run the Project Locally

1. Clone the repository

git clone https://github.com/YOUR_USERNAME/YOUR_REPO.git
cd YOUR_REPO

2. Set up environment variables

Create a .env file:

cp .env.example .env

Update values if needed.

3. Start all services with Docker Compose

docker-compose up --build

4. Access the application

Frontend: http://localhost:3000
API: http://localhost:8000
Redis: internal only (not exposed)

Running Health Checks

Each service includes a health endpoint:

/health → API health status
Docker health checks ensure services restart automatically if unhealthy

CI/CD Pipeline (GitHub Actions)

The project includes an automated pipeline:

Pipeline stages:

Lint (flake8, eslint, hadolint)
Test (unit tests with pytest)
Build Docker images
Security scan (Trivy)
Integration tests
Deployment

CI/CD Workflow Screenshot:

Running Tests Locally

To run tests manually:

pytest

For coverage report:

pytest --cov=app

Security Checks

The pipeline includes:

Docker image scanning with Trivy
Failing builds on critical vulnerabilities
No secrets stored in images or repository

Troubleshooting

Containers not starting?

Check logs:

docker-compose logs -f

Port already in use?

sudo lsof -i :3000

Key Learnings

This project helped reinforce:

How microservices communicate in real systems
Why containerization improves reliability
Importance of CI/CD automation
How infrastructure impacts application behavior

Conclusion

This project demonstrates how a multi-service application can be containerized, tested, and deployed in a structured and repeatable way.

Beyond just writing code, the focus was on building a system that can be reliably shipped and maintained.

Useful Links

GitHub Repository: https://github.com/AnitaAliCloud/hng14-stage2-devops
CI/CD Pipeline: (GitHub Actions tab in repo)

Building and Deploying a Simple REST API on a Linux VPS with Nginx

anitaalicloud — Mon, 01 Jun 2026 10:55:08 +0000

Introduction

I recently built and deployed a simple REST API on a Linux server to strengthen my understanding of how backend services are hosted and managed in real world environments.

The goal was not to build a complex application, but to understand how an API runs in production from server setup to deployment and reverse proxy configuration.

What I Built

I created a lightweight REST API with three endpoints:

/ → confirms the API is running
/health → checks service health
/me → returns basic profile information in JSON format

Each endpoint returns a JSON response and is optimized for fast response times.

Tech Stack

Node.js / Python (depending on your implementation)
Linux VPS (Ubuntu)
Nginx (reverse proxy)
PM2 / Systemd (process management)

How It Works

The application runs on a local server port and is not exposed directly to the internet.

Instead, Nginx handles incoming traffic and forwards it to the application.

Simple flow:

Client → Nginx → API Server (localhost)

This setup improves security, scalability, and control over traffic routing.

Deployment Approach

The process involved:

Setting up a Linux server environment
Running the API on a local port
Configuring Nginx as a reverse proxy
Using a process manager to keep the application running continuously

Once deployed, the API became publicly accessible through the server’s IP address or domain.

Key Learnings

This project helped me understand:

How APIs behave outside of local development
The role of reverse proxies in production systems
Why process managers are important for uptime
How Linux servers are used in real deployments

It was a simple project, but it gave me a clearer picture of how backend services are structured in real world systems.

Conclusion

Deploying this API helped bridge the gap between writing code and actually running it in a production like environment.

It reinforced the importance of understanding infrastructure, not just application logic.

How I Built SwiftDeploy: A Tool That Writes Its Own Infrastructure

anitaalicloud — Wed, 06 May 2026 19:28:10 +0000

A deep dive into declarative deployments, OPA policy gates, and chaos engineering from Stage 4A to 4B

Introduction

Most DevOps tasks ask you to configure infrastructure manually. This one asked me to build the tool that does it for me.

The result is SwiftDeploy which is a CLI tool that reads a single manifest.yaml file and generates your entire deployment stack from it. Nginx configs, Docker Compose files, policy checks, live metrics dashboards are all derived from one source of truth.

This post covers the full journey: the design decisions, the guardrails, the chaos, and the lessons learned.

The Architecture

Here is how all the pieces connect:

┌─────────────────────────────────────────────────────┐
│                    manifest.yaml                     │
│              (single source of truth)                │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
              ./swiftdeploy init
                       │
          ┌────────────┴────────────┐
          ▼                         ▼
     nginx.conf              docker-compose.yml
   (generated)                 (generated)
          │                         │
          ▼                         ▼
┌─────────────────────────────────────────────────────┐
│                   Docker Stack                       │
│                                                      │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐      │
│   │  Nginx   │───▶│   App    │    │   OPA    │      │
│   │  :8080   │    │  :3000   │    │  :8181   │      │
│   └──────────┘    └──────────┘    └──────────┘      │
│   (public)        (internal)      (internal)         │
└─────────────────────────────────────────────────────┘
          │                         ▲
          ▼                         │
     curl :8080              CLI queries OPA
    (your browser)          before deploy/promote

The key insight: you only ever touch manifest.yaml. The tool handles everything else.

Part 1 — The Design: A Tool That Writes Its Own Files

The Problem with Handwritten Config

When you write nginx.conf and docker-compose.yml by hand, you introduce drift. Change a port in one place and forget to update it in another. After a few weeks, nobody knows which file is the source of truth.

SwiftDeploy solves this with a three-layer system:

manifest.yaml          →    templates/*.tmpl    →    generated files
(VALUES)                    (STRUCTURE)              (VALUES + STRUCTURE)

The manifest.yaml holds all the values — ports, image names, modes, timeouts. The templates hold the structure — how nginx.conf and docker-compose.yml should look. The CLI combines them at runtime.

How `swiftdeploy init` Works

# Read manifest into a Python dict
m = yaml.safe_load(open("manifest.yaml"))

# Build a replacements map
replacements = {
    "{{NGINX_PORT}}":   str(m["nginx"]["port"]),
    "{{SERVICE_PORT}}": str(m["services"]["port"]),
    # ... etc
}

# Read template, replace placeholders, write output
with open("templates/nginx.conf.tmpl") as f:
    content = f.read()

for placeholder, value in replacements.items():
    content = content.replace(placeholder, value)

with open("nginx.conf", "w") as f:
    f.write(content)

Simple string replacement. No Jinja2, no templating engine — just Python's built-in str.replace(). The grader can delete nginx.conf and docker-compose.yml, run ./swiftdeploy init, and they regenerate perfectly every time.

The API Service

The API is a Python HTTP server using only the standard library — no Flask, no FastAPI. This keeps the Docker image under 60MB (well under the 300MB limit).

It runs in two modes controlled by a MODE environment variable injected by Docker Compose:

stable mode  →  normal behaviour
canary mode  →  adds X-Mode: canary header + activates /chaos endpoint

The same image runs both modes. The only difference is the environment variable.

The Nginx Reverse Proxy

Nginx sits in front of the app and adds:

X-Deployed-By: swiftdeploy header on every response
JSON error bodies on 502/503/504 (instead of ugly HTML)
Structured access logs in the required format
Forwards X-Mode header from the upstream app

Critically, the app port is never exposed directly. Only Nginx's port is mapped to the host. All traffic must flow through it.

Part 2 — The Guardrails: OPA Policy Enforcement

Why OPA?

The task required that the CLI never make allow/deny decisions itself. All logic must live in OPA (Open Policy Agent).

This matters because it separates concerns cleanly:

CLI  →  collects data, calls OPA, surfaces the result
OPA  →  owns all decision logic, never called by the app

If you want to change a policy, you edit a .rego file. You never touch the CLI. If you want to change a threshold, you edit data.json. You never touch the Rego files.

Policy Structure

Each policy domain owns exactly one question:

Infrastructure policy — Is the host healthy enough to deploy?

package infrastructure

import rego.v1

default allow := false

allow if {
    count(violations) == 0
}

violations contains msg if {
    input.disk_free_gb < data.infrastructure.min_disk_free_gb
    msg := sprintf(
        "Disk free (%.1fGB) is below minimum threshold (%.1fGB)",
        [input.disk_free_gb, data.infrastructure.min_disk_free_gb]
    )
}

violations contains msg if {
    input.cpu_load > data.infrastructure.max_cpu_load
    msg := sprintf(
        "CPU load (%.2f) exceeds maximum threshold (%.2f)",
        [input.cpu_load, data.infrastructure.max_cpu_load]
    )
}

Canary safety policy — Is the canary healthy enough to promote?

package canary

import rego.v1

default allow := false

allow if {
    count(violations) == 0
}

violations contains msg if {
    input.error_rate > data.canary.max_error_rate
    msg := sprintf(
        "Error rate (%.2f%%) exceeds maximum threshold (%.2f%%)",
        [input.error_rate * 100, data.canary.max_error_rate * 100]
    )
}

Threshold values live in data.json — never hardcoded in Rego:

{
  "infrastructure": {
    "min_disk_free_gb": 10.0,
    "max_cpu_load": 16.0,
    "min_mem_free_percent": 10.0
  },
  "canary": {
    "max_error_rate": 0.01,
    "max_p99_latency_ms": 500
  }
}

The Hard Gate in Action

When the CPU load exceeded the threshold, swiftdeploy deploy was blocked:

[deploy] Running OPA pre-deploy policy checks...
  Host stats: disk=80.08GB free, cpu_load=12.88, mem_free=50.0%
[policy] Checking Infrastructure...
  [BLOCK] Infrastructure policy FAILED:
    x CPU load (12.88) exceeds maximum threshold (2.00)
[deploy] BLOCKED by policy. Fix violations above before deploying.

The deploy never started. The CLI surfaced the exact violation reason from OPA — no guessing required.

OPA Isolation

OPA is intentionally isolated from public Nginx ingress. It runs on port 8181 inside the Docker network. It is NOT behind Nginx, and its port is only accessible to the CLI running on the host. A user hitting localhost:8080 cannot reach OPA.

Failure Handling

The CLI handles every distinct OPA failure mode differently:

except urllib.error.URLError as e:
    # OPA unreachable — warn but don't crash
    return {"allowed": False, "violations": [], 
            "error": f"OPA unreachable: {e.reason}"}

except json.JSONDecodeError:
    # OPA returned garbage — different message
    return {"allowed": False, "violations": [], 
            "error": "OPA returned invalid JSON"}

except Exception as e:
    # Catch-all — still doesn't crash
    return {"allowed": False, "violations": [], 
            "error": f"Unexpected OPA error: {e}"}

The CLI never crashes or hangs when OPA is unavailable. It warns the operator and continues.

Part 3 — The Chaos: Breaking Things on Purpose

The /metrics Endpoint

The API exposes a /metrics endpoint in Prometheus text format:

# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total{method="GET",path="/",status_code="200"} 42

# HELP http_request_duration_seconds Request latency
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_bucket{method="GET",path="/",le="0.005"} 38

# HELP app_mode Current deployment mode (0=stable, 1=canary)
# TYPE app_mode gauge
app_mode 1

# HELP chaos_active Current chaos state (0=none, 1=slow, 2=error)
# TYPE chaos_active gauge
chaos_active 0

No third-party libraries — pure Python calculating histogram buckets manually.

Injecting Chaos

After promoting to canary mode, chaos was injected:

# Slow mode — every request sleeps 3 seconds
curl -X POST http://localhost:8080/chaos \
  -H "Content-Type: application/json" \
  -d '{"mode": "slow", "duration": 3}'

# Error mode — 50% of requests return HTTP 500
curl -X POST http://localhost:8080/chaos \
  -H "Content-Type: application/json" \
  -d '{"mode": "error", "rate": 0.5}'

The Status Dashboard Capturing the Failure

With error mode active at 50%, the status dashboard showed:

=======================================================
  SwiftDeploy Status Dashboard
  2026-05-06T14:18:43Z
=======================================================

  Mode:        CANARY
  Uptime:      892s
  Req/s:       2.40
  P99 Latency: 250ms
  Error Rate:  48.20%

  Policy Compliance:
    + Infrastructure: PASSING
    x Canary Safety: FAILING
      -> Error rate (48.20%) exceeds maximum threshold (1.00%)

The canary safety policy immediately flagged the failure. Attempting to promote to stable at this point would have been blocked by OPA.

Recovery

curl -X POST http://localhost:8080/chaos \
  -H "Content-Type: application/json" \
  -d '{"mode": "recover"}'

Within one scrape cycle the dashboard showed error rate back to 0% and canary safety back to PASSING.

Part 4 — The Audit Trail

Every significant event is written to history.jsonl:

{"event": "deploy_success", "timestamp": "2026-05-06T13:53:39Z"}
{"event": "promote_success", "target": "canary", "timestamp": "2026-05-06T14:01:22Z"}
{"event": "status_scrape", "mode": "canary", "error_rate": 0.482, "timestamp": "2026-05-06T14:18:43Z"}

Running ./swiftdeploy audit parses this file and generates a clean Markdown report:

## Timeline
| Timestamp | Event | Details |
|---|---|---|
| 2026-05-06T13:53:39Z | Deploy | Stack deployed successfully |
| 2026-05-06T14:01:22Z | Promote | Mode switched to canary |

## Policy Violations
| Timestamp | Policy | Reason |
|---|---|---|
| 2026-05-06T14:18:43Z | Canary Safety | error_rate=48.20%, p99=250ms |

Lessons Learned

1. Single source of truth is worth the extra complexity
It felt like overkill to build a template engine just to generate two config files. But when the grader deletes your generated files and reruns init, you're grateful every value comes from one place.

2. OPA's syntax changes between versions
The latest OPA image requires import rego.v1 and the if/contains keywords. Older Rego syntax silently fails to load. Always check your OPA container logs first.

3. Start OPA before running policy checks
OPA is part of the stack, so it doesn't exist before docker compose up. The fix was to start OPA first as a separate step, wait 3 seconds for it to load policies, then run the pre-deploy check.

4. Chaos engineering reveals what metrics matter
Before injecting chaos, the /metrics endpoint felt like box-ticking. After watching the error rate spike to 48% in real time on the status dashboard while OPA simultaneously flagged the canary safety policy — the value became obvious.

5. Policy as code beats policy as documentation
A README saying "don't deploy if CPU load is above 2.0" gets ignored. A Rego file that blocks the deploy enforces it automatically.

The Full Subcommand Reference

./swiftdeploy init              # generate nginx.conf + docker-compose.yml
./swiftdeploy validate          # 5 pre-flight checks
./swiftdeploy deploy            # OPA check + start stack + health wait
./swiftdeploy promote canary    # switch to canary mode
./swiftdeploy promote stable    # switch back to stable
./swiftdeploy status            # live metrics + policy compliance dashboard
./swiftdeploy audit             # generate audit_report.md
./swiftdeploy teardown          # stop all containers
./swiftdeploy teardown --clean  # stop + delete generated files

Source Code

The full project is available on GitHub: https://github.com/AnitaAliCloud/hng4-devops

Built as part of the HNG DevOps Track — Stage 4A and 4B

How I Built an Anomaly Detection Engine for DDoS Protection

anitaalicloud — Tue, 28 Apr 2026 03:56:36 +0000

Introduction
Imagine you run a busy website. On a normal day, about 50 people visit per second. Then suddenly, 5,000 requests flood in every second from a single IP address. Your server crashes. Your real users can't access anything. This is a DDoS (Distributed Denial of Service) attack.
In this post, I'll explain how I built a tool that watches incoming traffic in real time, learns what "normal" looks like, and automatically blocks attackers before they can cause damage.

What Does the Tool Do?
My anomaly detection engine does 6 things automatically:

Watches every HTTP request coming into the server
Learns what normal traffic looks like over time
Detects when traffic becomes abnormal
Blocks the attacker using the Linux firewall
Alerts me on Slack within 10 seconds
Unbans the IP automatically after a timeout

The Architecture
Internet Traffic
↓
Nginx (logs every request as JSON)
↓
Nextcloud (the actual app)

Detector Daemon reads Nginx logs
↓
Sliding Window → tracks request rates
Rolling Baseline → learns normal traffic
Z-score Detection → spots anomalies
iptables → blocks attackers
Slack → sends alerts
Dashboard → shows live metrics

Part 1 — How the Sliding Window Works
Think of the sliding window like a 60 second camera 🎥
Every request that comes in gets a timestamp. We store these timestamps in a Python deque (double-ended queue) — one for each IP address and one for global traffic.

from collections import deque
import time

One deque per IP

ip_windows = {}

def record_request(ip):
now = time.time()

if ip not in ip_windows:
    ip_windows[ip] = deque()

# Add this request
ip_windows[ip].append(now)

# Remove requests older than 60 seconds from the LEFT
cutoff = now - 60
while ip_windows[ip] and ip_windows[ip][0] < cutoff:
    ip_windows[ip].popleft()

# Current rate = how many requests in last 60 seconds
current_rate = len(ip_windows[ip]) / 60
return current_rate

from collections import deque
import time

One deque per IP

ip_windows = {}

def record_request(ip):
now = time.time()

if ip not in ip_windows:
    ip_windows[ip] = deque()

# Add this request
ip_windows[ip].append(now)

# Remove requests older than 60 seconds from the LEFT
cutoff = now - 60
while ip_windows[ip] and ip_windows[ip][0] < cutoff:
    ip_windows[ip].popleft()

# Current rate = how many requests in last 60 seconds
current_rate = len(ip_windows[ip]) / 60
return current_rate

The magic is the eviction — old timestamps get removed from the left side of the deque automatically. So the deque always contains only the last 60 seconds of requests. The current rate is simply the length of the deque divided by 60.

Part 2 — How the Baseline Learns from Traffic
The baseline answers one question: "What is normal?"
We can't hardcode this because every website is different. A news site might normally get 1000 req/s. A small blog might get 2 req/s. So we let the system learn.
Every second we record how many requests came in. Every 60 seconds we look at the last 30 minutes of data and calculate:

import math

def recalculate_baseline(per_second_counts):
# Calculate average requests per second
mean = sum(per_second_counts) / len(per_second_counts)

# Calculate how much it normally varies
variance = sum((x - mean) ** 2 for x in per_second_counts) / len(per_second_counts)
stddev = math.sqrt(variance)

# Apply floors to prevent false alarms on quiet traffic
effective_mean = max(mean, 1.0)
effective_stddev = max(stddev, 1.0)

return effective_mean, effective_stddev

We also maintain per-hour slots — the system prefers the current hour's data when it has enough samples. This means the baseline adapts to time-of-day patterns. Rush hour traffic looks different from 3 AM traffic!

Part 3 — How the Detection Logic Makes a Decision
Once we have the baseline we use a Z-score to decide if current traffic is anomalous.
The Z-score answers: "How many standard deviations away from normal is this?"

def is_anomalous(current_rate, mean, stddev):
# Z-score calculation
z_score = (current_rate - mean) / stddev

# Rate multiplier
rate_multiplier = current_rate / mean

# Flag as anomalous if EITHER condition fires
if z_score > 3.0:
    return True, "z-score exceeded 3.0"

if rate_multiplier > 5.0:
    return True, "rate exceeded 5x baseline"

return False, None

Example:

Normal traffic: 50 req/s (mean=50, stddev=10)
Attack traffic: 5000 req/s from one IP
Z-score = (5000 - 50) / 10 = 495
495 > 3.0 → ANOMALY DETECTED! 🚨

We also detect error surges — if an IP is getting lots of 404/500 errors it might be scanning for vulnerabilities. In that case we tighten the thresholds automatically.

Part 4 — How iptables Blocks an IP
iptables is Linux's built-in firewall. It runs in the kernel and can drop packets before they even reach your application.
When we detect an attack:

import subprocess

def ban_ip(ip):
# Add a DROP rule — silently discard all packets from this IP
subprocess.run([
"iptables", "-A", "INPUT",
"-s", ip,
"-j", "DROP"
])
print(f"Banned {ip}")

def unban_ip(ip):
# Remove the DROP rule
subprocess.run([
"iptables", "-D", "INPUT",
"-s", ip,
"-j", "DROP"

])
print(f"Unbanned {ip}")
The -j DROP means "jump to DROP action" — the packet is silently discarded. The attacker doesn't even get an error message back. From their perspective the server just stopped responding.
Bans lift automatically on a backoff schedule:

1st offence → 10 minutes
2nd offence → 30 minutes
3rd offence → 2 hours
4th+ → permanent

Part 5 — The Live Dashboard
The dashboard is a simple web page that refreshes every 3 seconds showing:

Global requests per second
Currently banned IPs
Top 10 source IPs
CPU and memory usage
Current baseline mean and stddev
System uptime

It's built using Python's built-in http.server — no web framework needed!

Part 6 — Slack Alerts
When an IP gets banned, a Slack message arrives within 10 seconds:
🚨 IP BANNED
IP: 192.168.1.100
Condition: Anomalous request rate
Current Rate: 450.00 req/s
Baseline: 12.00 req/s
Ban Duration: 10 minutes
Timestamp: 2026-04-28 03:22:36 UTC
And when the ban expires:
✅ IP UNBANNED
IP: 192.168.1.100
Reason: ban-expired
Timestamp: 2026-04-28 03:32:36 UTC

What I Learned
Building this project taught me:

Z-scores are powerful — a simple maths formula can detect attacks that would be impossible to catch with hardcoded thresholds
Baselines must be dynamic — hardcoding "block if > 100 req/s" is wrong because normal traffic varies by time of day
iptables is incredibly fast — kernel-level packet dropping happens before the request even reaches Python
Threading needs care — shared data structures need locks to prevent race conditions
Deques are perfect for sliding windows — O(1) append and popleft make them ideal for real-time rate tracking

Try It Yourself
The full source code is available at:
https://github.com/AnitaAliCloud/hng-stage3-devops
The live dashboard is running at:
http://anitacloud.duckdns.org:8080

Built as part of the HNG14 DevOps internship programme