DEV Community: David Nwosu

Bridging the Gap: Converting ASCII Diagrams to Professional Images for AI-Driven Documentation

David Nwosu — Mon, 05 Jan 2026 21:55:00 +0000

As artificial intelligence tools become increasingly integrated into our creative workflows, a peculiar challenge has emerged: the disconnect between AI-generated text-based diagrams and the image formats required for professional documentation.

The Challenge

Modern AI assistants like ChatGPT, Claude, and GitHub Copilot excel at generating ASCII art, flowcharts, and text-based diagrams in markdown format. These tools can quickly produce complex visualizations using characters and symbols—perfect for code documentation, technical specifications, and research papers. However, these markdown diagrams often fail to render properly in Word documents, PowerPoint presentations, or PDF submissions. What looks perfectly aligned in a code editor becomes distorted in traditional document formats.

For researchers, technical writers, and developers working with AI-generated content, manually recreating these diagrams in traditional design tools wastes valuable time and introduces potential errors.

A Simple Solution

I developed a single-file web application that converts ASCII art and markdown diagrams directly into high-quality JPG images. The tool requires no installation, no server, and no external dependencies—just open the HTML file in any browser. You can try it out here

Key Features for Professional Use

The converter offers granular control over output quality: adjustable font sizes, customizable colors, configurable padding, and JPG quality settings ranging from standard to maximum compression. Live preview functionality ensures the output matches expectations before export. The monospace font options preserve the precise character alignment essential to ASCII diagrams.

Real-World Applications

Academic researchers can convert methodology flowcharts generated by AI assistants into submission-ready figures. Software developers can transform architecture diagrams from code comments into documentation images. Technical writers can standardize AI-generated process flows across multiple document formats. Students preparing thesis presentations can convert text-based models into professional slide graphics.

The Broader Impact

This tool represents a larger trend: creating bridges between AI-native content formats and traditional publishing requirements. As generative AI becomes more prevalent in creative and technical work, we need more solutions that seamlessly translate between these ecosystems.

The ASCII-to-image converter is open-source and freely available. It demonstrates that sometimes the most effective tools are the simplest ones—solving a specific pain point without unnecessary complexity.

Looking Forward

As AI assistants evolve, the content they generate will increasingly need to integrate with established workflows and formats. Tools that facilitate this integration, no matter how specialized, will become essential components of modern creative and technical pipelines.

For anyone working at the intersection of AI-generated content and traditional documentation, I invite you to try the converter and share your feedback. The goal is simple: make AI-assisted creativity more accessible and professional.

What challenges have you encountered when working with AI-generated content in professional contexts? I'd be interested to hear about other format conversion needs in the AI creative space.

Part 10: Scaling, Failure & Operating Like a Real Company

David Nwosu — Wed, 31 Dec 2025 01:25:12 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 20 minutes

Level: Intermediate to Advanced

The Journey So Far

We've come a long way:

Part 1: Deployed manually (SSH + npm start)

Part 2: Added process management (PM2)

Part 3: Containerized with Docker

Part 4: Orchestrated locally (Docker Compose)

Part 5: Broke things to feel the pain

Part 6: Moved to Kubernetes

Part 7: Automated infrastructure (Terraform)

Part 8: Packaged apps (Helm)

Part 9: Automated deployments (Argo CD)

Now what? You're still not done.

Your CTO walks over:

"Great! Now make it handle 10x traffic, cost 50% less, survive disasters, and keep hackers out."

This is Part 10. Production reality.

Scaling: Handling Real Load

Horizontal Pod Autoscaling (HPA)

Problem: Traffic spikes at 9 AM. 3 API pods can't handle it. Users see 500 errors.

Solution: Auto-scale based on CPU/memory usage.

# infrastructure/k8s/hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
      - type: Percent
        value: 50
        periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Pods
        value: 1
        periodSeconds: 60

What this does:

Starts with 3 replicas
Scales up when CPU > 70% or memory > 80%
Max 20 replicas (prevents runaway costs)
Scale up gradually (50% every 60s)
Scale down slowly (1 pod every 60s, wait 5 min before scaling down)

Apply:

kubectl apply -f infrastructure/k8s/hpa.yaml

Watch it work:

# Generate load
kubectl run -it --rm load-generator --image=busybox /bin/sh
# Inside pod:
while true; do wget -q -O- http://api-service:3000/api/v1/health; done

# Watch scaling
kubectl get hpa -w

Queue-Based Worker Scaling

Problem: 10,000 events arrive in 1 minute. Worker can't keep up.

Solution: Scale workers based on queue length.

Use KEDA (Kubernetes Event-Driven Autoscaling):

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-scaler
spec:
  scaleTargetRef:
    name: worker
  minReplicaCount: 2
  maxReplicaCount: 50
  triggers:
  - type: redis
    metadata:
      address: redis:6379
      listName: signal-processing-queue
      listLength: "10"

What this does:

Scales workers when queue has >10 items
1 worker per 10 jobs
Max 50 workers

Observability: Know What's Happening

Structured Logging

Bad logging:

console.log('Event received');
console.log('Error:', err);

Good logging:

import logger from './logger';

logger.info('Event received', {
  accountId: event.accountId,
  eventType: event.eventType,
  userId: event.userId,
  timestamp: Date.now(),
});

logger.error('Database connection failed', {
  error: err.message,
  stack: err.stack,
  retries: 3,
  service: 'api',
});

Why? You can search, filter, and aggregate structured logs.

Health Checks

Liveness probe: Is the container alive?

livenessProbe:
  httpGet:
    path: /health
    port: 3000
  initialDelaySeconds: 30
  periodSeconds: 10
  failureThreshold: 3

Readiness probe: Should we send traffic?

readinessProbe:
  httpGet:
    path: /health/ready
    port: 3000
  initialDelaySeconds: 5
  periodSeconds: 5
  failureThreshold: 2

Implement health endpoints:

// services/api/src/health.controller.ts
@Get('/health')
async liveness() {
  // Basic check: is the server running?
  return { status: 'ok', timestamp: Date.now() };
}

@Get('/health/ready')
async readiness() {
  // Deep check: can we serve traffic?
  const checks = await Promise.all([
    this.checkDatabase(),
    this.checkRedis(),
    this.checkElasticsearch(),
  ]);

  const allHealthy = checks.every(c => c.healthy);
  const status = allHealthy ? 200 : 503;

  return { status: allHealthy ? 'ready' : 'not ready', checks };
}

private async checkDatabase() {
  try {
    await this.db.query('SELECT 1');
    return { service: 'database', healthy: true };
  } catch (err) {
    return { service: 'database', healthy: false, error: err.message };
  }
}

Metrics with Prometheus

Expose metrics endpoint:

// services/api/src/metrics.controller.ts
import { Counter, Histogram, register } from 'prom-client';

const httpRequestsTotal = new Counter({
  name: 'http_requests_total',
  help: 'Total HTTP requests',
  labelNames: ['method', 'route', 'status'],
});

const httpRequestDuration = new Histogram({
  name: 'http_request_duration_seconds',
  help: 'HTTP request duration',
  labelNames: ['method', 'route'],
  buckets: [0.01, 0.05, 0.1, 0.5, 1, 2, 5],
});

@Get('/metrics')
async metrics() {
  return register.metrics();
}

// Middleware to track metrics
export function metricsMiddleware(req, res, next) {
  const start = Date.now();

  res.on('finish', () => {
    const duration = (Date.now() - start) / 1000;
    httpRequestsTotal.inc({ method: req.method, route: req.route?.path || 'unknown', status: res.statusCode });
    httpRequestDuration.observe({ method: req.method, route: req.route?.path || 'unknown' }, duration);
  });

  next();
}

Security: Defense in Depth

RBAC (Role-Based Access Control)

Don't give everyone cluster-admin.

# infrastructure/k8s/rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer-role
  namespace: sspp-prod
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: developer-binding
  namespace: sspp-prod
subjects:
- kind: User
  name: developer@company.com
roleRef:
  kind: Role
  name: developer-role
  apiGroup: rbac.authorization.k8s.io

Principle: Give minimum permissions needed. Developers can read logs, not delete production.

Network Policies

Don't let every pod talk to every pod.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: postgres-policy
spec:
  podSelector:
    matchLabels:
      app: postgres
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: api
    - podSelector:
        matchLabels:
          app: worker
    ports:
    - protocol: TCP
      port: 5432

Result: Only API and Worker can connect to PostgreSQL. Random pods can't steal data.

Secrets Management

Don't hardcode passwords.

apiVersion: v1
kind: Secret
metadata:
  name: database-credentials
type: Opaque
stringData:
  DATABASE_URL: postgresql://user:password@postgres:5432/sspp

Use in pods:

env:
- name: DATABASE_URL
  valueFrom:
    secretKeyRef:
      name: database-credentials
      key: DATABASE_URL

Better: Use external secrets management (AWS Secrets Manager, HashiCorp Vault, Sealed Secrets).

Image Security

Don't run as root.

FROM node:18-alpine

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001

# Set ownership
WORKDIR /app
COPY --chown=nodejs:nodejs package*.json ./
RUN npm ci --only=production
COPY --chown=nodejs:nodejs . .

# Run as non-root
USER nodejs
EXPOSE 3000
CMD ["node", "dist/main.js"]

Scan images:

# Use Trivy to scan for vulnerabilities
trivy image davidbrown77/sspp-api:latest

Cost Optimization

Right-Size Resources

Don't over-provision.

resources:
  requests:
    cpu: "100m"      # Minimum needed
    memory: "256Mi"
  limits:
    cpu: "1000m"     # Maximum allowed
    memory: "1Gi"

How to find right size:

Monitor actual usage with Prometheus
Set requests to P95 usage
Set limits to P99 + 20% buffer

Use Spot Instances

For non-critical workloads:

# infrastructure/terraform/main.tf
resource "linode_lke_cluster" "sspp" {
  label = "sspp-prod"
  k8s_version = "1.28"
  region = "us-east"

  pool {
    type = "g6-standard-4"
    count = 3
    # Use spot instances for worker pool
  }

  pool {
    type = "g6-standard-2"
    count = 5
    # Spot instances for batch jobs
    # 60-80% cheaper than on-demand
  }
}

Autoscale Down

Don't run empty clusters overnight.

# Scale down non-prod environments at night
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa-dev
spec:
  minReplicas: 0  # Scale to zero outside business hours
  maxReplicas: 10

Use Karpenter or cluster-autoscaler to remove unused nodes.

Disaster Recovery

Database Backups

Automate backups with CronJob:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: postgres-backup
spec:
  schedule: "0 2 * * *"  # 2 AM daily
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: postgres:15
            command:
            - /bin/bash
            - -c
            - |
              pg_dump $DATABASE_URL | gzip > /backups/backup-$(date +%Y%m%d-%H%M%S).sql.gz
              # Upload to S3/Object Storage
              aws s3 cp /backups/backup-$(date +%Y%m%d-%H%M%S).sql.gz s3://sspp-backups/
            env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: database-credentials
                  key: DATABASE_URL
          restartPolicy: OnFailure

Test restores regularly:

# Every quarter, restore backup to test environment
pg_restore -d sspp_test < backup-20251222.sql.gz

GitOps Disaster Recovery

Why GitOps helps:

Your entire cluster state is in Git. If cluster is destroyed:

# Recreate cluster with Terraform
cd infrastructure/terraform
terraform apply

# ArgoCD syncs everything from Git
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
argocd app create sspp --repo https://github.com/daviesbrown/sspp --path infrastructure/k8s --dest-server https://kubernetes.default.svc --dest-namespace sspp-prod
argocd app sync sspp

Recovery time: 15-20 minutes (mostly waiting for Terraform/ArgoCD).

Alerts: Know Before Users Do

Prometheus AlertManager

# infrastructure/k8s/alerts.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: sspp-alerts
spec:
  groups:
  - name: sspp
    interval: 30s
    rules:
    - alert: HighErrorRate
      expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.05
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: "High error rate detected"
        description: "{{ $value }}% of requests are failing"

    - alert: HighMemoryUsage
      expr: container_memory_usage_bytes / container_spec_memory_limit_bytes > 0.9
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: "Container using >90% memory"

    - alert: PodCrashLooping
      expr: rate(kube_pod_container_status_restarts_total[15m]) > 0
      labels:
        severity: critical
      annotations:
        summary: "Pod is crash looping"

Send to Slack/PagerDuty:

apiVersion: v1
kind: ConfigMap
metadata:
  name: alertmanager-config
data:
  alertmanager.yml: |
    route:
      receiver: 'slack'
    receivers:
    - name: 'slack'
      slack_configs:
      - api_url: 'https://hooks.slack.com/services/YOUR/WEBHOOK/URL'
        channel: '#alerts'
        title: '{{ .GroupLabels.alertname }}'
        text: '{{ range .Alerts }}{{ .Annotations.description }}{{ end }}'

When NOT to Scale

Scaling is not always the answer.

Optimize First

Before adding 10 more API pods:

Profile the code - Is there an N+1 query?
Add caching - Redis for hot data
Optimize queries - Add indexes
Use CDN - Offload static assets

Real example: Company spent $50k/month on compute. Fixed one database query. Cost dropped to $15k/month.

Know Your Limits

Your database doesn't scale horizontally easily.

Postgres has connection limits (~100-300)
Redis is single-threaded
Elasticsearch needs careful tuning

Solution: Connection pooling, read replicas, sharding (when necessary).

DevOps is Decision-Making Under Uncertainty

You've learned the tools:

✅ Docker, Kubernetes, Terraform, Helm, Argo CD
✅ Monitoring, logging, alerting
✅ Security, scaling, cost optimization

But tools don't make decisions. You do.

Production questions you'll face:

Should we scale now or optimize code first?
Is 99.9% uptime enough, or do we need 99.99%?
Do we need multi-region, or is one region acceptable?
Should we use managed PostgreSQL or run our own?
Is this alert worth waking someone up at 3 AM?

There are no perfect answers. Only tradeoffs.

Your job: Make informed tradeoffs based on:

Business requirements
Team size
Budget constraints
Risk tolerance

The Complete Architecture

What we built:

┌─────────────────────────────────────────────────────────┐
│                     USER REQUEST                         │
└────────────────────┬────────────────────────────────────┘
                     │
                     ▼
              ┌──────────────┐
              │ Load Balancer│  (Linode NodeBalancer)
              │   (HTTPS)    │
              └──────┬───────┘
                     │
      ┌──────────────┼──────────────┐
      │              │              │
      ▼              ▼              ▼
 ┌────────┐    ┌────────┐    ┌────────┐
 │ API Pod│    │ API Pod│    │ API Pod│  (Horizontal Pod Autoscaler)
 └────┬───┘    └────┬───┘    └────┬───┘
      │             │             │
      └──────────┬──┴─────────────┘
                 │
        ┌────────┴────────┐
        │                 │
        ▼                 ▼
   ┌─────────┐       ┌─────────┐
   │  Redis  │       │Postgres │  (StatefulSet)
   │ (Queue) │       │   (DB)  │
   └────┬────┘       └─────────┘
        │
        ▼
   ┌──────────┐
   │  Worker  │  (KEDA Autoscaler)
   │   Pods   │
   └────┬─────┘
        │
        ├──────────────┬───────────────┐
        ▼              ▼               ▼
   ┌─────────┐   ┌──────────┐   ┌──────────────┐
   │Postgres │   │  Redis   │   │Elasticsearch │
   └─────────┘   └──────────┘   └──────────────┘

────────────────────────────────────────────────────────
              OBSERVABILITY & CONTROL
────────────────────────────────────────────────────────
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│  Prometheus  │  │    Grafana   │  │  Argo CD     │
│  (Metrics)   │  │ (Dashboards) │  │  (GitOps)    │
└──────────────┘  └──────────────┘  └──────────────┘

────────────────────────────────────────────────────────
              INFRASTRUCTURE LAYER
────────────────────────────────────────────────────────
┌──────────────────────────────────────────────────────┐
│         Kubernetes Cluster (Linode LKE)               │
│  • 3-10 nodes (autoscaling)                          │
│  • Multiple availability zones                        │
│  • Managed control plane                             │
└──────────────────────────────────────────────────────┘
                     │
                     ▼
            ┌────────────────┐
            │   Terraform    │  (Infrastructure as Code)
            └────────────────┘

The Journey Recap

Part 1: Started with SSH and npm start (pain: everything dies)

Part 2: Added PM2 (pain: still manual, environment drift)

Part 3: Containerized with Docker (pain: local only)

Part 4: Local orchestration with Docker Compose (pain: no auto-recovery)

Part 5: Broke things intentionally (pain: realized we need orchestration)

Part 6: Kubernetes fundamentals (pain: too much YAML, manual clicks)

Part 7: Terraform for infrastructure (pain: still manual deploys)

Part 8: Helm for packaging (pain: no auto-sync)

Part 9: Argo CD for GitOps (pain: no visibility, security, cost control)

Part 10: Production hardening (scaling, observability, security, cost)

You're now running production SaaS infrastructure.

Try It Yourself

Complete production checklist:

[ ] Horizontal Pod Autoscaling configured
[ ] Resource limits on all containers
[ ] Health checks (liveness + readiness)
[ ] Structured logging with Winston
[ ] Metrics endpoint exposed
[ ] Prometheus + Grafana dashboards
[ ] Alerts configured (Slack/PagerDuty)
[ ] RBAC roles defined
[ ] Network policies applied
[ ] Secrets externalized
[ ] Database backups automated
[ ] Disaster recovery tested
[ ] Cost monitoring enabled
[ ] Security scanning in CI/CD
[ ] Documentation up to date

If you can check all boxes, you're production-ready.

What's Next?

You've completed the journey. You now understand:

Why tools exist (not just how to use them)
How to think about tradeoffs
What "production-ready" really means
How to operate, not just deploy

Where to go from here:

Multi-region deployments - Latency, disaster recovery
Service mesh - Istio, Linkerd for advanced networking
Chaos engineering - Intentional failure testing
FinOps - Advanced cost optimization
Compliance - SOC 2, HIPAA, GDPR for SaaS

But you have the foundation. Everything else builds on what you've learned.

Final Thoughts

DevOps is not about tools. It's about:

Understanding problems before solutions
Making systems reliable, not perfect
Balancing speed, cost, and safety
Operating with empathy (for users, teammates, on-call engineers)

You started with SSH and npm start.

You ended with GitOps-powered Kubernetes.

That's production DevOps.

Previous: Part 9: GitOps with Argo CD

About the Author

I built this 10-part series to demonstrate real DevOps thinking for my Proton.ai application. Every tool was introduced only after experiencing the pain it solves.

This is how production SaaS is built.

If you're hiring for DevOps/Platform roles and want someone who understands infrastructure (not just follows tutorials), let's talk.

Thank you for reading.

If this series helped you, please:

⭐ Star the GitHub repository
Share with others learning DevOps
Open issues with questions or feedback
Consider hiring me if you need this expertise on your team

From manual deployment to production-grade GitOps. You made it. 🚀

Part 9: GitOps with Argo CD - When Git Deploys Your App

David Nwosu — Wed, 31 Dec 2025 01:21:45 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 18 minutes

Level: Intermediate

The kubectl Apply Problem

Your Kubernetes cluster is running. You update an API deployment:

vim infrastructure/k8s/api.yaml
# Change replicas from 3 to 5

kubectl apply -f infrastructure/k8s/api.yaml

Two weeks later, a teammate asks:

"Hey, why do we have 5 API replicas?"

You: "I changed it two weeks ago."

"Where's the change tracked?"

You: "Uh... it's not. I ran kubectl apply locally."

"What if we need to rollback?"

You: "I'd have to... remember what it was before?"

The problems with manual kubectl apply:

❌ No audit trail - Who changed what, when, why?
❌ Configuration drift - Git says 3 replicas, cluster has 5
❌ No rollback - Can't revert to previous state
❌ Manual sync - Someone must run kubectl apply
❌ No approval process - Anyone with kubectl access can change anything
❌ Hard to debug - What's the desired state? What's actual state?

What you want:

"Git repository is the single source of truth. If it's in Git, it's deployed. If it's not in Git, it shouldn't exist."

This is GitOps.

What is GitOps?

GitOps: Using Git as the single source of truth for declarative infrastructure and applications.

Core principles:

Declarative - System state described in Git
Versioned - Git tracks all changes
Immutable - Git commits are immutable
Pulled automatically - System continuously syncs from Git

The workflow:

Developer → Git commit → Push to main → ArgoCD detects change → ArgoCD syncs cluster → Deployment updated

No kubectl commands. No manual intervention. Just Git.

Tools:

ArgoCD - GitOps for Kubernetes (we'll use this)
Flux - Another GitOps operator
Jenkins X - CI/CD with GitOps

Why ArgoCD?

✅ Best-in-class UI
✅ Multi-cluster support
✅ Application health monitoring
✅ Rollback with one click
✅ SSO integration
✅ RBAC built-in

ArgoCD Architecture

┌─────────────────────────────────────────────────────┐
│                     Git Repository                  │
│  infrastructure/k8s/                                │
│  ├── api.yaml                                       │
│  ├── worker.yaml                                    │
│  └── postgres.yaml                                  │
└──────────────────┬──────────────────────────────────┘
                   │
                   │ ArgoCD continuously polls
                   │ (every 3 minutes by default)
                   │
                   ▼
┌─────────────────────────────────────────────────────┐
│              ArgoCD Controller                      │
│  - Compares Git state with cluster state           │
│  - Detects drift                                    │
│  - Syncs automatically (if auto-sync enabled)      │
└──────────────────┬──────────────────────────────────┘
                   │
                   │ kubectl apply
                   │
                   ▼
┌─────────────────────────────────────────────────────┐
│            Kubernetes Cluster                       │
│  Running applications                               │
└─────────────────────────────────────────────────────┘

Key components:

Application Controller - Monitors Git repos, syncs cluster
Repo Server - Fetches manifests from Git
API Server - gRPC/REST API for management
Web UI - Visual dashboard
Dex - SSO integration (optional)

Installing ArgoCD

Step 1: Install ArgoCD

# Create namespace
kubectl create namespace argocd

# Install ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Wait for pods to be ready
kubectl wait --for=condition=Ready pods --all -n argocd --timeout=5m

Check installation:

kubectl get pods -n argocd

NAME                                  READY   STATUS    RESTARTS   AGE
argocd-application-controller-0       1/1     Running   0          2m
argocd-dex-server-5dd657bd9-xyz       1/1     Running   0          2m
argocd-redis-74cb89f466-abc           1/1     Running   0          2m
argocd-repo-server-6b8d9f5d4-def      1/1     Running   0          2m
argocd-server-7d9f8c5b4-ghi           1/1     Running   0          2m

Step 2: Access ArgoCD UI

Get admin password:

kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath="{.data.password}" | base64 -d; echo

Port forward to access UI:

kubectl port-forward svc/argocd-server -n argocd 8080:443

Open browser: https://localhost:8080

Login:

Username: admin
Password: [from command above]

You should see the ArgoCD dashboard! 🎉

Step 3: Install ArgoCD CLI

# macOS
brew install argocd

# Linux
curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
chmod +x /usr/local/bin/argocd

# Verify
argocd version

Login via CLI:

argocd login localhost:8080 --username admin --password [password] --insecure

Creating Your First ArgoCD Application

Step 1: Prepare Git Repository

Your repo structure:

davidbrown77/sspp/
├── infrastructure/
│   └── k8s/
│       ├── namespace.yaml
│       ├── api.yaml
│       ├── worker.yaml
│       ├── postgres.yaml
│       ├── redis.yaml
│       └── elasticsearch.yaml

Push to GitHub:

git add infrastructure/k8s/
git commit -m "Add Kubernetes manifests for ArgoCD"
git push origin main

Step 2: Create ArgoCD Application

Via UI:

Click "New App"
Fill in:
- Application Name: sspp-prod
- Project: default
- Sync Policy: Manual (we'll enable auto later)
- Repository URL: https://github.com/daviesbrown/sspp.git
- Revision: main
- Path: infrastructure/k8s
- Cluster: https://kubernetes.default.svc
- Namespace: sspp-prod
Click "Create"

Via CLI:

argocd app create sspp-prod \
  --repo https://github.com/daviesbrown/sspp.git \
  --path infrastructure/k8s \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace sspp-prod

Via YAML (GitOps way!):

# infrastructure/argocd/sspp-prod-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sspp-prod
  namespace: argocd
spec:
  project: default

  source:
    repoURL: https://github.com/daviesbrown/sspp.git
    targetRevision: main
    path: infrastructure/k8s

  destination:
    server: https://kubernetes.default.svc
    namespace: sspp-prod

  syncPolicy:
    automated:
      prune: true      # Delete resources not in Git
      selfHeal: true   # Sync when cluster state drifts
      allowEmpty: false
    syncOptions:
    - CreateNamespace=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

Apply:

kubectl apply -f infrastructure/argocd/sspp-prod-app.yaml

Step 3: Sync Application

In UI:

Click on sspp-prod app
Click "Sync"
Click "Synchronize"

Via CLI:

argocd app sync sspp-prod

Watch deployment:

argocd app wait sspp-prod --health

Output:

Name:               sspp-prod
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          sspp-prod
URL:                https://localhost:8080/applications/sspp-prod
Repo:               https://github.com/daviesbrown/sspp.git
Target:             main
Path:               infrastructure/k8s
SyncWindow:         Sync Allowed
Sync Policy:        Automated
Sync Status:        Synced to main (abc1234)
Health Status:      Healthy

GROUP  KIND         NAMESPACE   NAME          STATUS  HEALTH   HOOK
       Namespace    sspp-prod   sspp-prod     Synced                
       Service      sspp-prod   api           Synced  Healthy       
       Service      sspp-prod   postgres      Synced  Healthy       
apps   Deployment   sspp-prod   api           Synced  Healthy       
apps   Deployment   sspp-prod   worker        Synced  Healthy       
apps   Deployment   sspp-prod   postgres      Synced  Healthy

Your application is deployed via GitOps! 🚀

Auto-Sync: True GitOps

Enable auto-sync so Git changes deploy automatically:

syncPolicy:
  automated:
    prune: true      # Delete resources not in Git
    selfHeal: true   # Revert manual kubectl changes

Or via CLI:

argocd app set sspp-prod --sync-policy automated --auto-prune --self-heal

What this does:

Every 3 minutes, ArgoCD checks Git for changes
If Git changed, ArgoCD syncs cluster automatically
If someone runs kubectl apply, self-heal reverts it
If resource deleted from Git, prune deletes it from cluster

Test it:

# Edit deployment
vim infrastructure/k8s/api.yaml
# Change replicas: 3 → 5

git add infrastructure/k8s/api.yaml
git commit -m "Scale API to 5 replicas"
git push origin main

# Wait 3 minutes (or trigger manually)
argocd app sync sspp-prod

# Check
kubectl get deployment api -n sspp-prod
# DESIRED: 5

No kubectl commands. Just Git push. ✅

ArgoCD UI Features

Application Health

Health statuses:

Healthy - All resources running, ready
Progressing - Deployment rolling out
Degraded - Some Pods not ready
Suspended - Application suspended
Missing - Resources not found
Unknown - Health status unknown

Visual indicators:

🟢 Green - Healthy
🟡 Yellow - Progressing
🔴 Red - Degraded

Sync Status

Sync statuses:

Synced - Git matches cluster
OutOfSync - Git differs from cluster
Unknown - Can't determine sync status

Click on resource to see diff:

- replicas: 3
+ replicas: 5

Rollback

Click "History and Rollback":

See all previous syncs
Click "Rollback" to revert

Via CLI:

# List sync history
argocd app history sspp-prod

# Rollback to revision 3
argocd app rollback sspp-prod 3

App Diff

Compare Git vs Cluster:

argocd app diff sspp-prod

Output:

===== apps/Deployment sspp-prod/api ======
  spec:
-   replicas: 3
+   replicas: 5

Multi-Environment Setup

Best practice: One Git repo, multiple ArgoCD apps for different environments.

Repository Structure

davidbrown77/sspp/
├── infrastructure/
│   └── k8s/
│       ├── base/           # Shared resources
│       │   ├── api.yaml
│       │   ├── worker.yaml
│       │   └── postgres.yaml
│       ├── overlays/
│       │   ├── dev/        # Dev-specific
│       │   │   ├── kustomization.yaml
│       │   │   └── replicas.yaml
│       │   ├── staging/    # Staging-specific
│       │   │   ├── kustomization.yaml
│       │   │   └── replicas.yaml
│       │   └── prod/       # Prod-specific
│       │       ├── kustomization.yaml
│       │       └── replicas.yaml

Using Kustomize for environment-specific overrides:

# infrastructure/k8s/overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: sspp-prod

resources:
- ../../base

replicas:
- name: api
  count: 5
- name: worker
  count: 10

images:
- name: davidbrown77/sspp-api
  newTag: v2.1.0

commonLabels:
  environment: production

Create ArgoCD apps for each environment:

# infrastructure/argocd/dev-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sspp-dev
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/daviesbrown/sspp.git
    targetRevision: main
    path: infrastructure/k8s/overlays/dev
  destination:
    server: https://kubernetes.default.svc
    namespace: sspp-dev
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
---
# infrastructure/argocd/staging-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sspp-staging
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/daviesbrown/sspp.git
    targetRevision: main
    path: infrastructure/k8s/overlays/staging
  destination:
    server: https://kubernetes.default.svc
    namespace: sspp-staging
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
---
# infrastructure/argocd/prod-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sspp-prod
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/daviesbrown/sspp.git
    targetRevision: main
    path: infrastructure/k8s/overlays/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: sspp-prod
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

Deploy all environments:

kubectl apply -f infrastructure/argocd/

Now you have:

Dev environment (auto-synced from Git)
Staging environment (auto-synced from Git)
Prod environment (auto-synced from Git)

One Git push → all environments update automatically.

App of Apps Pattern

Problem: Managing many ArgoCD applications manually.

Solution: ArgoCD application that manages other applications.

# infrastructure/argocd/root-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sspp-apps
  namespace: argocd
spec:
  project: default

  source:
    repoURL: https://github.com/daviesbrown/sspp.git
    targetRevision: main
    path: infrastructure/argocd

  destination:
    server: https://kubernetes.default.svc
    namespace: argocd

  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Apply once:

kubectl apply -f infrastructure/argocd/root-app.yaml

Now:

sspp-apps manages all child applications
Add new app → commit to Git → ArgoCD creates it
Remove app → delete from Git → ArgoCD removes it

This is GitOps managing GitOps! 🤯

Progressive Delivery with ArgoCD

Blue/Green Deployments

Install ArgoCD Rollouts:

kubectl create namespace argo-rollouts
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml

Define Rollout:

# infrastructure/k8s/api-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: api
  namespace: sspp-prod
spec:
  replicas: 5
  revisionHistoryLimit: 2

  selector:
    matchLabels:
      app: api

  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: davidbrown77/sspp-api:v2.0.0
        ports:
        - containerPort: 3000

  strategy:
    blueGreen:
      activeService: api
      previewService: api-preview
      autoPromotionEnabled: false
      scaleDownDelaySeconds: 30

Services:

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: api
  ports:
  - port: 80
    targetPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: api-preview
spec:
  selector:
    app: api
  ports:
  - port: 80
    targetPort: 3000

Deploy new version:

# Update image tag in Git
vim infrastructure/k8s/api-rollout.yaml
# Change image: davidbrown77/sspp-api:v2.1.0

git commit -am "Deploy API v2.1.0"
git push

# ArgoCD syncs, creates preview environment
# Test preview: curl http://api-preview:80

# Promote to production
kubectl argo rollouts promote api -n sspp-prod

Canary Deployments

strategy:
  canary:
    steps:
    - setWeight: 10
    - pause: {duration: 5m}
    - setWeight: 25
    - pause: {duration: 5m}
    - setWeight: 50
    - pause: {duration: 5m}
    - setWeight: 75
    - pause: {duration: 5m}

What this does:

Deploy new version to 10% of Pods
Wait 5 minutes
If healthy, increase to 25%
Continue until 100%

If error rate spikes, rollback automatically.

ArgoCD Notifications

Get notified when deployments happen.

Install ArgoCD Notifications

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj-labs/argocd-notifications/stable/manifests/install.yaml

Configure Slack Notifications

# infrastructure/argocd/notifications-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
  namespace: argocd
data:
  service.slack: |
    token: $slack-token

  template.app-deployed: |
    message: |
      Application {{.app.metadata.name}} deployed!
      Revision: {{.app.status.sync.revision}}
      Author: {{.app.status.operationState.operation.initiatedBy.username}}
    slack:
      attachments: |
        [{
          "title": "{{.app.metadata.name}}",
          "title_link": "{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
          "color": "#18be52",
          "fields": [{
            "title": "Sync Status",
            "value": "{{.app.status.sync.status}}",
            "short": true
          }, {
            "title": "Health Status",
            "value": "{{.app.status.health.status}}",
            "short": true
          }]
        }]

  trigger.on-deployed: |
    - when: app.status.operationState.phase in ['Succeeded']
      send: [app-deployed]

Add annotations to applications:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sspp-prod
  annotations:
    notifications.argoproj.io/subscribe.on-deployed.slack: deployments-channel
spec:
  # ... rest of spec

Now Slack gets notified on every deployment! 📢

RBAC for ArgoCD

Control who can deploy what.

# infrastructure/argocd/rbac-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.default: role:readonly

  policy.csv: |
    # Developers can sync dev apps
    p, role:developer, applications, sync, sspp-dev, allow
    p, role:developer, applications, get, *, allow
    g, developers@example.com, role:developer

    # DevOps can sync all apps
    p, role:devops, applications, *, *, allow
    g, devops@example.com, role:devops

    # Admins can do everything
    p, role:admin, *, *, *, allow
    g, admins@example.com, role:admin

Now:

Developers can only deploy to dev
DevOps can deploy to all environments
Admins have full control

CI/CD Integration

Combine GitHub Actions + ArgoCD:

# .github/workflows/api.yml
name: API CI/CD

on:
  push:
    branches: [main]
    paths: ['services/api/**']

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Build and push Docker image
      run: |
        docker build -t davidbrown77/sspp-api:${{ github.sha }} services/api
        docker push davidbrown77/sspp-api:${{ github.sha }}

    - name: Update manifest
      run: |
        # Update image tag in Git
        sed -i "s|davidbrown77/sspp-api:.*|davidbrown77/sspp-api:${{ github.sha }}|" \
          infrastructure/k8s/overlays/prod/kustomization.yaml

        git config user.name "GitHub Actions"
        git config user.email "actions@github.com"
        git add infrastructure/k8s/overlays/prod/kustomization.yaml
        git commit -m "Update API image to ${{ github.sha }}"
        git push

    # ArgoCD detects change and deploys automatically!

Flow:

Push code → GitHub Actions
GitHub Actions builds Docker image
GitHub Actions updates manifest in Git
ArgoCD detects Git change
ArgoCD deploys new image

Complete automation! 🚀

Disaster Recovery with ArgoCD

Backup ArgoCD Configuration

# Export all applications
argocd app list -o yaml > argocd-apps-backup.yaml

# Export projects
kubectl get appprojects -n argocd -o yaml > argocd-projects-backup.yaml

# Export RBAC
kubectl get cm argocd-rbac-cm -n argocd -o yaml > argocd-rbac-backup.yaml

Restore ArgoCD

# Reinstall ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Restore applications
kubectl apply -f argocd-apps-backup.yaml
kubectl apply -f argocd-projects-backup.yaml
kubectl apply -f argocd-rbac-backup.yaml

# ArgoCD syncs everything from Git!

Because everything is in Git, recovery is fast.

ArgoCD Best Practices

1. Use App of Apps Pattern

One root app manages all child apps.

2. Enable Auto-Sync + Self-Heal

syncPolicy:
  automated:
    prune: true
    selfHeal: true

Git is the source of truth. Period.

3. Use Kustomize or Helm

Don't duplicate manifests. Use overlays.

4. Set Up Notifications

Slack/Email alerts for deployments.

5. Implement RBAC

Restrict who can deploy to production.

6. Use Projects for Multi-Tenancy

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: team-backend
spec:
  sourceRepos:
  - https://github.com/daviesbrown/sspp.git
  destinations:
  - namespace: 'backend-*'
    server: https://kubernetes.default.svc

7. Monitor ArgoCD Metrics

kubectl port-forward svc/argocd-metrics -n argocd 8082:8082
curl http://localhost:8082/metrics

Integrate with Prometheus.

What We Solved

✅ Git as source of truth - All configs in Git

✅ Automated deployments - Push → ArgoCD deploys

✅ Self-healing - Manual changes reverted automatically

✅ Audit trail - Git history = deployment history

✅ Easy rollback - One-click revert in UI

✅ Multi-environment - Dev, staging, prod from same repo

✅ Progressive delivery - Blue/green, canary deployments

✅ Notifications - Slack alerts for deployments

What's Next?

We have automated GitOps deployments! But production isn't just about deployment:

❌ No observability - Can't see what's happening in production

❌ Cost unknown - Don't know what we're spending

❌ Security gaps - Secrets in Git, no network policies

❌ No disaster recovery - What if everything crashes?

In Part 10 (final article), we'll tackle Production Operations.

You'll learn:

Observability stack (Prometheus, Grafana, Loki)
Cost optimization strategies (right-sizing, spot instances)
Security hardening (network policies, RBAC, secrets management)
Disaster recovery and backup strategies
Scaling beyond one cluster (multi-region, multi-cloud)

This is where you transition from "it works" to "it's production-ready."

The Complete DevOps Journey

You've now mastered:

Manual deployment (Part 1) - SSH, npm start
Process management (Part 2) - PM2
Containerization (Part 3) - Docker
Orchestration (Part 4) - Docker Compose
Why Kubernetes (Part 5) - Limitations of Compose
Kubernetes (Part 6) - Self-healing, scaling
Infrastructure as Code (Part 7) - Terraform
Helm Packaging (Part 8) - YAML templating
GitOps (Part 9) - ArgoCD
Production Operations (Part 10) - Scaling, security, cost

Almost there. One more article to production-grade DevOps.

Final Architecture

┌─────────────────────────────────────────────────────┐
│                  Git Repository                     │
│  - Application code (services/api, services/worker)│
│  - Kubernetes manifests (infrastructure/k8s)       │
│  - Terraform configs (infrastructure/terraform)    │
└───────┬─────────────────────────────────────┬───────┘
        │                                     │
        │ Push code                           │ Push manifests
        │                                     │
        ▼                                     ▼
┌──────────────────┐              ┌─────────────────────┐
│  GitHub Actions  │              │      ArgoCD         │
│  - Test code     │              │  - Detects changes  │
│  - Build image   │              │  - Syncs cluster    │
│  - Push to GHCR  │              │  - Self-heals       │
│  - Update Git    │              └─────────┬───────────┘
└──────────────────┘                        │
                                            │ kubectl apply
                                            │
                          ┌─────────────────▼──────────────┐
                          │     Kubernetes Cluster (LKE)   │
                          │  - API (2-10 replicas, HPA)    │
                          │  - Worker (3-15 replicas, HPA) │
                          │  - PostgreSQL (HA)             │
                          │  - Redis, Elasticsearch        │
                          │  - Observability stack         │
                          └────────────────────────────────┘
                                    │
                          Managed by Terraform
                                    │
                          ┌─────────▼──────────┐
                          │  Linode Cloud      │
                          │  - LKE cluster     │
                          │  - NodeBalancer    │
                          │  - Object storage  │
                          │  - DNS             │
                          └────────────────────┘

Everything automated. Everything in Git. Everything production-grade.

Try It Yourself

Final challenge: Set up complete GitOps workflow:

Install ArgoCD in your cluster
Create ArgoCD application pointing to your Git repo
Enable auto-sync and self-heal
Push a change to Git, watch ArgoCD deploy
Manually change a deployment with kubectl, watch self-heal revert it
Set up multi-environment (dev, staging, prod)
Implement blue/green deployment with Argo Rollouts
Add Slack notifications
Configure RBAC

Bonus: Implement app of apps pattern for managing all applications.

Discussion

Do you use ArgoCD? Flux? Manual kubectl? What's your deployment strategy?

Share on GitHub Discussions.

Previous: Part 8: Helm - Packaging Kubernetes Applications

Next: Part 10: Scaling, Failure & Operating Like a Real Company

About the Author

I built this 12-part series to document my complete DevOps journey from manual server deployment to production-grade, GitOps-powered Kubernetes infrastructure.

This series demonstrates:

Systems thinking - Understanding why tools exist, not just how to use them
Production experience - Real code, real failures, real solutions
Teaching ability - Complex topics explained clearly
DevOps mastery - CI/CD, IaC, GitOps, observability, security

This is my Proton.ai application.

GitHub: @daviesbrown
LinkedIn: David Nwosu Brown
Portfolio: Sales Signal Processing Platform

You started by SSH-ing into a server and running npm start.

You ended with a production-grade, cloud-native, GitOps-powered SaaS platform.

You're ready. Go build something amazing. 🚀

Part 8: Helm - Packaging Kubernetes Applications

David Nwosu — Wed, 31 Dec 2025 01:20:12 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 15 minutes

Level: Intermediate

The YAML Duplication Problem

In Part 6, we deployed our SSPP platform to Kubernetes. It works! But look at your k8s/ directory:

k8s/
├── api-deployment.yaml
├── api-service.yaml
├── api-configmap.yaml
├── worker-deployment.yaml
├── worker-configmap.yaml
├── redis-deployment.yaml
├── redis-service.yaml
├── postgres-statefulset.yaml
├── postgres-service.yaml
└── ...

Now your manager says:

"We need dev, staging, and prod environments."

Your first thought: Copy-paste all YAML files three times:

k8s/
├── dev/
│   ├── api-deployment.yaml  # replicas: 1, resources: small
│   ├── api-service.yaml
│   └── ...
├── staging/
│   ├── api-deployment.yaml  # replicas: 2, resources: medium
│   ├── api-service.yaml
│   └── ...
└── prod/
    ├── api-deployment.yaml  # replicas: 5, resources: large
    ├── api-service.yaml
    └── ...

What changes between environments?

Replica counts
Resource limits
Image tags
Database URLs
Domain names
Storage sizes

What stays the same?

Container ports
Health check paths
Service types
Label selectors
Volume mount paths

You're copying 80% identical YAML and changing 20%.

Then a bug is found: API health check path should be /health not /healthz.

Now you need to update it in 3 places. And you miss one. Staging is broken. Users are angry.

This is the YAML duplication problem.

What is Helm?

Helm is a package manager for Kubernetes applications.

Beginner mental model:

Helm is like apt-get (Linux) or Homebrew (Mac) for Kubernetes apps.

Instead of managing 20+ YAML files per environment, you create a Helm Chart—a template with variables:

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.name }}
spec:
  replicas: {{ .Values.replicas }}
  template:
    spec:
      containers:
      - name: api
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
        resources:
          limits:
            cpu: {{ .Values.resources.cpu }}
            memory: {{ .Values.resources.memory }}

Then you have different values files per environment:

# values-dev.yaml
name: sspp-api
replicas: 1
image:
  repository: davidbrown77/sspp-api
  tag: dev-latest
resources:
  cpu: "500m"
  memory: "512Mi"

# values-prod.yaml
name: sspp-api
replicas: 5
image:
  repository: davidbrown77/sspp-api
  tag: v1.2.3
resources:
  cpu: "2000m"
  memory: "4Gi"

Deploy with:

# Dev
helm install sspp-api ./charts/api -f values-dev.yaml

# Prod
helm install sspp-api ./charts/api -f values-prod.yaml

Same template, different values. DRY (Don't Repeat Yourself) for Kubernetes.

Helm Concepts

Charts

A Helm Chart is a package of Kubernetes manifests.

Structure:

charts/api/
├── Chart.yaml          # Metadata (name, version)
├── values.yaml         # Default values
├── templates/          # Templated Kubernetes manifests
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── configmap.yaml
│   └── ingress.yaml
└── charts/             # Dependencies (sub-charts)

Values

values.yaml defines default configuration:

replicaCount: 3

image:
  repository: davidbrown77/sspp-api
  tag: latest
  pullPolicy: IfNotPresent

service:
  type: ClusterIP
  port: 3000

resources:
  limits:
    cpu: 1000m
    memory: 1Gi
  requests:
    cpu: 500m
    memory: 512Mi

Override with:

helm install sspp-api ./charts/api \
  --set replicaCount=5 \
  --set image.tag=v1.2.3

Or use a values file:

helm install sspp-api ./charts/api -f prod-values.yaml

Templates

Templates use Go templating syntax:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "api.fullname" . }}
  labels:
    {{- include "api.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- include "api.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "api.selectorLabels" . | nindent 8 }}
    spec:
      containers:
      - name: {{ .Chart.Name }}
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
        ports:
        - name: http
          containerPort: {{ .Values.service.port }}

Template functions:

{{ .Values.replicaCount }} - Access values
{{ include "helper" . }} - Reuse templates
{{- if .Values.enabled }} - Conditionals
{{- range .Values.items }} - Loops

Creating a Helm Chart for SSPP API

Initialize Chart

cd infrastructure
helm create charts/api

This generates a basic chart structure.

Define Chart.yaml

apiVersion: v2
name: sspp-api
description: Sales Signal Processing Platform API
type: application
version: 1.0.0
appVersion: "1.0.0"

Create Templates

templates/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.name }}
  labels:
    app: {{ .Values.name }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Values.name }}
  template:
    metadata:
      labels:
        app: {{ .Values.name }}
    spec:
      containers:
      - name: api
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
        ports:
        - containerPort: 3000
        env:
        - name: DATABASE_URL
          value: {{ .Values.database.url }}
        - name: REDIS_URL
          value: {{ .Values.redis.url }}
        {{- if .Values.env }}
        {{- range $key, $value := .Values.env }}
        - name: {{ $key }}
          value: {{ $value | quote }}
        {{- end }}
        {{- end }}
        resources:
          limits:
            cpu: {{ .Values.resources.limits.cpu }}
            memory: {{ .Values.resources.limits.memory }}
          requests:
            cpu: {{ .Values.resources.requests.cpu }}
            memory: {{ .Values.resources.requests.memory }}
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 5

templates/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: {{ .Values.name }}
  labels:
    app: {{ .Values.name }}
spec:
  type: {{ .Values.service.type }}
  ports:
  - port: {{ .Values.service.port }}
    targetPort: 3000
    protocol: TCP
    name: http
  selector:
    app: {{ .Values.name }}

Define Values

values.yaml:

name: sspp-api
replicaCount: 3

image:
  repository: davidbrown77/sspp-api
  tag: latest
  pullPolicy: IfNotPresent

service:
  type: ClusterIP
  port: 3000

database:
  url: "postgresql://user:pass@postgres:5432/sspp"

redis:
  url: "redis://redis:6379"

resources:
  limits:
    cpu: "1000m"
    memory: "1Gi"
  requests:
    cpu: "500m"
    memory: "512Mi"

env: {}

values-dev.yaml:

name: sspp-api
replicaCount: 1
image:
  tag: dev-latest
resources:
  limits:
    cpu: "500m"
    memory: "512Mi"
  requests:
    cpu: "250m"
    memory: "256Mi"

values-prod.yaml:

name: sspp-api
replicaCount: 5
image:
  tag: v1.2.3
  pullPolicy: Always
resources:
  limits:
    cpu: "2000m"
    memory: "4Gi"
  requests:
    cpu: "1000m"
    memory: "2Gi"
env:
  LOG_LEVEL: "info"
  NODE_ENV: "production"

Deploying with Helm

Install

# Dev environment
helm install sspp-api ./charts/api -f values-dev.yaml

# Prod environment (different namespace)
helm install sspp-api ./charts/api \
  -f values-prod.yaml \
  -n production \
  --create-namespace

Upgrade

# Update image tag
helm upgrade sspp-api ./charts/api \
  --set image.tag=v1.3.0 \
  --reuse-values

Rollback

# Rollback to previous release
helm rollback sspp-api

# Rollback to specific revision
helm rollback sspp-api 3

List Releases

helm list
helm list -n production

Get Values

# See computed values
helm get values sspp-api

# See all values (including defaults)
helm get values sspp-api --all

Helm for Worker Service

Create a similar chart for the worker:

helm create charts/worker

Key differences from API:

No Service (worker doesn't expose HTTP)
Different environment variables
Different resource requirements

values.yaml:

name: sspp-worker
replicaCount: 2

image:
  repository: davidbrown77/sspp-worker
  tag: latest

redis:
  url: "redis://redis:6379"

database:
  url: "postgresql://user:pass@postgres:5432/sspp"

resources:
  limits:
    cpu: "1000m"
    memory: "2Gi"
  requests:
    cpu: "500m"
    memory: "1Gi"

Benefits of Helm

✅ DRY Principle:

One template, multiple environments
Change once, deploy everywhere

✅ Version Control:

Track changes to chart versions
Rollback to any previous release

✅ Parameterization:

Override any value at deployment
No hardcoded configuration

✅ Package Management:

Share charts via Helm repositories
Reuse community charts (PostgreSQL, Redis, etc.)

✅ Release Management:

Track deployment history
Easy rollbacks

What's Next?

Helm solved packaging and templating, but we still have deployment problems:

❌ Manual deployments - Someone must run helm upgrade

❌ No Git sync - Cluster state can drift from Git

❌ No automation - Still need CI/CD triggers

❌ Configuration drift - Manual kubectl changes go untracked

In Part 9, we'll add GitOps with ArgoCD.

You'll learn:

Git as single source of truth (not your laptop)
Automatic sync from Git → Cluster
Self-healing (ArgoCD reverts manual changes)
One-click rollbacks through UI
Deployment history and audit trails
Progressive delivery (blue/green, canary)

Spoiler: Push to Git → ArgoCD deploys automatically. This is GitOps.

Try It Yourself

Create Helm charts for all SSPP services:

API chart - Deployment + Service + Ingress
Worker chart - Deployment only
Redis chart - Or use bitnami/redis from Helm Hub
PostgreSQL chart - Or use bitnami/postgresql

Test multi-environment setup:

# Deploy dev
helm install sspp ./charts/api -f values-dev.yaml -n dev --create-namespace

# Deploy prod
helm install sspp ./charts/api -f values-prod.yaml -n production --create-namespace

# Compare
kubectl get pods -n dev
kubectl get pods -n production

Bonus: Package your chart and push to a Helm repository (GitHub Pages, ChartMuseum, or OCI registry).

Next: Automating Deployments

In Part 9, we'll solve the manual deployment problem:

"How do we automatically deploy when code is merged to Git?"

We'll use Argo CD to implement GitOps:

Git becomes the source of truth
Changes are automatically synced to cluster
Drift is detected and corrected
Deployments are auditable and traceable

But spoiler: GitOps solves deployment automation, but not the full picture. We'll still need observability and production hardening.

Previous: Part 7: Terraform - Making Infrastructure Repeatable

Next: Part 9: GitOps with Argo CD

About the Author

Building this series to demonstrate real DevOps thinking for my Proton.ai application. If you're hiring for platform engineering roles, let's connect.

GitHub: @daviesbrown
LinkedIn: David Nwosu

Part 7: Terraform - Making Infrastructure Repeatable

David Nwosu — Mon, 22 Dec 2025 23:35:58 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 18 minutes

Level: Intermediate

The ClickOps Problem

Your Kubernetes cluster is running perfectly. Traffic is growing. Your CTO says:

"We need a disaster recovery environment in another region."

Your thought process:

Log into Linode dashboard
Click "Create Cluster"
Fill out form (region, node count, instance type)
Wait 10 minutes
Download kubeconfig
Apply all Kubernetes manifests
Update DNS
Configure monitoring
Set up backups
Repeat for staging environment
Realize you made a typo in production and it's different from staging

Time: 2-3 hours

Error-prone: ✅ (what settings did I use for production?)

Reproducible: ❌ (did I enable autoscaling? what was the node size?)

Documented: ❌ (it's in my head)

Your manager: "Can you create a dev cluster for the new engineer?"

You: [internal screaming]

This is called "ClickOps"—managing infrastructure through web UI clicks.

Problems:

❌ Not reproducible - Can't recreate exact environment
❌ No version control - No history of changes
❌ No code review - No approval process
❌ Hard to scale - Can't manage 10+ environments
❌ No automation - Manual work for every change

Solution: Infrastructure as Code (IaC)

What is Infrastructure as Code?

Infrastructure as Code (IaC): Define infrastructure in code files, apply with CLI tools.

Instead of clicking:

# Clicks in Linode dashboard:
1. Create Kubernetes Cluster
2. Choose region: us-east
3. Select nodes: 3x g6-standard-4
4. Enable autoscaling: 3-10 nodes
5. Click "Create"

You write code:

# infrastructure/terraform/main.tf
resource "linode_lke_cluster" "sspp_prod" {
  label       = "sspp-prod"
  k8s_version = "1.28"
  region      = "us-east"

  pool {
    type  = "g6-standard-4"
    count = 3
    autoscaler {
      min = 3
      max = 10
    }
  }
}

Then apply:

terraform apply

Benefits:

✅ Version controlled - Git tracks all changes
✅ Reproducible - Spin up identical environments
✅ Code review - PRs for infrastructure changes
✅ Automated - CI/CD can apply changes
✅ Self-documenting - Code is the documentation

Why Terraform?

IaC Tools:

Terraform (HashiCorp) - Multi-cloud, largest ecosystem
Pulumi - Use real programming languages (TypeScript, Python)
CloudFormation - AWS-only, YAML
Ansible - Configuration management, also does provisioning
CDK (Cloud Development Kit) - AWS, defines CloudFormation

We're using Terraform because:

✅ Multi-cloud (Linode, AWS, GCP, Azure)
✅ Declarative (describe desired state)
✅ Large provider ecosystem
✅ Industry standard
✅ Free and open source

Terraform Fundamentals

Core Concepts

1. Providers: Plugins for cloud platforms

terraform {
  required_providers {
    linode = {
      source  = "linode/linode"
      version = "~> 2.9.0"
    }
  }
}

provider "linode" {
  token = var.linode_token
}

2. Resources: Infrastructure components

resource "linode_instance" "web" {
  label  = "web-server"
  region = "us-east"
  type   = "g6-standard-1"
  image  = "linode/ubuntu22.04"
}

3. Variables: Parameterize configuration

variable "environment" {
  type    = string
  default = "production"
}

resource "linode_lke_cluster" "cluster" {
  label = "sspp-${var.environment}"
}

4. Outputs: Extract values

output "cluster_id" {
  value = linode_lke_cluster.cluster.id
}

output "kubeconfig" {
  value     = linode_lke_cluster.cluster.kubeconfig
  sensitive = true
}

5. State: Terraform tracks what it created

terraform.tfstate  # Current infrastructure state

Terraform Workflow

# 1. Write configuration
vim main.tf

# 2. Initialize (download providers)
terraform init

# 3. Preview changes
terraform plan

# 4. Apply changes
terraform apply

# 5. Destroy infrastructure (cleanup)
terraform destroy

Building SSPP Infrastructure with Terraform

Project Structure

infrastructure/terraform/
├── main.tf           # Main configuration
├── variables.tf      # Input variables
├── outputs.tf        # Output values
├── versions.tf       # Provider versions
├── terraform.tfvars  # Variable values (gitignored)
├── backend.tf        # Remote state config
├── modules/
│   ├── lke-cluster/  # Kubernetes cluster module
│   ├── networking/   # VPC, firewall rules
│   └── monitoring/   # Monitoring setup
└── environments/
    ├── dev/
    ├── staging/
    └── prod/

Step 1: Provider Configuration

# infrastructure/terraform/versions.tf
terraform {
  required_version = ">= 1.6.0"

  required_providers {
    linode = {
      source  = "linode/linode"
      version = "~> 2.9.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.24.0"
    }
  }
}

provider "linode" {
  token = var.linode_token
}

provider "kubernetes" {
  host                   = base64decode(linode_lke_cluster.sspp.kubeconfig)
  cluster_ca_certificate = base64decode(linode_lke_cluster.sspp.api_endpoints[0].ca_certificate)
  token                  = linode_lke_cluster.sspp.api_endpoints[0].token
}

Step 2: Variables

# infrastructure/terraform/variables.tf
variable "linode_token" {
  description = "Linode API token"
  type        = string
  sensitive   = true
}

variable "environment" {
  description = "Environment name (dev, staging, prod)"
  type        = string
  default     = "prod"
}

variable "region" {
  description = "Linode region"
  type        = string
  default     = "us-east"
}

variable "k8s_version" {
  description = "Kubernetes version"
  type        = string
  default     = "1.28"
}

variable "node_type" {
  description = "Linode instance type for nodes"
  type        = string
  default     = "g6-standard-4"
}

variable "node_count" {
  description = "Initial node count"
  type        = number
  default     = 3
}

variable "autoscaler_min" {
  description = "Minimum nodes for autoscaling"
  type        = number
  default     = 3
}

variable "autoscaler_max" {
  description = "Maximum nodes for autoscaling"
  type        = number
  default     = 10
}

variable "tags" {
  description = "Tags for resources"
  type        = list(string)
  default     = ["sspp", "production"]
}

Step 3: LKE Cluster

# infrastructure/terraform/main.tf
resource "linode_lke_cluster" "sspp" {
  label       = "sspp-${var.environment}"
  k8s_version = var.k8s_version
  region      = var.region
  tags        = var.tags

  pool {
    type  = var.node_type
    count = var.node_count

    autoscaler {
      min = var.autoscaler_min
      max = var.autoscaler_max
    }
  }

  control_plane {
    high_availability = var.environment == "prod" ? true : false
  }
}

# Save kubeconfig to file
resource "local_file" "kubeconfig" {
  content         = base64decode(linode_lke_cluster.sspp.kubeconfig)
  filename        = "${path.module}/kubeconfig-${var.environment}"
  file_permission = "0600"
}

Step 4: NodeBalancer for LoadBalancer Services

# infrastructure/terraform/nodebalancer.tf
resource "linode_nodebalancer" "sspp_api" {
  label  = "sspp-api-${var.environment}"
  region = var.region
  tags   = var.tags
}

resource "linode_nodebalancer_config" "sspp_api_https" {
  nodebalancer_id = linode_nodebalancer.sspp_api.id
  port            = 443
  protocol        = "https"
  check           = "http"
  check_path      = "/api/v1/health"
  check_attempts  = 3
  check_timeout   = 5

  ssl_cert = var.ssl_certificate
  ssl_key  = var.ssl_private_key
}

# Output NodeBalancer IP
output "api_load_balancer_ip" {
  value       = linode_nodebalancer.sspp_api.ipv4
  description = "Load balancer IP for API service"
}

Step 5: Object Storage for Backups

# infrastructure/terraform/object-storage.tf
resource "linode_object_storage_bucket" "backups" {
  cluster = "us-east-1"
  label   = "sspp-backups-${var.environment}"

  lifecycle_rule {
    enabled = true

    expiration {
      days = 30
    }
  }
}

resource "linode_object_storage_key" "backup_access" {
  label = "sspp-backup-access-${var.environment}"

  bucket_access {
    bucket_name = linode_object_storage_bucket.backups.label
    cluster     = linode_object_storage_bucket.backups.cluster
    permissions = "read_write"
  }
}

output "backup_bucket_url" {
  value = "https://${linode_object_storage_bucket.backups.cluster}.linodeobjects.com/${linode_object_storage_bucket.backups.label}"
}

Step 6: Firewall Rules

# infrastructure/terraform/firewall.tf
resource "linode_firewall" "cluster" {
  label = "sspp-cluster-${var.environment}"
  tags  = var.tags

  inbound {
    label    = "allow-https"
    action   = "ACCEPT"
    protocol = "TCP"
    ports    = "443"
    ipv4     = ["0.0.0.0/0"]
    ipv6     = ["::/0"]
  }

  inbound {
    label    = "allow-http"
    action   = "ACCEPT"
    protocol = "TCP"
    ports    = "80"
    ipv4     = ["0.0.0.0/0"]
    ipv6     = ["::/0"]
  }

  inbound {
    label    = "allow-k8s-api"
    action   = "ACCEPT"
    protocol = "TCP"
    ports    = "6443"
    ipv4     = var.allowed_ip_ranges
  }

  outbound_policy = "ACCEPT"

  linodes = [for node in linode_lke_cluster.sspp.pool[0].nodes : node.instance_id]
}

Step 7: DNS Records

# infrastructure/terraform/dns.tf
resource "linode_domain" "sspp" {
  domain    = var.domain_name
  soa_email = var.admin_email
  type      = "master"
  tags      = var.tags
}

resource "linode_domain_record" "api" {
  domain_id   = linode_domain.sspp.id
  name        = "api"
  record_type = "A"
  target      = linode_nodebalancer.sspp_api.ipv4
  ttl_sec     = 300
}

resource "linode_domain_record" "wildcard" {
  domain_id   = linode_domain.sspp.id
  name        = "*"
  record_type = "A"
  target      = linode_nodebalancer.sspp_api.ipv4
  ttl_sec     = 300
}

Step 8: Outputs

# infrastructure/terraform/outputs.tf
output "cluster_id" {
  description = "LKE cluster ID"
  value       = linode_lke_cluster.sspp.id
}

output "cluster_endpoint" {
  description = "Kubernetes API endpoint"
  value       = linode_lke_cluster.sspp.api_endpoints[0].endpoint
}

output "kubeconfig_path" {
  description = "Path to kubeconfig file"
  value       = local_file.kubeconfig.filename
}

output "api_endpoint" {
  description = "API public endpoint"
  value       = "https://api.${var.domain_name}"
}

output "backup_bucket" {
  description = "S3-compatible backup bucket URL"
  value       = "https://${linode_object_storage_bucket.backups.cluster}.linodeobjects.com/${linode_object_storage_bucket.backups.label}"
}

output "backup_access_key" {
  description = "Backup bucket access key"
  value       = linode_object_storage_key.backup_access.access_key
  sensitive   = true
}

output "backup_secret_key" {
  description = "Backup bucket secret key"
  value       = linode_object_storage_key.backup_access.secret_key
  sensitive   = true
}

Remote State Management

Problem: terraform.tfstate is stored locally. If you lose it, Terraform can't manage your infrastructure.

Solution: Store state remotely (S3, Terraform Cloud).

# infrastructure/terraform/backend.tf
terraform {
  backend "s3" {
    bucket   = "sspp-terraform-state"
    key      = "prod/terraform.tfstate"
    region   = "us-east-1"
    encrypt  = true

    # State locking with DynamoDB
    dynamodb_table = "terraform-state-lock"
  }
}

Benefits:

✅ Team collaboration (shared state)
✅ State locking (prevents concurrent modifications)
✅ Encrypted at rest
✅ Version history

Alternative: Terraform Cloud

terraform {
  cloud {
    organization = "sspp"

    workspaces {
      name = "sspp-prod"
    }
  }
}

Using Terraform

Initialize

cd infrastructure/terraform

# Download providers
terraform init

Output:

Initializing the backend...
Initializing provider plugins...
- Finding linode/linode versions matching "~> 2.9.0"...
- Installing linode/linode v2.9.3...

Terraform has been successfully initialized!

Create Variable File

# infrastructure/terraform/terraform.tfvars
linode_token = "YOUR_LINODE_API_TOKEN"
environment  = "prod"
region       = "us-east"
domain_name  = "sspp.example.com"
admin_email  = "admin@example.com"

node_type       = "g6-standard-4"
node_count      = 3
autoscaler_min  = 3
autoscaler_max  = 10

tags = ["sspp", "production", "managed-by-terraform"]

Add to .gitignore:

echo "terraform.tfvars" >> .gitignore
echo "*.tfstate*" >> .gitignore
echo "kubeconfig-*" >> .gitignore

Plan (Preview Changes)

terraform plan

Output:

Terraform will perform the following actions:

  # linode_lke_cluster.sspp will be created
  + resource "linode_lke_cluster" "sspp" {
      + id          = (known after apply)
      + label       = "sspp-prod"
      + k8s_version = "1.28"
      + region      = "us-east"

      + pool {
          + count = 3
          + type  = "g6-standard-4"

          + autoscaler {
              + min = 3
              + max = 10
            }
        }
    }

  # linode_object_storage_bucket.backups will be created
  + resource "linode_object_storage_bucket" "backups" {
      + cluster = "us-east-1"
      + label   = "sspp-backups-prod"
    }

Plan: 8 to add, 0 to change, 0 to destroy.

Review carefully! This is your code review checkpoint.

Apply

terraform apply

Terraform asks for confirmation:

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Provisioning:

linode_lke_cluster.sspp: Creating...
linode_object_storage_bucket.backups: Creating...
linode_lke_cluster.sspp: Still creating... [10s elapsed]
linode_lke_cluster.sspp: Still creating... [5m0s elapsed]
linode_lke_cluster.sspp: Creation complete after 8m23s

Apply complete! Resources: 8 added, 0 changed, 0 destroyed.

Outputs:

cluster_id = "12345"
api_endpoint = "https://api.sspp.example.com"
kubeconfig_path = "./kubeconfig-prod"

Your infrastructure is now live! 🎉

Configure kubectl

export KUBECONFIG=$(terraform output -raw kubeconfig_path)
kubectl get nodes

Output:

NAME                        STATUS   ROLES    AGE   VERSION
lke12345-67890-abc123       Ready    <none>   5m    v1.28.3
lke12345-67890-def456       Ready    <none>   5m    v1.28.3
lke12345-67890-ghi789       Ready    <none>   5m    v1.28.3

Terraform Modules for Reusability

Problem: Duplicating configuration for dev, staging, prod.

Solution: Create reusable modules.

Module Structure

infrastructure/terraform/modules/lke-cluster/
├── main.tf
├── variables.tf
└── outputs.tf

Module definition:

# infrastructure/terraform/modules/lke-cluster/main.tf
resource "linode_lke_cluster" "cluster" {
  label       = var.cluster_name
  k8s_version = var.k8s_version
  region      = var.region
  tags        = var.tags

  pool {
    type  = var.node_type
    count = var.node_count

    autoscaler {
      min = var.autoscaler_min
      max = var.autoscaler_max
    }
  }

  control_plane {
    high_availability = var.high_availability
  }
}

Using the module:

# infrastructure/terraform/environments/prod/main.tf
module "prod_cluster" {
  source = "../../modules/lke-cluster"

  cluster_name      = "sspp-prod"
  region            = "us-east"
  k8s_version       = "1.28"
  node_type         = "g6-standard-4"
  node_count        = 3
  autoscaler_min    = 3
  autoscaler_max    = 10
  high_availability = true
  tags              = ["sspp", "prod"]
}

# infrastructure/terraform/environments/dev/main.tf
module "dev_cluster" {
  source = "../../modules/lke-cluster"

  cluster_name      = "sspp-dev"
  region            = "us-east"
  k8s_version       = "1.28"
  node_type         = "g6-standard-2"  # Smaller nodes
  node_count        = 2
  autoscaler_min    = 2
  autoscaler_max    = 5
  high_availability = false  # Dev doesn't need HA
  tags              = ["sspp", "dev"]
}

Benefits:

✅ DRY (Don't Repeat Yourself)
✅ Consistent configuration
✅ Easy to update (change module, affects all environments)

Terraform Best Practices

1. Use Variables for Everything

# ❌ Bad
resource "linode_lke_cluster" "cluster" {
  label  = "sspp-prod"
  region = "us-east"
}

# ✅ Good
resource "linode_lke_cluster" "cluster" {
  label  = "sspp-${var.environment}"
  region = var.region
}

2. Tag All Resources

tags = [
  "environment:${var.environment}",
  "managed-by:terraform",
  "project:sspp",
  "owner:devops-team"
]

Why: Cost tracking, resource filtering, compliance.

3. Use Remote State

# Never commit terraform.tfstate to git!
terraform {
  backend "s3" {
    bucket = "terraform-state"
    key    = "prod/terraform.tfstate"
  }
}

4. Lock Provider Versions

# ❌ Bad - Could break with provider updates
terraform {
  required_providers {
    linode = {
      source = "linode/linode"
    }
  }
}

# ✅ Good - Explicit version
terraform {
  required_providers {
    linode = {
      source  = "linode/linode"
      version = "~> 2.9.0"  # Allow 2.9.x, not 2.10.x
    }
  }
}

5. Use terraform fmt and validate

# Format code
terraform fmt -recursive

# Validate syntax
terraform validate

# Check for issues
tflint

6. Plan Before Apply

# Always review plan
terraform plan -out=tfplan

# Review changes
less tfplan

# Apply only if looks good
terraform apply tfplan

7. Use Workspaces for Environments

# Create workspaces
terraform workspace new dev
terraform workspace new staging
terraform workspace new prod

# Switch workspace
terraform workspace select prod

# List workspaces
terraform workspace list

Different state per workspace.

Terraform in CI/CD

GitHub Actions Workflow

# .github/workflows/terraform.yml
name: Terraform Infrastructure

on:
  push:
    branches: [main]
    paths:
    - 'infrastructure/terraform/**'
  pull_request:
    branches: [main]
    paths:
    - 'infrastructure/terraform/**'

env:
  TF_VERSION: 1.6.0

jobs:
  terraform:
    name: Terraform Plan & Apply
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: infrastructure/terraform

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v3
      with:
        terraform_version: ${{ env.TF_VERSION }}

    - name: Terraform Format Check
      run: terraform fmt -check -recursive

    - name: Terraform Init
      env:
        LINODE_TOKEN: ${{ secrets.LINODE_TOKEN }}
      run: terraform init

    - name: Terraform Validate
      run: terraform validate

    - name: Terraform Plan
      env:
        LINODE_TOKEN: ${{ secrets.LINODE_TOKEN }}
      run: |
        terraform plan -out=tfplan
        terraform show -no-color tfplan > plan.txt

    - name: Comment PR with Plan
      if: github.event_name == 'pull_request'
      uses: actions/github-script@v7
      with:
        script: |
          const fs = require('fs');
          const plan = fs.readFileSync('infrastructure/terraform/plan.txt', 'utf8');
          github.rest.issues.createComment({
            issue_number: context.issue.number,
            owner: context.repo.owner,
            repo: context.repo.repo,
            body: '```
{% endraw %}
terraform\n' + plan + '\n
{% raw %}
```'
          });

    - name: Terraform Apply
      if: github.ref == 'refs/heads/main' && github.event_name == 'push'
      env:
        LINODE_TOKEN: ${{ secrets.LINODE_TOKEN }}
      run: terraform apply -auto-approve tfplan

Workflow:

PR: Runs terraform plan, comments on PR
Merge to main: Runs terraform apply
Infrastructure changes are code-reviewed!

Managing Kubernetes Resources with Terraform

You can also manage Kubernetes resources with Terraform:

# infrastructure/terraform/kubernetes.tf
provider "kubernetes" {
  host                   = linode_lke_cluster.sspp.api_endpoints[0].endpoint
  token                  = linode_lke_cluster.sspp.api_endpoints[0].token
  cluster_ca_certificate = base64decode(linode_lke_cluster.sspp.api_endpoints[0].ca_certificate)
}

resource "kubernetes_namespace" "sspp_prod" {
  metadata {
    name = "sspp-prod"

    labels = {
      environment = "production"
      managed-by  = "terraform"
    }
  }
}

resource "kubernetes_secret" "sspp_secrets" {
  metadata {
    name      = "sspp-secrets"
    namespace = kubernetes_namespace.sspp_prod.metadata[0].name
  }

  data = {
    DB_PASSWORD = var.db_password
    JWT_SECRET  = var.jwt_secret
  }

  type = "Opaque"
}

But should you?

Pros:

Everything in one place
Terraform manages both cluster and apps

Cons:

Mixing infrastructure and application concerns
kubectl is faster for iteration
ArgoCD is better for GitOps

Best practice: Use Terraform for infrastructure (cluster, nodes), use Kubernetes manifests or ArgoCD for applications.

Real-World Example: Multi-Environment Setup

infrastructure/terraform/
├── modules/
│   └── lke-cluster/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
├── environments/
│   ├── dev/
│   │   ├── main.tf
│   │   ├── terraform.tfvars
│   │   └── backend.tf
│   ├── staging/
│   │   ├── main.tf
│   │   ├── terraform.tfvars
│   │   └── backend.tf
│   └── prod/
│       ├── main.tf
│       ├── terraform.tfvars
│       └── backend.tf

Deploy dev:

cd environments/dev
terraform init
terraform apply

Deploy staging:

cd environments/staging
terraform init
terraform apply

Deploy prod:

cd environments/prod
terraform init
terraform apply

Identical configuration, different parameters.

Cost Estimation with Infracost

How much will this cost?

# Install Infracost
brew install infracost

# Get cost estimate
infracost breakdown --path infrastructure/terraform

# Output:
# ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
# ┃ Resource                   ┃ Monthly cost ┃
# ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
# ┃ linode_lke_cluster.sspp    ┃              ┃
# ┃  └─ Node pool (3x g6-4)    ┃      $120.00 ┃
# ┃ linode_nodebalancer        ┃       $10.00 ┃
# ┃ object_storage_bucket      ┃        $5.00 ┃
# ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
# ┃ TOTAL                      ┃      $135.00 ┃
# ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛

Add to CI/CD:

- name: Cost estimate
  run: |
    infracost breakdown --path . --format json > cost.json
    infracost comment github --path cost.json

PR comments show cost changes!

What We Solved

✅ Reproducible infrastructure - Code defines everything

✅ Version controlled - Git tracks all changes

✅ Code reviewed - PRs for infrastructure changes

✅ Multi-environment - Dev, staging, prod from same code

✅ Self-documenting - Terraform files are the docs

✅ Automated - CI/CD applies changes

✅ Cost visible - Infracost shows price before apply

What's Next?

We have infrastructure as code with Terraform. But application deployment is still manual:

kubectl apply -f k8s/

Problems:

❌ No rollback mechanism
❌ No deployment history
❌ Manual sync between Git and cluster
❌ No automatic sync when manifests change

In Part 9, we'll add GitOps with ArgoCD:

Git as single source of truth
Automatic sync (Git → Cluster)
Deployment history and rollback
Multi-cluster management
Self-service deployments

Push to Git → ArgoCD deploys automatically.

Try It Yourself

Challenge: Create complete Terraform infrastructure:

Create Linode account (free $100 credit)
Get API token
Write Terraform configuration for LKE cluster
Add NodeBalancer, object storage, firewall
Create dev, staging, prod environments
Apply with Terraform
Deploy SSPP to each environment
Add cost estimation with Infracost
Set up CI/CD for Terraform

Bonus: Manage Kubernetes namespaces with Terraform.

Discussion

Do you use Terraform? Pulumi? CloudFormation? What's your IaC tool of choice?

Share on GitHub Discussions.

Previous: Part 6: Kubernetes Without Magic

Next: Part 8: Helm - Packaging Kubernetes Applications

About the Author

Building this series for my Proton.ai application to demonstrate real DevOps thinking.

GitHub: @daviesbrown
LinkedIn: David Nwosu

Part 6: Kubernetes from First Principles (No Magic)

David Nwosu — Mon, 22 Dec 2025 23:30:11 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 15 minutes

Level: Intermediate

The Kubernetes Mindset Shift

In Part 5, we broke Docker Compose in creative ways and felt the pain of manual orchestration. Now it's time to fix it.

But first, we need a mindset shift.

Before Kubernetes:

"I have a server. I'll SSH in and run containers on it."

With Kubernetes:

"I have a cluster. I'll declare what I want running. Kubernetes makes it happen."

This is the key insight: Kubernetes is declarative, not imperative.

You don't tell it how to do things. You tell it what you want, and it figures out the how.

# You write this
apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 5

# Kubernetes does this:
# ✓ Schedules 5 pods across nodes
# ✓ Monitors them continuously
# ✓ Restarts if they crash
# ✓ Redistributes if a node dies
# ✓ Rolls out updates gracefully

You describe desired state. Kubernetes maintains it.

Core Concepts (No Jargon)

Pods: The Smallest Unit

A Pod is one or more containers that run together on the same machine.

apiVersion: v1
kind: Pod
metadata:
  name: my-api
spec:
  containers:
  - name: api
    image: davidbrown77/sspp-api:latest
    ports:
    - containerPort: 3000

Think of a Pod as: A wrapper around containers that share:

Network (same IP address)
Storage (can mount same volumes)
Lifecycle (started/stopped together)

Why not just use containers directly? Because you need more control:

Sidecars: Add a logging container next to your app
Init containers: Run setup before main container starts
Shared volumes: Multiple containers access same files

Real-world analogy: A Pod is like a house. Containers are the rooms. They share the address (IP), utilities (network), and exist together.

Deployments: Managing Replicas

You almost never create Pods directly. You create a Deployment, which manages Pods for you.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: davidbrown77/sspp-api:latest

What this creates:

3 identical Pods running your API
ReplicaSet managing them
Automatic recreation if Pods die
Rolling update strategy built-in

The magic: Kill a Pod, Kubernetes immediately creates a new one. Always maintains 3 replicas.

Services: Stable Networking

Pods are ephemeral—they die and get recreated with new IP addresses.

Problem: How do other Pods find your API if its IP keeps changing?

Solution: A Service provides a stable DNS name and IP.

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: api
  ports:
  - port: 80
    targetPort: 3000
  type: LoadBalancer

What this does:

Creates a stable DNS name: api.sspp-dev.svc.cluster.local
Load balances requests across all Pods with label app: api
Exposes externally (type: LoadBalancer) via cloud provider's load balancer

Types of Services:

ClusterIP: Internal only (default)
NodePort: Exposes on each node's IP at a static port
LoadBalancer: Creates external load balancer (Linode NodeBalancer)

ConfigMaps & Secrets: Configuration Management

Problem: Hardcoding environment variables in Deployment YAML is bad:

❌ Can't reuse across environments (dev, staging, prod)
❌ Secrets visible in YAML files
❌ Config changes require redeploying entire app

Solution: Separate configuration from application code.

ConfigMaps: Non-Sensitive Configuration

A ConfigMap stores configuration data as key-value pairs.

apiVersion: v1
kind: ConfigMap
metadata:
  name: sspp-config
  namespace: sspp-dev
data:
  # Simple values
  NODE_ENV: "production"
  REDIS_PORT: "6379"
  ELASTICSEARCH_URL: "http://elasticsearch:9200"
  QUEUE_NAME: "sales-events"

  # Multi-line config files
  app.conf: |
    log_level=info
    max_connections=100
    timeout=30s

Use in Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  template:
    spec:
      containers:
      - name: api
        image: davidbrown77/sspp-api:latest
        envFrom:
        - configMapRef:
            name: sspp-config  # Import ALL keys as env vars
        # Or selectively:
        env:
        - name: NODE_ENV
          valueFrom:
            configMapKeyRef:
              name: sspp-config
              key: NODE_ENV

Secrets: Sensitive Data

A Secret stores sensitive data (passwords, tokens, keys) with base64 encoding.

Create from command line:

# From literal values
kubectl create secret generic sspp-secrets \
  --from-literal=DB_PASSWORD=sspp_password \
  --from-literal=REDIS_PASSWORD=redis_secret \
  -n sspp-dev

# From files
kubectl create secret generic api-tls \
  --from-file=tls.crt=./cert.pem \
  --from-file=tls.key=./key.pem \
  -n sspp-dev

Or define in YAML (base64-encode first):

# Encode secrets
echo -n 'sspp_password' | base64
# Output: c3NwcF9wYXNzd29yZA==

apiVersion: v1
kind: Secret
metadata:
  name: sspp-secrets
  namespace: sspp-dev
type: Opaque
data:
  DB_PASSWORD: c3NwcF9wYXNzd29yZA==  # base64 encoded
  REDIS_PASSWORD: cmVkaXNfc2VjcmV0==

Use in Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  template:
    spec:
      containers:
      - name: api
        image: davidbrown77/sspp-api:latest
        env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: sspp-secrets
              key: DB_PASSWORD
        # Or mount as files:
        volumeMounts:
        - name: secret-volume
          mountPath: /etc/secrets
          readOnly: true
      volumes:
      - name: secret-volume
        secret:
          secretName: sspp-secrets

Access mounted secret:

// In your app
const password = fs.readFileSync('/etc/secrets/DB_PASSWORD', 'utf8');

Key differences:

Feature	ConfigMap	Secret
Use case	Non-sensitive config	Passwords, tokens, keys
Encoding	Plain text	Base64
Storage	etcd (plain)	etcd (can be encrypted)
Size limit	1MB	1MB
Best for	URLs, ports, flags	Credentials, certificates

Setting Up Kubernetes on Linode

We'll use Linode Kubernetes Engine (LKE) for production-grade Kubernetes without the setup pain.

Create a Cluster

Log into Linode Cloud Manager
Click "Kubernetes" → "Create Cluster"
Configure:
- Cluster Label: sspp-cluster
- Region: Choose closest to you
- Kubernetes Version: 1.28 (or latest stable)
- Node Pools:
  - Pool: standard-4gb (2 CPU, 4GB RAM)
  - Count: 3 nodes
Click "Create Cluster"

Wait 5-10 minutes for cluster provisioning.

Configure kubectl

Download your kubeconfig:

# Install kubectl (if not already installed)
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/

# Download kubeconfig from Linode dashboard
# Or use Linode CLI:
linode-cli lke kubeconfig-view <cluster-id> > ~/.kube/config

# Verify connection
kubectl get nodes

Output:

NAME                        STATUS   ROLES    AGE   VERSION
lke12345-67890-abcdef1234   Ready    <none>   5m    v1.28.0
lke12345-67890-abcdef5678   Ready    <none>   5m    v1.28.0
lke12345-67890-abcdef9012   Ready    <none>   5m    v1.28.0

You now have a 3-node Kubernetes cluster. 🎉

Deploying SSPP to Kubernetes

Let's deploy our entire stack: PostgreSQL, Redis, Elasticsearch, API, and Worker.

Step 1: Create Namespace

Namespaces isolate resources (dev, staging, prod).

kubectl create namespace sspp-dev

# Set as default
kubectl config set-context --current --namespace=sspp-dev

Step 2: Deploy PostgreSQL

# infrastructure/k8s/postgres.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  namespace: sspp-dev
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: linode-block-storage
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: sspp-dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15-alpine
        env:
        - name: POSTGRES_DB
          value: sales_signals
        - name: POSTGRES_USER
          value: sspp_user
        - name: POSTGRES_PASSWORD
          value: sspp_password
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
      volumes:
      - name: postgres-storage
        persistentVolumeClaim:
          claimName: postgres-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: sspp-dev
spec:
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432
  type: ClusterIP

Key points:

PersistentVolumeClaim: Requests 10GB storage (data survives Pod restarts)
Deployment: 1 replica (databases don't horizontally scale easily)
Service: ClusterIP (internal only, not exposed externally)

Deploy it:

kubectl apply -f infrastructure/k8s/postgres.yaml

# Check status
kubectl get pods
kubectl get pvc
kubectl get svc

Step 3: Deploy Redis

# infrastructure/k8s/redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: sspp-dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7-alpine
        ports:
        - containerPort: 6379
---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: sspp-dev
spec:
  selector:
    app: redis
  ports:
  - port: 6379
    targetPort: 6379
  type: ClusterIP

Deploy:

kubectl apply -f infrastructure/k8s/redis.yaml

Step 4: Deploy Elasticsearch

# infrastructure/k8s/elasticsearch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: elasticsearch
  namespace: sspp-dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: elasticsearch
        image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
        env:
        - name: discovery.type
          value: single-node
        - name: xpack.security.enabled
          value: "false"
        - name: ES_JAVA_OPTS
          value: "-Xms512m -Xmx512m"
        ports:
        - containerPort: 9200
        - containerPort: 9300
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: sspp-dev
spec:
  selector:
    app: elasticsearch
  ports:
  - name: http
    port: 9200
    targetPort: 9200
  - name: transport
    port: 9300
    targetPort: 9300
  type: ClusterIP

Deploy:

kubectl apply -f infrastructure/k8s/elasticsearch.yaml

Step 5: Deploy API Service

# infrastructure/k8s/api.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: sspp-dev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: davidbrown77/sspp-api:latest
        ports:
        - containerPort: 3000
        env:
        - name: NODE_ENV
          value: production
        - name: PORT
          value: "3000"
        - name: DB_HOST
          value: postgres
        - name: DB_PORT
          value: "5432"
        - name: DB_NAME
          value: sales_signals
        - name: DB_USER
          value: sspp_user
        - name: DB_PASSWORD
          value: sspp_password
        - name: REDIS_HOST
          value: redis
        - name: REDIS_PORT
          value: "6379"
        - name: ELASTICSEARCH_URL
          value: http://elasticsearch:9200
        resources:
          requests:
            memory: "256Mi"
            cpu: "200m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /api/v1/health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /api/v1/health
            port: 3000
          initialDelaySeconds: 10
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: api
  namespace: sspp-dev
spec:
  selector:
    app: api
  ports:
  - port: 80
    targetPort: 3000
  type: LoadBalancer

Key features:

2 replicas: Load balanced automatically
Resource limits: Prevents resource hogging
Health checks: Kubernetes won't route traffic to unhealthy Pods
LoadBalancer: Exposes API publicly via Linode NodeBalancer

Deploy:

kubectl apply -f infrastructure/k8s/api.yaml

# Get external IP (takes 1-2 minutes)
kubectl get svc api

Output:

NAME   TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE
api    LoadBalancer   10.128.45.123   45.79.123.45     80:30123/TCP   2m

Your API is now publicly accessible at http://45.79.123.45!

Step 6: Deploy Worker Service

# infrastructure/k8s/worker.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: worker
  namespace: sspp-dev
spec:
  replicas: 3
  selector:
    matchLabels:
      app: worker
  template:
    metadata:
      labels:
        app: worker
    spec:
      containers:
      - name: worker
        image: davidbrown77/sspp-worker:latest
        env:
        - name: NODE_ENV
          value: production
        - name: DB_HOST
          value: postgres
        - name: DB_PORT
          value: "5432"
        - name: DB_NAME
          value: sales_signals
        - name: DB_USER
          value: sspp_user
        - name: DB_PASSWORD
          value: sspp_password
        - name: REDIS_HOST
          value: redis
        - name: REDIS_PORT
          value: "6379"
        - name: ELASTICSEARCH_URL
          value: http://elasticsearch:9200
        - name: QUEUE_NAME
          value: sales-events
        resources:
          requests:
            memory: "256Mi"
            cpu: "200m"
          limits:
            memory: "512Mi"
            cpu: "500m"

Deploy:

kubectl apply -f infrastructure/k8s/worker.yaml

Managing Multiple Environments

Real production setup: You need dev, staging, and production environments.

Strategy 1: Namespaces (Simple)

Separate by namespace:

# Create namespaces
kubectl create namespace sspp-dev
kubectl create namespace sspp-staging
kubectl create namespace sspp-prod

Deploy to each:

# Dev
kubectl apply -f infrastructure/k8s/ -n sspp-dev

# Staging
kubectl apply -f infrastructure/k8s/ -n sspp-staging

# Production
kubectl apply -f infrastructure/k8s/ -n sspp-prod

Create environment-specific ConfigMaps:

# config-dev.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: sspp-config
  namespace: sspp-dev
data:
  NODE_ENV: "development"
  LOG_LEVEL: "debug"
  REDIS_HOST: "redis.sspp-dev"
---
# config-prod.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: sspp-config
  namespace: sspp-prod
data:
  NODE_ENV: "production"
  LOG_LEVEL: "info"
  REDIS_HOST: "redis.sspp-prod"

Apply:

kubectl apply -f config-dev.yaml
kubectl apply -f config-prod.yaml

Strategy 2: Kustomize (Better)

Kustomize is built into kubectl and lets you customize YAML without templates.

Directory structure:

infrastructure/k8s/
├── base/
│   ├── kustomization.yaml
│   ├── deployment.yaml
│   ├── service.yaml
│   └── configmap.yaml
├── overlays/
│   ├── dev/
│   │   ├── kustomization.yaml
│   │   └── patch-replicas.yaml
│   ├── staging/
│   │   └── kustomization.yaml
│   └── prod/
│       ├── kustomization.yaml
│       └── patch-replicas.yaml

base/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- deployment.yaml
- service.yaml
- configmap.yaml

commonLabels:
  app: sspp-api

overlays/dev/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: sspp-dev

resources:
- ../../base

patchesStrategicMerge:
- patch-replicas.yaml

configMapGenerator:
- name: sspp-config
  literals:
  - NODE_ENV=development
  - LOG_LEVEL=debug

overlays/dev/patch-replicas.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 1  # Dev only needs 1 replica

overlays/prod/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: sspp-prod

resources:
- ../../base

patchesStrategicMerge:
- patch-replicas.yaml

configMapGenerator:
- name: sspp-config
  literals:
  - NODE_ENV=production
  - LOG_LEVEL=info

secretGenerator:
- name: sspp-secrets
  literals:
  - DB_PASSWORD=actual_prod_password

overlays/prod/patch-replicas.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 5  # Production needs 5 replicas

Deploy with Kustomize:

# Dev
kubectl apply -k infrastructure/k8s/overlays/dev

# Staging
kubectl apply -k infrastructure/k8s/overlays/staging

# Production
kubectl apply -k infrastructure/k8s/overlays/prod

Benefits:

✅ DRY (Don't Repeat Yourself) - base shared across environments
✅ Environment-specific overrides
✅ No templating language to learn
✅ Native kubectl support

Strategy 3: Separate Clusters (Production-Grade)

For serious production:

┌─────────────────┐
│  Dev Cluster    │  - 1-2 nodes, shared resources
│  sspp-dev       │  - Developers can break things
└─────────────────┘

┌─────────────────┐
│ Staging Cluster │  - Mirrors production size
│  sspp-staging   │  - Pre-release testing
└─────────────────┘

┌─────────────────┐
│  Prod Cluster   │  - High availability (3+ nodes)
│  sspp-prod      │  - Strict access control
└─────────────────┘

Why separate clusters?

🔒 Security: Compromised dev can't affect prod
💰 Cost: Dev uses cheaper spot instances
🎯 Blast radius: Experiments stay contained
📊 Resource isolation: Dev load doesn't impact prod

Deploy to multiple clusters:

# Switch context
kubectl config use-context lke-dev
kubectl apply -k overlays/dev

kubectl config use-context lke-staging  
kubectl apply -k overlays/staging

kubectl config use-context lke-prod
kubectl apply -k overlays/prod

Best Practices for Environment Management

✅ Never hardcode secrets in YAML

# Bad
env:
- name: DB_PASSWORD
  value: "password123"  # ❌ Visible in Git

# Good
env:
- name: DB_PASSWORD
  valueFrom:
    secretKeyRef:
      name: db-secrets
      key: password  # ✅ Stored separately

✅ Use external secret managers for production

# Use AWS Secrets Manager, Vault, or Sealed Secrets
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: sspp-secrets
spec:
  secretStoreRef:
    name: aws-secrets-manager
  target:
    name: sspp-secrets
  data:
  - secretKey: DB_PASSWORD
    remoteRef:
      key: prod/sspp/db-password

✅ Namespace quotas prevent resource hogging

apiVersion: v1
kind: ResourceQuota
metadata:
  name: dev-quota
  namespace: sspp-dev
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    persistentvolumeclaims: "5"

✅ RBAC for access control

# Developers can deploy to dev, but not prod
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: developers
  namespace: sspp-dev
subjects:
- kind: Group
  name: developers
roleRef:
  kind: ClusterRole
  name: edit  # Can create/update resources

Refactoring API Deployment with ConfigMaps & Secrets

Let's refactor our API deployment to use proper configuration management:

1. Create ConfigMap:

# infrastructure/k8s/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: sspp-config
  namespace: sspp-dev
data:
  NODE_ENV: "production"
  PORT: "3000"
  DB_HOST: "postgres"
  DB_PORT: "5432"
  DB_NAME: "sales_signals"
  REDIS_HOST: "redis"
  REDIS_PORT: "6379"
  ELASTICSEARCH_URL: "http://elasticsearch:9200"

2. Create Secret:

kubectl create secret generic sspp-secrets \
  --from-literal=DB_USER=sspp_user \
  --from-literal=DB_PASSWORD=sspp_password \
  -n sspp-dev

3. Update API Deployment:

# infrastructure/k8s/api.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: sspp-dev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: davidbrown77/sspp-api:latest
        ports:
        - containerPort: 3000
        envFrom:
        - configMapRef:
            name: sspp-config  # Import all non-sensitive config
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: sspp-secrets
              key: DB_USER
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: sspp-secrets
              key: DB_PASSWORD
        resources:
          requests:
            memory: "256Mi"
            cpu: "200m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /api/v1/health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /api/v1/health
            port: 3000
          initialDelaySeconds: 10
          periodSeconds: 5

4. Deploy:

kubectl apply -f infrastructure/k8s/configmap.yaml
kubectl apply -f infrastructure/k8s/api.yaml

# Verify
kubectl get configmap sspp-config -n sspp-dev -o yaml
kubectl get secret sspp-secrets -n sspp-dev

Benefits of this approach:

✅ Configuration separate from code
✅ Easy to update config without redeploying image
✅ Secrets not visible in YAML files
✅ Same deployment YAML works across environments (just swap ConfigMap/Secret)

Testing the Deployment

Check Everything is Running

kubectl get pods

NAME                             READY   STATUS    RESTARTS   AGE
api-7d8f9c6b5-abc12              1/1     Running   0          5m
api-7d8f9c6b5-def34              1/1     Running   0          5m
postgres-6c8d7f5b4-ghi56         1/1     Running   0          8m
redis-5b7c6d4a3-jkl78            1/1     Running   0          7m
elasticsearch-4a6b5c3d2-mno90    1/1     Running   0          7m
worker-3c5d4e2f1-pqr12           1/1     Running   0          4m
worker-3c5d4e2f1-stu34           1/1     Running   0          4m
worker-3c5d4e2f1-vwx56           1/1     Running   0          4m

Perfect! All Pods are Running.

Send an Event

# Get external IP
EXTERNAL_IP=$(kubectl get svc api -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

# Send event
curl -X POST http://$EXTERNAL_IP/api/v1/events \
  -H "Content-Type: application/json" \
  -d '{
    "accountId": "acct_k8s_001",
    "userId": "user_k8s_001",
    "eventType": "email_sent",
    "timestamp": "2025-12-22T15:00:00Z",
    "metadata": {
      "campaign": "Kubernetes_Launch"
    }
  }'

Response:

{
  "status": "accepted",
  "jobId": "1",
  "message": "Event queued for processing"
}

Verify Processing

# Check worker logs
kubectl logs -l app=worker --tail=20

# Expected output:
# info: Processing job 1 {"accountId":"acct_k8s_001",...}
# info: Signal stored in PostgreSQL {"signalId":1}
# info: Signal indexed in Elasticsearch {"signalId":1}
# info: Job 1 completed successfully

End-to-end flow working! 🎊

Kubernetes Superpowers

Now let's see what Kubernetes can do that Docker Compose can't.

1. Self-Healing

Kill a Pod:

# List pods
kubectl get pods

# Delete one
kubectl delete pod api-7d8f9c6b5-abc12

# Check again immediately
kubectl get pods

Output:

NAME                    READY   STATUS              RESTARTS   AGE
api-7d8f9c6b5-abc12     1/1     Terminating         0          10m
api-7d8f9c6b5-xyz99     0/1     ContainerCreating   0          1s
api-7d8f9c6b5-def34     1/1     Running             0          10m

Kubernetes immediately created a replacement! No manual intervention.

2. Horizontal Scaling

Scale up:

kubectl scale deployment api --replicas=5

# Check
kubectl get pods -l app=api

Output:

NAME                    READY   STATUS    RESTARTS   AGE
api-7d8f9c6b5-abc12     1/1     Running   0          1m
api-7d8f9c6b5-def34     1/1     Running   0          1m
api-7d8f9c6b5-ghi56     1/1     Running   0          1m
api-7d8f9c6b5-jkl78     1/1     Running   0          10s
api-7d8f9c6b5-mno90     1/1     Running   0          10s

5 API instances in 10 seconds! Try doing that with Docker Compose.

Scale down:

kubectl scale deployment api --replicas=2

Kubernetes gracefully terminates 3 Pods.

3. Rolling Updates (Zero Downtime)

Update the image:

kubectl set image deployment/api api=davidbrown77/sspp-api:v2.0.0

# Watch the rollout
kubectl rollout status deployment/api

What happens:

Kubernetes creates 1 new Pod with v2.0.0
Waits for it to be healthy (readiness probe)
Terminates 1 old Pod
Repeats until all Pods are updated

No downtime. Traffic always routes to healthy Pods.

4. Rollback

New version has a bug? Rollback instantly:

kubectl rollout undo deployment/api

# Or rollback to specific revision
kubectl rollout undo deployment/api --to-revision=2

Rollback time: 30-60 seconds.

Compare to manual deployment: 5-10 minutes (SSH, git pull, rebuild, restart, pray).

5. Auto-Scaling (HPA)

Create a Horizontal Pod Autoscaler:

kubectl autoscale deployment api \
  --cpu-percent=70 \
  --min=2 \
  --max=10

What this does:

Monitors CPU usage
If average CPU > 70%, scale up
If CPU < 70%, scale down
Min 2 Pods, max 10 Pods

Load test it:

# Generate traffic
for i in {1..1000}; do
  curl -X POST http://$EXTERNAL_IP/api/v1/events \
    -H "Content-Type: application/json" \
    -d '{"accountId":"load_test","userId":"user_'$i'","eventType":"email_sent","timestamp":"2025-12-22T15:00:00Z","metadata":{}}' &
done

# Watch scaling
kubectl get hpa -w

Output:

NAME   REFERENCE        TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
api    Deployment/api   85%/70%   2         10        5          2m

Kubernetes scaled to 5 Pods automatically!

Essential kubectl Commands

Pods

# List pods
kubectl get pods

# Describe pod (detailed info)
kubectl describe pod <pod-name>

# Logs
kubectl logs <pod-name>
kubectl logs -f <pod-name>  # Follow

# Execute command
kubectl exec -it <pod-name> -- sh

# Delete pod
kubectl delete pod <pod-name>

Deployments

# List deployments
kubectl get deployments

# Describe deployment
kubectl describe deployment <name>

# Scale
kubectl scale deployment <name> --replicas=5

# Update image
kubectl set image deployment/<name> <container>=<image>

# Rollout status
kubectl rollout status deployment/<name>

# Rollback
kubectl rollout undo deployment/<name>

# Delete deployment
kubectl delete deployment <name>

Services

# List services
kubectl get svc

# Describe service
kubectl describe svc <name>

# Delete service
kubectl delete svc <name>

Everything

# List all resources
kubectl get all

# Delete everything in namespace
kubectl delete all --all

What We Solved

✅ Self-healing - Pods restart automatically

✅ Multi-server - Cluster of nodes, not single server

✅ Load balancing - Built-in Service resource

✅ Horizontal scaling - kubectl scale or HPA

✅ Rolling updates - Zero-downtime deployments

✅ Rollback - One-command revert

✅ Health checks - Traffic only to healthy Pods

✅ Declarative - YAML manifests = desired state

What's Next?

Our Kubernetes deployment works, but we have a critical problem:

❌ Manual infrastructure setup - We created the cluster by clicking buttons

❌ Not reproducible - Can't recreate this infrastructure reliably

❌ No version control - Infrastructure changes aren't tracked

❌ Drift risk - Production and dev environments diverge over time

In Part 7, we'll fix this with Terraform (Infrastructure as Code).

You'll learn:

Define infrastructure in code (not clicks)
Version control your entire stack
Recreate environments identically
Manage multiple environments (dev, staging, prod)
Treat infrastructure like application code

Spoiler: Delete your entire cluster, run terraform apply, and it rebuilds perfectly.

Try It Yourself

Challenge: Deploy SSPP to Linode Kubernetes Engine:

Create LKE cluster (3 nodes)
Deploy all services (postgres, redis, elasticsearch, api, worker)
Send 100 events
Kill Pods and watch them resurrect
Scale API to 10 replicas
Roll out a new image version

Bonus: Set up HPA and load test it.

Discussion

What was your "aha!" moment with Kubernetes?

Share on GitHub Discussions.

Previous: Part 5: Why Containers Still Fail in Production

Next: Part 7: Terraform - Making Infrastructure Repeatable

About the Author

Building this series for my Proton.ai application to demonstrate real DevOps thinking.

GitHub: @daviesbrown
LinkedIn: David Nwosu

Part 5: From One Server to Many - The Need for Orchestration

David Nwosu — Mon, 22 Dec 2025 23:26:27 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 11 minutes

Level: Intermediate

The Production Reality Check

Your SSPP platform is live! Docker Compose works beautifully on your local machine and even on your single production server.

Then Black Friday hits. Traffic spikes 50x.

What do you do?

You can't just run docker-compose up --scale worker=50 because:

One server doesn't have 50x the resources
The database would be overwhelmed
You'd need multiple servers

So you start manually:

# Rent 5 more Linode servers
# SSH into each one
# Install Docker on each
# Copy docker-compose.yml to each
# Modify each to avoid port conflicts
# Start containers manually
# Configure a load balancer somehow
# Hope nothing breaks

Time to scale: 3-4 hours (if you're fast and lucky)

By the time you're done, Black Friday is over.

Failure Scenario 1: Container Crashes

Let's simulate a production crash:

# Start your stack
docker-compose up -d

# Kill the API container
docker kill sspp-api

What happens?

The API is dead. Docker Compose doesn't restart it automatically.

Check status:

docker-compose ps

NAME            STATE
sspp-api        Exited (137)
sspp-worker     Up
sspp-postgres   Up
sspp-redis      Up

Users see 500 errors. Your on-call phone explodes. 📱💥

Manual fix:

docker-compose up -d api

Downtime: 2-10 minutes (detection + SSH + restart)

In a production system, you need automatic recovery.

Failure Scenario 2: Server Crashes

Even worse—the entire server goes down:

# Simulate server crash (don't actually run this!)
sudo reboot -f

What happens?

API: Dead
Worker: Dead
PostgreSQL: Dead (data persisted in volumes, but service down)
Redis queue: Empty (all jobs lost)
Users: Angry

Manual recovery:

# Wait for server to boot (~2 minutes)
# SSH in
docker-compose up -d
# Wait for services to start (~30 seconds)
# Hope data is intact

Downtime: 3-5 minutes minimum

Lost data: All queued jobs

Failure Scenario 3: Rolling Update Gone Wrong

You need to deploy a critical bug fix:

# Build new image
docker-compose build api

# Restart with new image
docker-compose up -d api

What happens?

Old API container stops (connections dropped)
New API container starts
5-30 seconds of downtime while it boots
If the new version has a bug, you need to manually rollback

The deployment strategy:

No blue/green deployment
No canary releases
No gradual rollout
Just... restart and pray 🙏

Failure Scenario 4: Manual Scaling Nightmare

Traffic is increasing. You need 5 API instances across 3 servers:

Server 1 (docker-compose.yml):

services:
  api:
    ports:
      - "3000:3000"  # Occupies port 3000

Server 2 (docker-compose.yml):

services:
  api:
    ports:
      - "3000:3000"  # Same port—works because different server

But how do users reach them? You need a load balancer:

         ┌──────────────┐
         │ Load Balancer│
         │  (HAProxy?)  │
         └───────┬──────┘
                 │
      ┌──────────┼──────────┐
      ▼          ▼          ▼
  Server 1   Server 2   Server 3
   API:3000   API:3000   API:3000

Manual steps:

Install HAProxy
Configure health checks
Add server IPs manually
Restart HAProxy when adding/removing servers
Handle SSL termination
Monitor everything

Time to set up: 2-4 hours

Maintenance burden: High

Error-prone: Very

Failure Scenario 5: Database Connection Limits

Your PostgreSQL server has a max_connections limit (default: 100).

With 10 API instances and 10 Worker instances, each holding 10 connections:

10 APIs × 10 connections = 100
10 Workers × 10 connections = 100
Total = 200 connections
Max allowed = 100

Result: Half your containers can't connect to the database.

Manual fix:

Configure connection pooling in each service
Increase PostgreSQL max_connections
Restart everything
Hope you calculated correctly

What You Need (But Don't Have)

At this point, you realize you need:

Self-healing: Automatically restart failed containers
Auto-scaling: Add/remove instances based on load
Load balancing: Distribute traffic across instances
Service discovery: Containers find each other dynamically
Rolling updates: Deploy without downtime
Rollback capability: Revert bad deploys instantly
Health checks: Don't route traffic to sick containers
Resource limits: Prevent one container from starving others
Secrets management: No passwords in plain text
Multi-server orchestration: Run across many machines

Docker Compose gives you none of these in production.

The Orchestration Gap

Docker Compose is great for development:

Single server
Manual starts/stops
Simple networking
Quick iteration

But terrible for production:

No multi-server support
No automatic recovery
No scaling logic
No deployment strategies
No resource management
No production-grade networking

You've hit the orchestration wall.

The Emotional Journey

Stage 1: Denial

"Docker Compose works fine. I'll just run it on a big server."

Stage 2: Anger

"Why is this so hard?! I just want to run containers!"

Stage 3: Bargaining

"Maybe I can script this with bash and cron jobs?"

Stage 4: Depression

"I'm spending 80% of my time managing infrastructure, 20% building features."

Stage 5: Acceptance

"I need an orchestrator. I need Kubernetes."

Why Kubernetes Exists

Kubernetes solves all the problems we just experienced:

Problem	Docker Compose	Kubernetes
Auto-restart	❌ Manual	✅ Automatic
Multi-server	❌ Single server	✅ Cluster of servers
Load balancing	❌ Manual HAProxy	✅ Built-in Service
Scaling	❌ Manual `--scale`	✅ Auto-scaling (HPA)
Rolling updates	❌ Restart (downtime)	✅ Zero-downtime
Rollback	❌ Manual	✅ One command
Health checks	⚠️ Basic	✅ Advanced (liveness, readiness)
Secrets	❌ Plain text	✅ Encrypted
Resource limits	⚠️ Basic	✅ Fine-grained
Service discovery	⚠️ DNS-based	✅ Dynamic

Kubernetes is Docker Compose for production, multiplied by 1000.

But Why Not Just... [Alternative]?

"Why not Docker Swarm?"

Docker Swarm is simpler than Kubernetes, but:

Smaller ecosystem
Fewer features (no HPA, limited RBAC)
Less adoption (most tools target K8s)
Docker Inc. de-prioritized it

Use case: Small teams, simple apps.

"Why not managed services (AWS ECS, Cloud Run)?"

Managed services work great, but:

Vendor lock-in (can't easily move)
Limited customization
Higher costs at scale
Not portable (can't run locally)

Use case: Fully bought into one cloud provider.

"Why not Nomad?"

HashiCorp Nomad is excellent, but:

Smaller community
Fewer integrations
Less tooling
Harder to hire for

Use case: Already using HashiCorp stack (Terraform, Vault, Consul).

"Why Kubernetes?"

Industry standard (most jobs require it)
Huge ecosystem (tools for everything)
Cloud-agnostic (AWS, GCP, Azure, Linode)
Local development (k3s, Minikube, Kind)
Portable (same manifests everywhere)

Kubernetes won the orchestration war.

What You'll Learn in Part 6

In the next article, we'll deploy SSPP to Kubernetes on Linode.

But we won't just throw kubectl commands at you.

We'll explain:

What Pods, Deployments, and Services actually are
Why Kubernetes seems complicated (and how to think about it)
How to run Kubernetes locally (k3s) before going to production
Real deployment strategies (rolling updates, blue/green)
How our SSPP manifests work

No magic. No copy-paste. Just understanding.

The Mindset Shift

Before Kubernetes, you think:

"I have a server. I'll put containers on it."

After Kubernetes, you think:

"I have a cluster. I'll declare what I want running. Kubernetes makes it happen."

It's declarative infrastructure:

# You say what you want
apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 5  # I want 5 API instances

# Kubernetes makes it happen
# - Schedules 5 pods
# - Distributes across servers
# - Monitors them
# - Restarts if they die
# - Scales up/down dynamically

You describe the desired state. Kubernetes maintains it.

Try It Yourself (Before Part 6)

Challenge: Break Docker Compose in creative ways:

Kill containers—see if they restart (they won't)
Overload the API—see if it auto-scales (it won't)
Deploy a new version—see if there's downtime (there will be)
Simulate high CPU—see if K8s would help (it would)

Write down your frustrations. They'll make Part 6 more satisfying.

Discussion

What production incident convinced you that you needed orchestration?

Share your war stories on GitHub Discussions.

Previous: Part 4: Running Multiple Services Locally with Docker Compose

Next: Part 6: Kubernetes from First Principles (No Magic)

About the Author

Documenting real DevOps journey for Proton.ai application. Connect with me:

GitHub: @daviesbrown
LinkedIn: David Nwosu Brown

Part 4: Running Multiple Services Locally with Docker Compose

David Nwosu — Mon, 22 Dec 2025 23:21:01 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 12 minutes

Level: Beginner to Intermediate

The Problem: Container Orchestration by Hand

In Part 3, we containerized our API and Worker services. Success!

But to run the full stack, you need to:

# Start PostgreSQL
docker run -d --name postgres -e POSTGRES_PASSWORD=pass postgres:15-alpine

# Start Redis  
docker run -d --name redis redis:7-alpine

# Start Elasticsearch
docker run -d --name elasticsearch -e "discovery.type=single-node" elasticsearch:8.11.0

# Wait for them to be ready (how long? 🤷)
sleep 30

# Start API (with 10+ environment variables)
docker run -d --name api -p 3000:3000 \
  -e DB_HOST=172.17.0.1 \
  -e DB_PORT=5432 \
  -e REDIS_HOST=172.17.0.1 \
  # ... 8 more -e flags
  sspp-api:latest

# Start Worker (with another 10+ environment variables)
docker run -d --name worker \
  -e DB_HOST=172.17.0.1 \
  # ... more -e flags
  sspp-worker:latest

This is painful:

5 separate docker run commands
Manual networking configuration
Environment variables repeated everywhere
No startup order control
Stopping everything requires 5 separate commands

There must be a better way.

Enter Docker Compose

Docker Compose lets you define your entire application stack in a single YAML file:

services:
  api:
    # ...
  worker:
    # ...
  postgres:
    # ...
  redis:
    # ...
  elasticsearch:
    # ...

Then start everything with:

docker-compose up

One command. Entire stack. Magic.

Installing Docker Compose

Docker Compose should already be installed with Docker Desktop (Mac/Windows).

On Linux:

# Install
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Make executable
sudo chmod +x /usr/local/bin/docker-compose

# Verify
docker-compose --version

Creating docker-compose.yml

In your project root (/opt/sspp):

cd /opt/sspp
nano docker-compose.yml

version: '3.8'

services:
  # PostgreSQL Database
  postgres:
    image: postgres:15-alpine
    container_name: sspp-postgres
    environment:
      POSTGRES_DB: sales_signals
      POSTGRES_USER: sspp_user
      POSTGRES_PASSWORD: sspp_password
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./infrastructure/database/init.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U sspp_user -d sales_signals"]
      interval: 10s
      timeout: 5s
      retries: 5

  # Redis Queue & Cache
  redis:
    image: redis:7-alpine
    container_name: sspp-redis
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5

  # Elasticsearch for Search & Analytics
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
    container_name: sspp-elasticsearch
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - elasticsearch_data:/usr/share/elasticsearch/data
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:9200/_cluster/health || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 5

  # API Service
  api:
    build:
      context: ./services/api
      dockerfile: Dockerfile
    image: davidbrown77/sspp-api:latest
    container_name: sspp-api
    ports:
      - "3000:3000"
    environment:
      NODE_ENV: production
      PORT: 3000
      DB_HOST: postgres
      DB_PORT: 5432
      DB_NAME: sales_signals
      DB_USER: sspp_user
      DB_PASSWORD: sspp_password
      REDIS_HOST: redis
      REDIS_PORT: 6379
      ELASTICSEARCH_URL: http://elasticsearch:9200
      QUEUE_NAME: sales-events
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
      elasticsearch:
        condition: service_healthy
    restart: unless-stopped

  # Worker Service
  worker:
    build:
      context: ./services/worker
      dockerfile: Dockerfile
    image: davidbrown77/sspp-worker:latest
    container_name: sspp-worker
    environment:
      NODE_ENV: production
      DB_HOST: postgres
      DB_PORT: 5432
      DB_NAME: sales_signals
      DB_USER: sspp_user
      DB_PASSWORD: sspp_password
      REDIS_HOST: redis
      REDIS_PORT: 6379
      ELASTICSEARCH_URL: http://elasticsearch:9200
      QUEUE_NAME: sales-events
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
      elasticsearch:
        condition: service_healthy
    restart: unless-stopped

volumes:
  postgres_data:
  redis_data:
  elasticsearch_data:

Let's break down the key concepts:

Key Concepts

1. Services

Each service is a container:

services:
  postgres:
    image: postgres:15-alpine
    # ...

  api:
    build: ./services/api
    # ...

Two ways to define services:

image = Use pre-built image from Docker Hub
build = Build from Dockerfile

2. Networking (Automatic!)

Docker Compose creates a default network where services can reach each other by name:

environment:
  DB_HOST: postgres    # Not 172.17.0.1!
  REDIS_HOST: redis    # Not localhost!

Magic: postgres resolves to the PostgreSQL container's IP automatically.

3. Volumes (Persistent Data)

Without volumes, data disappears when containers stop:

volumes:
  - postgres_data:/var/lib/postgresql/data

This creates a named volume that persists between container restarts.

Named volumes:

volumes:
  postgres_data:
  redis_data:
  elasticsearch_data:

Docker manages these volumes for you.

4. Health Checks

Tell Docker Compose to wait for services to be ready:

healthcheck:
  test: ["CMD-SHELL", "pg_isready -U sspp_user -d sales_signals"]
  interval: 10s
  timeout: 5s
  retries: 5

Why this matters: API won't start until PostgreSQL is accepting connections.

5. Dependency Order

depends_on:
  postgres:
    condition: service_healthy
  redis:
    condition: service_healthy

API starts only after:

PostgreSQL health check passes
Redis health check passes
Elasticsearch health check passes

6. Restart Policies

restart: unless-stopped

Options:

no = Never restart
always = Always restart (even after manual stop)
on-failure = Only on error
unless-stopped = Always, unless manually stopped

7. Environment Variables

Define once, use everywhere:

environment:
  NODE_ENV: production
  DB_HOST: postgres
  DB_PORT: 5432

Or use .env files:

env_file:
  - .env.production

Running Your Stack

Start Everything

docker-compose up

Output:

Creating network "sspp_default" with the default driver
Creating volume "sspp_postgres_data" with default driver
Creating volume "sspp_redis_data" with default driver
Creating volume "sspp_elasticsearch_data" with default driver
Creating sspp-postgres ... done
Creating sspp-redis ... done
Creating sspp-elasticsearch ... done
Waiting for postgres to be healthy...
Waiting for redis to be healthy...
Waiting for elasticsearch to be healthy...
Creating sspp-api ... done
Creating sspp-worker ... done
Attaching to sspp-postgres, sspp-redis, sspp-elasticsearch, sspp-api, sspp-worker

Watch the logs stream in real-time. Press Ctrl+C to stop.

Start in Background (Detached)

docker-compose up -d

Check status:

docker-compose ps

Output:

        Name                      Command               State           Ports
-------------------------------------------------------------------------------------
sspp-api              docker-entrypoint.sh pnpm ...   Up      0.0.0.0:3000->3000/tcp
sspp-elasticsearch    /bin/tini -- /usr/local/b ...   Up      0.0.0.0:9200->9200/tcp
sspp-postgres         docker-entrypoint.sh postgres   Up      0.0.0.0:5432->5432/tcp
sspp-redis            docker-entrypoint.sh redis ...  Up      0.0.0.0:6379->6379/tcp
sspp-worker           docker-entrypoint.sh pnpm ...   Up

All services running! 🎉

Testing the Full Stack

1. Check API Health

curl http://localhost:3000/api/v1/health

Response:

{
  "status": "ok",
  "timestamp": "2025-12-22T14:00:00.000Z"
}

2. Send an Event

curl -X POST http://localhost:3000/api/v1/events \
  -H "Content-Type: application/json" \
  -d '{
    "accountId": "acct_001",
    "userId": "user_001",
    "eventType": "email_sent",
    "timestamp": "2025-12-22T14:00:00Z",
    "metadata": {
      "campaign": "Q4_Outreach",
      "recipientDomain": "acmecorp.com"
    }
  }'

Response:

{
  "status": "accepted",
  "jobId": "1",
  "message": "Event queued for processing"
}

3. Check Worker Processed It

docker-compose logs worker

Output:

sspp-worker | info: Processing job 1 {"accountId":"acct_001",...}
sspp-worker | info: Signal stored in PostgreSQL {"signalId":1}
sspp-worker | info: Signal indexed in Elasticsearch {"signalId":1}
sspp-worker | info: Job 1 completed successfully

4. Verify Data in PostgreSQL

docker-compose exec postgres psql -U sspp_user -d sales_signals -c "SELECT * FROM sales_signals;"

Output:

 id | account_id | user_id  | event_type | signal_type | signal_score | ...
----+------------+----------+------------+-------------+--------------+-----
  1 | acct_001   | user_001 | email_sent | outreach    |         0.10 | ...

Complete end-to-end flow verified! 🎊

Docker Compose Commands

Start/Stop

# Start all services
docker-compose up

# Start in background
docker-compose up -d

# Stop all services
docker-compose down

# Stop and remove volumes (delete data!)
docker-compose down -v

# Restart all services
docker-compose restart

# Restart specific service
docker-compose restart api

Logs

# View all logs
docker-compose logs

# Follow logs (live tail)
docker-compose logs -f

# Specific service
docker-compose logs api

# Last 50 lines
docker-compose logs --tail 50 api

Build

# Build all images
docker-compose build

# Build specific service
docker-compose build api

# Force rebuild (no cache)
docker-compose build --no-cache

# Build and start
docker-compose up --build

Exec (Run Commands in Containers)

# Open shell in API container
docker-compose exec api sh

# Run one-off command
docker-compose exec postgres psql -U sspp_user -d sales_signals

# Run command in new container (not running)
docker-compose run --rm api pnpm test

Scale Services

# Run 3 worker instances
docker-compose up --scale worker=3

# Check it
docker-compose ps

Output:

sspp_worker_1   Up
sspp_worker_2   Up
sspp_worker_3   Up

Parallel event processing!

Advanced: Multiple Compose Files

You can have different configurations for different environments.

Base config (docker-compose.yml):

services:
  api:
    build: ./services/api
    environment:
      NODE_ENV: ${NODE_ENV:-production}

Development overrides (docker-compose.dev.yml):

services:
  api:
    volumes:
      - ./services/api:/app  # Live code reload
    environment:
      NODE_ENV: development
    command: pnpm run start:dev

Run with override:

docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

Or create an alias:

alias dc-dev='docker-compose -f docker-compose.yml -f docker-compose.dev.yml'
dc-dev up

Environment Variables

Method 1: Inline in docker-compose.yml

environment:
  NODE_ENV: production
  PORT: 3000

Method 2: .env File

Create .env:

NODE_ENV=production
PORT=3000
DB_PASSWORD=supersecret

Reference in docker-compose.yml:

environment:
  NODE_ENV: ${NODE_ENV}
  PORT: ${PORT}
  DB_PASSWORD: ${DB_PASSWORD}

Method 3: env_file

services:
  api:
    env_file:
      - .env
      - .env.local

Precedence: Later files override earlier ones.

Real-World Tips

1. Use Named Volumes for Data

volumes:
  postgres_data:    # Named volume (Docker-managed)
  - ./logs:/logs    # Bind mount (host directory)

Named volumes are portable. Bind mounts are machine-specific.

2. Always Set Health Checks

Don't use depends_on without health checks:

# ❌ Bad (starts immediately, might not be ready)
depends_on:
  - postgres

# ✅ Good (waits for health check)
depends_on:
  postgres:
    condition: service_healthy

3. Pin Image Versions

# ❌ Don't do this
image: postgres:latest

# ✅ Do this
image: postgres:15.5-alpine

4. Use Multi-Stage Builds

We already do this in our Dockerfiles (builder + production stages).

5. .dockerignore Everything Unnecessary

cat > .dockerignore <<EOF
node_modules
.git
.env*
*.log
dist
coverage
.vscode
.idea
EOF

What We Solved

✅ One-command startup - docker-compose up

✅ Automatic networking - Services find each other by name

✅ Startup ordering - Health checks ensure readiness

✅ Environment management - Centralized config

✅ Volume persistence - Data survives container restarts

✅ Easy scaling - --scale worker=5

✅ Local-prod parity - Same stack, same behavior

What We Didn't Solve

❌ Production orchestration - Compose is for dev/local, not production at scale

❌ Multi-server deployment - Compose runs on one machine

❌ Auto-scaling - Manual --scale isn't dynamic

❌ Self-healing - If a container dies, it won't restart automatically (in production)

❌ Load balancing - No built-in request distribution

❌ Zero-downtime deploys - Restart = brief downtime

Docker Compose is amazing for local development. But production needs more power.

What's Next?

We've mastered Docker Compose for local development. But how do you run this in production?

In Part 5, we'll expose the limitations of Docker Compose and simulate production failures to understand why we need orchestration.

You'll learn:

What happens when containers die in production
Manual scaling nightmares
Why deployment strategies matter
The emotional journey to accepting Kubernetes

Spoiler: You'll want orchestration. Badly.

Try It Yourself

Challenge: Get the full SSPP stack running with Docker Compose:

Create docker-compose.yml with all 5 services
Start it with docker-compose up -d
Send 10 events via the API
Scale workers to 3 instances
Verify all events processed in PostgreSQL

Bonus: Create a docker-compose.dev.yml with live reload.

Discussion

How do you structure your Docker Compose files? Single file or multi-file?

Join the conversation on GitHub Discussions.

Previous: Part 3: Dependency Hell - Why Docker Exists

Next: Part 5: From One Server to Many - The Need for Orchestration

About the Author

Writing this series to demonstrate production infrastructure thinking for my Proton.ai application.

GitHub: @daviesbrown
LinkedIn: David Nwosu Brown

Part 3: Dependency Hell - Why Docker Exists

David Nwosu — Mon, 22 Dec 2025 23:15:36 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 14 minutes

Level: Beginner to Intermediate

The "Works on My Machine" Problem

It's Monday morning. Your coworker tries to deploy a critical bug fix to production.

They SSH into the server, pull the latest code, and restart the app with PM2.

The app crashes.

Error: Cannot find module 'pg'

"That's weird," they say. "It works on my machine."

They run npm install. Still crashes.

Error: The module '/opt/sspp/node_modules/bcrypt/...' was compiled against a different Node.js version

Now they're rebuilding native modules. Still failing.

After 90 minutes of debugging, they discover:

Production has Node 16.x (they have 18.x)
Production has different OpenSSL version (native module incompatibility)
Production PostgreSQL is 14, code uses 15 features
Someone manually edited files on the server (never committed to git)

The bug fix still isn't deployed. Users are angry.

This is dependency hell—and it kills productivity.

What Are Containers?

Containers solve the "works on my machine" problem by packaging your entire runtime environment:

Your code
All dependencies (node_modules, system libraries)
The exact runtime (specific Node.js version)
System tools (curl, git, whatever you need)

Everything your app needs to run, bundled into a single, portable package called a container image.

Containers vs Virtual Machines

Virtual Machines:

┌─────────────────────────────────┐
│         Application             │
├─────────────────────────────────┤
│       Node.js + Dependencies    │
├─────────────────────────────────┤
│      Guest OS (Ubuntu)          │  ← Full OS copy
├─────────────────────────────────┤
│         Hypervisor              │  ← Virtualization layer
├─────────────────────────────────┤
│      Host OS (Linux)            │
├─────────────────────────────────┤
│         Hardware                │
└─────────────────────────────────┘

Containers:

┌─────────────────────────────────┐
│         Application             │
├─────────────────────────────────┤
│       Node.js + Dependencies    │
├─────────────────────────────────┤
│      Container Runtime (Docker) │  ← Lightweight isolation
├─────────────────────────────────┤
│      Host OS (Linux)            │
├─────────────────────────────────┤
│         Hardware                │
└─────────────────────────────────┘

Key Differences:

Aspect	Virtual Machine	Container
Size	GBs (full OS)	MBs (just your app)
Startup	Minutes	Seconds
Isolation	Strong (separate kernel)	Process-level
Overhead	High (full OS per VM)	Minimal
Portability	Moderate	High

The magic: Containers share the host OS kernel but isolate everything else.

Installing Docker

On your Linode server:

# Update packages
apt update

# Install prerequisites
apt install -y apt-transport-https ca-certificates curl software-properties-common

# Add Docker's official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

# Add Docker repository
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

# Install Docker
apt update
apt install -y docker-ce docker-ce-cli containerd.io

# Start Docker
systemctl start docker
systemctl enable docker

# Verify
docker --version
docker run hello-world

Output:

Hello from Docker!
This message shows that your installation appears to be working correctly.

Building Our First Container: The API Service

Step 1: Create a Dockerfile

A Dockerfile is a recipe for building a container image.

cd /opt/sspp/services/api
nano Dockerfile

# Stage 1: Builder
FROM node:18-alpine AS builder

# Enable pnpm
RUN corepack enable && corepack prepare pnpm@latest --activate

WORKDIR /app

# Copy dependency files
COPY package.json pnpm-lock.yaml* ./

# Install dependencies
RUN pnpm install --frozen-lockfile

# Copy source code
COPY . .

# Build TypeScript to JavaScript
RUN pnpm run build

# Stage 2: Production
FROM node:18-alpine

# Enable pnpm
RUN corepack enable && corepack prepare pnpm@latest --activate

WORKDIR /app

# Copy dependency files
COPY package.json pnpm-lock.yaml* ./

# Install ONLY production dependencies
RUN pnpm install --prod --frozen-lockfile

# Copy built application from builder stage
COPY --from=builder /app/dist ./dist

# Expose port
EXPOSE 3000

# Run the app
CMD ["pnpm", "run", "start:prod"]

Let's break this down:

Multi-Stage Build

We use two stages to keep the final image small:

Builder stage: Has dev dependencies, compiles TypeScript
Production stage: Only runtime dependencies, no build tools

Why? The final image is 50-70% smaller.

Base Image: `node:18-alpine`

node:18 = Node.js version 18
alpine = Minimal Linux distro (~5MB vs ~100MB for Ubuntu-based)

WORKDIR

Sets the working directory inside the container to /app.

COPY

Copies files from your local filesystem into the image.

COPY package.json pnpm-lock.yaml* ./

The * makes pnpm-lock.yaml optional (if it doesn't exist, no error).

RUN

Executes commands during image build:

RUN pnpm install --frozen-lockfile

--frozen-lockfile ensures exact dependency versions (reproducible builds).

EXPOSE

Documents that the container listens on port 3000 (doesn't actually publish it).

CMD

The command to run when the container starts:

CMD ["pnpm", "run", "start:prod"]

Step 2: Build the Image

docker build -t sspp-api:latest .

What happens:

Docker reads the Dockerfile
Pulls the node:18-alpine base image (if not cached)
Runs each instruction (RUN, COPY, etc.)
Creates layers (each instruction = one layer)
Tags the final image as sspp-api:latest

This takes 2-5 minutes the first time. Subsequent builds are faster (cached layers).

Output:

[+] Building 123.4s (17/17) FINISHED
 => [internal] load build definition from Dockerfile
 => [internal] load .dockerignore
 => [builder 1/6] FROM docker.io/library/node:18-alpine
 => [builder 2/6] RUN corepack enable && corepack prepare pnpm@latest --activate
 => [builder 3/6] COPY package.json pnpm-lock.yaml* ./
 => [builder 4/6] RUN pnpm install --frozen-lockfile
 => [builder 5/6] COPY . .
 => [builder 6/6] RUN pnpm run build
 => [stage-1 2/5] RUN corepack enable && corepack prepare pnpm@latest --activate
 => [stage-1 3/5] COPY package.json pnpm-lock.yaml* ./
 => [stage-1 4/5] RUN pnpm install --prod --frozen-lockfile
 => [stage-1 5/5] COPY --from=builder /app/dist ./dist
 => exporting to image
 => => naming to docker.io/library/sspp-api:latest

Verify the image:

docker images

REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
sspp-api     latest    a1b2c3d4e5f6   30 seconds ago   185MB

Step 3: Run the Container

docker run -d \
  --name sspp-api \
  -p 3000:3000 \
  -e NODE_ENV=production \
  -e DB_HOST=172.17.0.1 \
  -e DB_PORT=5432 \
  -e DB_NAME=sales_signals \
  -e DB_USER=sspp_user \
  -e DB_PASSWORD=sspp_password \
  -e REDIS_HOST=172.17.0.1 \
  -e REDIS_PORT=6379 \
  -e ELASTICSEARCH_URL=http://172.17.0.1:9200 \
  sspp-api:latest

Flags explained:

-d = Detached (run in background)
--name sspp-api = Container name (for easy reference)
-p 3000:3000 = Port mapping (host:container)
-e KEY=value = Environment variables
sspp-api:latest = Image to run

What's 172.17.0.1? That's the Docker bridge network gateway—how containers reach the host machine's services (PostgreSQL, Redis).

Check if it's running:

docker ps

CONTAINER ID   IMAGE              COMMAND                  CREATED          STATUS          PORTS                    NAMES
f8a9b1c2d3e4   sspp-api:latest    "docker-entrypoint..."   10 seconds ago   Up 9 seconds    0.0.0.0:3000->3000/tcp   sspp-api

Test it:

curl http://localhost:3000/api/v1/health

Output:

{
  "status": "ok",
  "timestamp": "2025-12-22T12:00:00.000Z"
}

🎉 Your containerized API is running!

Step 4: View Logs

docker logs sspp-api

# Live tail
docker logs -f sspp-api

# Last 50 lines
docker logs --tail 50 sspp-api

Building the Worker Container

Same process for the worker service:

cd /opt/sspp/services/worker
nano Dockerfile

FROM node:18-alpine AS builder

RUN corepack enable && corepack prepare pnpm@latest --activate

WORKDIR /app

COPY package.json pnpm-lock.yaml* ./
RUN pnpm install --frozen-lockfile

COPY . .
RUN pnpm run build

FROM node:18-alpine

RUN corepack enable && corepack prepare pnpm@latest --activate

WORKDIR /app

COPY package.json pnpm-lock.yaml* ./
RUN pnpm install --prod --frozen-lockfile

COPY --from=builder /app/dist ./dist

CMD ["pnpm", "start"]

Build and run:

docker build -t sspp-worker:latest .

docker run -d \
  --name sspp-worker \
  -e NODE_ENV=production \
  -e DB_HOST=172.17.0.1 \
  -e DB_PORT=5432 \
  -e DB_NAME=sales_signals \
  -e DB_USER=sspp_user \
  -e DB_PASSWORD=sspp_password \
  -e REDIS_HOST=172.17.0.1 \
  -e REDIS_PORT=6379 \
  -e ELASTICSEARCH_URL=http://172.17.0.1:9200 \
  -e QUEUE_NAME=sales-events \
  sspp-worker:latest

Check status:

docker ps

CONTAINER ID   IMAGE                  COMMAND       CREATED          STATUS          PORTS                    NAMES
f8a9b1c2d3e4   sspp-api:latest        "..."         5 minutes ago    Up 5 minutes    0.0.0.0:3000->3000/tcp   sspp-api
a1b2c3d4e5f6   sspp-worker:latest     "..."         10 seconds ago   Up 9 seconds                             sspp-worker

What We Just Accomplished

1. Reproducible Builds

Anyone can build the exact same image:

git clone https://github.com/daviesbrown/sspp
cd sspp/services/api
docker build -t sspp-api:latest .

Same code + same Dockerfile = same image. Always.

2. Isolated Dependencies

Each container has its own:

Node.js version
npm/pnpm version
System libraries
Environment variables

No more version conflicts.

3. Portable

Build on your Mac, run on Linux. Build on dev, run on prod. It's the same image.

4. Lightweight

docker images

REPOSITORY      TAG       SIZE
sspp-api        latest    185MB
sspp-worker     latest    178MB

Compare to a full Ubuntu VM: 2-5GB.

Docker Layer Caching

Docker is smart about rebuilding. Each instruction creates a layer:

FROM node:18-alpine              # Layer 1 (cached if unchanged)
COPY package.json ./             # Layer 2 (cached if files unchanged)
RUN pnpm install                 # Layer 3 (cached if layer 2 unchanged)
COPY . .                         # Layer 4 (cached if files unchanged)
RUN pnpm build                   # Layer 5 (cached if layer 4 unchanged)

Order matters! Put frequently-changing files (source code) after rarely-changing files (dependencies).

Good:

COPY package.json ./     # Changes rarely
RUN pnpm install         # Cached most of the time
COPY . .                 # Changes often

Bad:

COPY . .                 # Changes often
RUN pnpm install         # Runs every time (slow!)

Common Docker Commands

Images

# List images
docker images

# Remove image
docker rmi sspp-api:latest

# Remove unused images
docker image prune

# Remove ALL images
docker rmi $(docker images -q)

Containers

# List running containers
docker ps

# List all containers (including stopped)
docker ps -a

# Stop container
docker stop sspp-api

# Start stopped container
docker start sspp-api

# Restart container
docker restart sspp-api

# Remove container
docker rm sspp-api

# Force remove (even if running)
docker rm -f sspp-api

# Remove all stopped containers
docker container prune

Logs & Debugging

# View logs
docker logs sspp-api

# Execute command in running container
docker exec -it sspp-api sh

# Inspect container details
docker inspect sspp-api

# View resource usage
docker stats sspp-api

What We Solved

✅ "Works on my machine" - Same environment everywhere

✅ Dependency conflicts - Each container is isolated

✅ Version management - Exact Node.js, system libs

✅ Reproducible builds - Same Dockerfile = same image

✅ Portability - Run anywhere Docker runs

✅ Lightweight - Much smaller than VMs

What We Didn't Solve

❌ Multi-container coordination - Manual networking, port management

❌ Service discovery - How does API find Redis? Hard-coded IPs

❌ Volume management - What about database data persistence?

❌ Environment variables - Still passing 10+ -e flags per container

❌ Startup order - What if PostgreSQL isn't ready yet?

❌ Scaling - Running multiple workers is manual

We're running containers, but managing them is still tedious.

Real-World Docker Tips

1. Use `.dockerignore`

Prevent copying unnecessary files:

cat > .dockerignore <<EOF
node_modules
.git
.env
*.log
dist
coverage
EOF

2. Don't Run as Root

Security best practice:

# Create non-root user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup

# Switch to that user
USER appuser

3. Use Specific Tags

# ❌ Don't use 'latest'
FROM node:latest

# ✅ Use specific version
FROM node:18.19.0-alpine3.19

4. Health Checks

Tell Docker how to check if your app is healthy:

HEALTHCHECK --interval=30s --timeout=3s --start-period=40s \
  CMD node -e "require('http').get('http://localhost:3000/health', (r) => process.exit(r.statusCode === 200 ? 0 : 1))"

5. Multi-Stage Builds Always

Keep final images small by separating build and runtime stages.

What's Next?

We've containerized our services! But running them individually with docker run doesn't scale.

In Part 4, we'll use Docker Compose to:

Manage multiple containers together
Define networking automatically
Set environment variables in one place
Control startup order
Run the entire stack with one command

Spoiler: docker-compose up and your entire system (API, Worker, PostgreSQL, Redis, Elasticsearch) starts in perfect harmony.

Try It Yourself

Challenge: Containerize both API and Worker services, then:

Build images for both
Run them with proper environment variables
Send an event to the API
Watch the Worker process it (check logs)
Verify data in PostgreSQL

Bonus: Modify the Dockerfile to add a health check endpoint.

Discussion

What's your Docker horror story? Or success story?

Share on GitHub Discussions.

Previous: Part 2: Process Managers - Keeping Your App Alive with PM2

Next: Part 4: Running Multiple Services Locally with Docker Compose

About the Author

Documenting real DevOps infrastructure for my Proton.ai application. Hiring? Let's connect.

GitHub: @daviesbrown
LinkedIn: David Nwosu

Part 2: Process Managers - Keeping Your App Alive with PM2

David Nwosu — Mon, 22 Dec 2025 23:11:19 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 10 minutes

Level: Beginner-friendly

Quick Recap

In Part 1, we deployed our Sales Signal Processing Platform to a Linode server the manual way. It worked... until:

We closed our SSH session (app died)
The app crashed (stayed dead)
The server rebooted (app didn't restart)

Today's mission: Keep the app alive without babysitting it.

The Problem: Processes Are Fragile

Let's simulate what happens in production.

SSH into your server and start the API:

cd /opt/sspp/services/api
npm start &

Now kill it on purpose:

# Find the process ID
ps aux | grep node

# Kill it
kill -9 <pid>

Test the API:

curl http://localhost:3000/api/v1/health

Dead. And it's not coming back.

In production, processes die for many reasons:

Unhandled exceptions
Memory leaks (OOM killer strikes)
Dependency failures (database connection lost)
Random cosmic rays (yes, really)

You need something that automatically restarts your app.

Enter PM2: Process Manager 2

PM2 is a production-grade process manager for Node.js applications. Think of it as a babysitter that:

Keeps your app running - Restarts on crash
Survives reboots - Starts on system boot
Manages logs - Aggregates stdout/stderr
Monitors resources - CPU, memory usage
Zero-downtime reloads - Update without dropping connections

Why PM2? It's battle-tested, actively maintained, and used by thousands of companies in production.

Installation

SSH into your server:

# Install PM2 globally
npm install -g pm2

# Verify
pm2 --version

Simple. Now let's use it.

Running Your App with PM2

Basic Usage

Instead of npm start, use PM2:

cd /opt/sspp/services/api

# Start the app
pm2 start npm --name "sspp-api" -- start

# Check status
pm2 status

Output:

┌─────┬──────────────┬─────────┬──────┬───────┬────────┬─────────┬────────┬──────┬───────────┬──────────┐
│ id  │ name         │ mode    │ ↺    │ status│ cpu    │ memory  │
├─────┼──────────────┼─────────┼──────┼───────┼────────┼─────────┤
│ 0   │ sspp-api     │ fork    │ 0    │ online│ 0%     │ 45.2mb  │
└─────┴──────────────┴─────────┴──────┴───────┴────────┴─────────┘

Your app is now:

Named (no more anonymous PIDs)
Monitored (PM2 watches it)
Managed (can be controlled by name)

Better: Use an Ecosystem File

Create a PM2 configuration file:

cd /opt/sspp
cat > ecosystem.config.js <<'EOF'
module.exports = {
  apps: [
    {
      name: 'sspp-api',
      cwd: '/opt/sspp/services/api',
      script: 'npm',
      args: 'start',
      instances: 1,
      autorestart: true,
      watch: false,
      max_memory_restart: '500M',
      env: {
        NODE_ENV: 'production',
        PORT: 3000,
        DB_HOST: 'localhost',
        DB_PORT: 5432,
        DB_NAME: 'sales_signals',
        DB_USER: 'sspp_user',
        DB_PASSWORD: 'sspp_password',
        REDIS_HOST: 'localhost',
        REDIS_PORT: 6379,
        ELASTICSEARCH_URL: 'http://localhost:9200',
      },
      error_file: '/var/log/sspp/api-error.log',
      out_file: '/var/log/sspp/api-out.log',
      time: true,
    },
    {
      name: 'sspp-worker',
      cwd: '/opt/sspp/services/worker',
      script: 'npm',
      args: 'start',
      instances: 2,
      autorestart: true,
      watch: false,
      max_memory_restart: '500M',
      env: {
        NODE_ENV: 'production',
        DB_HOST: 'localhost',
        DB_PORT: 5432,
        DB_NAME: 'sales_signals',
        DB_USER: 'sspp_user',
        DB_PASSWORD: 'sspp_password',
        REDIS_HOST: 'localhost',
        REDIS_PORT: 6379,
        ELASTICSEARCH_URL: 'http://localhost:9200',
        QUEUE_NAME: 'sales-events',
      },
      error_file: '/var/log/sspp/worker-error.log',
      out_file: '/var/log/sspp/worker-out.log',
      time: true,
    },
  ],
};
EOF

What this does:

Defines both services (API + Worker) in one place
Sets environment variables (no more .env files to manage)
Configures resources (max memory before restart)
Organizes logs (separate files for each service)
Runs multiple workers (2 worker instances for parallel processing)

Create log directory:

mkdir -p /var/log/sspp

Start everything:

pm2 start ecosystem.config.js

# Check status
pm2 status

Output:

┌─────┬──────────────┬─────────┬──────┬───────┬────────┬─────────┐
│ id  │ name         │ mode    │ ↺    │ status│ cpu    │ memory  │
├─────┼──────────────┼─────────┼──────┼───────┼────────┼─────────┤
│ 0   │ sspp-api     │ fork    │ 0    │ online│ 1.2%   │ 48.3mb  │
│ 1   │ sspp-worker  │ fork    │ 0    │ online│ 0.8%   │ 42.1mb  │
│ 2   │ sspp-worker  │ fork    │ 0    │ online│ 0.7%   │ 41.8mb  │
└─────┴──────────────┴─────────┴──────┴───────┴────────┴─────────┘

Now you have:

1 API instance
2 Worker instances (for parallel event processing)
All managed by PM2

Testing Auto-Restart

Let's intentionally crash the API:

# Find the process ID
pm2 list

# Kill the API process
pm2 delete sspp-api
pm2 start ecosystem.config.js --only sspp-api

# Now kill it brutally
kill -9 $(pgrep -f "sspp-api")

Wait 1 second, then check:

pm2 status

The ↺ (restart count) increases! PM2 automatically restarted it.

Test the API:

curl http://localhost:3000/api/v1/health

Still alive. 🎉

Startup Script: Survive Reboots

The app survives crashes now. But what about server reboots?

# Generate startup script
pm2 startup systemd

# Follow the command it prints (looks like):
# sudo env PATH=$PATH:/usr/bin pm2 startup systemd -u root --hp /root

Run that sudo command it generates.

Save the current PM2 process list:

pm2 save

This creates /root/.pm2/dump.pm2 with your process configuration.

Test it:

# Reboot the server
sudo reboot

Wait 30 seconds, SSH back in:

pm2 status

Your apps are running! Without you doing anything.

Managing Your Apps

View Logs

# All logs (combined)
pm2 logs

# Specific app
pm2 logs sspp-api

# Last 100 lines
pm2 logs sspp-api --lines 100

# Live tail
pm2 logs sspp-worker --lines 0

Monitor Resources

pm2 monit

This opens an interactive dashboard showing:

CPU usage
Memory usage
Logs (live stream)

Press Ctrl+C to exit.

Restart/Reload

# Restart (kills and starts)
pm2 restart sspp-api

# Reload (zero-downtime, only works for cluster mode)
pm2 reload sspp-api

# Restart all
pm2 restart all

Stop/Delete

# Stop (keeps in PM2 list)
pm2 stop sspp-api

# Delete (removes from PM2 list)
pm2 delete sspp-api

# Stop all
pm2 stop all

# Delete all
pm2 delete all

Cluster Mode (Bonus: Load Balancing)

PM2 can run multiple instances of your app and load-balance between them:

// In ecosystem.config.js
{
  name: 'sspp-api',
  script: './dist/main.js', // Direct script, not npm
  instances: 4, // Or 'max' for CPU count
  exec_mode: 'cluster', // Enable cluster mode
  // ... rest of config
}

Restart PM2:

pm2 delete all
pm2 start ecosystem.config.js

Now you have 4 API instances behind PM2's built-in load balancer.

Why this matters:

Utilizes all CPU cores
Automatic load distribution
Zero-downtime reloads (one instance at a time)

What We Solved

With PM2, we fixed:

✅ Automatic restart on crash - App crashes are now recoverable

✅ Startup on boot - Server reboots don't kill your service

✅ Log management - Centralized, timestamped logs

✅ Resource monitoring - Know when memory leaks happen

✅ Process naming - No more searching for PIDs

✅ Multi-instance management - Run workers in parallel

What We Didn't Solve

PM2 is great, but it doesn't solve:

❌ "Works on my machine" - Still manual dependency installation

❌ Environment consistency - Different Node versions, OS differences

❌ Multi-server scaling - PM2 is single-server only

❌ Deployment strategy - Still manual git pull, restart

❌ Rollback capability - No version management

❌ Network complexity - How do API and Worker discover services?

❌ Resource isolation - Apps can steal CPU/memory from each other

PM2 is a massive improvement over raw processes. But we're still managing dependencies manually, and we can't easily scale to multiple servers.

Real-World PM2 Tips

1. Always Use Ecosystem Files

Don't run pm2 start with inline arguments. Use ecosystem.config.js:

# ❌ Don't do this
pm2 start npm --name api -- start

# ✅ Do this
pm2 start ecosystem.config.js

2. Set Memory Limits

Prevent runaway processes:

{
  max_memory_restart: '500M', // Restart if memory exceeds 500MB
}

3. Use Absolute Paths

Relative paths break when PM2 restarts:

{
  cwd: '/opt/sspp/services/api', // Absolute path
  script: 'npm', // Not '../../../node_modules/...'
}

4. Separate Logs

Don't dump everything to one file:

{
  error_file: '/var/log/sspp/api-error.log',
  out_file: '/var/log/sspp/api-out.log',
}

5. Use Log Rotation

Logs grow forever. Set up rotation:

pm2 install pm2-logrotate
pm2 set pm2-logrotate:max_size 10M
pm2 set pm2-logrotate:retain 7

Production Checklist

Before going live with PM2:

[ ] Ecosystem file configured
[ ] Startup script installed (pm2 startup)
[ ] Process list saved (pm2 save)
[ ] Memory limits set
[ ] Log rotation enabled
[ ] Monitoring alerts configured (e.g., PM2 Plus)

What's Next?

PM2 solves the "keep it running" problem beautifully. But we're still stuck with:

Manual dependency management (Node, PostgreSQL, Redis, Elasticsearch)
"Works on my machine" syndrome (different environments)
Single-server limitations (can't easily scale horizontally)

In Part 3, we'll tackle these by introducing Docker—containers that package your entire application environment.

What PM2 Does NOT Fix

PM2 solves process management, but let's be honest about what's still broken:

✅ What PM2 fixes:

Auto-restarts on crash
Survives SSH disconnects
Starts on server boot
Basic logging and monitoring

❌ What PM2 does NOT fix:

Environment consistency - Still manually installing Node, PostgreSQL, Redis
Infrastructure drift - Every server is a unique snowflake
Scaling - Can't easily add more servers
Dependency conflicts - Node v16 on this server, v18 on that one
Reproducibility - "Works on my machine" still exists (just less obviously)
Onboarding - New devs still need 2+ hours of setup
Rollbacks - No easy way to undo deployments

The hidden danger:

PM2 makes things feel professional, which can hide deeper problems.

Try It Yourself

Experience what breaks next:

Set up PM2 for both API and Worker services
Enable startup script: pm2 startup && pm2 save
Now try to set up a second server identically
Notice how many steps you have to remember
Notice how easy it is to have version mismatches

This pain is important. It's why Docker exists.

Next: Making Environments Consistent

In Part 3, we'll solve the "works on my machine" problem:

"How do I package my app so it runs the same everywhere?"

We'll use Docker to:

Freeze dependencies in time
Eliminate environment drift
Make onboarding instant
Enable reliable rollbacks

But spoiler: Docker solves packaging, not operations. We'll discover what breaks when you have multiple containers.

Previous: Part 1: The Default Way - Putting an App on a Server

Next: Part 3: Docker - Freezing the Application in Time

About the Author

Building this series to demonstrate real DevOps thinking for my Proton.ai application. If you're hiring for platform engineering roles, let's connect.

GitHub: @daviesbrown
LinkedIn: David Nwosu

Part 1: The Default Way - Putting an App on a Server (And Why It Breaks)

David Nwosu — Mon, 22 Dec 2025 23:09:33 +0000

Series: From "Just Put It on a Server" to Production DevOps

Reading time: 12 minutes

Level: Beginner-friendly

Introduction

Every developer has a moment where they think: "I've built this amazing app locally. Now I just need to put it on a server somewhere."

Sounds simple, right?

You rent a server, upload your code, run npm start, and boom—you're live.

This article is about what happens next.

We're going to deploy a real application—the Sales Signal Processing Platform (SSPP)—the old-school way. No Docker. No Kubernetes. No CI/CD magic. Just you, a Linux server, and SSH.

By the end, you'll understand why modern DevOps tools exist—not because someone said they're "best practices," but because you'll feel the pain they solve.

What We're Building

The Sales Signal Processing Platform is an event-driven system that:

Accepts events via REST API (email sent, page viewed, meeting booked)
Queues them in Redis for async processing
Calculates signal scores (how "hot" a sales lead is)
Stores data in PostgreSQL
Indexes in Elasticsearch for search/analytics

Why this project?

It's not a toy. It's a real-world SaaS pattern used by companies like Segment, Mixpanel, and sales intelligence platforms.

Architecture (Simple Version):

┌─────────────┐      ┌─────────────┐      ┌─────────────┐
│   Client    │─────▶│  API Service │─────▶│  Redis Queue│
│ (HTTP POST) │      │  (NestJS)    │      │             │
└─────────────┘      └─────────────┘      └──────┬──────┘
                                                   │
                                                   ▼
                                           ┌─────────────┐
                                           │   Worker    │
                                           │  (Node.js)  │
                                           └──────┬──────┘
                                                  │
                              ┌──────────────────┼───────────────┐
                              ▼                  ▼               ▼
                        ┌──────────┐      ┌──────────┐   ┌─────────────┐
                        │PostgreSQL│      │  Redis   │   │Elasticsearch│
                        └──────────┘      └──────────┘   └─────────────┘

Tech Stack:

API: NestJS (TypeScript)
Worker: Node.js (TypeScript)
Databases: PostgreSQL 15, Redis 7, Elasticsearch 8

Step 1: Rent a Server (The Easy Part)

We're using Linode (now Akamai Cloud) because:

Simple pricing ($5-$10/month gets you started)
No vendor lock-in (real VMs, not proprietary services)
Shows you understand infrastructure fundamentals

Create a Linode Instance:

Sign up at linode.com
Create a new Linode:
- Distribution: Ubuntu 22.04 LTS
- Plan: Shared CPU - Nanode 1GB ($5/mo)
- Region: Choose closest to you
- Label: sspp-server-01
Set a root password (save it somewhere secure)
Boot it up

What you just got:

A real computer running Linux somewhere in a datacenter
A public IP address (e.g., 45.79.123.45)
SSH access via port 22

Step 2: SSH Into Your Server

SSH (Secure Shell) is how you control remote servers.

ssh root@45.79.123.45

First time? You'll see:

The authenticity of host '45.79.123.45' can't be established.
ED25519 key fingerprint is SHA256:...
Are you sure you want to continue connecting (yes/no)?

Type yes. This saves the server's "fingerprint" so you know it's the same server next time.

You're now controlling a computer 500+ miles away.

Type pwd (print working directory):

/root

You're in the home directory of the root user—the superuser who can do anything.

Step 3: Install Dependencies (The Manual Way)

Our app needs:

Node.js (runtime for our JavaScript/TypeScript)
PostgreSQL (database)
Redis (queue and cache)
Elasticsearch (search/analytics)

Install Node.js

# Update package list
apt update

# Install Node.js 18.x
curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
apt install -y nodejs

# Verify
node --version  # Should show v18.x.x
npm --version   # Should show v9.x.x

Install PostgreSQL

# Install PostgreSQL
apt install -y postgresql postgresql-contrib

# Start the service
systemctl start postgresql
systemctl enable postgresql  # Auto-start on boot

# Create database and user
sudo -u postgres psql <<EOF
CREATE DATABASE sales_signals;
CREATE USER sspp_user WITH PASSWORD 'sspp_password';
GRANT ALL PRIVILEGES ON DATABASE sales_signals TO sspp_user;
\q
EOF

Install Redis

# Install Redis
apt install -y redis-server

# Start it
systemctl start redis-server
systemctl enable redis-server

# Test
redis-cli ping  # Should return "PONG"

Install Elasticsearch

# Install dependencies
apt install -y apt-transport-https

# Add Elasticsearch GPG key
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

# Add repository
echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" > /etc/apt/sources.list.d/elastic-8.x.list

# Install
apt update
apt install -y elasticsearch

# Start it
systemctl start elasticsearch
systemctl enable elasticsearch

# Test (wait 30 seconds for startup)
curl localhost:9200

How long did this take? Probably 15-20 minutes.

What happens if you need to do this again? You repeat every command. Manually.

Step 4: Deploy Your Application

Now let's get our code onto the server.

Clone the Repository

cd /opt
git clone https://github.com/daviesbrown/sspp.git
cd sspp

Install API Dependencies

cd /opt/sspp/services/api
npm install

This takes 2-5 minutes. Watch the terminal spam scroll by.

Build the API

npm run build

TypeScript compiles to JavaScript in the dist/ folder.

Create Environment Variables

cat > .env <<EOF
NODE_ENV=production
PORT=3000
DB_HOST=localhost
DB_PORT=5432
DB_NAME=sales_signals
DB_USER=sspp_user
DB_PASSWORD=sspp_password
REDIS_HOST=localhost
REDIS_PORT=6379
ELASTICSEARCH_URL=http://localhost:9200
EOF

Run the API

npm start

Output:

[Nest] 12345  - 12/22/2025, 10:30:15 AM     LOG [NestFactory] Starting Nest application...
[Nest] 12345  - 12/22/2025, 10:30:15 AM     LOG [InstanceLoader] AppModule dependencies initialized
[Nest] 12345  - 12/22/2025, 10:30:16 AM     LOG [RoutesResolver] EventsController {/api/v1/events}
[Nest] 12345  - 12/22/2025, 10:30:16 AM     LOG [NestApplication] Nest application successfully started
[Nest] 12345  - 12/22/2025, 10:30:16 AM     LOG API listening on port 3000

🎉 Your app is running!

Step 5: Test It

Open another terminal on your local machine (not the server):

# Test the API
curl http://45.79.123.45:3000/api/v1/health

# Expected response:
{
  "status": "ok",
  "timestamp": "2025-12-22T10:31:00.000Z"
}

Send an event:

curl -X POST http://45.79.123.45:3000/api/v1/events \
  -H "Content-Type: application/json" \
  -d '{
    "accountId": "acct_001",
    "userId": "user_001",
    "eventType": "email_sent",
    "timestamp": "2025-12-22T10:00:00Z",
    "metadata": {
      "campaign": "Q4_Outreach"
    }
  }'

# Response:
{
  "status": "accepted",
  "jobId": "1",
  "message": "Event queued for processing"
}

IT WORKS! 🎊

You're a full-stack DevOps engineer now, right?

Step 6: Reality Hits

Go back to your SSH session and press Ctrl+C to stop the app.

Now try accessing the API again:

curl http://45.79.123.45:3000/api/v1/health

Output:

curl: (7) Failed to connect to 45.79.123.45 port 3000: Connection refused

Your app is dead.

Why?

When you run npm start in your SSH session:

It starts a foreground process
That process is attached to your SSH session
When you stop it (Ctrl+C) or disconnect, the process dies

Let's try running it in the background:

npm start &

The & runs it in the background. You get your terminal back.

Now close your SSH session (disconnect).

Wait 10 seconds, then test again:

curl http://45.79.123.45:3000/api/v1/health

Still dead.

Why?

When you disconnect from SSH, the shell sends a SIGHUP (hangup signal) to all child processes, which kills them.

Step 7: The "Just Keep It Running" Attempts

Attempt 1: Use `nohup`

nohup npm start &

nohup (no hangup) ignores the SIGHUP signal.

This works! But:

No automatic restart if it crashes
No logs management (they dump to nohup.out)
No process monitoring

Attempt 2: Use `screen` or `tmux`

These create persistent terminal sessions:

screen -S api
npm start
# Press Ctrl+A, then D to detach

Better! But:

Still manual
No restart on crash
No resource limits
You have to remember screen commands

Step 8: The Real Problems

Let's say you got it "working" with nohup or screen. Here are the problems you'll hit in production:

Problem 1: No Automatic Restart

Your app crashes (memory leak, unhandled exception). It's just... down. Until you SSH in and restart it manually.

Problem 2: No Startup on Reboot

Server reboots (Linode maintenance, kernel update). Your app doesn't start automatically.

Problem 3: "Works on My Machine"

Your coworker tries to deploy. They have Node 16 installed, you have Node 18. Different npm versions. Different PostgreSQL versions. It breaks.

Problem 4: Manual Scaling

You get featured on Product Hunt. Traffic spikes 50x. You need to:

Rent 5 more servers
SSH into each one
Repeat all the installation steps
Set up a load balancer manually
Pray nothing goes wrong

Time to scale: 2-4 hours (if you're fast)

Problem 5: Zero Deployment Strategy

You need to deploy a bug fix. To avoid downtime:

Deploy to a new server
Switch DNS/load balancer
Drain old connections
Hope nothing broke

Or you just restart the app and accept 5-30 seconds of downtime.

Problem 6: No Rollback

The new version has a critical bug. To rollback:

SSH in
git checkout old commit
npm install (hope dependencies didn't change)
Restart

Time to rollback: 5-10 minutes (plus panic time)

Problem 7: Configuration Management

You have 3 environments: dev, staging, production.

Each needs different:

Database credentials
API keys
Feature flags
Resource limits

Your solution: .env files, config.dev.json, config.prod.json, remembering which is which.

Problem 8: Security

Your server is wide open:

PostgreSQL accepting connections from anywhere
No firewall rules
Root SSH access enabled
Passwords in plaintext .env files
No SSL certificates

What We Learned

We successfully deployed an app to a server the manual way. And we discovered:

Process management is hard - Apps need to survive crashes and reboots
Environment consistency is critical - "Works on my machine" kills teams
Scaling is manual - Copying setup to multiple servers doesn't scale
Deployments are risky - No safety net, no rollback strategy
Security is an afterthought - When you're fighting fires, security waits

This is why DevOps exists.

Every tool you've heard of—Docker, Kubernetes, CI/CD, Infrastructure as Code—solves one of these problems.

But you can't appreciate the solution until you've felt the pain.

What's Next?

In Part 2, we'll solve the first problem: keeping your app alive.

We'll use PM2 (Process Manager 2) to:

Automatically restart crashed processes
Start apps on server reboot
Manage logs properly
Monitor resource usage

This will make our deployment slightly better. But it won't solve the deeper problems.

Spoiler: We'll eventually replace all of this with Docker and Kubernetes. But first, we need to understand what life was like before containers.

Try It Yourself

Don't just read—do it:

Spin up a Linode (or DigitalOcean Droplet, AWS EC2, etc.)
Follow this guide exactly
Get the app running
Try to kill it in creative ways:
- kill -9 <pid>
- Disconnect SSH
- Reboot the server
Notice how often it stays dead

Bonus challenge: Try to deploy the Worker service too (hint: it's even harder because it's a background service with no HTTP endpoint).

Discussion

What have your "just put it on a server" horror stories been?

Drop a comment or open an issue on GitHub.

Next: Part 2: Process Managers - Keeping Your App Alive with PM2

About the Author

I'm building this series to show real-world DevOps thinking for my Proton.ai application. If you're hiring for DevOps/Platform roles and want someone who understands infrastructure (not just follows tutorials), let's talk.

GitHub: @daviesbrown
LinkedIn: David Nwosu