Abdallateef Shohdy

Posted on Oct 12, 2025

🚀 Deploy Node.js Apps on AWS EC2 - Complete Guide

#aws #tutorial #node #devops

📑 Table of Contents

Architecture Overview
Request Flow Explained
systemd vs PM2
Step by Step Deployment Guide
Process Management Deep Dive
Monitoring and Logs
Troubleshooting and Best Practices
Summary

Architecture Overview

Complete Server Architecture

┌─────────────────────────────────────────────────────────┐
│                    Internet (External World)            │
│                   https://yourdomain.com                │
└────────────────────────┬────────────────────────────────┘
                         │
                         │ DNS Resolution
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│              AWS EC2 Instance (Ubuntu)                  │
│              Public IP: xx.xx.xx.xx                     │
│                                                         │
│  ┌────────────────────────────────────────────────┐     │
│  │         Caddy (Reverse Proxy)                  │     │
│  │         Port 80 (HTTP) → 443 (HTTPS)           │     │
│  │         ✅ Auto SSL Certificate                │     │
│  └──────────────┬─────────────────┬───────────────┘     │
│                 │                 │                     │
│                 │                 │                     │
│     ┌───────────▼──────────┐  ┌──▼──────────────-┐      │
│     │   Frontend (Next.js) │  │  Backend (Node)  │       │
│     │   Port: 3000         │  │  Port: 5000      │      │
│     │   Managed by:        │  │  Managed by:     │      │
│     │   • systemd OR       │  │  • systemd OR    │      │
│     │   • PM2              │  │  • PM2           │      │
│     └──────────────────────┘  └──────────────────┘      │
│                                                         │
│  ┌────────────────────────────────────────────────┐     │
│  │         Operating System Layer                 │     │
│  │         • systemd (init system)                │     │
│  │         • Process monitoring                   │     │
│  │         • Auto-restart on crash                │     │
│  └────────────────────────────────────────────────┘     │
└─────────────────────────────────────────────────────────┘

Key Components Explained

Component	Purpose	Port	Technology
Caddy	Reverse Proxy & SSL	80, 443	Go-based web server
Frontend	User Interface	3000	Next.js / React
Backend	API / Business Logic	5000	Node.js / Express
systemd/PM2	Process Manager	N/A	Linux service / Node tool

Request Flow Explained

How a Request Travels Through Your Server

Step 1: User Request
┌─────────────────┐
│   User Browser  │  User types: https://yourdomain.com
│   (Chrome/etc)  │
└────────┬────────┘
         │
         │ ① DNS Lookup (domain → IP address)
         │
         ▼
┌─────────────────┐
│   DNS Server    │  Returns: 54.123.45.67 (EC2 Public IP)
└────────┬────────┘
         │
         │ ② HTTPS Request to IP:443
         │
         ▼
┌─────────────────────────────────────────┐
│         AWS EC2 Instance                │
│                                         │
│  Step 3: Caddy Receives Request         │
│  ┌───────────────────────────────┐     │
│  │  Caddy (Port 443)             │     │
│  │  • Terminates SSL             │     │
│  │  • Checks Caddyfile rules     │     │
│  │  • Decides where to route     │     │
│  └──────────┬────────────────────┘     │
│             │                           │
│   ┌─────────┴──────────┐               │
│   │                    │               │
│   │ If route = /       │ If route = /api│
│   │                    │               │
│   ▼                    ▼               │
│  ┌─────────┐      ┌─────────┐         │
│  │Frontend │      │Backend  │         │
│  │localhost│      │localhost│         │
│  │:3000    │      │:5000    │         │
│  │         │      │         │         │
│  │Next.js  │      │Node.js  │         │
│  │App      │      │Express  │         │
│  └────┬────┘      └────┬────┘         │
│       │                │               │
│       │ ④ Process      │ ④ Process     │
│       │   Request      │   Request     │
│       │                │               │
│       ▼                ▼               │
│  ┌─────────┐      ┌─────────┐         │
│  │HTML/CSS/│      │JSON     │         │
│  │JS       │      │Response │         │
│  └────┬────┘      └────┬────┘         │
│       │                │               │
│       └────────┬───────┘               │
│                │                       │
│  ⑤ Response    │                       │
│  back through  │                       │
│  Caddy         │                       │
│  ┌─────────────▼─────────────┐        │
│  │  Caddy adds headers       │        │
│  │  • Security headers       │        │
│  │  • Compression           │        │
│  └─────────────┬─────────────┘        │
└────────────────┼──────────────────────┘
                 │
                 │ ⑥ HTTPS Response
                 │
                 ▼
         ┌─────────────────┐
         │   User Browser  │  Renders the page
         └─────────────────┘

Request Flow Examples

Example 1: Frontend Request

User → https://yourdomain.com
     ↓
Caddy receives on :443
     ↓
Caddyfile rule: yourdomain.com {
    reverse_proxy localhost:3000
}
     ↓
Forwards to Next.js on localhost:3000
     ↓
Next.js returns HTML
     ↓
Caddy sends response back to user

Example 2: API Request

User → https://api.yourdomain.com/users
     ↓
Caddy receives on :443
     ↓
Caddyfile rule: api.yourdomain.com {
    reverse_proxy localhost:5000
}
     ↓
Forwards to Node.js backend on localhost:5000
     ↓
Express processes /users endpoint
     ↓
Returns JSON data
     ↓
Caddy sends response back to user

systemd vs PM2

What is systemd?

systemd is the init system for Linux. It's responsible for:

Starting services when the server boots
Managing all system processes
Monitoring and restarting crashed processes
Managing system logs

┌─────────────────────────────────────┐
│     Linux Boot Process              │
│                                     │
│  ① Kernel loads                     │
│       ↓                             │
│  ② systemd starts (PID 1)           │
│       ↓                             │
│  ③ systemd reads service files      │
│     /etc/systemd/system/*.service   │
│       ↓                             │
│  ④ Starts enabled services          │
│     • backend.service               │
│     • frontend.service              │
│     • caddy.service                 │
└─────────────────────────────────────┘

What is PM2?

PM2 is a process manager specifically for Node.js applications. Features:

Cluster mode (multi-core CPU usage)
Zero-downtime reload
Built-in load balancer
Rich monitoring dashboard
Log management

┌─────────────────────────────────────┐
│     PM2 Architecture                │
│                                     │
│  PM2 Daemon (God Process)           │
│       │                             │
│       ├── App 1 (backend)           │
│       │    ├── Instance 1           │
│       │    ├── Instance 2           │
│       │    └── Instance 3           │
│       │                             │
│       └── App 2 (frontend)          │
│            └── Instance 1           │
└─────────────────────────────────────┘

Comparison Table

Feature	systemd	PM2
Platform	Linux only	Cross-platform
Language	Any	Node.js focused
Clustering	Manual setup	Built-in
Zero-downtime reload	❌	✅
Monitoring UI	journalctl (CLI)	Web dashboard
Memory usage	Very low	Moderate
Complexity	Medium	Easy
Auto-start on boot	✅ Native	✅ Via systemd
Log rotation	✅ Built-in	✅ Built-in
Process restart	✅	✅
Load balancing	❌	✅
Best for	Production servers	Node.js apps

When to Use What?

Use systemd when:

You want minimal dependencies
You're comfortable with Linux tools
You have non-Node.js services too
You want tight OS integration
You prefer standard Linux tools

Use PM2 when:

You need cluster mode (multi-core)
You want zero-downtime deployments
You need advanced monitoring
You want easier log management
Your team prefers Node.js tools

Use BOTH when:

You want PM2 features + systemd reliability
PM2 managed by systemd for auto-restart
Best of both worlds approach

Step by Step Deployment Guide

Prerequisites Checklist

[ ] AWS EC2 instance running Ubuntu 22.04+
[ ] SSH key for access
[ ] Domain name configured (optional but recommended)
[ ] Security Group allows ports: 22 (SSH), 80 (HTTP), 443 (HTTPS)
[ ] Git repositories ready (backend & frontend)

Part 1: Initial Server Setup

1.1 Connect to EC2

# From your local machine
ssh -i /path/to/your-key.pem ubuntu@your-ec2-ip

# Example:
ssh -i ~/aws-keys/my-app-key.pem ubuntu@54.123.45.67

1.2 Update System

sudo apt update && sudo apt upgrade -y

1.3 Install Essential Tools

sudo apt install -y git curl build-essential

1.4 Configure Firewall

# Enable firewall
sudo ufw enable

# Allow SSH (important - don't lock yourself out!)
sudo ufw allow 22/tcp

# Allow HTTP & HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Check status
sudo ufw status

Part 2: Install Node.js

# Install Node.js LTS (v20.x)
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt install -y nodejs

# Verify installation
node -v  # Should show v20.x.x
npm -v   # Should show 10.x.x

Part 3: Clone and Setup Backend

3.1 Clone Repository

cd /home/ubuntu
git clone https://github.com/yourusername/backend-app.git
cd backend-app

3.2 Create Environment File

nano .env

Add your configuration:

PORT=5000
NODE_ENV=production

# Database
MONGO_URI=mongodb+srv://username:password@cluster.mongodb.net/dbname
# or
DATABASE_URL=postgresql://user:pass@host:5432/db

# Security
JWT_SECRET=your-super-secret-jwt-key-change-this
JWT_EXPIRE=7d

# API Keys
STRIPE_SECRET_KEY=sk_live_xxxxx
SENDGRID_API_KEY=SG.xxxxx

# CORS
ALLOWED_ORIGINS=https://yourdomain.com,https://www.yourdomain.com

Security tip: Make sure .env is in .gitignore!

3.3 Install Dependencies

npm install --production

3.4 Test Backend Locally

# Test run
npm start

# You should see something like:
# Server running on port 5000

Press Ctrl + C to stop.

Part 4: Clone and Setup Frontend

4.1 Clone Repository

cd /home/ubuntu
git clone https://github.com/yourusername/frontend-app.git
cd frontend-app

4.2 Create Environment File

nano .env.production

Add your configuration:

# API Configuration
NEXT_PUBLIC_API_URL=https://api.yourdomain.com
NEXT_PUBLIC_API_TIMEOUT=10000

# App Configuration
NEXT_PUBLIC_APP_NAME=My Awesome App
NEXT_PUBLIC_APP_URL=https://yourdomain.com

# Analytics (optional)
NEXT_PUBLIC_GA_ID=G-XXXXXXXXXX

# Feature Flags
NEXT_PUBLIC_ENABLE_FEATURE_X=true

4.3 Install and Build

npm install
npm run build

# This creates the .next folder with optimized production build

4.4 Test Frontend Locally

npm start

# Should start on port 3000

Press Ctrl + C to stop.

Part 5: Process Management - Option A (systemd)

5.1 Create Backend Service

sudo nano /etc/systemd/system/backend.service

Add this configuration:

[Unit]
Description=Backend API Service
Documentation=https://github.com/yourusername/backend-app
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/backend-app
EnvironmentFile=/home/ubuntu/backend-app/.env
ExecStart=/usr/bin/node /home/ubuntu/backend-app/src/server.js

# Restart policy
Restart=always
RestartSec=10

# Resource limits
LimitNOFILE=65536

# Security
NoNewPrivileges=true
PrivateTmp=true

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=backend

[Install]
WantedBy=multi-user.target

5.2 Create Frontend Service

sudo nano /etc/systemd/system/frontend.service

Add this configuration:

[Unit]
Description=Frontend Next.js Application
Documentation=https://github.com/yourusername/frontend-app
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/frontend-app
Environment="NODE_ENV=production"
ExecStart=/usr/bin/npm start

# Restart policy
Restart=always
RestartSec=10

# Resource limits
LimitNOFILE=65536

# Security
NoNewPrivileges=true
PrivateTmp=true

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=frontend

[Install]
WantedBy=multi-user.target

5.3 Enable and Start Services

# Reload systemd to read new service files
sudo systemctl daemon-reload

# Enable services (start on boot)
sudo systemctl enable backend
sudo systemctl enable frontend

# Start services now
sudo systemctl start backend
sudo systemctl start frontend

# Check status
sudo systemctl status backend
sudo systemctl status frontend

5.4 Useful systemd Commands

# View logs
sudo journalctl -u backend -f        # Follow backend logs
sudo journalctl -u frontend -f       # Follow frontend logs
sudo journalctl -u backend --since "1 hour ago"
sudo journalctl -u backend -n 100    # Last 100 lines

# Service management
sudo systemctl restart backend       # Restart service
sudo systemctl stop backend          # Stop service
sudo systemctl start backend         # Start service
sudo systemctl disable backend       # Don't start on boot

# System status
systemctl list-units --type=service  # List all services
systemctl is-active backend          # Check if running
systemctl is-enabled backend         # Check if starts on boot

Part 6: Process Management - Option B (PM2)

6.1 Install PM2 Globally

sudo npm install -g pm2

6.2 Start Applications with PM2

# Start backend
cd /home/ubuntu/backend-app
pm2 start src/server.js --name backend

# Start frontend
cd /home/ubuntu/frontend-app
pm2 start npm --name frontend -- start

# Or for cluster mode (multiple instances):
pm2 start src/server.js --name backend -i max  # Uses all CPU cores

6.3 Configure PM2 with ecosystem.config.js

Create a config file for better management:

cd /home/ubuntu
nano ecosystem.config.js

Add this configuration:

module.exports = {
  apps: [
    {
      name: 'backend',
      cwd: '/home/ubuntu/backend-app',
      script: 'src/server.js',
      instances: 2,  // or 'max' for all CPU cores
      exec_mode: 'cluster',
      env: {
        NODE_ENV: 'production',
        PORT: 5000
      },
      error_file: '/home/ubuntu/logs/backend-error.log',
      out_file: '/home/ubuntu/logs/backend-out.log',
      log_date_format: 'YYYY-MM-DD HH:mm:ss Z',
      merge_logs: true,
      autorestart: true,
      max_memory_restart: '1G',
      watch: false
    },
    {
      name: 'frontend',
      cwd: '/home/ubuntu/frontend-app',
      script: 'npm',
      args: 'start',
      instances: 1,
      exec_mode: 'fork',
      env: {
        NODE_ENV: 'production',
        PORT: 3000
      },
      error_file: '/home/ubuntu/logs/frontend-error.log',
      out_file: '/home/ubuntu/logs/frontend-out.log',
      log_date_format: 'YYYY-MM-DD HH:mm:ss Z',
      merge_logs: true,
      autorestart: true,
      max_memory_restart: '500M',
      watch: false
    }
  ]
};

6.4 Start with Ecosystem File

# Create logs directory
mkdir -p /home/ubuntu/logs

# Start all apps
pm2 start ecosystem.config.js

# Save PM2 process list
pm2 save

# Make PM2 start on system boot
pm2 startup systemd -u ubuntu --hp /home/ubuntu
# Run the command it outputs (starts with sudo)

6.5 Useful PM2 Commands

# Process management
pm2 list                    # List all processes
pm2 status                  # Same as list
pm2 restart backend         # Restart specific app
pm2 restart all             # Restart all apps
pm2 reload backend          # Zero-downtime reload
pm2 stop backend            # Stop app
pm2 delete backend          # Remove app from PM2

# Monitoring
pm2 monit                   # Real-time monitoring dashboard
pm2 logs                    # View all logs
pm2 logs backend            # View specific app logs
pm2 logs backend --lines 100  # Last 100 lines
pm2 flush                   # Clear all logs

# Information
pm2 describe backend        # Detailed info about app
pm2 show backend            # Same as describe

# Cluster management
pm2 scale backend 4         # Scale to 4 instances
pm2 scale backend +2        # Add 2 more instances

# Updates
pm2 update                  # Update PM2 daemon
pm2 save                    # Save current process list

Part 7: Install and Configure Caddy

7.1 Install Caddy

# Install dependencies
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https

# Add Caddy repository
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | \
  sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg

curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | \
  sudo tee /etc/apt/sources.list.d/caddy-stable.list

# Update and install
sudo apt update
sudo apt install caddy -y

7.2 Configure Caddyfile

sudo nano /etc/caddy/Caddyfile

Basic Configuration:

# Frontend
yourdomain.com {
    reverse_proxy localhost:3000

    # Optional: Compression
    encode gzip

    # Optional: Security headers
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
        X-XSS-Protection "1; mode=block"
    }
}

# Backend API
api.yourdomain.com {
    reverse_proxy localhost:5000

    encode gzip

    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        X-Content-Type-Options "nosniff"
    }
}

Advanced Configuration with Load Balancing:

# Frontend with multiple instances
yourdomain.com {
    reverse_proxy localhost:3000 localhost:3001 {
        lb_policy round_robin
        health_uri /
        health_interval 10s
    }

    encode gzip zstd

    header {
        Strict-Transport-Security "max-age=31536000"
        X-Content-Type-Options "nosniff"
    }

    # Rate limiting
    rate_limit {
        zone static_rl {
            key {remote_host}
            window 1m
            events 100
        }
    }
}

# Backend API with authentication
api.yourdomain.com {
    reverse_proxy localhost:5000 localhost:5001 {
        lb_policy least_conn
        health_uri /health
        health_interval 30s
    }

    encode gzip

    # CORS headers
    header {
        Access-Control-Allow-Origin "https://yourdomain.com"
        Access-Control-Allow-Methods "GET, POST, PUT, DELETE"
        Access-Control-Allow-Headers "Content-Type, Authorization"
    }
}

7.3 Validate and Restart Caddy

# Validate configuration
sudo caddy validate --config /etc/caddy/Caddyfile

# If valid, reload Caddy
sudo systemctl reload caddy

# Check status
sudo systemctl status caddy

# View logs
sudo journalctl -u caddy -f

7.4 Caddy will automatically:

Get SSL certificates from Let's Encrypt
Renew certificates before expiration
Redirect HTTP to HTTPS
Handle TLS termination

Process Management Deep Dive

How systemd Manages Processes

System Boot Sequence:
──────────────────────

① BIOS/UEFI
    ↓
② Bootloader (GRUB)
    ↓
③ Linux Kernel loads
    ↓
④ systemd starts (PID 1)
    ↓
⑤ systemd reads target
   (multi-user.target)
    ↓
⑥ Starts all enabled services
   • backend.service
   • frontend.service
   • caddy.service
   • ssh.service
   • etc.

What happens when a process crashes?

Process Flow with systemd:
─────────────────────────

① App running (PID 1234)
    ↓
② App crashes (exit code 1)
    ↓
③ systemd detects termination
    ↓
④ Checks Restart= policy
   (Restart=always)
    ↓
⑤ Waits RestartSec seconds (10s)
    ↓
⑥ Starts app again (new PID 1567)
    ↓
⑦ Logs event to journald

How PM2 Manages Processes

PM2 Architecture:
────────────────

┌──────────────────────────┐
│   PM2 Daemon (God)       │  ← Main PM2 process
│   PID: 789               │
└────────┬─────────────────┘
         │
         ├─→ ┌──────────────────┐
         │   │ backend (cluster)│
         │   ├── Worker 1 (1234)│
         │   ├── Worker 2 (1235)│
         │   ├── Worker 3 (1236)│
         │   └── Worker 4 (1237)│
         │   └──────────────────┘
         │
         └─→ ┌──────────────────┐
             │ frontend (fork)  │
             └── Process (1238) │
             └──────────────────┘

Cluster Mode Explained:

Without Cluster (1 instance):
────────────────────────────
CPU Core 1: [████████] Backend running
CPU Core 2: [        ] Idle
CPU Core 3: [        ] Idle
CPU Core 4: [        ] Idle

With Cluster Mode (4 instances):
───────────────────────────────
CPU Core 1: [████████] Backend Worker 1
CPU Core 2: [████████] Backend Worker 2
CPU Core 3: [████████] Backend Worker 3
CPU Core 4: [████████] Backend Worker 4

Load Balancer distributes requests across all workers

What happens when a PM2 process crashes?

Process Flow with PM2:
─────────────────────

① Worker 2 crashes
    ↓
② PM2 God process detects
    ↓
③ Immediately spawns new worker
    ↓
④ New worker starts in <1 second
    ↓
⑤ Logs crash to PM2 logs
    ↓
⑥ Cluster continues serving traffic
   (Workers 1, 3, 4 still running)

Monitoring and Logs

systemd Monitoring

Check Service Status

# Quick status check
sudo systemctl status backend
sudo systemctl status frontend

# Is the service running?
systemctl is-active backend   # Returns: active or inactive

# Will it start on boot?
systemctl is-enabled backend  # Returns: enabled or disabled

View Logs

# View all logs for a service
sudo journalctl -u backend

# Follow logs in real-time (like tail -f)
sudo journalctl -u backend -f

# Show last 50 lines
sudo journalctl -u backend -n 50

# Show logs from last hour
sudo journalctl -u backend --since "1 hour ago"

# Show logs from specific time
sudo journalctl -u backend --since "2024-03-20 10:00:00"

# Show logs between times
sudo journalctl -u backend --since "2024-03-20" --until "2024-03-21"

# Show only errors
sudo journalctl -u backend -p err

# Export logs to file
sudo journalctl -u backend > backend-logs.txt

PM2 Monitoring

Monitor Dashboard

# Real-time monitoring (press Ctrl+C to exit)
pm2 monit

# Shows:
# - CPU usage
# - Memory usage
# - Process status
# - Logs in real-time

Check Status

# List all processes
pm2 list

# Detailed info
pm2 describe backend

# JSON output (for scripts)
pm2 jlist

View Logs

# All logs
pm2 logs

# Specific app
pm2 logs backend

# Last 100 lines
pm2 logs backend --lines 100

# Follow logs
pm2 logs backend -f

# Error logs only
pm2 logs backend --err

# Clear all logs
pm2 flush

System Information

# PM2 system info
pm2 info

# Show PM2 version
pm2 -v

# Show memory usage
pm2 list | grep MB

Troubleshooting and Best Practices

Common Issues and Solutions

Issue 1: Service Won't Start

Symptoms:

sudo systemctl status backend
# Shows: failed (Result: exit-code)

Solutions:

Check the logs:

sudo journalctl -u backend -n 50

Common causes:
- Missing .env file
- Wrong file paths in service file
- Port already in use
- Missing dependencies
Test manually:

cd /home/ubuntu/backend-app
node src/server.js
# See what error appears

Check permissions:

ls -la /home/ubuntu/backend-app
# Make sure ubuntu user owns the files
sudo chown -R ubuntu:ubuntu /home/ubuntu/backend-app

Issue 2: Port Already in Use

Symptoms:

Error: listen EADDRINUSE: address already in use :::5000

Solutions:

Find what's using the port:

sudo lsof -i :5000
sudo netstat -tulpn | grep 5000

Kill the process:

sudo kill -9 <PID>

Or change your app's port in .env

Issue 3: Caddy Can't Get SSL Certificate

Symptoms:

certificate retrieval failed

Solutions:

Make sure DNS is pointing to your server:

dig yourdomain.com
nslookup yourdomain.com

Check if ports 80 and 443 are open:

sudo ufw status
# Should show 80/tcp and 443/tcp ALLOW

Check Caddy logs:

sudo journalctl -u caddy -f

Verify domain ownership:

# Make sure A record points to your EC2 IP
curl -I http://yourdomain.com

Test Caddy config:

sudo caddy validate --config /etc/caddy/Caddyfile

Issue 4: High Memory Usage

Symptoms:

Server becomes slow
Out of memory errors
Applications crash randomly

Solutions:

Check memory usage:

free -h
htop  # Install with: sudo apt install htop

Find memory-hungry processes:

ps aux --sort=-%mem | head -n 10

For systemd, add memory limits:

[Service]
MemoryMax=512M
MemoryHigh=400M

For PM2, configure max memory restart:

max_memory_restart: '500M'

Optimize Node.js memory:

# In your service file or PM2 config
NODE_OPTIONS="--max-old-space-size=512"

Issue 5: Application Crashes After Deployment

Symptoms:

App works locally but crashes on server
"Module not found" errors

Solutions:

Check Node.js version matches:

node -v
# Compare with your local version

Reinstall dependencies:

cd /home/ubuntu/backend-app
rm -rf node_modules package-lock.json
npm install --production

Check for missing environment variables:

# Add console.log to verify .env is loaded
console.log('Environment:', process.env.NODE_ENV);

Verify file permissions:

sudo chown -R ubuntu:ubuntu /home/ubuntu/backend-app
chmod -R 755 /home/ubuntu/backend-app

Issue 6: Frontend Shows 502 Bad Gateway

Symptoms:

Caddy returns 502 error
"upstream connect error"

Solutions:

Check if frontend is actually running:

curl localhost:3000
# Should return HTML

sudo systemctl status frontend
# Should show "active (running)"

Check frontend logs:

sudo journalctl -u frontend -n 50

Verify port in Caddyfile matches your app:

# In Caddyfile should be:
reverse_proxy localhost:3000

# In your Next.js app:
PORT=3000 npm start

Test connection manually:

telnet localhost 3000
# Should connect successfully

Best Practices

Security Best Practices

Never expose .env files:

# Make sure .env is in .gitignore
echo ".env" >> .gitignore
echo ".env.*" >> .gitignore

# Set proper permissions
chmod 600 /home/ubuntu/backend-app/.env

Use environment-specific secrets:

# Different secrets for dev/staging/production
JWT_SECRET_DEV=dev-secret-key
JWT_SECRET_PROD=super-complex-prod-key-12345

Keep dependencies updated:

# Check for vulnerabilities
npm audit

# Fix automatically (test first!)
npm audit fix

# Update dependencies
npm update

Use SSH keys only (disable password auth):

sudo nano /etc/ssh/sshd_config

# Set these values:
PasswordAuthentication no
PermitRootLogin no
PubkeyAuthentication yes

sudo systemctl restart sshd

Configure fail2ban to prevent brute force:

sudo apt install fail2ban -y
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Performance Best Practices

Use PM2 cluster mode for CPU-intensive apps:

{
  name: 'backend',
  instances: 'max',  // Uses all CPU cores
  exec_mode: 'cluster'
}

Enable Caddy compression:

yourdomain.com {
    encode gzip zstd
    reverse_proxy localhost:3000
}

Use production builds:

# Next.js
NODE_ENV=production npm run build

# React
npm run build

Implement health checks:

// In your Express app
app.get('/health', (req, res) => {
  res.status(200).json({ 
    status: 'ok', 
    timestamp: new Date().toISOString() 
  });
});

Use CDN for static assets:
Images, CSS, JS should be on S3 + CloudFront
Reduces server load
Faster delivery worldwide

Deployment Best Practices

Use Git tags for versions:

# Tag your releases
git tag -a v1.0.0 -m "Production release 1.0.0"
git push origin v1.0.0

# Deploy specific version
git checkout v1.0.0

Create deployment script:

nano /home/ubuntu/deploy-backend.sh

Add:

#!/bin/bash
set -e  # Exit on error

echo "🚀 Starting deployment..."

cd /home/ubuntu/backend-app

echo "📥 Pulling latest code..."
git pull origin main

echo "📦 Installing dependencies..."
npm install --production

echo "🧪 Running tests..."
npm test

echo "🔄 Restarting service..."
sudo systemctl restart backend

echo "✅ Deployment complete!"

Make executable:

chmod +x /home/ubuntu/deploy-backend.sh

Run:

./deploy-backend.sh

Implement zero-downtime deployment with PM2:

# Instead of restart, use reload
pm2 reload backend

# This:
# 1. Starts new instances
# 2. Waits for them to be ready
# 3. Stops old instances
# 4. Zero downtime!

Keep backup of previous version:

# Before deploying
cp -r /home/ubuntu/backend-app /home/ubuntu/backend-app.backup

# If deployment fails, rollback:
rm -rf /home/ubuntu/backend-app
mv /home/ubuntu/backend-app.backup /home/ubuntu/backend-app
sudo systemctl restart backend

Use separate staging environment:

# Different ports for staging
STAGING_PORT=5001
PRODUCTION_PORT=5000

# Different subdomains
staging.yourdomain.com → localhost:5001
api.yourdomain.com → localhost:5000

Monitoring Best Practices

Set up log rotation:

For systemd:

sudo nano /etc/systemd/journald.conf

# Add these settings:
SystemMaxUse=500M
SystemKeepFree=1G
MaxRetentionSec=1week

For PM2:

{
  log_date_format: 'YYYY-MM-DD HH:mm:ss',
  merge_logs: true,
  max_size: '10M',
  retain: 7  // Keep 7 log files
}

Monitor disk space:

# Check disk usage
df -h

# Find large files
du -sh /home/ubuntu/* | sort -rh | head -n 10

# Clean up PM2 logs
pm2 flush

# Clean up system logs
sudo journalctl --vacuum-time=7d

Set up alerts (optional):

# Install monitoring tools
sudo apt install monitoring-plugins -y

# Or use cloud monitoring:
# - AWS CloudWatch
# - Datadog
# - New Relic

Regular backups:

# Backup script
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
tar -czf /home/ubuntu/backups/app-$DATE.tar.gz \
  /home/ubuntu/backend-app \
  /home/ubuntu/frontend-app \
  /etc/caddy/Caddyfile

# Keep only last 7 backups
ls -t /home/ubuntu/backups/*.tar.gz | tail -n +8 | xargs rm -f

Add to crontab:

crontab -e

# Run backup daily at 2 AM
0 2 * * * /home/ubuntu/backup.sh

Deployment Workflow Comparison

Manual Deployment (Basic)

┌─────────────────────────────────────────┐
│ 1. Developer pushes code to GitHub     │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 2. SSH into EC2 server                  │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 3. cd /home/ubuntu/backend-app          │
│    git pull origin main                 │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 4. npm install                          │
│    npm run build (if needed)            │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 5. sudo systemctl restart backend       │
│    OR                                   │
│    pm2 reload backend                   │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 6. Test the application                 │
└─────────────────────────────────────────┘

⏱️ Time: 5-10 minutes
👤 Manual effort required
⚠️ Potential downtime

Automated Deployment with GitHub Actions (Advanced)

# .github/workflows/deploy.yml
name: Deploy to EC2

on:
  push:
    branches: [ main ]

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Deploy to EC2
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.EC2_HOST }}
        username: ubuntu
        key: ${{ secrets.EC2_SSH_KEY }}
        script: |
          cd /home/ubuntu/backend-app
          git pull origin main
          npm install --production
          pm2 reload backend

┌─────────────────────────────────────────┐
│ 1. Developer pushes to GitHub           │
└──────────────────┬──────────────────────┘
                   │
                   ↓ (automatic)
┌─────────────────────────────────────────┐
│ 2. GitHub Actions triggered             │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 3. Runs tests automatically             │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 4. SSH to EC2 and deploy                │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 5. Zero-downtime reload with PM2        │
└──────────────────┬──────────────────────┘
                   │
                   ↓
┌─────────────────────────────────────────┐
│ 6. Send notification (Discord/Slack)    │
└─────────────────────────────────────────┘

⏱️ Time: 2-3 minutes
👤 Zero manual effort
✅ No downtime
🔔 Automatic notifications

System Architecture Diagrams

Complete Request Flow with All Components

                    🌍 Internet
                        │
                        │ HTTPS Request
                        │ https://yourdomain.com/api/users
                        │
                        ▼
        ┌───────────────────────────────────┐
        │      AWS Security Group           │
        │   • Allow 80 (HTTP)               │
        │   • Allow 443 (HTTPS)             │
        │   • Allow 22 (SSH - your IP only) │
        └───────────────┬───────────────────┘
                        │
                        ▼
        ┌───────────────────────────────────┐
        │         UFW Firewall              │
        │   • Port 80 → Allow               │
        │   • Port 443 → Allow              │
        │   • Port 22 → Allow               │
        │   • Other ports → Deny            │
        └───────────────┬───────────────────┘
                        │
                        ▼
        ┌───────────────────────────────────┐
        │    Caddy (Reverse Proxy)          │
        │    • Receives on port 443         │
        │    • Terminates SSL/TLS           │
        │    • Checks Caddyfile rules       │
        │    • Routes to backend/frontend   │
        └───────────────┬───────────────────┘
                        │
        ┌───────────────┴───────────────┐
        │                               │
        │ if /api/*                     │ if /*
        ▼                               ▼
┌──────────────┐                ┌──────────────┐
│   Backend    │                │   Frontend   │
│   Node.js    │                │   Next.js    │
│              │                │              │
│ Port: 5000   │                │ Port: 3000   │
│              │                │              │
│ Managed by:  │                │ Managed by:  │
│ ┌──────────┐ │                │ ┌──────────┐ │
│ │ systemd  │ │                │ │ systemd  │ │
│ │   OR     │ │                │ │   OR     │ │
│ │   PM2    │ │                │ │   PM2    │ │
│ └──────────┘ │                │ └──────────┘ │
│              │                │              │
│ ┌──────────┐ │                │              │
│ │ Process  │ │                │              │
│ │ Monitor  │ │                │              │
│ │ • Auto   │ │                │              │
│ │ restart  │ │                │              │
│ │ • Logs   │ │                │              │
│ └──────────┘ │                │              │
└──────┬───────┘                └──────────────┘
       │
       │ Database queries
       ▼
┌─────────────────┐
│   Database      │
│ • MongoDB       │
│ • PostgreSQL    │
│ • MySQL         │
│ (External)      │
└─────────────────┘

Process Lifecycle with systemd

System Boot
    │
    ▼
┌───────────────────────────────────────┐
│ systemd (PID 1) starts                │
│ Reads: /etc/systemd/system/*.service  │
└───────────────┬───────────────────────┘
                │
    ┌───────────┼───────────┐
    │           │           │
    ▼           ▼           ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ backend │ │frontend │ │  caddy  │
│ service │ │ service │ │ service │
└────┬────┘ └────┬────┘ └────┬────┘
     │           │           │
     │ Starts    │ Starts    │ Starts
     │ Process   │ Process   │ Process
     ▼           ▼           ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ node    │ │ npm     │ │ caddy   │
│ server  │ │ start   │ │ run     │
│ PID:1234│ │ PID:1235│ │ PID:1236│
└────┬────┘ └────┬────┘ └────┬────┘
     │           │           │
     │ Running   │ Running   │ Running
     │           │           │
     ▼           ▼           ▼
┌────────────────────────────────────┐
│     systemd monitors all           │
│     • Collects logs (journald)     │
│     • Monitors health              │
│     • Auto-restart on crash        │
└────────────────────────────────────┘

If process crashes:

Process Crash Detected
        │
        ▼
┌───────────────────┐
│ systemd detects   │
│ process exit      │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Check Restart=    │
│ policy            │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Wait RestartSec   │
│ (default 10s)     │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Start new process │
│ (new PID)         │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Log to journald   │
└───────────────────┘

Process Lifecycle with PM2

PM2 Startup
    │
    ▼
┌───────────────────────────────────────┐
│ PM2 Daemon (God Process) starts       │
│ Reads: ecosystem.config.js            │
│ PID: 789                              │
└───────────────┬───────────────────────┘
                │
    ┌───────────┴───────────┐
    │                       │
    ▼                       ▼
┌─────────────────┐   ┌──────────────┐
│ Backend App     │   │ Frontend App │
│ • Cluster Mode  │   │ • Fork Mode  │
│ • 4 instances   │   │ • 1 instance │
└────┬────────────┘   └──────┬───────┘
     │                       │
     ├──┬──┬──┬──            │
     ▼  ▼  ▼  ▼              ▼
    W1 W2 W3 W4             W1
    PID PID PID PID         PID
    1234 1235 1236 1237     1238

All workers report to PM2 Daemon

Request distribution in Cluster Mode:

Incoming Requests
        │
        ▼
┌────────────────────┐
│ PM2 Load Balancer  │
│ (Round Robin)      │
└────────┬───────────┘
         │
    ┌────┼────┬────┬────┐
    │    │    │    │    │
    ▼    ▼    ▼    ▼    ▼
   W1   W2   W3   W4   W1
  Req1 Req2 Req3 Req4 Req5

Each worker handles requests independently
If one crashes, others continue serving

Quick Reference Commands

systemd Commands Cheat Sheet

# Service Management
sudo systemctl start <service>      # Start service
sudo systemctl stop <service>       # Stop service
sudo systemctl restart <service>    # Restart service
sudo systemctl reload <service>     # Reload config
sudo systemctl status <service>     # Check status

# Enable/Disable (auto-start on boot)
sudo systemctl enable <service>     # Enable auto-start
sudo systemctl disable <service>    # Disable auto-start
sudo systemctl is-enabled <service> # Check if enabled

# Logs
sudo journalctl -u <service>        # All logs
sudo journalctl -u <service> -f     # Follow logs
sudo journalctl -u <service> -n 50  # Last 50 lines
sudo journalctl -u <service> --since "1 hour ago"
sudo journalctl -u <service> --since "2024-03-20 10:00"

# System Management
sudo systemctl daemon-reload        # Reload service files
sudo systemctl list-units          # List all units
sudo systemctl list-units --failed # Failed services

PM2 Commands Cheat Sheet

# Start/Stop
pm2 start app.js --name myapp      # Start app
pm2 start ecosystem.config.js      # Start from config
pm2 stop myapp                     # Stop app
pm2 restart myapp                  # Restart app
pm2 reload myapp                   # Zero-downtime reload
pm2 delete myapp                   # Remove from PM2

# Cluster Mode
pm2 start app.js -i 4              # 4 instances
pm2 start app.js -i max            # All CPU cores
pm2 scale myapp 4                  # Scale to 4 instances
pm2 scale myapp +2                 # Add 2 instances

# Information
pm2 list                           # List all apps
pm2 describe myapp                 # Detailed info
pm2 monit                         # Real-time monitor

# Logs
pm2 logs                          # All logs
pm2 logs myapp                    # App logs
pm2 logs myapp --lines 100        # Last 100 lines
pm2 flush                         # Clear logs

# Startup
pm2 startup                       # Generate startup script
pm2 save                          # Save process list
pm2 resurrect                     # Restore saved processes

# Updates
pm2 update                        # Update PM2 daemon
pm2 reset myapp                   # Reset restart counter

Git Deployment Commands

# Pull latest changes
cd /home/ubuntu/backend-app
git pull origin main

# Check current branch/version
git branch                        # Current branch
git log -1                        # Last commit
git status                        # Working directory status

# Deploy specific version
git fetch --tags
git checkout v1.0.0
git checkout main                 # Back to main

# Undo changes (dangerous!)
git reset --hard origin/main      # Reset to remote
git clean -fd                     # Remove untracked files

Server Maintenance Commands

# Disk Space
df -h                             # Disk usage
du -sh /path/*                    # Directory sizes
ncdu /home/ubuntu                 # Interactive disk usage

# Memory
free -h                           # Memory usage
htop                             # Interactive process viewer
ps aux --sort=-%mem | head       # Top memory processes

# Processes
ps aux                           # All processes
ps aux | grep node               # Find node processes
kill -9 <PID>                    # Kill process
pkill -f "node"                  # Kill by name

# Network
netstat -tulpn                   # Open ports
ss -tulpn                        # Socket statistics
lsof -i :3000                    # What's on port 3000

# Logs
tail -f /var/log/syslog          # System log
tail -f /var/log/nginx/error.log # Nginx errors
sudo journalctl -f               # All system logs

Summary

Key Takeaways

Architecture: Internet → DNS → EC2 → Caddy → Backend/Frontend → Database
Process Managers:
- systemd: Native Linux, minimal overhead, works with any language
- PM2: Node.js focused, cluster mode, zero-downtime, better monitoring
Deployment Flow:
- Clone repo → Install dependencies → Build (if needed) → Configure service → Start → Monitor
Best Practices:
- Always use environment variables for secrets
- Enable auto-restart for reliability
- Monitor logs regularly
- Keep backups
- Use HTTPS everywhere (Caddy handles this)
- Implement proper error handling
Troubleshooting:
- Check logs first (journalctl or pm2 logs)
- Verify ports are available
- Test manually before automating
- Keep services updated

Quick Start Checklist

[ ] EC2 instance running with SSH access
[ ] Node.js installed
[ ] Git repositories cloned
[ ] Environment files (.env) configured
[ ] Dependencies installed (npm install)
[ ] Production build created (npm run build)
[ ] Process manager configured (systemd or PM2)
[ ] Services enabled and started
[ ] Caddy installed and configured
[ ] DNS pointing to EC2 IP
[ ] SSL certificate obtained (automatic with Caddy)
[ ] Firewall configured (ports 80, 443, 22)
[ ] Services tested and monitored
[ ] Deployment script created
[ ] Backup strategy in place

You're now ready to deploy and manage production applications on AWS EC2! 🚀