DEV Community: Cyril Sebastian

Keep It Running: Email Keep It Running: Email Infrastructure Operations Guide

Cyril Sebastian — Fri, 27 Mar 2026 14:59:53 +0000

Making email infrastructure a pro.

Daily Operations (5 Minutes)

The Morning Health Check

Create this script and run it daily:

cat > ~/email-health.sh <<'EOF'
#!/bin/bash
YESTERDAY=$(date -d "yesterday" +"%b %d")

echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "Email Health ($YESTERDAY)"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

# Service status
systemctl is-active --quiet postfix && echo "Postfix" || echo "Postfix DOWN"
systemctl is-active --quiet ses-logger && echo "Logger" || echo "Logger DOWN"

# Email stats
SENT=$(grep "$YESTERDAY" /var/log/postfix/postfix.log 2>/dev/null | grep -c "status=sent")
DELIVERED=$(grep "$YESTERDAY" /var/log/postfix/mail.log 2>/dev/null | grep -c "status=delivered")
BOUNCED=$(grep "$YESTERDAY" /var/log/postfix/mail.log 2>/dev/null | grep -c "status=bounced")

echo ""
echo "📊 Volume"
echo "   Sent: $SENT"
echo "   Delivered: $DELIVERED"
echo "   Bounced: $BOUNCED"

if [ $SENT -gt 0 ]; then
  DELIVERY_RATE=$((DELIVERED * 100 / SENT))
  BOUNCE_RATE=$((BOUNCED * 100 / SENT))
  echo ""
  echo "📈 Rates"
  echo "   Delivery: ${DELIVERY_RATE}%"
  echo "   Bounce: ${BOUNCE_RATE}%"

  [ $BOUNCE_RATE -gt 5 ] && echo "   ⚠️  High bounce rate!"
fi

# Queue status
QUEUE=$(mailq | tail -1 | awk '{print $5}')
[ "$QUEUE" = "empty" ] && echo "" && echo "Queue empty" || echo "" && echo "⚠️  Queue: $QUEUE messages"

echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
EOF

chmod +x ~/email-health.sh

Run it:

./email-health.sh

Automate it (runs at 9 AM, emails you results):

(crontab -l 2>/dev/null; echo "0 9 * * * ~/email-health.sh | mail -s 'Email Health Report' admin@yourdomain.com") | crontab -

Essential Monitoring

Real-Time Log Watching

Monitor live email flow:

# Watch everything
sudo tail -f /var/log/postfix/*.log

# Watch only delivered emails
sudo tail -f /var/log/postfix/mail.log | grep --line-buffered "status=delivered"

# Watch bounces
sudo tail -f /var/log/postfix/mail.log | grep --line-buffered "status=bounced"

Key Metrics to Track

Metric	Target	Alert If
Delivery Rate	>95%	<90%
Bounce Rate	<5%	>10%
Queue Size	0	>100
Service Uptime	99.9%	Any downtime

Quick metric checks:

# Today's delivery rate
SENT=$(grep "$(date +%b\ %d)" /var/log/postfix/postfix.log | grep -c "status=sent")
DELIVERED=$(grep "$(date +%b\ %d)" /var/log/postfix/mail.log | grep -c "status=delivered")
echo "Delivery rate: $((DELIVERED * 100 / SENT))%"

# Average delivery time (milliseconds)
grep "status=delivered" /var/log/postfix/mail.log | \
  grep -oP 'delay=\K\d+' | \
  awk '{sum+=$1; n++} END {print "Avg delay: " sum/n "ms"}'

# Top recipient domains
grep "status=delivered" /var/log/postfix/mail.log | \
  grep -oP 'to=<[^@]+@\K[^>]+' | \
  sort | uniq -c | sort -rn | head -5

Common Operations

Adding New Senders

# 1. Edit whitelist
sudo vim /etc/postfix/allowed_senders

# Add line:
# newsender@yourdomain.com    OK

# 2. Rebuild database
sudo postmap /etc/postfix/allowed_senders

# 3. Reload (no restart needed!)
sudo systemctl reload postfix

# 4. Test
echo "Test" | mail -s "Test" -r newsender@yourdomain.com test@example.com

No downtime! Reload picks up changes instantly.

Removing Senders

# 1. Comment out or remove from whitelist
sudo vim /etc/postfix/allowed_senders
# #oldsender@yourdomain.com    OK

# 2. Rebuild and reload
sudo postmap /etc/postfix/allowed_senders
sudo systemctl reload postfix

# 3. Verify rejection
echo "Test" | mail -s "Test" -r oldsender@yourdomain.com test@example.com
# Should see: "Sender address rejected"

Managing Mail Queue

View queue:

mailq

Flush queue (retry all deferred emails):

sudo postqueue -f

Delete specific email:

# Get queue ID from mailq
sudo postsuper -d QUEUE_ID

Delete all queued emails:

sudo postsuper -d ALL

Delete only deferred emails:

sudo postsuper -d ALL deferred

Searching Email History

Find specific email:

grep "user@example.com" /var/log/postfix/*.log

Find by sender:

grep "from=<sender@yourdomain.com>" /var/log/postfix/postfix.log

Find bounces to specific domain:

grep "gmail.com" /var/log/postfix/mail.log | grep "bounced"

Get complete email journey:

# Get message ID from sent log
MSG_ID=$(grep "user@example.com" /var/log/postfix/postfix.log | grep -oP 'status=sent \(250 Ok \K[^)]+' | head -1)

# Find all events for that message
grep "$MSG_ID" /var/log/postfix/*.log

Troubleshooting Guide

Problem 1: Postfix Won't Start

Symptoms:

sudo systemctl start postfix
# Job for postfix.service failed

Fix:

# 1. Check config syntax
sudo postfix check

# 2. View detailed error
sudo journalctl -u postfix -n 20 --no-pager

# 3. Common issues:

# Port in use?
sudo lsof -i :25
# Kill conflicting process: sudo systemctl stop sendmail

# Permission issue?
sudo chown -R postfix:postfix /var/log/postfix
sudo chown -R postfix:postfix /var/spool/postfix

# Check line number from 'postfix check' output
sudo vim /etc/postfix/main.cf +LINE_NUMBER

Problem 2: Emails Stuck in Queue

Diagnosis:

mailq  # Shows queued emails
sudo tail -100 /var/log/postfix/postfix.log | grep "status=deferred"

Common causes and fixes:

Wrong SES credentials:

# Verify credentials
sudo postmap -q "[email-smtp.ap-south-1.amazonaws.com]:587" /etc/postfix/sasl_passwd

# Update if needed
sudo vim /etc/postfix/sasl_passwd
sudo postmap /etc/postfix/sasl_passwd
sudo systemctl restart postfix

Network blocked:

# Test SES connectivity
telnet email-smtp.ap-south-1.amazonaws.com 587

# Check security group allows outbound 587
# Check route table has internet gateway

SES quota exceeded:

aws ses get-send-quota --region ap-south-1
# If near limit, wait or request increase

After fixing, flush the queue:

sudo postqueue -f

Problem 3: Logger Service Keeps Crashing

Check logs:

sudo journalctl -u ses-logger -n 50 --no-pager
sudo tail -50 /var/log/ses-logger-error.log

Common fixes:

boto3 missing:

python3 -c "import boto3" || sudo yum install -y python3-boto3
sudo systemctl restart ses-logger

Wrong queue URL:

# Get correct URL
QUEUE_URL=$(aws sqs get-queue-url --queue-name ses-events-queue --region ap-south-1 --query 'QueueUrl' --output text)

# Update service
sudo sed -i "s|Environment=\"SQS_QUEUE_URL=.*\"|Environment=\"SQS_QUEUE_URL=$QUEUE_URL\"|" /etc/systemd/system/ses-logger.service

sudo systemctl daemon-reload
sudo systemctl restart ses-logger

IAM permissions:

# Verify role attached
aws sts get-caller-identity

# Should show: PostfixSESLogger role
# If not, reattach IAM instance profile

Problem 4: No Delivery Events in Logs

Diagnosis:

# 1. Check SQS queue has messages
aws sqs get-queue-attributes \
  --queue-url "$(aws sqs get-queue-url --queue-name ses-events-queue --region ap-south-1 --query 'QueueUrl' --output text)" \
  --attribute-names ApproximateNumberOfMessages \
  --region ap-south-1

If messages are accumulating:

Logger not processing → Check sudo journalctl -u ses-logger
Restart logger → sudo systemctl restart ses-logger

If no messages in queue:

# 2. Verify SES publishing to SNS
aws ses get-identity-notification-attributes \
  --identities yourdomain.com \
  --region ap-south-1

# Should show all three topics configured

# 3. Reconfigure if needed
SNS_ARN=$(aws sns list-topics --region ap-south-1 --query "Topics[?contains(TopicArn, 'ses-events-topic')].TopicArn | [0]" --output text)

for EVENT in Delivery Bounce Complaint; do
  aws ses set-identity-notification-topic \
    --identity yourdomain.com \
    --notification-type $EVENT \
    --sns-topic "$SNS_ARN" \
    --region ap-south-1
done

Problem 5: High Bounce Rate (>10%)

Analyze bounce reasons:

grep "status=bounced" /var/log/postfix/mail.log | \
  grep -oP 'reason=\(\K[^\)]+' | \
  sort | uniq -c | sort -rn | head -10

Common reasons:

"User unknown" (invalid addresses):

# Extract bounced addresses
grep "status=bounced" /var/log/postfix/mail.log | \
  grep "bounce_type=Permanent" | \
  grep -oP 'to=<\K[^>]+' | \
  sort -u > bounced_addresses.txt

# Remove from your mailing list

"Mailbox full":

Temporary issue, will resolve
Retry after 24 hours

"550 Spam":

Review email content
Check SPF/DKIM/DMARC setup
Verify sender reputation

Problem 6: Emails Going to Spam

Verification checklist:

# 1. Check SPF
dig +short TXT yourdomain.com | grep spf
# Should include: include:amazonses.com

# 2. Check DKIM
aws ses get-identity-dkim-attributes \
  --identities yourdomain.com \
  --region ap-south-1
# Should show: DkimEnabled=true, Status=Success

# 3. Check DMARC
dig +short TXT _dmarc.yourdomain.com
# Should return DMARC policy

# 4. Check SES reputation
aws ses get-account-sending-enabled --region ap-south-1
# Should be enabled

Content checklist:

Avoid spam trigger words (FREE!, ACT NOW!)
Include unsubscribe link
Balance text/image ratio (60% text minimum)
Use a consistent "From" name and address
Authenticate with SPF/DKIM/DMARC

Performance Optimization

Postfix Tuning

For higher throughput:

sudo vim /etc/postfix/main.cf

Add/update:

# Increase concurrent deliveries
default_destination_concurrency_limit = 50
default_destination_recipient_limit = 50

# Reduce queue lifetime
maximal_queue_lifetime = 1d
bounce_queue_lifetime = 1d

# Connection caching
smtp_connection_cache_on_demand = yes
smtp_connection_cache_destinations = email-smtp.ap-south-1.amazonaws.com

Reload:

sudo systemctl reload postfix

Logger Optimization

For high volume (>1000 events/min):

Edit /usr/local/bin/ses_logger.py:

# Increase batch size
response = sqs.receive_message(
    QueueUrl=queue_url,
    MaxNumberOfMessages=30,  # Up from 10
    WaitTimeSeconds=20
)

Restart:

sudo systemctl restart ses-logger

Scaling Strategies

When to Scale

Metric	Scale Trigger
CPU Usage	Sustained >70%
Emails/day	>40,000 (80% of quota)
Queue Size	Sustained >100
Memory	>80% used

Vertical Scaling (Bigger Instance)

Current performance by instance:

Instance	vCPU	RAM	Emails/day
t3a.small	2	2GB	10,000
t3a.medium	2	4GB	50,000
t3a.large	2	8GB	100,000
c6a.xlarge	4	8GB	500,000

Security Hardening

Restrict Relay Access

Tighten network access:

sudo vim /etc/postfix/main.cf

# Only specific IPs
mynetworks = 127.0.0.1, 10.10.3.125

# Or specific subnet
mynetworks = 10.10.0.0/21

Rate Limiting

Prevent abuse:

sudo vim /etc/postfix/main.cf

# Max 100 connections/min per client
smtpd_client_connection_rate_limit = 100

# Max 100 emails/min per client
smtpd_client_message_rate_limit = 100

Monitor IAM Usage

Enable CloudTrail for audit:

aws cloudtrail create-trail \
  --name email-infrastructure-audit \
  --s3-bucket-name my-audit-logs

Resources

AWS Documentation:

Postfix:

Series Complete! 🎉

Part 1: Architecture & Design
Part 2: Implementation Guide
Part 3: Operations ← . You just finished this

🔗 If this helped or resonated with you, connect with me on LinkedIn. Let’s learn and grow together.

👉 Stay tuned for more behind-the-scenes write-ups and system design breakdowns.

From Zero to Production: Building Postfix + AWS SES in 2 Hours

Cyril Sebastian — Tue, 17 Mar 2026 17:49:24 +0000

What You Need Before Starting

AWS account with SES access
EC2 instance (t3a.medium, Amazon Linux 2023)
Your domain's DNS access
2 hours of focused time

That's it. Everything else, we'll build together.

Phase 1: AWS SES Setup (15 mins)

Step 1: Verify Your Domain

AWS Console → SES → Verified Identities → Create Identity

Identity type: Domain
Domain: yourdomain.com

Add the TXT record SES provides to your DNS:

Type: TXT
Name: _amazonses.yourdomain.com
Value: [provided by SES]

Verify it worked:

aws ses get-identity-verification-attributes \
  --identities yourdomain.com \
  --region ap-south-1

Look for "VerificationStatus": "Success"

Step 2: Configure DKIM (5 mins)

In SES Console:

Click your domain → DKIM tab → Edit
Enable Easy DKIM → Save

Add the 3 CNAME records SES provides to your DNS.

Verify:

aws ses get-identity-dkim-attributes \
  --identities yourdomain.com \
  --region ap-south-1

Should show "DkimEnabled": true

Step 3: SPF + DMARC (3 mins)

Add SPF record:

Type: TXT
Name: yourdomain.com
Value: "v=spf1 include:amazonses.com ~all"

Add DMARC record:

Type: TXT  
Name: _dmarc.yourdomain.com
Value: "v=DMARC1; p=quarantine; rua=mailto:dmarc@yourdomain.com"

Step 4: Get SMTP Credentials (2 mins)

SES Console → SMTP Settings → Create SMTP Credentials

Save these immediately (you won't see them again):

Username: AKAWSSAMPLEEXAMPLE
Password: wJalrXUtnuTde/EXAMPLE

Phase 2: Postfix Setup (20 mins)

SSH into your server and let's configure Postfix.

Install Postfix

sudo yum update -y
sudo yum install -y postfix cyrus-sasl-plain mailx
sudo systemctl enable postfix

Configure SES Credentials

sudo vim /etc/postfix/sasl_passwd

Add this line (use YOUR credentials):

[email-smtp.ap-south-1.amazonaws.com]:587 YOUR_USERNAME:YOUR_PASSWORD

Secure it:

sudo chmod 600 /etc/postfix/sasl_passwd
sudo postmap /etc/postfix/sasl_passwd

Configure Postfix Main Settings

sudo vim /etc/postfix/main.cf

Replace entire contents with:

# Basic Settings
myhostname = mail.yourdomain.com
mydomain = yourdomain.com
myorigin = $mydomain
inet_interfaces = all
mynetworks = 127.0.0.0/8, 10.0.0.0/16
mydestination = 

# AWS SES Relay
relayhost = [email-smtp.ap-south-1.amazonaws.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

# TLS Security
smtp_use_tls = yes
smtp_tls_security_level = encrypt
smtp_tls_CAfile = /etc/ssl/certs/ca-bundle.crt

# Sender Validation
smtpd_sender_restrictions =
    check_sender_access hash:/etc/postfix/allowed_senders,
    reject

smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination

# Logging
maillog_file = /var/log/postfix/postfix.log

Create Sender Whitelist

sudo vim /etc/postfix/allowed_senders

Add approved senders:

info@yourdomain.com         OK
noreply@yourdomain.com      OK
@yourdomain.com             REJECT Not authorized

Compile and setup:

sudo postmap /etc/postfix/allowed_senders
sudo mkdir -p /var/log/postfix
sudo chown postfix:postfix /var/log/postfix

Start Postfix

sudo postfix check  # Should output nothing
sudo systemctl start postfix
sudo systemctl status postfix

Test it:

echo "Test" | mail -s "Test Email" -r info@yourdomain.com your-email@example.com
sudo tail -f /var/log/postfix/postfix.log

Look for status=sent (250 Ok...)

Phase 3: Event Pipeline (25 mins)

This is where we set up bounce/delivery tracking.

Create IAM Role

# Trust policy
cat trust-policy.json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "ec2.amazonaws.com"},
    "Action": "sts:AssumeRole"
  }]
}


# Create role
aws iam create-role \
  --role-name PostfixSESLogger \
  --assume-role-policy-document file://trust-policy.json

# Permissions policy
cat policy.json 
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "sqs:ReceiveMessage",
      "sqs:DeleteMessage",
      "sqs:GetQueueUrl"
    ],
    "Resource": "arn:aws:sqs:ap-south-1:*:ses-events-queue"
  }]
}


# Attach policy
aws iam put-role-policy \
  --role-name PostfixSESLogger \
  --policy-name SESLogging \
  --policy-document file:///policy.json

# Create instance profile
aws iam create-instance-profile --instance-profile-name PostfixSESLogger
aws iam add-role-to-instance-profile \
  --instance-profile-name PostfixSESLogger \
  --role-name PostfixSESLogger

# Attach to instance
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 associate-iam-instance-profile \
  --instance-id $INSTANCE_ID \
  --iam-instance-profile Name=PostfixSESLogger

Wait 10 seconds, then verify:

aws sts get-caller-identity  # Should show the role

Create SQS Queue

QUEUE_URL=$(aws sqs create-queue \
  --queue-name ses-events-queue \
  --region ap-south-1 \
  --query 'QueueUrl' \
  --output text)

QUEUE_ARN=$(aws sqs get-queue-attributes \
  --queue-url "$QUEUE_URL" \
  --attribute-names QueueArn \
  --region ap-south-1 \
  --query 'Attributes.QueueArn' \
  --output text)

echo "Queue URL: $QUEUE_URL"
echo "Queue ARN: $QUEUE_ARN"

Create SNS Topic and Subscribe SQS

SNS_ARN=$(aws sns create-topic \
  --name ses-events-topic \
  --region ap-south-1 \
  --query 'TopicArn' \
  --output text)

# Allow SNS to send to SQS
cat sqs-policy.json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "sns.amazonaws.com"},
    "Action": "sqs:SendMessage",
    "Resource": "$QUEUE_ARN",
    "Condition": {"ArnEquals": {"aws:SourceArn": "$SNS_ARN"}}
  }]
}


aws sqs set-queue-attributes \
  --queue-url "$QUEUE_URL" \
  --attributes Policy="$(cat /tmp/sqs-policy.json)" \
  --region ap-south-1

# Subscribe SQS to SNS
aws sns subscribe \
  --topic-arn "$SNS_ARN" \
  --protocol sqs \
  --notification-endpoint "$QUEUE_ARN" \
  --region ap-south-1

Configure SES to Publish Events

for EVENT in Delivery Bounce Complaint; do
  aws ses set-identity-notification-topic \
    --identity yourdomain.com \
    --notification-type $EVENT \
    --sns-topic "$SNS_ARN" \
    --region ap-south-1
done

# Disable email forwarding
aws ses set-identity-feedback-forwarding-enabled \
  --identity yourdomain.com \
  --no-forwarding-enabled \
  --region ap-south-1

Phase 4: Logger Deployment (30 mins)

Install Dependencies

sudo yum install -y python3-boto3
python3 -c "import boto3; print('✓ boto3 installed')"

Create Logger Script

sudo nano /usr/local/bin/ses_logger.py

Paste this complete script:

#!/usr/bin/env python3
import boto3, json, syslog, os, sys
from datetime import datetime

REGION = 'ap-south-1'
syslog.openlog('postfix/ses-events', logoption=syslog.LOG_PID, facility=syslog.LOG_MAIL)

def log_event(msg_id, event_type, recipient, details):
    log = f"{msg_id}: to=<{recipient}>, relay=amazonses.com, {details}"
    level = syslog.LOG_WARNING if event_type == "Bounce" else syslog.LOG_INFO
    syslog.syslog(level, log)
    print(f"[{datetime.now():%Y-%m-%d %H:%M:%S}] {event_type}: {log}")

def process_event(message):
    try:
        event = json.loads(message)
        event_type = event.get('notificationType')
        mail = event.get('mail', {})
        msg_id = mail.get('messageId', 'UNKNOWN')

        if event_type == 'Delivery':
            delivery = event.get('delivery', {})
            for recipient in delivery.get('recipients', []):
                delay = delivery.get('processingTimeMillis', 0)
                details = f"dsn=2.0.0, status=delivered, delay={delay}ms"
                log_event(msg_id, event_type, recipient, details)

        elif event_type == 'Bounce':
            bounce = event.get('bounce', {})
            for r in bounce.get('bouncedRecipients', []):
                details = f"dsn=5.0.0, status=bounced, type={bounce.get('bounceType')}"
                log_event(msg_id, event_type, r.get('emailAddress'), details)

        return True
    except Exception as e:
        print(f"Error: {e}", file=sys.stderr)
        return False

def main():
    queue_url = os.environ.get('SQS_QUEUE_URL')
    if not queue_url:
        sys.exit("ERROR: SQS_QUEUE_URL not set")

    print(f"SES Logger Started\nQueue: {queue_url}\n")
    sqs = boto3.client('sqs', region_name=REGION)

    while True:
        try:
            response = sqs.receive_message(
                QueueUrl=queue_url,
                MaxNumberOfMessages=10,
                WaitTimeSeconds=20
            )

            for message in response.get('Messages', []):
                body = json.loads(message['Body'])
                if process_event(body.get('Message')):
                    sqs.delete_message(
                        QueueUrl=queue_url,
                        ReceiptHandle=message['ReceiptHandle']
                    )
        except KeyboardInterrupt:
            break
        except Exception as e:
            print(f"Error: {e}", file=sys.stderr)

if __name__ == '__main__':
    main()

Make executable:

sudo chmod +x /usr/local/bin/ses_logger.py

Create Systemd Service

QUEUE_URL=$(aws sqs get-queue-url --queue-name ses-events-queue --region ap-south-1 --query 'QueueUrl' --output text)

sudo vim /etc/systemd/system/ses-logger.service
[Unit]
Description=SES Event Logger
After=network.target

[Service]
Type=simple
User=root
Environment="SQS_QUEUE_URL=$QUEUE_URL"
Environment="AWS_DEFAULT_REGION=ap-south-1"
ExecStart=/usr/bin/python3 /usr/local/bin/ses_logger.py
Restart=always
RestartSec=10
StandardOutput=append:/var/log/postfix/ses-logger.log
StandardError=append:/var/log/postfix/ses-logger-error.log

[Install]
WantedBy=multi-user.target

Start Logger

sudo systemctl daemon-reload
sudo systemctl enable ses-logger
sudo systemctl start ses-logger
sudo systemctl status ses-logger

Should show Active: active (running)

View logs:

sudo tail -f /var/log/postfix/ses-logger.log

Phase 5: Testing (20 mins)

Test 1: Complete Flow

Send test email:

echo "Test from infrastructure" | \
  mail -s "Test Email" -r info@yourdomain.com your-email@example.com

Watch both logs:

# Terminal 1 - Sent status (immediate)
sudo tail -f /var/log/postfix/postfix.log | grep "status=sent"

# Terminal 2 - Delivered status (10-30 sec delay)
sudo tail -f /var/log/postfix/mail.log | grep "status=delivered"

Expected:

# Postfix log (immediate):
status=sent (250 Ok 0109019c...)

# Mail log (after 10-30 seconds):
status=delivered, delay=3558ms

Test 2: Bounce Detection

# Use SES bounce simulator
echo "Bounce test" | \
  mail -s "Bounce Test" -r info@yourdomain.com bounce@simulator.amazonses.com

# Watch for bounce (1-2 mins)
sudo tail -f /var/log/postfix/mail.log | grep "bounced"

Expected: status=bounced, type=Permanent

Test 3: Sender Validation

# Try unauthorized sender
echo "Should fail" | \
  mail -s "Test" -r unauthorized@yourdomain.com test@example.com

# Check rejection
sudo tail /var/log/postfix/postfix.log | grep reject

Expected: Sender address rejected: Access denied

What You Built

In 2 hours, you created:

- Postfix SMTP relay with sender validation

- AWS SES integration with DKIM/SPF/DMARC

- Real-time tracking for delivery and bounces

- Unified logging - both "sent" and "delivered" in one place

- Cost-effective - ~$30/month vs $90+ for SaaS

Quick Reference

Restart services:

sudo systemctl restart postfix
sudo systemctl restart ses-logger

View logs:

sudo tail -f /var/log/postfix/postfix.log  # Sent
sudo tail -f /var/log/postfix/mail.log     # Delivered

Check queue:

mailq

Search for email:

grep "user@example.com" /var/log/postfix/*.log

Common Issues

Email stuck in queue?

# Check why
sudo tail -50 /var/log/postfix/postfix.log | grep deferred
# Flush after fixing
sudo postqueue -f

Logger not running?

# Check errors
sudo journalctl -u ses-logger -n 50
# Restart
sudo systemctl restart ses-logger

No delivery events?

# Check SQS has messages
aws sqs get-queue-attributes \
  --queue-url "YOUR_QUEUE_URL" \
  --attribute-names ApproximateNumberOfMessages

What's Next?

Read Part 3: Operations & Troubleshooting

🔗 If this helped or resonated with you, connect with me on LinkedIn. Let’s learn and grow together.

👉 Stay tuned for more behind-the-scenes write-ups and system design breakdowns.

Building Production Email Infrastructure with Postfix + AWS SES: Architecture & Design

Cyril Sebastian — Thu, 12 Mar 2026 18:23:20 +0000

Learn how to build a scalable, cost-effective email infrastructure using Postfix and AWS SES with complete bounce tracking. Part 1 covers architecture decisions, security, and design patterns.

Part 1: Architecture & Design Decisions

Series Navigation

Part 1: Architecture & Design ← You are here

Part 2: Implementation Guide

Part 3: Operations & Troubleshooting

TL;DR

What we're building: A production email system combining Postfix SMTP relay with AWS SES, complete with real-time bounce tracking, all visible in unified logs.

Why it matters: Track emails from send → delivery → bounce in one log file, works in private subnets, costs ~$30/month.

Who it's for: DevOps engineers, backend developers, and SREs managing email infrastructure.

Introduction

Have you ever sent an email and wondered: "Did it actually reach the inbox? Or did it bounce? When?"

Traditional SMTP relays tell you when they sent the email, but not when it was delivered. This gap creates a blind spot in your infrastructure.

What We're Building

An email infrastructure that provides:

Complete visibility: Both "sent" and "delivered" statuses
Real-time bounce tracking: Know immediately when emails fail
Unified logging: Everything in one log file (grep-friendly!)
Private subnet compatible: No public endpoints needed
Cost-effective: ~$30/month for 50,000 emails
Enterprise deliverability: AWS SES's 99.9% delivery rate

Who This Series Is For

DevOps Engineers looking to build a reliable email infrastructure

Backend Developers integrating email into applications

SREs need observability into email delivery

Series Overview

Part 1 (this post): Understand the architecture and design decisions

Part 2: Step-by-step implementation guide

Part 3: Operations, monitoring, and troubleshooting

The Problem: Email Observability

Traditional SMTP Relay Limitations

When you send an email through a standard SMTP relay:

postfix: status=sent (250 Ok)

But "sent" doesn't mean "delivered"! It just means your mail server handed the email to the next server.

What You Don't Know

- Did it reach the recipient's inbox?

- Did it bounce?

- Was it marked as spam?

- How long did the delivery take?

- Which ISP was slow?

This creates a visibility gap in your infrastructure.

Solution: The Best of Both Worlds

Your App → Postfix → SES → Recipient
     ↓         ↓        ↓
   Logs    Logs    SNS→SQS→Logger
                         ↓
                  Unified Logs ✅

This approach delivers:

Both statuses: "sent" (Postfix) AND "delivered" (SES)

Private subnet: No public endpoints required

Cost-effective: ~$30/month

Standard SMTP: No code changes needed

Unix-friendly: Logs searchable with grep/awk

Full control: Own your infrastructure

Architecture Overview

The Big Picture

╔══════════════════════════════════╗
║     Application Layer            ║
║  "Send email to user@example.com"║
╚════════════╤═════════════════════╝
             │ SMTP • Port 25
             ▼
╔══════════════════════════════════╗
║      Postfix (Private Subnet)    ║
║  ├─ ✓ Sender authorized          ║
║  ├─ ➡ Forwarding to SES...       ║
║  └─ 📋 LOG: status=sent          ║
╚════════════╤═════════════════════╝
             │ SMTP + TLS • Port 587
             ▼
╔══════════════════════════════════╗
║     AWS Simple Email Service    ║
║  "Delivering to recipient..."    ║
╚════════════╤═════════════════════╝
             │
     ┌───────┴───────┐
     ▼               ▼
╔════════════╗ ╔════════════════════╗
║           ║ ║    Event Flow     ║
║  Recipient ║ ║  SNS → SQS → Logger║
║  Mail Server║╚══════════╤═════════╝
╚════════════╝            │
                          ▼
                 ╔════════════════════╗
                ║    Your Logs ✓    ║
                ║  ├─   sent        ║
                ║  ├─   delivered   ║
                ║  └─    bounced     ║
                 ╚════════════════════╝

Data Flow Summary

Application sends email to Postfix via SMTP
Postfix validates sender, relays to SES
SES delivers email to recipient
SES publishes event (delivery/bounce) to SNS
SNS forwards event to SQS queue
Python logger polls SQS, writes to syslog
Result: Both "sent" and "delivered" in your logs!

Core Components Explained

1. Postfix: The Smart Relay

Role: SMTP relay with sender validation and forwarding logic

What it does:

Receives emails from your application
Validates sender addresses against the whitelist
Forwards to AWS SES via authenticated SMTP
Logs "sent" status immediately

Why Postfix?

Industry standard: Powers millions of servers

Highly configurable: Fine-grained control over routing

Excellent logging: Detailed, parseable log format

Battle-tested: Decades of production use

Performance: Handles thousands of concurrent connections

Key Configuration:

# Sender validation (whitelist)
smtpd_sender_restrictions = 
    check_sender_access hash:/etc/postfix/allowed_senders,
    reject

# Only approved senders can use relay
# Example whitelist:
# info@example.com    OK
# noreply@example.com OK
# @example.com        REJECT

Log Output Example:

Feb 25 00:05:15 mail postfix/smtp[123]: ABC123: 
  to=<user@example.com>, 
  relay=email-smtp.ap-south-1.amazonaws.com:587, 
  delay=0.14, 
  dsn=2.0.0, 
  status=sent (250 Ok 0109019c...)

What this tells you:

Postfix accepted the email
Email forwarded to SES
SES accepted the email (250 Ok)
Took 0.14 seconds

2. AWS SES: The Delivery Engine

Role: Actual email delivery to recipients

What it does:

Delivers emails to recipient mail servers
Handles DKIM signing for authentication
Manages IP reputation
Publishes delivery/bounce events

Why SES?

99.9% deliverability: Enterprise-grade infrastructure

Global reach: AWS's worldwide network

Pay-as-you-go: $0.10 per 1,000 emails

Built-in auth: Automatic SPF, DKIM, DMARC

Event publishing: Real-time delivery notifications

Scalable: From 100 to millions of emails

Event Types:

Delivery - Email reached the recipient's inbox
Bounce - Email rejected (permanent or temporary)
Complaint - Recipient marked as spam

Why not use SES directly?

While SES has an API, using Postfix as a relay provides:

Standard SMTP interface (no code changes)
Sender validation
Easy provider switching
Centralized configuration
Better logging

3. The Event Pipeline: SNS → SQS → Logger

This is the secret sauce that brings delivery confirmations into your logs.

SNS (Simple Notification Service)

Role: Event broadcaster from SES

Flow:

SES delivers email
    ↓
SES publishes event to SNS
    ↓
SNS broadcasts to subscribers

Why SNS?

Real-time notifications
Fan-out to multiple destinations
Native SES integration
Filter by event type

SQS (Simple Queue Service)

Role: Message buffer between SNS and logger

Sample Event:

{
  "notificationType": "Delivery",
  "mail": {
    "messageId": "0109019c...",
    "destination": ["user@example.com"]
  },
  "delivery": {
    "timestamp": "2026-02-25T00:05:18.000Z",
    "smtpResponse": "250 ok dirdel",
    "processingTimeMillis": 3558
  }
}

Why SQS instead of HTTP webhooks?

This is a critical design decision:

HTTP Webhook Approach:

SES → SNS → HTTP POST to your server
                      ↓
                Need public endpoint
                Need ALB/Load balancer  
                Security concerns
                Webhook authentication

SQS Polling Approach:

SES → SNS → SQS Queue
              ↓
      Python script polls (outbound only)
              ↓
        No public endpoint needed!
        Works in private subnet
        Messages buffered if logger down
        IAM-based authentication

Benefits of SQS:

Private subnet compatible: Polling is outbound-only

Resilient: Messages buffered for 14 days

No ALB needed: Saves $18/month

More reliable: No missed webhooks

IAM auth: No webhook secrets to manage

4. The Logger: Python + Syslog

Role: Poll SQS and write events to Postfix logs

What it does:

Polls SQS every 20 seconds (long polling)
Parses SES delivery/bounce events
Writes to syslog (same facility as Postfix)
Deletes processed messages from the queue

Code Snippet:

import boto3, syslog

# Initialize SQS client (uses IAM role automatically)
sqs = boto3.client('sqs', region_name='ap-south-1')

# Poll queue (long polling reduces API calls)
response = sqs.receive_message(
    QueueUrl=queue_url,
    MaxNumberOfMessages=10,
    WaitTimeSeconds=20  # Wait up to 20s for messages
)

# Process each event
for message in response.get('Messages', []):
    event = parse_ses_event(message)

    # Write to syslog (appears in Postfix logs!)
    syslog.openlog('postfix/ses-events', 
                   facility=syslog.LOG_MAIL)
    syslog.syslog(syslog.LOG_INFO, 
                  f"{msg_id}: to=<{recipient}>, "
                  f"status=delivered")

    # Remove from queue
    sqs.delete_message(...)

Why Syslog?

Same log file: Appears alongside Postfix logs

Auto-rotation: System handles log management

Searchable: Standard Unix tools (grep, awk)

Integration: Works with existing log aggregators

Familiar format: Same as Postfix log entries

Log Output:

Feb 25 00:05:19 mail postfix/ses-events[456]: 0109019c...: 
  to=<user@example.com>, 
  relay=amazonses.com, 
  dsn=2.0.0, 
  status=delivered, 
  delay=3607ms,
  response=(250 ok dirdel)

What this tells you:

Email successfully delivered
Took 3.6 seconds from SES to the inbox
Final SMTP response from recipient server

The Complete Email Journey

Let's trace a single email through the entire system with precise timings.

T+0ms: Application Sends Email

import smtplib
from email.mime.text import MIMEText

msg = MIMEText("Hello World")
msg['From'] = "info@example.com"
msg['To'] = "user@example.com"

s = smtplib.SMTP("10.0.0.23", 25)
s.sendmail("info@example.com", "user@example.com", msg.as_string())
s.quit()

What happens: SMTP connection to Postfix relay

T+10ms: Postfix Validates & Queues

Checks performed:

Is sender in whitelist? (info@example.com → OK)
Is sender from allowed network? (10.0.0.0/16 → OK)
Queue email for delivery

Log entries:

postfix/smtpd[123]: connect from ip-10-10-3-125
postfix/smtpd[123]: ABC123: client=ip-10-0-0-125
postfix/cleanup[124]: ABC123: message-id=<...>
postfix/qmgr[125]: ABC123: from=<info@example.com>, size=432

T+150ms: Postfix → SES Relay

Process:

Establish TLS 1.3 connection to SES
Authenticate with SMTP credentials
Transmit email content
Receive confirmation

Log entry (FIRST "sent" status!):

postfix/smtp[126]: Trusted TLS connection established to 
  email-smtp.ap-south-1.amazonaws.com:587: 
  TLSv1.3 with cipher TLS_AES_256_GCM_SHA384

postfix/smtp[126]: ABC123: to=<user@example.com>, 
  relay=email-smtp.ap-south-1.amazonaws.com:587, 
  delay=0.14, 
  status=sent (250 Ok 0109019c...)

What you know at this point:

Email left your infrastructure
SES accepted the email
Total time: 150ms

T+1500ms: SES Processes & Delivers

SES internal process:

Add DKIM signature
Perform SPF/DMARC checks
Select optimal sending IP
Connect to the recipient's mail server
Deliver email
Receive final confirmation

This happens entirely within AWS - you don't see these steps

T+1550ms: SES Publishes Event to SNS

Event generated:

{
  "notificationType": "Delivery",
  "mail": {
    "timestamp": "2026-02-25T00:05:15.140Z",
    "messageId": "0109019c...",
    "source": "info@example.com",
    "destination": ["user@example.com"]
  },
  "delivery": {
    "timestamp": "2026-02-25T00:05:18.698Z",
    "recipients": ["user@example.com"],
    "smtpResponse": "250 ok dirdel",
    "processingTimeMillis": 3558,
    "remoteMtaIp": "74.198.68.21",
    "reportingMTA": "a8-123.smtp-out.amazonses.com"
  }
}

Published to SNS topic: ses-events-topic

T+1600ms: SNS → SQS Forward

SNS wraps the event:

{
  "Type": "Notification",
  "MessageId": "...",
  "TopicArn": "arn:aws:sns:ap-south-1:...:ses-events-topic",
  "Message": "{\"notificationType\":\"Delivery\",...}",
  "Timestamp": "2026-02-25T00:05:18.750Z"
}

Delivered to SQS queue: ses-events-queue

T+5000ms: Logger Polls & Processes

Logger wakes up (polls every 20 seconds with long polling)

Process:

Retrieve message from SQS
Parse SNS wrapper
Extract SES event
Format for syslog
Write to log file
Delete message from queue

Log entry (SECOND "delivered" status!):

Feb 25 00:05:19 mail postfix/ses-events[456]: 0109019c...: 
  to=<user@example.com>, 
  relay=amazonses.com, 
  dsn=2.0.0, 
  status=delivered, 
  delay=3558ms,
  response=(250 ok dirdel)

What you know now:

Email delivered to inbox
Delivery took 3.5 seconds
Final confirmation from Gmail

Final Result: Unified Logs

Complete journey in logs:

# T+150ms - Postfix → SES
Feb 25 00:05:15 mail postfix/smtp[126]: ABC123: 
  to=<user@example.com>, 
  status=sent (250 Ok)

# T+5000ms - SES → Inbox confirmed
Feb 25 00:05:19 mail postfix/ses-events[456]: 0109019c...: 
  to=<user@example.com>, 
  status=delivered, 
  delay=3558ms

Search for any email:

grep "user@example.com" /var/log/postfix/*.log

Output:

postfix.log:  status=sent (handed to SES)
mail.log:     status=delivered (reached inbox)

Complete visibility from send to delivery!

Why Private Subnet?

Security Benefits:

Reduced attack surface: No public IP = can't be scanned

No direct internet access: Blocks many attack vectors

Network-level isolation: Additional security layer

AWS best practice: Recommended architecture

Compliance-friendly: Easier to meet security requirements

How it works:

┌─────────────────────────────────┐
│     Private Subnet             │
│                                 │
│  ┌──────────┐                  │
│  │ Postfix  │ (No public IP)   │
│  └────┬─────┘                  │
│       │ Outbound HTTPS only    │
└───────┼─────────────────────────┘
        │
        ↓ Via NAT Gateway
  ┌─────────────┐
  │   AWS SES   │
  │   AWS SQS   │
  └─────────────┘

Key point: Both SES and SQS are "pull" services:

Postfix initiates connection to SES (outbound)
Logger initiates connection to SQS (outbound)
No inbound connections needed!

SQS Polling Benefits:

SES → SNS → SQS ← Python polls (outbound only)

Simple infrastructure:

No public endpoints
No TLS cert management
No inbound firewall rules
IAM authentication (built-in)
Automatic retry (queue buffering)

Security Architecture

Network Security

Multi-layer defense:

┌─────────────────────────────────────┐
│        Private Subnet               │
│  ┌──────────────────────────┐      │
│  │ Security Group           │      │
│  │ - Port 25: 10.0.0.0/21 │      │
│  │ - Port 22: Admin IPs     │      │
│  │ - Outbound: All          │      │
│  │                          │      │
│  │   ┌──────────┐          │      │
│  │   │ Postfix  │          │      │
│  │   │ (Private)│          │      │
│  │   └────┬─────┘          │      │
│  └────────┼─────────────────┘      │
└───────────┼─────────────────────────┘
            │ Outbound only
            ↓ (TLS 1.3)
      ┌─────────────┐
      │   AWS SES   │
      │  (Public)   │
      └─────────────┘

Security controls:

Network isolation: Private subnet, no public IP
Sender validation: Whitelist checks before relay
Encryption: TLS 1.3 to SES
Authentication: SMTP credentials for SES

Additional Resources

AWS Documentation

Postfix Resources

Official Postfix Documentation

Email Authentication

DMARC.org

About This Series

This is Part 1 of a 3-part series on building production email infrastructure:

Part 1: Architecture & Design ← You just read this
Part 2: Implementation Guide - Step-by-step setup
Part 3: Operations & Troubleshooting - Day-to-day management

Next in series: Part 2: Implementation Guide

🔗 If this helped or resonated with you, connect with me on LinkedIn. Let’s learn and grow together.

👉 Stay tuned for more behind-the-scenes write-ups and system design breakdowns.

Run Docker Remotely with Portainer Instead of Docker Desktop

Cyril Sebastian — Tue, 29 Jul 2025 04:41:00 +0000

Managing Docker remotely, especially on a Linux server, is a powerful alternative to using Docker Desktop on your local machine. In this guide, we'll walk through how to:

Host and manage Docker containers remotely.
Use Portainer as a graphical interface to control Docker.
Replace Docker Desktop with a more flexible and lightweight remote workflow.

Whether you're a DevOps engineer, a cloud enthusiast, or simply exploring alternatives to Docker Desktop, this setup will enhance your productivity.

🧱 Why Portainer Instead of Docker Desktop?

Docker Desktop is great, but it has limitations:

Requires a GUI and lots of system resources.
Licensing costs for teams.
Tied to your local machine.

Portainer is:

Lightweight
Web-based
Platform-agnostic
Perfect for headless servers and remote access.

🗂 Step-by-Step Guide

1. Connect to Your Server

SSH into your remote server:

ssh your-user@your-server-ip

Switch to root if necessary:

sudo -i

2. Create a Directory for Docker Configs

Well organize Portainer and any other services under /opt/docker:

mkdir -p /opt/docker && cd /opt/docker

3. Prepare `portainer.yml` File

Create a Docker Compose file for Portainer:

vim portainer.yml

Paste the following content:

version: '3.8'

services:
  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    restart: unless-stopped
    ports:
      - "9000:9000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data

volumes:
  portainer_data:

Save and exit (Ctrl + O, Enter, Ctrl + X).

4. Start Portainer

Now launch Portainer with:

docker compose -f portainer.yml up -d

You should see:

 Container Portainer started

To follow logs:

docker compose -f portainer.yml logs -f

5. Access Portainer in a Browser

Open your browser and go to:

http://<your-server-ip>:9000

You'll be prompted to set up an admin password and connect to your local Docker environment (it'll detect the Docker socket you mapped).

🔍 Confirm It's Working

To check if Portainer is running:

docker ps

Expected output:

CONTAINER ID IMAGE PORTS NAMESxxxxxxx portainer/portainer-ce 0.0.0.0:9000->9000/tcp portainer

You can also view logs anytime:

docker logs portainer

Run Docker Remotely Like It's Local

You can run any Docker CLI commands remotely, without directly accessing your lab machine.

Option 1: Set a Permanent Remote Docker Context

Edit your .zshrc file on your Mac:

echo 'export DOCKER_HOST=ssh://username@remote_ip' >> ~/.zshrc source ~/.zshrc

Now, you can run any Docker CLI command like:

docker ps docker compose up -d docker logs <container>

and it'll execute on the remote machine automatically. No ssh, no fuss.

Conclusion

You now have a clean and efficient way to manage Docker remotely without relying on Docker Desktop. Using Portainer , you gain:

A lightweight web UI
Easy container and image management
Portability across systems
Remote team-friendly access

This approach is ideal for remote development, headless servers, or teams building CI/CD and DevOps pipelines.

Self-Hosted Streaming with Jellyfin and ngrok – A Personal Weekend Project

Cyril Sebastian — Thu, 10 Jul 2025 17:37:29 +0000

It all started with a simple need: I wanted to watch some old movies and personal family videos stored on my desktop. My niece had also backed up a bunch of vacation clips from her phone onto my machine, and now that she's back home, I needed an easy way to send them back to her without juggling USB drives or cloud uploads.

Being someone from a DevOps/SRE background, these small personal needs often spiral into fun infrastructure experiments. So, I figured, why not use this as a chance to self-host a media server?

That's when I turned to Jellyfin, a free, open-source media streaming solution. Paired with ngrok, I could securely share access with family and now had my very own private Netflix running directly from my Linux desktop. This was one of those side projects that started as a practical fix but turned into an unexpectedly enjoyable weekend build.

In this post, I'll walk you through exactly how I set it all up using Docker and ngrok, clean, simple, and DevOps-style.

📌 Why Jellyfin?

As someone who values open-source tools and self-hosted alternatives, Jellyfin hits the sweet spot:

Fully open-source
No telemetry or licensing headaches
Supports local media, subtitles, transcoding, and even user profiles

I had a folder full of personal and family videos collecting digital dust. Jellyfin gave them a Netflix-like interface without the surveillance.

🐳 Step 1: Run Jellyfin in Docker

To keep things clean and reproducible (DevOps mantra!), I went with Docker.

docker run -d \ --name jellyfin \ -p 8096:8096 \ -v /home/user/Documents/p2p:/media \ jellyfin/jellyfin

A few things to note:

-d runs it in the background
Port 8096 is Jellyfin's default web UI
The -v mount points my local media directory into the container at /media

Once the container spins up, hit http://localhost:8096 on your browser and follow the setup wizard. You'll be able to:

Create an admin account
Add your media libraries
Configure transcoding and user access

Simple and smooth.

🌍 Step 2: Access Jellyfin from Anywhere Using `ngrok`

Since I didn't want to mess with router port forwarding or dynamic DNS at home (and certainly not expose ports to the internet unsafely), ngrok was the perfect plug-and-play solution.

Install ngrok

wget https://bin.equinox.io/c/bNyj1mQVY4c/ngrok-v3-stable-linux-amd64.tgztar -xvf ngrok-v3-stable-linux-amd64.tgzsudo mv ngrok /usr/local/binngrok version

You'll need an ngrok account to get an auth token. Then:

ngrok config add-authtoken <YOUR_AUTH_TOKEN>

Create a Tunnel for Port 8096

ngrok http 8096

Boom! You'll get a public HTTPS URL https://abc123.ngrok.io that tunnels securely to your local Jellyfin instance.

Bonus: Protect it with Basic Auth

To prevent unauthorized access, use basic auth:

ngrok http -auth="username:password" 8096

Replace with your credentials. Now, even if someone stumbles upon the link, they'll need to authenticate first.

📦 Alternate: Run ngrok in Docker

If you like consistency (like me), you might prefer running ngrok in a container too:

docker run -it \ -e NGROK_AUTHTOKEN=<YOUR_AUTH_TOKEN> \ ngrok/ngrok http 8096

Optional: include -auth="username:password" in the command above if you want the same security via Docker.

🔒 A Quick Word on Security

This setup is intended for personal use. If you're planning to stream across multiple users or set up a family server, consider:

Running Jellyfin behind a proper reverse proxy (like Nginx)
Using a free domain with Let's Encrypt certs
Disabling public tunnels when not in use
Not exposing write access to the mounted media directory (read-only is safer)

🎯 Real-World Use Cases

Personal Netflix Clone Watch your ripped DVDs or archived home videos from anywhere.
Test Media Playback Over Slow Networks Useful if you're tuning transcoding profiles for a home server.
Portable Demos Great for showing media apps at meetups or events without deploying to a cloud server.
Media Backup Viewer Remote preview of a NAS or cold storage drive contents.

🧠 Final Thoughts

For anyone in DevOps, self-hosting isn't just about saving money. It's about owning your stack, learning through tinkering, and reusing familiar tools (like Docker, tunneling, and logs) in a low-stress, real-life scenario.

This Jellyfin + ngrok combo was a less than an hour-long weekend project, but the satisfaction of seeing your own media beautifully indexed and remotely accessible is real.

Give it a try. This might just become your favorite side gig for relaxing after a long sprint.

If you found this helpful or tried something similar, I'd love to hear about your setup, tweaks, or war stories. Leave a comment on Hashnode or connect via LinkedIn, I'm always happy to connect and geek out about self-hosting and home lab fun.

Happy streaming! 🎥🍿

ImageCredits: Photo by Marques Kaspbrak on Unsplash

My learnings after migrating the infrastructure from GCP to AWS.

Cyril Sebastian — Wed, 02 Jul 2025 13:57:41 +0000

Cyril Sebastian

Jun 30 '25

Planning a Cloud Migration? 10 Lessons from Production Cutovers

#aws #gcp #devops #awschallenge

Comments

5 min read

Planning a Cloud Migration? 10 Lessons from Production Cutovers

Cyril Sebastian — Mon, 30 Jun 2025 10:51:50 +0000

Moving a production-grade application serving millions of daily users from one cloud provider to another is a high-stakes operation. After executing a complete GCP to AWS migrationincluding 21TB of data, MongoDB replica sets, MySQL clusters, and Apache Solr search infrastructure, here are ten critical lessons that separate successful migrations from costly disasters. Connect with me on LinkedIn for more real-world DevOps insights.

1. Hidden Dependencies Surface During Cutover Weekend

Even though we had tools for the job, coordinating syncs across staging, pre-prod, and production environments required careful orchestration and monitoring. Your application architecture diagram rarely tells the complete story. During our migration, we discovered hardcoded GCP configurations buried deep in environment variables and application configs that weren't documented anywhere. Network monitoring weeks before cutover revealed missed communication patterns, including internal DNS dependencies that required AWS Route 53 private hosted zones to replicate GCP's automatic internal DNS resolution.

2. Data Transfer Costs Are a Hidden Budget Killer

Data Transfer Out from GCP was a hidden but substantial cost center; plan for this when budgeting. Our 21TB migration taught us that egress charges can dramatically exceed your initial estimates. We saved thousands of dollars by identifying GCS buckets containing temporary compliance logs with auto-expiry policies and choosing to let them expire in GCP rather than migrating unnecessary data. Plan for these costs early and audit your data to ensure migration is necessary.

3. Database Replication Lag Is Your Biggest Enemy

One of the major blockers encountered was replication lag during promotion drills, especially under active write-heavy workloads. Our MySQL master-slave setup experienced significant lag during peak traffic. The solution was implementing iptables-based rules at the OS level to block application write traffic, allowing replication to catch up safely before cutover. This gave us a clean buffer for promotion without the risk of in-flight transactions.

4. Storage Behavior Differs Dramatically Between Clouds

Lazy loading of EBS volumes made autoscaling unreliable for time-sensitive indexing. Our Apache Solr migration revealed that AWS EBS volumes suffer from lazy loading, meaning data wasn't instantly accessible upon instance boot-up. In GCP, persistent volumes are mounted seamlessly with boot disks, enabling autoscaling. In AWS, we had to abandon autoscaling for Solr and use scheduled start/stop scripts instead. Factor in rebuild times and whether AWS Fast Snapshot Restore fits your budget.

The IOPS and throughput planning complexity were another major difference. GCP used to take care of IOPS and throughput by choosing SSD disks, and increasing the disk size if more IOPS and throughput are required. But in AWS, we had to plan accordingly as per the usage, especially for databases. EBS gp3 volumes require explicit IOPS and throughput provisioning separate from storage capacity, meaning our database performance tuning became a multi-dimensional optimization problem rather than GCP's simpler disk-size-based scaling.

5. Load Balancer Architectures Require Complete Rethinking

The differences between GCP's global HTTPS Load Balancer and AWS Application Load Balancer go beyond simple configuration. GCP's URL maps allowed expressive, path-based routing across services. AWS required translating these into listener rules and target groups, often resulting in more granular configurations. We moved from static public IPs to CNAME-based routing, requiring DNS strategy adjustments and SSL certificate management changes through AWS Certificate Manager.

6. Security Models Force Architectural Changes

All EC2 instances (except load balancers) were placed in private subnets for enhanced security. We had to implement bastion host access, update CORS headers for CloudFront integration, and create explicit firewall rules using iptables to control MySQL access during migration. AWS's security group model required translating GCP firewall rules while adding WAF integration for DDoS protection.

7. Network Restrictions Create Unexpected Blockers

AWS restricts outbound SMTP traffic on port 25 by default to prevent abuse. This is not the case in GCP, so ensure to factor this into your cutover timeline if you're migrating mail servers. Our Postfix mail servers required explicit AWS Support requests to open port 25, adding weeks to our timeline.

Beyond port restrictions, we discovered that our maximum utilized servers with low configurations couldn't be placed on burstable VM instances like t3 due to CPU credit limitations and network throttling. High-traffic applications that ran smoothly on GCP's custom CPU/RAM configurations suffered performance degradation when mapped to AWS burstable instances. We had to carefully analyze baseline vs. burst performance patterns and move critical workloads to dedicated instance types like m6i or c6i to avoid throttling during peak loads.

8. Rollback Plans Must Account for Cloud-Specific Behaviors

We implemented iptables-based rules at the OS level to block application write traffic, allowing replication to catch up safely before cutover. Our rollback strategy included controlled write freezes and hybrid MongoDB nodes that acted as bridges between cloud environments. The key was testing promotion and demotion scenarios multiple times, not just hoping data backups would suffice.

9. Monitoring Blind Spots Emerge in Cloud Transitions

We experienced monitoring gaps during the most critical phases when existing tools didn't translate to the new environment. Setting up CloudWatch, maintaining Nagios compatibility, and ensuring Grafana dashboards worked across both environments simultaneously was crucial. Establish baseline performance metrics in both clouds and create real-time visibility before cutover day.

10. Post-Migration Optimization Is Where Real Value Emerges

This migration tested our nerves and processes, but ultimately, it left us with better observability, tighter security, and an infrastructure we could proudly call production-grade. After successful cutover, we rightsized EC2 instances using historical metrics, implemented Savings Plans for reserved workloads, and enabled S3 lifecycle policies. The migration wasn't just about changing providers, it forced us to modernize our entire infrastructure approach.

The Reality of Production Cutovers

No plan survives contact without flexibility. Our experience proved that even with meticulous planning, live production environments will surprise you. We faced MySQL promotion lags, discovered hardcoded configurations requiring emergency patches, and dealt with Solr performance issues under load. The key was having a flexible team ready to improvise while maintaining strict rollback readiness.

The most important lesson? Migration is more than lift-and-shift; it's evolve or expire. Successful cloud migration requires embracing architectural differences rather than fighting them. Plan thoroughly, test relentlessly, and be prepared to adapt quickly when reality differs from your runbook.

Your cloud migration is an opportunity to build better infrastructure, not just move existing problems to a new provider. Approach it as a chance to evolve your entire operational model, and you'll emerge stronger on the other side.

In Part 2 of my GCP to AWS migration blog series, I’m sharing the _unfiltered reality_—the hiccups, missed assumptions, and how we stabilized production in real time.

Cyril Sebastian — Mon, 09 Jun 2025 08:25:17 +0000

Cyril Sebastian

Jun 5 '25

GCP to AWS Migration – Part 2: Real Cutover, Issues & Recovery

#aws #gcp #devops #awschallenge

Comments

5 min read

Migrated from #GCP to #AWS recently and wrote about it. It's battle-tested architecture + transfer optimizations. Part 1: https://dev.to/cyrilsebastian/gcp-to-aws-migration-part-1-architecture-data-transfer-infrastructure-setup-oi7

Cyril Sebastian — Fri, 06 Jun 2025 05:18:58 +0000

GCP to AWS Migration – Part 2: Real Cutover, Issues & Recovery

Cyril Sebastian — Thu, 05 Jun 2025 08:28:12 +0000

🚀 Start of Part 2: The Real Cutover & Beyond

While Part 1 laid the architectural and data groundwork, Part 2 is where the real-world complexity kicked in.

We faced:

Database promotions that didnt go as rehearsed,
Lazy-loaded Solr indexes fighting with EBS latency,
Hardcoded GCP configs in the dark corners of our stack,
And the high-stakes pressure of a real-time production cutover.

If Part 1 was planning and theory, Part 2 was execution and improvisation.

Lets dive into the live switch, the challenges we didnt see coming, and how we turned them into lessons and long-term wins.

⚙️ Phase 4: Application & Infrastructure Layer Adaptation

As part of the migration, significant adjustments were required in both the application configuration and infrastructure setup to align with AWS's architecture and security practices.

Key Changes & Adaptations

Private Networking & Bastion Access
CORS & S3 Policy Updates
Application Configuration Updates
Internal DNS Transition
Static Asset Delivery via CloudFront
Security Hardening with WAF

Firewall Rules: Securing AWS MySQL from Legacy Sources

To ensure controlled access to the new MySQL server in AWS , we hardened the instance using explicit iptables rules. These rules:

Blocked direct MySQL access from legacy or untrusted subnets (e.g., GCP App subnets)
Allowed SSH access only from trusted bastion/admin IPs during the migration window

FIREWALL RULES FLOW:

[Blocked Sources] ──❌──┐
                        │
10.AAA.0.0/22     ──────┤
10.BBB.248.0/21   ──────┤
                        ├─── DROP:3306 ───┐
                        │                 │
[Allowed Sources] ──✅──┤                 ▼
                        │         ┌─────────────────┐
10.AAA.0.4/32     ──────┤         │ 10.BBB.CCC.223  │
10.BBB.248.158/32 ──────┤         │  MySQL Server   │
10.BBB.251.107/32 ──────┼─ACCEPT──│     (AWS)       │
10.BBB.253.9/32   ──────┤  :22    │                 │
                        │         └─────────────────┘

Legend:

10.AAA.x.x = Source network (GCP)
10.BBB.CCC.223 = Target MySQL server in AWS
IPs like 10.BBB.248.158 = Bastion or trusted admin IPs allowed for SSH

This rule-based approach gave us an extra layer of protection beyond AWS Security Groups during the critical migration phase.

🌐 Load Balancer Differences: GCP vs AWS

During the migration, we encountered significant differences in how load balancing is handled between GCP and AWS. This required architectural adjustments and deeper routing, SSL, and compute scaling planning.

📊 Comparison Overview

Feature	GCP HTTPS Load Balancer	AWS Application Load Balancer (ALB)
Scope	Global by default	Regional
TLS/SSL	Wildcard SSL was uploaded to GCP	Managed manually via AWS Certificate Manager (ACM)
Routing Logic	URL Maps	Target Groups with Listener Rules
IP Type	Static Public IP	CNAME with DNS-based routing
Backend Integration	Global Load Balancer → MIG (Managed Instance Groups)	ALB → Target Group → ASG (Auto Scaling Group)

🧩 Key Migration Notes

Static IP vs DNS Routing
Routing Mechanism Differences
SSL/TLS Certificates
Application-Specific Custom Rules

📮 Special Case: Postfix & Port 25 Restrictions

To migrate our Postfix mail servers that use port 25 for SMTP, we had to:

Submit an explicit request to AWS Support for port 25 to be opened (outbound) on our AWS account in the specific region.
This was a prerequisite for creating a Network Load Balancer (NLB) that could pass traffic directly to the Postfix instances.

Note : AWS restricts outbound SMTP traffic on port 25 by default to prevent abuse. This is not the case in GCP, so ensure to factor this into your cutover timeline if you're migrating mail servers.

🔍 Phase 5: Apache Solr Migration

Apache Solr powered our platform's search functionality with complex indexing and fast response times. Migrating it to AWS introduced both architectural and operational complexities.

🛠 Migration Strategy

AMI Creation Was Non-Trivial :

We created custom AMIs for Solr nodes with large EBS volumes. However, this surfaced two key challenges:
No AWS FSR :

AWS Fast Snapshot Restore (FSR) could have helpedbut was ruled out due to budget constraints. Without FSR, we observed delayed volume readiness post-launch.
Index Rebuild from Source DB :

Post-migration, we rebuilt Solr indexes from source data stored in MongoDB and MySQL, ensuring consistency and avoiding partial data issues.
Master-Slave Architecture :

We finalized a standalone Solr master-slave setup on EC2 after a dedicated PoC. This provided better control compared to GCP's managed instance groups.

🏗 GCP vs AWS Deployment Model

Feature	GCP MIGs	AWS EC2 Standalone
Deployment	Solr slaves ran in Managed Instance Groups	Solr nodes deployed on standalone EC2s
Volume Attachment	Persistent volumes mounted with boot disk	EBS volumes suffered from lazy loading, slowing boot
Autoscaling	Fully autoscaled Solr slaves based on demand	Autoscaling impractical due to volume readiness delays
Cost Management	On-demand scaling saved costs	Used EC2 scheduling (shutdown/startup) to control spend

⚡ Operational Decision: No Autoscaling for Solr in AWS

In GCP, autoscaling Solr slaves was seamless—new instances booted with attached volumes and joined the cluster dynamically.

However, in AWS:

Lazy loading of EBS volumes made autoscaling unreliable for time-sensitive indexing.

Instead, we:

Kept EC2 nodes in a fixed topology
Used scheduled start/stop scripts (via cron) to manage uptime during peak/off-peak hours

Lessons Learned

Solr migrations need deep consideration of disk behavior in AWS. If you're not using FSR, do not assume volume availability equals data availability. Factor in rebuild times, cost impact, and whether autoscaling truly benefits your workload.

🛑 The Cutover Weekend

We declared a deployment freeze 7 days before the migration to maintain stability and reduce last-minute surprises.

Pre-Cutover Checklist

TTL reduced to 60 seconds to allow quick DNS propagation.
Final S3 and database sync performed.
Checksums validated for critical data.
Route 53 routing policies configured to mimic GCPs internal DNS.
CloudWatch , Nagios, and Grafana set up for monitoring.
Final fallback snapshot captured.
A comprehensive cutover runbook was prepared with clear task ownership and escalation paths.

🕒 Cutover Timeline

Time Slot	Task
Hour 1	Final S3 + DB sync
Hour 2–3	DB failover and validation
Hour 4	DNS switch from GCP to Route 53
Hour 5–6	Traffic validation + rollback readiness

😮 Unexpected Issues (and What We Did)

Problem	Solution
MySQL master switch had lag	Improved replica promotion playbook
Hardcoded GCP configs found	Emergency patching of ENV & redeploy
Solr slow to boot under load	Temporarily pre-warmed EC2 nodes

🚀 Post-Migration Optimizations

Rightsized EC2 instances using historical metrics
Committed to Savings Plans for reserved workloads
Enabled and tuned S3 lifecycle policies
Set up automated AMI rotations and DB snapshots

🧠 End of Part 2: Final Thoughts & Whats Next

This journey from GCP to AWS wasnt just about swapping clouds, it was a masterclass in operational resilience , cross-team coordination , and cloud-native rethinking.

We learned that:

No plan survives contact without flexibility.
Owning your infrastructure also means owning your edge cases.
Migration is more than lift-and-shift, it's evolve or expire.

Smooth seas never made a skilled sailor.

Franklin D. Roosevelt

This migration tested our nerves and processes, but ultimately, it left us with better observability, tighter security, and an infrastructure we could proudly call production-grade.

🔗 If this helped or resonated with you, connect with me on LinkedIn. Let's learn and grow together.

👉 Stay tuned for more behind-the-scenes write-ups and system design breakdowns.

GCP to AWS Migration – Part 1: Architecture, Data Transfer & Infrastructure Setup

Cyril Sebastian — Tue, 03 Jun 2025 19:16:24 +0000

Migrating a production-grade, high-traffic application from Google Cloud to AWS is no small feat. Here's how our DevOps and engineering teams executed.

🧭 Why We Migrated: Business Drivers Behind the Move

Our platform, serving millions of daily users, was running smoothly on GCP. However, evolving business goals, pricing considerations, and long-term cloud ecosystem alignment led us to migrate to AWS.

Key components of our GCP-based stack:

Web Tier : Next.js frontend + Django backend
Databases : MongoDB replica sets, MySQL clusters
Asynchronous Services : Redis, RabbitMQ
Search : Apache Solr for full-text search
Infrastructure : GCP Compute Engine VMs, managed instance groups, HTTPS Load Balancer
Storage : 21 TB of data in Google Cloud Storage (GCS)

📋 Step 0: Creating a Migration Runbook

We treated this as a mission-critical project. Our runbook included:

Stakeholders : CTO, DevOps Lead, Database Architect, Application Owners
Timeline : 8 weeks from planning to cutover
Phases : Network Setup Data Migration Database Sync Application Migration Cutover
Rollback Plan : Prepared and rehearsed with timelines for failback

🛠 Infrastructure Mapping: GCP vs AWS

Challenges in Mapping:

GCP allows custom CPU and RAM, AWS uses fixed instance types (t3, m6i, r6g, etc)
IOPS differences between GCP SSDs and AWS EBS gp3 required tuning
Cost model varies significantly (especially egress from GCP)

We used AWS Pricing Calculator and GCP Pricing Calculator to simulate monthly billing and select cost-optimized instance types.

🌐 Phase 1: AWS Network Infrastructure Setup

AWS Network Infrastructure (ap-south-1)

                            ┌──────────────────────┐
                            │    GCP / DC VPC      │
                            └────────┬─────────────┘
                                     │
                           ┌─────────▼─────────┐
                           │ Site-to-Site VPN  │
                           └─────────┬─────────┘
                                     │
                           ┌─────────▼─────────┐
                           │      VPC          │
                           │  (ap-south-1)     │
                           └─────────┬─────────┘
                                     │
          ┌──────────────────────────┼─────────────────────────────┐
          │                          │                             │
┌─────────▼─────────┐     ┌──────────▼──────────┐       ┌──────────▼──────────┐
│ Public Subnet AZ1 │     │ Public Subnet AZ2   │       │ Public Subnet AZ3   │
│ - Bastion Host    │     │ - NAT Gateway       │       │ - Internet Gateway  │
└─────────┬─────────┘     └──────────┬──────────┘       └──────────┬──────────┘
          │                          │                             │
          ▼                          ▼                             ▼
┌────────────────┐        ┌────────────────┐              ┌────────────────┐
│Private Subnet 1│        │Private Subnet 2│              │Private Subnet 3│
│App / DB Tier   │        │App / DB Tier   │              │App / DB Tier   │
└────────────────┘        └────────────────┘              └────────────────┘

                 Security Groups + NACLs as per GCP mapping
                 VPC Flow Logs → CloudWatch Logs

🧩 Components Breakdown

📦 Phase 2: Data Migration (GCS S3)

We migrated over 21 TB of user-generated and application asset data from Google Cloud Storage (GCS) to Amazon S3. Given the scale, this phase required surgical precision in planning, execution, and cost control.

Tools & Techniques Used

AWS DataSync:

Chosen for its efficiency, security, and ability to handle large-scale object transfers.
Service Account HMAC Credentials :

Used for secure bucket-to-bucket authentication between GCP and AWS.
Phased Sync Strategy :

Note: We carefully validated checksums and object counts after each sync phase to ensure data integrity and avoid overwriting unchanged files.

💡 Smart Optimization Decisions

Selective Data Migration :
Delta Awareness :

Post-Migration S3 Tuning

After the bulk migration was completed, we fine-tuned our S3 environment for cost optimization, data hygiene , and long-term sustainability.

Lifecycle Policies Implemented :
Automatic archival of infrequently accessed data to S3 Glacier.
Expiry rules for:
Configured S3 Incomplete Multipart Upload Aborts :

Lessons Learned

Data Volume Data Complexity :

Even though we had tools for the job, coordinating syncs across staging, pre-prod, and production environments required careful orchestration and monitoring.
Egress and DTO Costs :

Data Transfer Out from GCP was a hidden but substantial cost center plan ahead for this when budgeting.
S3 Behavior Is Not GCS :

We had to adjust application logic and IAM policies post-migration to align with S3 object handling, access policies, and permissions model.

🗄 Phase 3: Database Migration

MongoDB Migration

Migrating MongoDB from GCP to AWS was one of the most sensitive components of the move due to its role in powering real-time operations and user sessions.

Our Strategy :

Replica Set Initialization : Set up MongoDB replica sets on AWS EC2 instances to mirror the topology running in GCP.
Oplog-Based Sync : Enabled oplog-based replication between AWS and GCP MongoDB nodes to ensure near real-time data synchronization without full data dumps.
Hybrid Node Integration : Deployed a MongoDB node in AWS , directly connected to the GCP replica set , acting as a bridge before full cutover.
iptables for Controlled Access : Used iptables rules to restrict write access during the sync period. This allowed inter-DB synchronization traffic only , blocking application-level writes and ensuring data consistency before switchover.
Failover Testing : Conducted multiple failover and promotion drills to validate readiness, with rollback plans in place.

Key Takeaway : Setting up a hybrid node and controlling access at the OS level allowed us to minimize data drift and test production-grade failovers without service disruption.

MySQL Migration

The MySQL component required careful orchestration to ensure transactional consistency and minimal downtime.

Our Approach :

Master-Slave Topology : Established a classic master-slave setup on AWS EC2 instances to replicate data from the GCP-hosted MySQL master.
Replication Lag Challenges : One of the major blockers encountered was replication lag during promotion drills, especially under active write-heavy workloads.
Controlled Write Freeze : We implemented iptables-based rules at the OS level to block application write traffic , allowing replication to catch up safely before cutover.
Promotion Strategy :

Key Takeaway : Blocking writes via iptables provided a clean buffer for promotion without the risk of in-flight transactions, making the cutover smooth and predictable.

End of Part 1: Setting the Stage for Migration

Youve seen how we architected an AWS environment from scratch, replicated critical systems like MongoDB and MySQL, and seamlessly migrated over 21 TB of assets from GCP to S3all while optimizing for cost, security, and scalability.

But this was just the calm before the storm.

"Give me six hours to chop down a tree and I will spend the first four sharpening the axe."

Abraham Lincoln

We were well-prepared. But would the systemsand the teamhold up during live cutover?

In Part 2: The Real Cutover & Beyond , well step into the fire:

What went wrong,
What we had to patch live,
And what we did to walk away from it stronger.

👉 Don't miss it. Follow me on LinkedIn for more deep-dive case studies and real-world DevOps/CloudOps stories like this.

Akamai Staging vs. Production: How to Set Up Environments Efficiently

Cyril Sebastian — Thu, 26 Dec 2024 12:39:14 +0000

Content Delivery Networks (CDNs) have become indispensable in modern web applications to ensure faster load times, enhanced security, and improved global performance. Among the leading CDN providers, Akamai stands out with its robust platform and mature features.

This article covers the basics of testing rules in a staging environment and deploying changes to the production environment after setting up rules in Akamai properties.

Why Use a CDN?

A CDN is a distributed network of servers that caches content closer to the end users. Here are the key reasons to use a CDN:

Reduced Latency : By serving requests from the nearest edge servers, CDNs significantly decrease load times.
Scalability : Seamlessly handles high traffic during peak usage by distributing the load.
Enhanced Security : Protects against DDoS attacks, provides Web Application Firewalls (WAF), and mitigates other security threats.
Cost Optimization : Reduces the burden on origin servers by caching static assets and efficiently managing traffic.

Best Practices for Routing Requests in Akamai.

When configuring Akamai, follow these best practices for optimal results:

Use Appropriate Cache Keys : Ensure caching is configured for specific query strings, cookies, or headers to serve accurate content.
Implement Cache Invalidation Policies : Use Akamais Content Purge to remove outdated content.

 akamai purge --edgerc ~/.edgerc --section default invalidate "www.example.com/path1" "www.example.com/path2"

Enable Gzip or Brotli Compression : Compress text-based assets (e.g., HTML, CSS, JS) to reduce bandwidth usage.
Strategic Traffic Routing : Utilize geo-blocking and geo-routing to enhance performance and minimize latency by routing users to the nearest server.
Monitor Cache Hit/Miss Ratios : Use the Akamais dashboard or logs to analyze and improve caching efficiency.

Setting Up Akamai Staging and Production Environments

Akamai allows testing configurations in a staging environment before deploying them live. Here's how to set it up:

1. Staging Environment Setup

The staging environment mimics production, ensuring safe testing. Follow these steps:

Determine the Staging URL : Add the Akamai -staging suffix to your Edge Hostname:

example.com.edgesuite-staging.net
example.com.edgekey-staging.net

Perform DNS Lookup : Use dig commands to resolve IPv4 and IPv6 addresses from staging Edge Hostname:

dig +short example.com.edgesuite-staging.net  
dig +short example.com.edgesuite-staging.net AAAA

Update /etc/hosts: Map the resolved IP address to the staging URL. Prefer IPv6 if IPv4 causes issues:

2600:XXXX:X:X::XXXX:XXXX example.com.edgesuite-staging.net

Test Staging with curl: Confirm the request routes to Akamais staging servers:

 curl -vvv -L \ 
 -H 'User-Agent: Safari: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Safari/605.1.15' \
 -H "Pragma: akamai-x-cache-on" -H "Pragma: akamai-x-check-cacheable" -H "x-akamai-internal: true" -H "Pragma: akamai-x-cache-remote-on" \ 
 -H "Pragma: akamai-x-get-cache-key" -H "Pragma: akamai-x-get-extracted-values" -H "Pragma: akamai-x-get-ssl-client-session-id" \
 -H "Pragma: akamai-x-get-true-cache-key" -H "Pragma: akamai-x-serial-no" -H "Pragma: akamai-x-get-request-id" -H "Pragma: X-Akamai-CacheTrack" \
 -I "https://example.com.edgesuite-staging.net"

2. Verifying Akamai Requests

Response Headers : Check for Akamai-specific headers:
- X-Cache: TCP_HIT (served from cache)
- X-Cache: TCP_MISS (fetched from origin server)
- X-Akamai-Staging: ESSL (indicates staging environment)
Browser Developer Tools :
- Open DevTools (Ctrl+Shift+I or Cmd+Option+I).
- Inspect response headers under the Network tab for Akamai-specific details.

3. Deploying to Production

Once testing is complete, promote the configuration to production via Akamai's property manager. Activating the changes ensures they are live without downtime.

Conclusion

Akamai's staging and production environments simplify the process of testing and deploying secure, high-performing web applications. By adhering to best practices like optimizing caching, compressing assets, and leveraging Akamai's robust tools, you can ensure your web application delivers a seamless experience to users globally.