DEV Community: Yashvi Kothari

Hands-On Lab Guide: L1 Support Engineer Skills

Yashvi Kothari — Mon, 17 Aug 2026 09:36:54 +0000

🛠️ Hands-On Lab Guide: L1 Support Engineer Skills

Lab 1: Windows Server + IIS Setup (Core L1 Skill)

Objective

Launch a Windows Server EC2, install IIS, deploy a test website, and practice troubleshooting.

Step-by-Step

1. Launch a Windows EC2 Instance

AWS Console → EC2 → Launch Instance

Name:              L1-Lab-Windows-IIS
AMI:               Microsoft Windows Server 2022 Base
Instance type:     t3.medium
Key pair:          Create new → "l1-lab-key" → Download .pem
Security Group:    Create new with these rules:
                   - RDP (port 3389) → My IP
                   - HTTP (port 80) → 0.0.0.0/0
                   - HTTPS (port 443) → 0.0.0.0/0

Click "Launch Instance"

2. Connect via RDP

Password:
6xr&;(LfLeaZyayHGSpjA9c@!OgA$xBF
Private IP address
172.31.7.190

1. Select your instance → Click "Connect"
2. Go to "RDP Client" tab
3. Click "Get password" → Upload your .pem key file → Decrypt
4. Download the RDP file
5. Open it with Remote Desktop Connection
6. Enter the decrypted password
7. You're now on a Windows Server!

3. Install IIS (Internet Information Services)

Open PowerShell as Administrator on the Windows Server:

# Install IIS with management tools
Install-WindowsFeature -Name Web-Server -IncludeManagementTools

# Verify IIS is running
Get-Service W3SVC

# Check IIS is serving the default page
Invoke-WebRequest -Uri http://localhost -UseBasicParsing | Select-Object StatusCode

4. Deploy a Test Website

# Navigate to the default website directory
cd C:\inetpub\wwwroot

# Backup the default page
Rename-Item .\iisstart.htm .\iisstart.htm.bak

# Create a custom test page
@"
<!DOCTYPE html>
<html>
<head><title>L1 Lab - Test Application</title></head>
<body>
    <h1>L1 Support Lab - Application Running</h1>
    <p>Server: $(hostname)</p>
    <p>Time: <span id="time"></span></p>
    <p>Status: <span style="color:green;">HEALTHY</span></p>
    <script>
        document.getElementById('time').textContent = new Date().toISOString();
    </script>
</body>
</html>
"@ | Out-File -FilePath .\index.html -Encoding UTF8

5. Verify — Open a browser and navigate to your EC2 Public IP:

http://<your-ec2-public-ip>

You should see your test page. ✅

🔥 Troubleshooting Exercises

Exercise A: Simulate "Website Down" (IIS Stopped)

# Stop IIS
Stop-Service W3SVC

# Now try accessing the website in browser — it fails!
# This simulates a P1 incident: "Website is down"

# How to diagnose:
Get-Service W3SVC  # Shows "Stopped"

# How to fix:
Start-Service W3SVC

# Verify:
Invoke-WebRequest -Uri http://localhost -UseBasicParsing | Select-Object StatusCode

Exercise B: Simulate "Port Blocked"

# Block port 80 in Windows Firewall
New-NetFirewallRule -DisplayName "Block HTTP" -Direction Inbound -LocalPort 80 -Protocol TCP -Action Block

# Website now unreachable from outside, but IIS is running!
# This simulates: "IIS is running but website not accessible"

# Diagnose:
Get-NetFirewallRule -DisplayName "Block HTTP"

# Fix:
Remove-NetFirewallRule -DisplayName "Block HTTP"

Exercise C: Check Event Viewer for Errors

# View recent IIS-related events
Get-EventLog -LogName System -Source "IIS*" -Newest 10

# View application errors
Get-EventLog -LogName Application -EntryType Error -Newest 20

# This is how you find error evidence for incident reports

Lab 2: Linux Server + Log Analysis (Core L1 Skill)

Objective

Launch a Linux EC2, practice essential commands, analyze logs, and simulate issues.

Step-by-Step

1. Launch a Linux EC2 Instance

AWS Console → EC2 → Launch Instance

Name:              L1-Lab-Linux
AMI:               Amazon Linux 2023
Instance type:     t3.micro (save your t3.medium for IIS)
Key pair:          Use same "l1-lab-key"
Security Group:    Allow SSH (port 22) from My IP

Click "Launch Instance"

2. Connect via SSH

From your local terminal (or use EC2 Instance Connect in the browser):

# If using terminal (convert .pem permissions first on Linux/Mac)
ssh -i "l1-lab-key.pem" ec2-user@<your-linux-public-ip>

# Or: AWS Console → Select Instance → Connect → EC2 Instance Connect → Connect

3. Essential L1 Commands — Practice Each One

# ===== SYSTEM INFORMATION =====

# Check who you are
whoami

# Check server hostname
hostname

# Check OS version
cat /etc/os-release

# Check uptime (how long server has been running)
uptime

# ===== CPU & MEMORY =====

# Check CPU usage (press 'q' to exit)
top

# Check memory usage
free -m

# Detailed memory info
cat /proc/meminfo | head -5

# ===== DISK SPACE =====

# Check disk usage (human-readable)
df -h

# Check which directories are using most space
du -sh /var/log/*

# ===== PROCESSES =====

# List all running processes
ps aux

# Find a specific process
ps aux | grep httpd

# Count running processes
ps aux | wc -l

# ===== NETWORK =====

# Check network interfaces
ip addr show

# Check open ports
ss -tlnp

# Test if a remote service is reachable
curl -I http://google.com

4. Log Analysis — The Core L1 Skill

# ===== VIEWING LOGS =====

# View system logs
sudo cat /var/log/messages | tail -50

# Follow logs in real-time (CRITICAL L1 SKILL)
sudo tail -f /var/log/messages
# (Press Ctrl+C to stop)

# Search logs for errors
sudo grep -i "error" /var/log/messages

# Search for specific timestamps
sudo grep "Aug 17" /var/log/messages

# Count how many errors occurred
sudo grep -ic "error" /var/log/messages

# ===== JOURNALCTL (systemd logs) =====

# View recent logs
sudo journalctl -n 50

# View logs since a specific time
sudo journalctl --since "1 hour ago"

# View logs for a specific service
sudo journalctl -u sshd -n 20

# Follow logs live
sudo journalctl -f

5. Simulate and Analyze Errors

# Create a fake application log for practice
sudo bash -c 'cat > /var/log/test-app.log << EOF
2026-08-17 10:00:01 INFO  Application started successfully
2026-08-17 10:00:02 INFO  Database connection established
2026-08-17 10:05:15 WARN  Slow query detected: 3200ms on /api/users
2026-08-17 10:05:16 WARN  Connection pool usage at 85%
2026-08-17 10:10:30 ERROR Database connection timeout after 30000ms
2026-08-17 10:10:31 ERROR java.sql.SQLException: Cannot acquire connection from pool
2026-08-17 10:10:32 ERROR HTTP 500 returned for GET /api/users/123
2026-08-17 10:10:33 ERROR HTTP 500 returned for POST /api/orders
2026-08-17 10:10:34 WARN  Retry attempt 1 for database connection
2026-08-17 10:10:35 WARN  Retry attempt 2 for database connection
2026-08-17 10:10:36 ERROR All retry attempts exhausted. Service degraded.
2026-08-17 10:15:00 INFO  Database connection re-established
2026-08-17 10:15:01 INFO  Service recovered. All endpoints healthy.
EOF'

# Now practice analysis:

# Find all errors
grep "ERROR" /var/log/test-app.log

# Find when errors started
grep "ERROR" /var/log/test-app.log | head -1

# Find when service recovered
grep "recovered" /var/log/test-app.log

# Count errors vs warnings
echo "Errors: $(grep -c 'ERROR' /var/log/test-app.log)"
echo "Warnings: $(grep -c 'WARN' /var/log/test-app.log)"

# Find the root cause pattern
grep -E "timeout|connection" /var/log/test-app.log

[!TIP]
Practice narrating your findings out loud:
"Errors started at 10:10:30. The root cause appears to be database connection timeout. The connection pool was already at 85% utilization at 10:05. This caused HTTP 500 errors on multiple endpoints. Service auto-recovered at 10:15 after database connection was re-established."
This is exactly how you'd communicate during a P1 bridge call.

Lab 3: AWS CloudWatch Monitoring (Monitoring Skill)

Objective

Set up CloudWatch dashboards and alarms for your EC2 instances.

Step-by-Step

1. View Default EC2 Metrics

AWS Console → CloudWatch → Metrics → All metrics

Click: EC2 → Per-Instance Metrics

Find your instances. Select:
  ✅ CPUUtilization
  ✅ NetworkIn
  ✅ NetworkOut
  ✅ StatusCheckFailed

Click "Graphed metrics" tab to see the charts

2. Create a CloudWatch Dashboard

CloudWatch → Dashboards → Create Dashboard

Name: "L1-Lab-Monitoring"

Add Widget → Line chart → Select:
  - EC2 > Per-Instance > CPUUtilization (for both instances)

Add Widget → Number → Select:
  - EC2 > Per-Instance > StatusCheckFailed

Add Widget → Line chart → Select:
  - EC2 > Per-Instance > NetworkIn + NetworkOut

Save Dashboard

3. Create a CloudWatch Alarm (P1 Simulation)

CloudWatch → Alarms → Create Alarm

Select Metric: EC2 → Per-Instance → CPUUtilization
Select your Windows IIS instance

Conditions:
  Threshold type: Static
  Greater than: 80
  Period: 1 minute
  Datapoints: 1 out of 1

Notification:
  Create new SNS topic: "L1-Lab-Alerts"
  Email: your-email@example.com

Alarm name: "High-CPU-IIS-Server"
Description: "CPU above 80% on IIS production server"

Create Alarm

4. Trigger the Alarm (Simulate High CPU)

RDP into your Windows Server and run:

# Simulate CPU load (will spike CPU to ~100%)
# Run this in PowerShell:
$cores = (Get-WmiObject Win32_Processor).NumberOfLogicalProcessors
1..$cores | ForEach-Object {
    Start-Job -ScriptBlock {
        $end = (Get-Date).AddMinutes(3)
        while ((Get-Date) -lt $end) { [math]::Sqrt(12345) | Out-Null }
    }
}

# Watch CPU spike in Task Manager
# Check CloudWatch — the alarm should trigger within 2-3 minutes

# Stop the load after testing:
Get-Job | Stop-Job
Get-Job | Remove-Job

5. Check the Alarm

CloudWatch → Alarms → "High-CPU-IIS-Server"

State should change: OK → In alarm

This is exactly what happens in production.
You'd see this alarm, investigate CPU usage, and open an incident.

[!NOTE]
In real-world L1 support, you'd receive this alarm via email/Slack/PagerDuty, investigate the cause (runaway process? traffic spike? deployment?), and either fix it or escalate.

Lab 4: REST API Troubleshooting with Postman + curl

Objective

Test APIs, understand HTTP status codes, and diagnose API failures.

Step-by-Step (No EC2 needed — use your local machine)

1. Install Postman
Download from: https://www.postman.com/downloads/

2. Test a Public API

Method: GET
URL: https://jsonplaceholder.typicode.com/posts/1

Click Send

Expected:
  Status: 200 OK
  Body: JSON with a post object (userId, id, title, body)

3. Test Different Status Codes

# 200 - Success
GET https://jsonplaceholder.typicode.com/posts/1

# 404 - Not Found
GET https://jsonplaceholder.typicode.com/posts/99999

# Test another API for different codes:
GET https://httpstat.us/200    → Returns 200
GET https://httpstat.us/401    → Returns 401 Unauthorized
GET https://httpstat.us/403    → Returns 403 Forbidden
GET https://httpstat.us/500    → Returns 500 Internal Server Error
GET https://httpstat.us/502    → Returns 502 Bad Gateway
GET https://httpstat.us/503    → Returns 503 Service Unavailable

4. Test POST Request

Method: POST
URL: https://jsonplaceholder.typicode.com/posts
Headers:
  Content-Type: application/json
Body (raw JSON):
{
  "title": "Test from L1 Lab",
  "body": "Testing API creation",
  "userId": 1
}

Click Send

Expected:
  Status: 201 Created
  Body: JSON with id: 101

5. Using curl from Command Line

# GET request
curl -v https://jsonplaceholder.typicode.com/posts/1

# POST request
curl -X POST https://jsonplaceholder.typicode.com/posts \
  -H "Content-Type: application/json" \
  -d '{"title":"Test","body":"Testing","userId":1}'

# Check only status code
curl -s -o /dev/null -w "%{http_code}" https://jsonplaceholder.typicode.com/posts/1

# Check response headers
curl -I https://jsonplaceholder.typicode.com/posts/1

# Test timeout behavior (useful for troubleshooting)
curl --max-time 5 https://httpstat.us/200?sleep=10000
# This will timeout after 5 seconds — simulates API timeout

6. Practice Scenario: "Client reports API not working"

Client says: "The /users endpoint is returning errors"

Your investigation steps:

1. Test the endpoint:
   GET https://jsonplaceholder.typicode.com/users

2. Check status code — is it 200, 401, 500, 503?

3. Check response time — Headers tab in Postman shows duration

4. Check response body — any error messages?

5. Test with different parameters:
   GET https://jsonplaceholder.typicode.com/users/1
   GET https://jsonplaceholder.typicode.com/users?email=Sincere@april.biz

6. Document your findings:
   "Tested /users endpoint at 14:30 UTC. 
    Status: 200 OK. 
    Response time: 245ms. 
    Data returned correctly for all test cases.
    Unable to reproduce client's issue.
    Requesting client to share:
    - Exact URL they're hitting
    - Request headers (especially Authorization)
    - Screenshot of the error
    - Browser/client they're using"

Lab 5: Writing an Incident Report + RCA

Objective

Practice writing a real incident report from the issues you simulated.

Template

# Incident Report

**Incident ID:** INC-2026-0817-001
**Priority:** P2 - High
**Status:** Resolved

## Summary
IIS web server on production EC2 instance became unreachable 
at 10:10 UTC on August 17, 2026. Root cause identified as 
IIS service failure. Service restored at 10:25 UTC.
Total downtime: 15 minutes.

## Timeline
| Time (UTC) | Event |
|------------|-------|
| 10:10 | CloudWatch alarm triggered: StatusCheckFailed |
| 10:12 | L1 engineer acknowledged the incident |
| 10:13 | Client notification sent: "Investigating reported downtime" |
| 10:15 | RDP to server confirmed IIS service stopped |
| 10:16 | Event Viewer checked — unexpected service termination logged |
| 10:18 | IIS service restarted |
| 10:20 | Website verified accessible |
| 10:22 | CloudWatch alarm returned to OK state |
| 10:25 | Client notified: "Service restored, monitoring for stability" |

## Root Cause
The IIS (W3SVC) service stopped unexpectedly due to [cause].
Event Viewer logs showed [specific error details].

## Impact
- All users accessing the web application were affected
- Duration: 15 minutes
- No data loss occurred

## Resolution
IIS service was restarted manually via PowerShell:
`Start-Service W3SVC`

## Preventive Measures
1. Configure IIS service auto-recovery (restart on failure)
2. Add CloudWatch alarm for W3SVC service status
3. Investigate root cause of service termination
4. Add automated health check endpoint monitoring

## Lessons Learned
- Need automated service recovery configuration
- CloudWatch alarm detected issue 2 minutes before client report
- Communication template was used effectively

[!TIP]
Write at least 3 RCA reports using different scenarios from these labs. This is a frequently asked skill in L1 support interviews.

Lab Checklist — Track Your Progress

#	Lab	Skill	Done?
1	Windows EC2 + IIS	Server management, IIS administration	☐
2	Linux EC2 + Logs	CLI commands, log analysis	☐
3	CloudWatch Monitoring	Metrics, dashboards, alarms	☐
4	Postman + curl	API testing, status codes	☐
5	Incident Report + RCA	Documentation, communication	☐

Quick Reference: Key Commands

Windows (PowerShell)

Get-Service W3SVC                    # Check IIS status
Start-Service W3SVC                  # Start IIS
Stop-Service W3SVC                   # Stop IIS
Restart-Service W3SVC                # Restart IIS
Get-EventLog -LogName System -Newest 20   # Recent system events
Get-Process | Sort-Object CPU -Desc | Select -First 10  # Top CPU processes
Test-NetConnection google.com -Port 443   # Test network connectivity

Linux (Bash)

top                    # CPU/Memory usage (live)
free -m                # Memory usage
df -h                  # Disk space
ps aux                 # Running processes
tail -f /var/log/messages   # Follow logs
grep "ERROR" logfile   # Search for errors
journalctl -f          # Follow system logs
ss -tlnp               # Open ports
curl -I url            # Check HTTP response
systemctl status httpd # Check service status

AWS CLI (Bonus)

# Describe your instances
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId,State.Name,PublicIpAddress]' --output table

# Get CloudWatch CPU metric
aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=YOUR_INSTANCE_ID \
  --start-time 2026-08-17T00:00:00Z \
  --end-time 2026-08-17T23:59:59Z \
  --period 300 \
  --statistics Average

[!IMPORTANT]
Sandbox Time Management: sandbox sessions expire. Prioritize:

Lab 1 (IIS) — takes ~30 min

Lab 2 (Linux) — takes ~20 min

Lab 3 (CloudWatch) — takes ~15 min

Labs 4 and 5 don't need the sandbox. Do them on your local machine anytime.

The L1 Support Engineer Nobody Talks About (But Every Company Desperately Needs)

Yashvi Kothari — Mon, 17 Aug 2026 08:52:29 +0000

The L1 Support Engineer Nobody Talks About (But Every Company Desperately Needs)

How to become the person who keeps the internet running at 3 AM — and why that's a career superpower.

Here's a truth most tech influencers won't tell you:

The fastest path into tech isn't learning to code. It's learning to troubleshoot.

While thousands of people are grinding LeetCode, a quieter army of L1 Support Engineers is keeping production systems alive for millions of users. They're the first responders of the digital world. No glory. No fanfare. Just a Slack notification at 2 AM and a client in London who can't access their dashboard.

And here's the kicker — the best ones get promoted faster than most junior developers.

What Is an L1 Support Engineer, Really?

Strip away the jargon. An L1 Support Engineer is a technical detective with a customer-service heart.

You're not writing the code. You're figuring out why the code broke. You're the person who:

Reads the error log when everyone else panics
Asks the right questions when the client says "everything is down"
Knows the difference between "the server is slow" and "the database connection pool is exhausted"

Naval would say: "An L1 engineer is leveraged. One person, monitoring systems that serve thousands."

Ogilvy would say: "The consumer isn't a moron. She's your client. Treat her with respect, even at 3 AM."

The 7 Skills That Separate Good L1 Engineers From Great Ones

1. You Understand the Stack (Even If You Didn't Build It)

You don't need to build a web application. You need to understand how one works:

Client (Browser) → DNS → Load Balancer → Web Server (IIS/Nginx) → Application → Database

When a client says "the app is broken," a great L1 engineer mentally walks this chain. Where did it break?

Practical Step: Set up a Windows Server on AWS with IIS. Deploy a simple web app. Break it intentionally. Fix it. That's the entire training.

2. You Can Read Logs Like a Detective Reads Crime Scenes

Logs are your evidence. Every error tells a story.

[2026-08-17 03:14:22 ERROR] System.Data.SqlClient.SqlException: 
Timeout expired. The timeout period elapsed prior to completion 
of the operation or the server is not responding.

A junior reads this and says: "Database error."
A great L1 reads this and says: "The database query is timing out — could be a long-running query, connection pool exhaustion, or the DB server itself is under load. Let me check CloudWatch metrics for the RDS instance."

Practical Step: SSH into a Linux server. Run tail -f /var/log/syslog. Break something. Watch the logs scream. Learn to hear what they're saying.

3. You Speak REST API

Every modern application talks in APIs. You need to understand the conversation.

Method	Purpose	Example
GET	Retrieve data	`GET /api/users/123`
POST	Create data	`POST /api/orders`
PUT	Update data	`PUT /api/users/123`
DELETE	Remove data	`DELETE /api/orders/456`

Status codes are your vocabulary:

200 — All good
401 — Authentication failed (wrong credentials)
403 — Forbidden (right credentials, wrong permissions)
404 — Resource not found
500 — Server broke internally
502 — Bad Gateway (upstream server failed)
503 — Service unavailable (server overloaded or down)

Practical Step: Open Postman. Hit a public API like https://jsonplaceholder.typicode.com/posts. Change the endpoint to something that doesn't exist. Observe the status codes. Now you speak the language.

4. You Know Enough SQL to Be Dangerous (In a Good Way)

You're not writing stored procedures. You're running SELECT queries to verify data.

-- Check if a user exists
SELECT * FROM Users WHERE Email = 'client@example.com';

-- Check recent orders
SELECT TOP 10 * FROM Orders ORDER BY CreatedDate DESC;

-- Check for failed transactions in the last hour
SELECT * FROM Transactions 
WHERE Status = 'FAILED' 
AND CreatedDate >= DATEADD(HOUR, -1, GETDATE());

Practical Step: Install SQL Server Management Studio (SSMS). Connect to a local or cloud database. Write 10 SELECT queries. You'll use these daily.

5. You Understand Environments (And Never Confuse Them)

This is where careers are saved or destroyed:

Development → QA/Testing → UAT (User Acceptance) → Staging → Production

The golden rule: You NEVER make changes directly in Production. Ever.

Each environment has its own:

Server/URL
Database
Configuration
Access controls

A real-world scenario: Client reports a bug. You reproduce it in UAT. You confirm it exists in Production. You do NOT fix it in Production. You log it, escalate it, and the fix flows through the pipeline.

6. You Master the Incident Priority Matrix

Not all fires are equal. Knowing which fire to fight first is the job.

Priority	Description	Example	SLA Response
P1	Critical — System down, all users affected	Production app completely inaccessible	15 minutes
P2	High — Major feature broken, many users affected	Payment processing failing	30 minutes
P3	Medium — Feature degraded, workaround exists	Reports loading slowly	4 hours
P4	Low — Minor issue, cosmetic or enhancement	Button color wrong on one page	24 hours

The communication framework during a P1:

1. Acknowledge → "We are aware of the issue and investigating."
2. Update (every 15-30 min) → "Root cause identified. The team is working on a fix."
3. Resolution → "The issue has been resolved. Services are restored."
4. RCA (within 24-48h) → "Root Cause Analysis report with preventive measures."

Practical Step: Write a mock RCA report. Take a real-world outage (AWS, Azure, any public postmortem) and rewrite their RCA in your own words. This is how you build the muscle.

7. You Monitor Before Things Break

The best L1 engineers don't wait for the phone to ring. They watch the dashboards.

Key metrics to watch:

CPU utilization — above 80% for sustained periods = trouble
Memory usage — if it keeps climbing without dropping, you have a memory leak
Disk space — databases and logs eat disk space silently
Response time — if your API goes from 200ms to 2000ms, something changed
Error rate — a sudden spike in 5xx errors means something broke
Active connections — connection pool exhaustion kills apps quietly

Tools of the trade:

AWS CloudWatch (if you're on AWS)
Azure Monitor / Application Insights (if you're on Azure)
Grafana + Prometheus (open-source, powerful)
New Relic / ELK Stack (enterprise-grade observability)

The Career Path Nobody Draws For You

L1 Support Engineer
    ↓
L2 Support Engineer (deeper troubleshooting, some scripting)
    ↓
    ├── DevOps Engineer (automation, CI/CD, infrastructure)
    ├── Cloud Engineer (AWS/Azure architecture)
    ├── SRE (Site Reliability Engineering)
    └── Solutions Architect (client-facing technical leadership)

Every great SRE I know started in support. Every great DevOps engineer I know started by reading logs at midnight. The path is real. The opportunity is real.

The Mindset Shift

Most people see L1 Support as a stepping stone. Something to survive.

The best engineers see it as a training ground.

You're learning:

How production systems actually behave (not how textbooks say they should)
How to communicate under pressure (a skill worth more than any certification)
How to think systematically when everything is on fire
How to own a problem from first ring to final resolution

Naval's principle applies perfectly here: "Specific knowledge is knowledge you cannot be trained for. It's found by pursuing your genuine curiosity."

An L1 Support role, done right, gives you the most specific knowledge in tech — how real systems fail, and how real teams fix them.

Your 30-Day Action Plan

Week	Focus	Actions
Week 1	Foundation	Set up AWS EC2 (Windows + Linux), install IIS, deploy a test app, learn basic CLI commands
Week 2	Monitoring & Logs	Set up CloudWatch dashboards, practice log analysis, learn to read error patterns
Week 3	APIs & Database	Use Postman to test APIs, write 20+ SQL queries, understand connection strings
Week 4	Incident Management	Write 3 mock RCA reports, practice client communication templates, simulate P1 response

The Bottom Line

The world doesn't need more people who can write "Hello World."

The world needs people who can figure out why "Hello World" stopped working at 3 AM, fix it, and explain what happened to a client in London who just lost $10,000 per minute of downtime.

That person is an L1 Support Engineer.

That person might be you.

If this helped you, share it with someone who's trying to break into tech. The support path is underrated, undershared, and wildly effective.

How to Think Like a System Administrator: The Practical Interview Playbook

Yashvi Kothari — Wed, 12 Aug 2026 11:33:27 +0000

At 11:00 PM, the phone rings.

A client says:

“The website is down.”

This is where a junior administrator starts guessing.

A strong system administrator starts asking questions.

Is the server reachable?

Is the web server running?

Is PHP-FPM healthy?

Is the database responding?

Is the disk full?

Did someone deploy something 10 minutes ago?

Are there errors in the logs?

The difference between a beginner and an experienced SysAdmin isn't how many commands they memorized.

It's how systematically they think when something breaks.

The SysAdmin Mental Model

Almost every production incident can be approached with the same pattern:

Verify → Isolate → Inspect → Fix → Verify → Prevent

Don't restart everything.

Don't randomly change configuration.

Don't immediately blame the application.

Find the failure.

Then fix the failure.

Scenario #1: The Website Is Down

Imagine a production website stops responding at 11 PM.

Your first step isn't:

sudo systemctl restart nginx

Your first step is verification.

Try:

curl -I https://example.com

Check connectivity:

ping SERVER_IP

Then connect to the server:

ssh user@SERVER_IP

Check the basic health:

uptime
free -h
df -h
top

Now inspect the web server:

systemctl status nginx

If you're using Apache:

systemctl status apache2

Then check logs.

For Nginx:

tail -50 /var/log/nginx/error.log

For PHP-FPM:

tail -50 /var/log/php8.2-fpm.log

If the site returns 502 Bad Gateway, investigate the upstream service.

For example:

systemctl status php8.2-fpm

Then check the database:

systemctl status mysql

And finally:

df -h

Because a surprisingly large number of “application problems” are actually disk-full problems.

The interview answer

Don't simply say:

“I would restart Nginx.”

Say:

“First I would verify the outage and determine whether it's isolated to one website or affects the entire server. Then I'd check server resources, web-server status, application services, database connectivity and logs. I would identify the root cause before making changes, verify the fix, and document the incident.”

That sounds like an administrator.

Scenario #2: The Server Is Slow

“It's slow” is not a diagnosis.

It's a symptom.

Start with:

uptime
free -h
htop

Check CPU:

ps aux --sort=-%cpu | head

Check memory:

ps aux --sort=-%mem | head

Check disk I/O:

iostat

Then investigate the web server.

How much traffic are you receiving?

Which IPs are generating the most requests?

For example:

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

Then investigate the database.

SHOW PROCESSLIST;

A slow website might actually be:

Nginx → PHP-FPM → MySQL → slow query

Restarting Nginx won't fix that.

This is why troubleshooting must follow the request path.

Scenario #3: SSL Certificate Expired

Users see:

“Your connection is not private.”

Check the certificate:

echo | openssl s_client \
-connect example.com:443 2>/dev/null \
| openssl x509 -noout -dates

For Let's Encrypt:

sudo certbot renew --dry-run

If renewal is required:

sudo certbot renew

Then validate Nginx before reloading:

sudo nginx -t
sudo systemctl reload nginx

The important interview point isn't knowing Certbot.

It's knowing the sequence:

Identify → Renew → Validate → Reload → Verify → Automate

And always test automatic renewal before assuming it works.

Scenario #4: Disk Is 95% Full

This is where many administrators make dangerous mistakes.

Don't immediately start deleting files.

First identify the problem.

df -h

Then:

sudo du -sh /* 2>/dev/null | sort -rh | head -20

Drill deeper:

sudo du -sh /var/* 2>/dev/null | sort -rh | head -20

Common culprits include:

Nginx/Apache logs
systemd journal logs
database binary logs
old backups
temporary files
application-generated files

Clean safely.

For example:

sudo apt clean
sudo journalctl --vacuum-time=7d

Then investigate log rotation.

The real solution isn't:

“I deleted some files.”

It's:

“I identified the source of disk growth and implemented log rotation, backup retention and monitoring so the problem doesn't return.”

That's the difference between incident response and systems administration.

Scenario #5: Database Connection Errors

Multiple websites suddenly report:

Cannot connect to database.

Start with:

sudo systemctl status mysql

Check the logs:

sudo tail -50 /var/log/mysql/error.log

Check disk:

df -h

Check memory:

free -h

Look for OOM events:

dmesg | grep -i "oom\|killed process"

If MySQL is running but applications can't connect, investigate:

credentials
database existence
user permissions
connection limits
host restrictions
application configuration

Test directly:

mysql -u appuser -p -h localhost database_name

Then:

SHOW STATUS LIKE 'Threads_connected';

And:

SHOW GRANTS FOR 'appuser'@'localhost';

Again, don't jump to:

systemctl restart mysql

Understand why connections are failing.

Scenario #6: Migrating 15 Websites

This is where planning matters more than commands.

A migration should look something like:

Before migration

Inventory:

domains
DNS records
databases
application versions
PHP versions
web-server configuration
SSL certificates
cron jobs
firewall rules
backups

Lower DNS TTL before migration.

For example:

300 seconds

Prepare the new server.

Then perform an initial data synchronization.

For example:

rsync -avz /var/www/ user@NEW_SERVER:/var/www/

Import databases.

Test websites against the new server before changing DNS.

A useful technique is testing through the local hosts file.

Then:

Final sync
Final database migration
DNS change
Verify applications
Monitor logs
Keep the old server available as rollback

The goal isn't simply:

“Move the websites.”

The goal is:

Move the websites while minimizing risk and maintaining a rollback path.

Scenario #7: The Server Is Under Attack

First determine what you're dealing with.

Look at traffic:

awk '{print $1}' /var/log/nginx/access.log \
| sort | uniq -c | sort -rn | head -20

Check connections:

ss -s

For HTTP connections:

ss -ant | grep ':80' | wc -l

But remember:

Not every traffic spike is a DDoS attack.

It could be:

legitimate traffic
a broken client
a crawler
a brute-force attack
an application bug
a real DDoS

For application-layer attacks, rate limiting can help.

For larger attacks, move protection upstream.

A service such as Cloudflare can provide:

WAF
rate limiting
bot protection
DDoS mitigation

The key interview answer:

“I would identify the attack pattern first, apply the least disruptive mitigation available, involve the provider or upstream DDoS protection when necessary, and then implement permanent controls.”

The 10 Commands I Want Every SysAdmin to Know

If you remember nothing else, remember these:

uptime
free -h
df -h
htop
ss -tlnp
systemctl status nginx
journalctl -u nginx
nginx -t
tail -f /var/log/nginx/error.log
lsof -i :80

They answer some of the most important questions:

Is the server healthy?

uptime
free -h
df -h

What's consuming resources?

htop

What's listening?

ss -tlnp

Is the service healthy?

systemctl status nginx

What went wrong?

journalctl -u nginx

Is my configuration valid?

nginx -t

What's happening right now?

tail -f /var/log/nginx/error.log

What's using the port?

lsof -i :80

The Two-Week Interview Plan

Don't spend two weeks just reading.

Build.

Week 1

Day 1: Linux commands

Day 2: Users, permissions and SSH

Day 3: Apache

Day 4: Nginx

Day 5: DNS

Day 6: SSL

Day 7: PHP and PHP-FPM

Week 2

Day 8: MySQL/MariaDB

Day 9: Monitoring

Day 10: Firewall and security

Day 11: Backups and cron

Day 12: Docker

Day 13: Cloud and Git

Day 14: Scenario practice

Every day:

1 hour theory

2 hours hands-on

1 hour interview practice

And don't just read answers.

Say them out loud.

Build Your Own Production-Like Lab

A cheap Ubuntu VM is enough to learn the fundamentals.

Install:

Nginx
PHP-FPM
MySQL
Docker
Git
UFW
Fail2ban

Then deliberately break things.

Stop Nginx.

Fill the disk.

Break a configuration file.

Stop PHP-FPM.

Change a database password.

Break DNS.

Expire a certificate.

Kill a process.

Then recover.

That is where real learning happens.

The Golden Rule of Troubleshooting

When something breaks:

Don't panic.

Don't guess.

Don't restart everything.

Use:

Verify → Isolate → Inspect → Fix → Verify → Prevent

And remember five things interviewers want to hear:

1. Systematic thinking

You investigate before changing things.

2. Security

You don't use root unnecessarily.

You don't expose databases publicly.

You don't give developers root access.

3. Backups

Before risky changes:

Have a rollback plan.

4. Communication

During incidents, stakeholders need updates.

5. Prevention

Fixing today's problem isn't enough.

Ask:

“How do we make sure this doesn't happen again?”

That's what separates a command runner from a system administrator.

Final Interview Advice

When asked:

“Tell me about your experience.”

Don't give a list of technologies.

Tell a story.

Instead of:

“I know Linux, Nginx, MySQL, AWS and Docker.”

Say:

“I manage Linux-based servers and web applications, including Nginx/Apache, PHP-FPM and databases. I handle deployments, SSL, DNS, backups, monitoring and security. When incidents occur, I troubleshoot systematically using logs, system metrics and service health checks, then focus on preventing the same issue from happening again.”

Technology tells them what you know.

Problem-solving tells them why they should hire you.

The best SysAdmins aren't the people who know the most commands.

They're the people who can stay calm at 11 PM, find the signal inside the noise, restore service safely, and explain exactly what happened afterward.

Learn the commands.

Practice the scenarios.

Build the habit of thinking in systems.

That's how you become interview-ready.

Part 7: Scenario Questions, Study Plan & Quick Reference

7.1 Scenario-Based Interview Questions (MOST IMPORTANT)

[!IMPORTANT]
Interviewers love scenario questions because they test real-world problem-solving. Practice explaining these OUT LOUD.

Scenario 1: Website is Down

Q: "It's 11 PM and a client reports their website is down. Walk me through your troubleshooting process."

Answer:

1. VERIFY the issue
   - Try accessing the site myself from browser + curl
   - Check if it's site-specific or all sites on the server
   - Ping the server IP

2. CHECK server connectivity
   - SSH into the server (if can't → check provider's console)
   - Run: uptime, free -h, df -h, top

3. CHECK web server
   - systemctl status nginx (or apache2)
   - If stopped → systemctl start nginx → check logs for why it stopped

4. CHECK error logs
   - tail -50 /var/log/nginx/error.log
   - tail -50 /var/log/php8.2-fpm.log

5. CHECK PHP-FPM (if PHP site)
   - systemctl status php8.2-fpm
   - If 502 → restart PHP-FPM

6. CHECK database
   - systemctl status mysql
   - If site shows DB error → restart MySQL, check credentials

7. CHECK disk space
   - df -h → if full → find and clean large files/logs

8. CHECK for recent changes
   - Did someone deploy code? Change config? Update server?

9. COMMUNICATE
   - Update client with status and ETA
   - Document the incident

Scenario 2: Server is Slow

Q: "Users report the website is very slow. How do you investigate?"

Answer:

1. CHECK server resources
   - htop → CPU usage, RAM usage, load average
   - free -h → is swap being used heavily?
   - iostat → disk I/O bottleneck?

2. CHECK what's consuming resources
   - ps aux --sort=-%cpu | head → top CPU processes
   - ps aux --sort=-%mem | head → top RAM processes

3. CHECK web server
   - Access log analysis: are there too many requests? Bot traffic?
   - awk '{print $1}' access.log | sort | uniq -c | sort -rn | head

4. CHECK database
   - SHOW PROCESSLIST → stuck/slow queries?
   - Enable slow query log
   - Check if indexes are missing

5. CHECK PHP
   - PHP-FPM status → are all workers busy?
   - Check PHP error log for warnings
   - Check if OPcache is enabled

6. CHECK network
   - Is bandwidth maxed? → nload, vnstat
   - Is there a DDoS attack? → check access logs for patterns

7. QUICK FIXES
   - Kill stuck processes
   - Restart PHP-FPM (clears memory leaks)
   - Enable caching (Redis, OPcache, Nginx cache)
   - Block abusive IPs with fail2ban or UFW

8. LONG-TERM
   - Scale server (more CPU/RAM)
   - Add CDN (Cloudflare)
   - Optimize database queries
   - Implement application-level caching

Scenario 3: SSL Certificate Expired

Q: "A client's SSL certificate has expired and users are seeing security warnings. What do you do?"

Answer:

1. IDENTIFY which certificate
   - echo | openssl s_client -connect domain.com:443 2>/dev/null | openssl x509 -noout -dates

2. RENEW the certificate
   - If Let's Encrypt: sudo certbot renew --force-renewal
   - If paid cert: generate new CSR, submit to provider, download new cert

3. INSTALL/VERIFY
   - sudo nginx -t && sudo systemctl reload nginx
   - Verify in browser: check padlock icon

4. PREVENT recurrence
   - Verify certbot timer: sudo systemctl status certbot.timer
   - Or set up monitoring: SSL expiry alert 30 days before
   - Test auto-renewal: sudo certbot renew --dry-run

Scenario 4: Disk Space Full

Q: "You receive an alert that the server's disk is 95% full. What steps do you take?"

Answer:

1. IDENTIFY what's consuming space
   - df -h (which partition is full?)
   - du -sh /* | sort -rh | head -20
   - Then drill down: du -sh /var/* | sort -rh | head -20

2. QUICK CLEANUP
   - sudo apt clean                    # Clear apt cache
   - sudo journalctl --vacuum-time=7d  # Clear old systemd logs
   - find /var/log -name "*.gz" -delete # Delete compressed old logs
   - find /tmp -type f -mtime +7 -delete # Clean temp files
   - Check /var/log for huge log files → truncate or rotate

3. CHECK for common culprits
   - Large log files: /var/log/apache2/access.log, error.log
   - Old backups still on server
   - MySQL binary logs: PURGE BINARY LOGS BEFORE '2024-01-01'
   - Mail queue: mailq → clear if large

4. PREVENT recurrence
   - Set up logrotate for all logs
   - Move backups to S3/remote storage
   - Set up disk space alert at 80%
   - Add cron to clean old files

Scenario 5: Database Connection Errors

Q: "Multiple websites are showing 'Cannot connect to database'. How do you fix this?"

Answer:

1. CHECK MySQL status
   - sudo systemctl status mysql
   - If stopped → sudo systemctl start mysql
   - Check: sudo tail -50 /var/log/mysql/error.log

2. IF MySQL won't start
   - Check disk space: df -h
   - Check RAM: free -h (OOM killer may have killed MySQL)
   - Check: sudo dmesg | grep -i "oom\|mysql"
   - Try: sudo mysqld_safe &

3. IF MySQL is running but connections fail
   - Check max connections: SHOW STATUS LIKE 'Threads_connected';
   - If maxed: kill idle connections, increase max_connections
   - Check credentials in website config files (.env, wp-config.php)
   - Test: mysql -u appuser -p -h localhost database_name

4. IF one site works but another doesn't
   - Check database-specific credentials
   - Check if specific database exists
   - Check user permissions: SHOW GRANTS FOR 'user'@'localhost';

Scenario 6: Server Migration

Q: "We need to migrate 15 websites from an old server to a new server with zero downtime. How?"

Answer:

1. PLAN (1-2 days before)
   - Inventory: list all domains, databases, configs, SSL certs
   - Set up new server with same software stack
   - Lower DNS TTL to 300 seconds (do this early!)

2. PREPARE NEW SERVER
   - Install: OS updates, Nginx/Apache, PHP (same version), MySQL
   - Configure: firewall, SSH, fail2ban
   - Set up same PHP extensions and configurations

3. SYNC DATA (night before)
   - rsync website files (first sync - bulk transfer)
   - Export all databases
   - Copy SSL certificates, configs, crontabs

4. MIGRATION DAY
   - Final rsync (only changed files, very fast)
   - Final database dump and import
   - Copy latest configs
   - Test each site on new server (use hosts file to test)

5. DNS SWITCH
   - Update A records to new server IP
   - Monitor propagation: dnschecker.org

6. VERIFY
   - Test all 15 sites on new server
   - Test SSL, forms, databases, email
   - Monitor error logs for 24-48 hours

7. POST-MIGRATION
   - Keep old server running 7 days as fallback
   - Raise DNS TTL back to normal
   - Update all documentation

Scenario 7: Server Under Attack

Q: "The server is being hit by a DDoS attack. What do you do?"

Answer:

1. IDENTIFY the attack
   - Check access logs for patterns:
     awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20
   - Check: ss -s (total connections)
   - Check: netstat -an | grep :80 | wc -l

2. IMMEDIATE ACTIONS
   - Block top attacking IPs:
     sudo ufw deny from ATTACKER_IP
   - Enable rate limiting in Nginx:
     limit_req_zone $binary_remote_addr zone=one:10m rate=10r/s;

3. CLOUDFLARE (if available)
   - Enable "Under Attack Mode"
   - Set up rate limiting rules
   - Block suspicious countries/ASNs
   - Enable Bot Fight Mode

4. SERVER-LEVEL
   - Configure fail2ban for HTTP flood
   - Increase worker_connections in Nginx
   - Enable connection limiting

5. LONG-TERM
   - Set up Cloudflare on all domains
   - Configure WAF rules
   - Set up monitoring alerts for traffic spikes

Scenario	Key Actions
PHP upgrade needed	Install new PHP alongside old, test sites, update Nginx/Apache fastcgi_pass, remove old version
Email not being delivered	Check SPF/DKIM/DMARC, check mail logs, check blacklist status (mxtoolbox), verify port 25/587 open
Cron job not running	Check `grep CRON /var/log/syslog`, verify path, check permissions, test command manually
MySQL table crashed	`REPAIR TABLE tablename`, or restore from backup if repair fails
Server rebooted unexpectedly	Check `last reboot`, check `dmesg`, check `/var/log/syslog`, check OOM killer, check provider status
New website deployment	Create vhost, set up DNS, create DB + user, deploy code, fix permissions, install SSL, test
Developer needs database access	Create MySQL user with limited permissions, never give root. `GRANT SELECT, INSERT, UPDATE on db.* TO 'dev'@'ip'`
Backup restoration needed	Verify backup integrity, create fresh DB, import SQL, update config, fix permissions, test

7.2 Two-Week Study Plan (Zero to Interview-Ready)

Week 1: Core Skills

Day	Topic	Practice
Day 1	Linux commands, file operations	Set up Ubuntu VM, practice all commands from Part 1
Day 2	Users, permissions, SSH	Create users, set permissions, configure SSH keys & hardening
Day 3	Apache configuration	Install Apache, create 3 virtual hosts, set up .htaccess
Day 4	Nginx configuration	Install Nginx, create 3 server blocks, set up reverse proxy
Day 5	DNS & Domains	Set up a domain with all record types (use free domains or Cloudflare)
Day 6	SSL Certificates	Install Let's Encrypt on both Apache & Nginx, practice troubleshooting
Day 7	PHP & PHP-FPM	Install PHP, configure php.ini, set up PHP-FPM with Nginx

Week 2: Advanced Skills

Day	Topic	Practice
Day 8	MySQL/MariaDB	Install, create databases/users, backup/restore, slow query log
Day 9	Server monitoring	Practice htop, disk analysis, log analysis, set up monitoring
Day 10	Firewall & Security	Configure UFW, install fail2ban, SSH hardening, security audit
Day 11	Backup & Cron	Write backup scripts, set up cron jobs, test restore
Day 12	Docker basics	Install Docker, run containers, create docker-compose.yml
Day 13	Cloud & Git	Deploy on DigitalOcean, set up Git deployment workflow
Day 14	Scenario practice	Review all scenarios, practice explaining solutions out loud

Daily Routine

Morning (1 hour):   Read theory for today's topic
Afternoon (2 hours): Hands-on practice on VM/VPS
Evening (1 hour):    Review interview Q&A, practice explaining answers

7.3 Hands-On Lab Setup

Option A: Local VM (Free)

# 1. Download VirtualBox: virtualbox.org
# 2. Download Ubuntu Server 22.04 LTS ISO
# 3. Create VM: 2 CPU, 2GB RAM, 20GB disk
# 4. Install Ubuntu Server
# 5. Practice everything locally

# Network setup for SSH from host:
# VirtualBox → Settings → Network → Adapter 1 → Bridged Adapter
# OR Port Forwarding: Host 2222 → Guest 22
ssh -p 2222 user@localhost

Option B: DigitalOcean (Paid but realistic)

# 1. Sign up at digitalocean.com (get $200 free credit with referral)
# 2. Create a $6/month Droplet: Ubuntu 22.04 LTS
# 3. SSH in: ssh root@YOUR_DROPLET_IP
# 4. Practice everything on real cloud server
# 5. Destroy when done to stop billing

7.4 Critical Commands Cheat Sheet

# === SERVER STATUS ===
uptime                          # Uptime + load average
free -h                         # Memory usage
df -h                           # Disk space
htop                            # Interactive process viewer
ss -tlnp                        # Listening ports

# === SERVICE MANAGEMENT ===
systemctl status nginx          # Check service
systemctl restart nginx         # Restart service
systemctl reload nginx          # Reload config (zero downtime)
systemctl enable nginx          # Start on boot
journalctl -u nginx -f          # Follow service logs

# === WEB SERVER ===
nginx -t                        # Test Nginx config
apache2ctl configtest           # Test Apache config
tail -f /var/log/nginx/error.log       # Follow error log

# === DNS ===
dig example.com                 # DNS lookup
dig @8.8.8.8 example.com       # Lookup via Google DNS
dig example.com MX              # Mail records
dig TXT _dmarc.example.com     # DMARC record

# === SSL ===
certbot --nginx -d example.com  # Get SSL cert
certbot renew --dry-run         # Test renewal
openssl s_client -connect example.com:443  # Check SSL

# === DATABASE ===
mysql -u root -p                # Login
mysqldump -u root -p dbname | gzip > backup.sql.gz   # Backup
gunzip < backup.sql.gz | mysql -u root -p dbname      # Restore
SHOW PROCESSLIST;               # Running queries

# === FIREWALL ===
ufw status                     # Check firewall
ufw allow 80/tcp               # Allow HTTP
ufw deny from 1.2.3.4          # Block IP
fail2ban-client status sshd    # Check banned IPs

# === FILES & PERMISSIONS ===
chown -R www-data:www-data /var/www/    # Fix ownership
chmod 755 /var/www/html                 # Dir permissions
chmod 644 /var/www/html/index.html      # File permissions
find /var/www -type f -perm 0777        # Find insecure files

# === TROUBLESHOOTING ===
tail -f /var/log/syslog         # System log
dmesg | tail                    # Kernel messages
grep "error" /var/log/nginx/error.log   # Search for errors
lsof -i :80                    # What's using port 80

7.5 Documentation Template

Server Inventory Template

# Server: production-web-01

## Server Details
- Provider: DigitalOcean
- IP: 192.168.1.100
- OS: Ubuntu 22.04 LTS
- CPU: 4 vCPUs
- RAM: 8 GB
- Disk: 160 GB SSD

## Software Stack
- Nginx 1.22
- PHP 8.2-FPM
- MySQL 8.0
- Redis 7.0
- Node.js 20 LTS

## Hosted Domains
| Domain | Type | Document Root | Database | SSL |
|--------|------|---------------|----------|-----|
| example.com | WordPress | /var/www/example.com | example_db | Let's Encrypt |
| app.company.com | Laravel | /var/www/app | app_db | Let's Encrypt |
| api.company.com | Node.js | Port 3000 | N/A | Let's Encrypt |

## Backups
- Database: Daily at 2 AM, retained 30 days
- Files: Weekly Sunday 3 AM, retained 12 weeks
- Location: S3 bucket: s3://company-backups/production/

## Access
- SSH: Port 2222, key-based only
- Users: john (sudo), deploy (limited)

## Monitoring
- UptimeRobot: All domains monitored
- Disk alert: > 80%
- CPU alert: > 90% for 5 minutes

## Last Incidents
| Date | Issue | Resolution | Duration |
|------|-------|------------|----------|
| 2024-01-15 | Disk full | Cleaned old logs | 15 min |
| 2024-02-03 | MySQL crashed | OOM, increased RAM | 30 min |

Incident Report Template

# Incident Report: [Brief Description]

## Summary
- Date/Time: 2024-01-15 23:30 IST
- Duration: 45 minutes
- Impact: example.com was unreachable
- Severity: High

## Timeline
- 23:30 - Alert received: example.com is down
- 23:32 - SSHed into server
- 23:33 - Found: Nginx running but PHP-FPM stopped
- 23:34 - Checked logs: OOM killer killed PHP-FPM
- 23:35 - Restarted PHP-FPM, site came back up
- 23:40 - Investigated: memory leak in custom plugin
- 00:15 - Increased server RAM, disabled problematic plugin

## Root Cause
A WordPress plugin had a memory leak, causing PHP-FPM workers to consume
all available RAM. The OOM killer terminated PHP-FPM processes.

## Resolution
1. Restarted PHP-FPM
2. Disabled the problematic plugin
3. Increased server RAM from 4GB to 8GB
4. Set pm.max_requests = 500 to restart workers periodically

## Prevention
- Set up memory usage alerting at 80%
- Added pm.max_requests to PHP-FPM config
- Scheduled weekly plugin updates and review

7.6 Final Tips for the Interview

[!TIP]
How to answer "Tell me about your experience":

"I have X years managing Linux servers running Ubuntu with Nginx/Apache, PHP-FPM, and MySQL"

"I manage Y domains across Z servers on [DigitalOcean/AWS]"

"I handle SSL certificates, DNS management, server security, and automated backups"

"I monitor server health including CPU, RAM, disk, and uptime using [tools]"

Mention specific scenarios you've resolved

[!IMPORTANT]
Key behaviors interviewers look for:

Systematic troubleshooting - Don't jump to random fixes. Check logs first

Security mindset - Always mention security considerations

Backup awareness - Always mention backup before making changes

Communication - Mention updating stakeholders during incidents

Documentation - Mention documenting changes and incidents

Proactive approach - Mention monitoring, alerts, and prevention

[!CAUTION]
Common interview mistakes to avoid:

Don't say "I would Google it" - show you know the commands

Don't say "rm -rf /" as a joke - shows carelessness

Don't say "just restart the server" for everything - shows lack of depth

Don't admit to using root for everything - shows bad security practice

Don't forget to mention testing config before restarting (nginx -t, apache2ctl configtest)

The Linux Server Administrator’s Modern Survival Guide: Cloud, Docker, Git, CI/CD and Production Infrastructure

Yashvi Kothari — Wed, 12 Aug 2026 11:29:38 +0000

The Linux Server Administrator’s Modern Survival Guide: Cloud, Docker, Git, CI/CD and Production Infrastructure

Most server administrators start with:

ssh

Then they discover that modern infrastructure is much bigger than SSH.

You need to understand cloud servers, containers, Git, CI/CD, reverse proxies, databases, caching, monitoring, backups, and disaster recovery.

The good news?

You don't need to memorize hundreds of commands.

You need to understand how the pieces fit together.

That is the difference between someone who can execute commands and someone who can operate production infrastructure.

The Modern Server Stack

A production application might look like this:

Users
   ↓
Cloudflare / CDN
   ↓
Load Balancer
   ↓
Nginx
   ↓
Application
   ↓
Redis
   ↓
Database
   ↓
Object Storage / Backups

GitHub
   ↓
CI/CD
   ↓
Deployment
   ↓
Monitoring
   ↓
Alerts

Every layer solves a different problem.

Cloud provides infrastructure.

Nginx handles web traffic.

Docker packages applications.

Git manages source code.

CI/CD automates delivery.

Redis accelerates frequently accessed data.

Prometheus/Grafana tells you what is happening.

Backups and disaster recovery make sure one failure doesn't become a catastrophe.

Learn the architecture first.

The commands become easier afterward.

1. Cloud: Stop Thinking in Terms of Servers

AWS, DigitalOcean, and Azure all provide similar fundamental building blocks.

You need to understand:

Compute
Storage
Networking
DNS
Identity
Firewalls
Load balancing
Monitoring
Backups

For example, on AWS:

EC2       → Compute
S3        → Object storage
RDS       → Managed database
Route 53  → DNS
ELB       → Load balancing
CloudFront → CDN
IAM       → Identity and access
VPC       → Networking

The important interview question isn't:

"What is EC2?"

It's:

"How would you deploy and protect an application running on EC2?"

A strong answer includes:

EC2
 ↓
Security Group
 ↓
Nginx
 ↓
Application
 ↓
Database
 ↓
S3 backups
 ↓
CloudWatch monitoring

That's infrastructure thinking.

2. Security Groups Are Not Just Another Firewall

A common interview mistake is saying:

"Security Groups are AWS firewalls."

That's technically correct, but incomplete.

A better answer:

A Security Group controls network traffic to and from AWS resources such as EC2 instances through defined inbound and outbound rules.

For example:

aws ec2 authorize-security-group-ingress \
    --group-id sg-12345 \
    --protocol tcp \
    --port 22 \
    --cidr 203.0.113.0/24

But don't blindly open SSH to:

0.0.0.0/0

Production security starts with one principle:

Allow only what is required, from only where it is required.

3. Docker: Package the Application, Not the Server

Before containers, deploying an application often meant:

Install OS
Install runtime
Install dependencies
Configure application
Configure services
Fix permissions
Hope nothing conflicts

Docker changes the model.

You package the application and its environment into an image.

Then run that image consistently.

docker pull nginx

docker run -d \
  --name my-nginx \
  -p 80:80 \
  nginx

Now understand the difference:

Image

The packaged blueprint.

Container

A running instance of that image.

Volume

Persistent storage.

Network

Communication between containers.

This distinction appears constantly in interviews.

4. Containers Are Not Virtual Machines

This question is almost guaranteed in DevOps interviews.

VM

Hardware
   ↓
Hypervisor
   ↓
Guest OS
   ↓
Application

Container

Hardware
   ↓
Host OS / Kernel
   ↓
Container Runtime
   ↓
Container
   ↓
Application

Containers generally start faster and use fewer resources because they share the host kernel.

But that doesn't mean:

"Containers are always better."

The correct engineering answer is:

Use the abstraction that fits the workload, isolation requirements, operational model, and scalability needs.

5. Docker Compose: One Application, Multiple Services

Real applications rarely contain only one process.

You might have:

Nginx
PHP
MySQL
Redis

Docker Compose lets you describe that architecture as code.

services:
  nginx:
    image: nginx:latest

  php:
    image: php:8.2-fpm

  mysql:
    image: mysql:8.0

  redis:
    image: redis:alpine

Then:

docker compose up -d

One command.

Four services.

Repeatable infrastructure.

That is the real value of containers:

consistency.

6. Git Is Infrastructure Knowledge

Git isn't just for developers.

If you're managing servers, you should understand:

git clone
git status
git pull
git fetch
git add
git commit
git push
git diff
git log
git stash
git revert

But don't just memorize commands.

Understand this:

Working Directory
       ↓
Staging Area
       ↓
Commit
       ↓
Remote Repository

And understand the difference between:

git fetch

and:

git pull

fetch downloads remote changes without modifying your current working tree.

pull generally performs a fetch followed by integration into your current branch.

That distinction matters during production troubleshooting.

7. CI/CD Turns Deployment Into a Pipeline

Without automation:

Developer
   ↓
SSH server
   ↓
git pull
   ↓
install dependencies
   ↓
restart service
   ↓
hope

With CI/CD:

Git Push
   ↓
Build
   ↓
Test
   ↓
Package
   ↓
Deploy
   ↓
Health Check
   ↓
Monitoring

A simple GitHub Actions workflow might deploy after a push to main.

But production CI/CD should ideally also include:

Automated tests
Security scanning
Build verification
Deployment approval where appropriate
Secrets management
Health checks
Rollback strategy

The goal isn't simply:

"Deploy automatically."

The goal is:

Deploy safely and repeatably.

8. Python and Node.js Need a Process Model

An application running manually from a terminal is not production architecture.

For Python:

Nginx
  ↓
Gunicorn
  ↓
Flask/Django

For Node.js:

Nginx
  ↓
Node.js

A process manager such as PM2 can help manage Node.js processes.

For Python applications, systemd plus Gunicorn is a common pattern.

The important principle is:

Your application should survive logout, crashes, and server reboots.

That means:

systemctl enable myapp

and:

Restart=always

are not cosmetic configuration.

They are operational resilience.

9. Redis: Fast Doesn't Mean Permanent

Redis is commonly used for:

Caching
Sessions
Queues
Counters
Temporary application state
Real-time workloads

A basic test:

redis-cli ping

Expected:

PONG

You should also understand dangerous commands.

For example:

redis-cli FLUSHALL

can delete all Redis data.

And:

redis-cli KEYS "*"

can become expensive on large production datasets.

A good administrator doesn't just know commands.

They know which commands not to run casually.

10. Elasticsearch: Search Is Different From a Database

Elasticsearch is designed around searching and analyzing large amounts of data.

Useful operational endpoints include:

curl localhost:9200

curl \
  "localhost:9200/_cluster/health?pretty"

curl \
  "localhost:9200/_cat/indices?v"

When troubleshooting, don't ask only:

"Is Elasticsearch running?"

Ask:

Is the service running?
Is port 9200 listening?
Is the cluster healthy?
Are nodes connected?
Are indices available?
Is disk space sufficient?
Are requests succeeding?

This is the difference between service checking and system troubleshooting.

11. Monitoring: If You Can't See It, You Can't Operate It

A server can be "up" while the application is effectively down.

For example:

CPU = 40%
RAM = 50%
Disk = 70%

Everything looks healthy.

But:

Database connections = exhausted

The application is down.

This is why production monitoring needs multiple layers.

Prometheus

Collects and stores metrics.

Grafana

Visualizes metrics.

Node Exporter

Exposes Linux system metrics.

Zabbix

Provides broader infrastructure monitoring and alerting.

The key lesson:

Monitoring should measure user-impacting health, not just server health.

12. Cloudflare Is More Than a CDN

Cloudflare can provide:

DNS
CDN
DDoS protection
WAF
TLS
Caching
Traffic filtering

A common architecture:

User
 ↓
Cloudflare
 ↓
Load Balancer
 ↓
Nginx
 ↓
Application

For production TLS, understand:

Flexible

Browser → HTTPS

Cloudflare → HTTP

Usually not desirable for secure applications.

Full

HTTPS between Cloudflare and origin, including self-signed certificates.

Full Strict

HTTPS end-to-end with a valid trusted origin certificate.

For production:

Full (Strict) is generally the preferred model.

13. Disaster Recovery Is Part of Being a Good Administrator

The worst backup is the one you've never tested.

A strong backup strategy follows the:

3-2-1 Rule

3 copies
2 different storage types
1 offsite copy

For example:

Production DB
      ↓
Local backup
      ↓
Cloud object storage
      ↓
Different region/provider

But backups alone aren't enough.

You need recovery procedures.

Ask:

"If this server disappears right now, how long until the application is running again?"

That question introduces two critical concepts:

RTO — Recovery Time Objective

How quickly must the service be restored?

RPO — Recovery Point Objective

How much data loss is acceptable?

Production infrastructure is not about preventing every failure.

It's about recovering intelligently when failure happens.

14. High Traffic Requires Architecture, Not Just Bigger Servers

When traffic increases, many administrators immediately think:

"Buy a bigger server."

That's vertical scaling.

Sometimes it works.

But eventually you need horizontal scaling.

                  Load Balancer
                 /      |      \
                /       |       \
            App 1     App 2     App 3
               \        |        /
                \       |       /
                   Database
                      |
                  Read Replica

Then add:

CDN
Caching
Redis
Connection pooling
Database replicas
Autoscaling

The goal is not to make one machine infinitely powerful.

The goal is to remove single points of failure and distribute work.

15. The Most Important Skill: Troubleshooting

Commands are easy to memorize.

Troubleshooting is harder.

Suppose a website returns:

502 Bad Gateway

Don't randomly restart everything.

Follow the path:

Client
 ↓
DNS
 ↓
Cloudflare
 ↓
Load Balancer
 ↓
Nginx
 ↓
Application
 ↓
Database

Check each layer.

For example:

dig example.com

curl -I https://example.com

systemctl status nginx

nginx -t

ss -tulpn

curl http://127.0.0.1:3000

journalctl -u myapp -n 100

docker ps

docker logs my-container

You're narrowing the failure domain.

That's what good troubleshooting looks like.

16. The Interview Mental Model

When an interviewer gives you a production problem, don't jump immediately to commands.

Use this framework:

1. Observe

What is the exact symptom?

2. Scope

Is one user affected?

One server?

One service?

The entire application?

3. Check

Look at:

Logs
Processes
Ports
CPU
Memory
Disk
Network
Dependencies
Configuration

4. Isolate

Find the first layer where reality differs from expectation.

5. Fix

Make the smallest safe change.

6. Verify

Confirm the service works.

7. Prevent

Add monitoring, automation, documentation, or configuration changes to prevent recurrence.

This framework works across:

Linux
AWS
Docker
Nginx
MySQL
Redis
Kubernetes
CI/CD

Final Lesson

You don't become a strong Linux or DevOps engineer by memorizing 1,000 commands.

You become one by understanding relationships.

Cloud
 ↓
Network
 ↓
Load Balancer
 ↓
Web Server
 ↓
Application
 ↓
Cache
 ↓
Database
 ↓
Storage
 ↓
Monitoring
 ↓
Backup
 ↓
Recovery

Then automate the boring parts.

Monitor the important parts.

Secure the dangerous parts.

Document the critical parts.

And test your recovery before production forces you to.

The best administrator isn't the person who knows the most commands.

It's the person who can look at a broken system, reduce the problem to a few possibilities, find the real cause, fix it safely, and make sure it doesn't happen again.

That is the skill worth building.

Part 6: Cloud Platforms, Docker, Git, CI/CD & Preferred Skills

6.1 Cloud Platforms (AWS, DigitalOcean, Azure)

AWS (Amazon Web Services)

# Key AWS services for server admins:
# EC2       - Virtual servers (VPS)
# S3        - Object storage (backups, static files)
# RDS       - Managed databases
# Route53   - DNS management
# ELB       - Load balancer
# CloudFront - CDN
# IAM       - User access management
# VPC       - Virtual private network

# AWS CLI setup
sudo apt install awscli
aws configure
# Enter: Access Key ID, Secret Access Key, Region, Output format

# EC2 operations
aws ec2 describe-instances                          # List all instances
aws ec2 start-instances --instance-ids i-1234567    # Start instance
aws ec2 stop-instances --instance-ids i-1234567     # Stop instance
aws ec2 reboot-instances --instance-ids i-1234567   # Reboot instance

# S3 operations (backups)
aws s3 ls                                           # List buckets
aws s3 ls s3://my-bucket/                          # List bucket contents
aws s3 cp backup.tar.gz s3://my-bucket/backups/    # Upload file
aws s3 sync /backups/ s3://my-bucket/backups/      # Sync directory
aws s3 cp s3://my-bucket/backups/backup.tar.gz ./  # Download file

# SSH to EC2
ssh -i ~/.ssh/my-key.pem ubuntu@ec2-ip-address
# Default users: ubuntu (Ubuntu), ec2-user (Amazon Linux), admin (Debian)

# Security Groups (AWS firewall)
# Configure in AWS Console or CLI:
aws ec2 authorize-security-group-ingress \
    --group-id sg-12345 \
    --protocol tcp \
    --port 22 \
    --cidr 203.0.113.0/24

DigitalOcean

# Key concepts:
# Droplets   - Virtual servers
# Spaces     - Object storage (like S3)
# Managed DB - Managed databases
# Load Balancers, Firewalls, Domains

# doctl CLI setup
sudo snap install doctl
doctl auth init                         # Enter API token

# Droplet operations
doctl compute droplet list             # List all droplets
doctl compute droplet create my-server \
    --size s-2vcpu-4gb \
    --image ubuntu-22-04-x64 \
    --region blr1                       # Create droplet

# Firewall
doctl compute firewall create \
    --name my-firewall \
    --inbound-rules "protocol:tcp,ports:22,address:0.0.0.0/0 protocol:tcp,ports:80,address:0.0.0.0/0 protocol:tcp,ports:443,address:0.0.0.0/0"

# Snapshots (backup)
doctl compute droplet-action snapshot DROPLET_ID --snapshot-name "pre-migration-backup"

# DigitalOcean console access
# Through web console when SSH is locked out

Azure

# Key services:
# Virtual Machines  - Compute
# Blob Storage      - Object storage
# Azure SQL         - Managed databases
# Azure DNS         - DNS management
# Azure CDN         - Content delivery

# Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
az login

# VM operations
az vm list --output table
az vm start --name myVM --resource-group myRG
az vm stop --name myVM --resource-group myRG
az vm restart --name myVM --resource-group myRG

VPS vs Cloud vs Dedicated Servers

Feature	VPS	Cloud (AWS/DO/Azure)	Dedicated
Cost	Low ($5-50/mo)	Pay-as-you-go	High ($100+/mo)
Scalability	Limited	Instant scaling	Hardware limited
Performance	Shared resources	Variable	Full resources
Management	Self-managed	Managed options	Self-managed
Use case	Small-medium sites	Scalable apps	High-performance

Interview Q&A: Cloud

Question	Answer
Difference between VPS and cloud?	VPS is a fixed virtual server. Cloud is scalable infrastructure (can add/remove resources instantly)
What is an EC2 instance?	AWS virtual server. You choose CPU, RAM, storage, OS
What is S3?	AWS object storage. Unlimited storage for files, backups, static assets
What are Security Groups in AWS?	Virtual firewall rules for EC2 instances. Control inbound/outbound traffic
How do you access a cloud server?	SSH with key pair. AWS uses .pem files, DigitalOcean uses SSH keys
What is auto-scaling?	Automatically adds/removes servers based on traffic/load

6.2 Cloudflare & CDN

Cloudflare Setup

# Setup steps:
# 1. Create Cloudflare account
# 2. Add your domain
# 3. Change nameservers at domain registrar to Cloudflare NS
# 4. Configure DNS records in Cloudflare dashboard

# Cloudflare provides:
# - CDN (cache static files at edge locations)
# - DDoS protection
# - WAF (Web Application Firewall)
# - Free SSL (Flexible, Full, Full Strict)
# - DNS management
# - Page Rules
# - Caching Rules

Cloudflare SSL Modes

Mode	Description	When to use
Off	No SSL	Never
Flexible	HTTPS browser→CF, HTTP CF→server	No SSL on origin (not recommended)
Full	HTTPS everywhere, self-signed OK	Self-signed cert on origin
Full (Strict)	HTTPS everywhere, valid cert required	Recommended with Let's Encrypt

Cloudflare Configuration Tips

# Page Rules examples:
# Force HTTPS: URL pattern *.example.com/* → Always Use HTTPS
# Cache everything: URL pattern *.example.com/static/* → Cache Level: Cache Everything

# Common Cloudflare DNS settings:
# Type | Name | Value           | Proxy Status
# A    | @    | YOUR_SERVER_IP  | Proxied (orange cloud)
# A    | www  | YOUR_SERVER_IP  | Proxied (orange cloud)
# A    | api  | YOUR_SERVER_IP  | DNS only (grey cloud - for websockets/SSH)

# Getting real visitor IP behind Cloudflare (Nginx)
# Install ngx_http_realip_module, then:
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
# ... (add all Cloudflare IP ranges)
real_ip_header CF-Connecting-IP;

# Apache: Install mod_remoteip
RemoteIPHeader CF-Connecting-IP

Interview Q&A: CDN

Question	Answer
What is a CDN?	Content Delivery Network. Caches content at edge servers globally for faster delivery
What does Cloudflare do?	CDN, DDoS protection, WAF, DNS, SSL, performance optimization
What is "Full (Strict)" SSL?	Encrypts traffic end-to-end and requires a valid SSL certificate on the origin server
How do you see the real client IP behind Cloudflare?	Use `CF-Connecting-IP` header. Configure `real_ip_header` in Nginx or `mod_remoteip` in Apache
How to bypass Cloudflare for debugging?	Use DNS-only mode (grey cloud) or add server IP to local hosts file
What are Cloudflare Page Rules?	URL-based rules for caching, redirects, SSL mode, and other behavior per URL pattern

6.3 Docker & Containerization

Docker Basics

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER         # Add user to docker group
# Log out and log back in

# Docker commands
docker ps                              # List running containers
docker ps -a                           # List all containers (including stopped)
docker images                          # List images
docker pull nginx                      # Download image
docker pull php:8.2-fpm                # Specific version

# Run containers
docker run -d --name my-nginx -p 80:80 nginx          # Run Nginx
docker run -d --name my-mysql -p 3306:3306 \
    -e MYSQL_ROOT_PASSWORD=secret \
    -v mysql_data:/var/lib/mysql mysql:8.0              # Run MySQL with volume

# Container management
docker start container_name
docker stop container_name
docker restart container_name
docker rm container_name               # Remove stopped container
docker rm -f container_name            # Force remove running container
docker logs container_name             # View logs
docker logs -f container_name          # Follow logs
docker exec -it container_name bash    # Shell into container
docker inspect container_name          # Detailed info

# Clean up
docker system prune -a                 # Remove unused images, containers, networks
docker volume prune                    # Remove unused volumes

Docker Compose (Multi-Container Apps)

# docker-compose.yml
version: '3.8'

services:
  nginx:
    image: nginx:latest
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./html:/var/www/html
    depends_on:
      - php
    restart: always

  php:
    image: php:8.2-fpm
    volumes:
      - ./html:/var/www/html
    restart: always

  mysql:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: myapp
      MYSQL_USER: appuser
      MYSQL_PASSWORD: apppassword
    volumes:
      - mysql_data:/var/lib/mysql
    ports:
      - "3306:3306"
    restart: always

  redis:
    image: redis:alpine
    ports:
      - "6379:6379"
    restart: always

volumes:
  mysql_data:

# Docker Compose commands
docker compose up -d                   # Start all services
docker compose down                    # Stop all services
docker compose restart                 # Restart all services
docker compose logs                    # View all logs
docker compose logs -f php             # Follow specific service logs
docker compose exec php bash           # Shell into specific service
docker compose ps                      # List services
docker compose pull                    # Pull latest images
docker compose build                   # Build images (if using Dockerfile)

Dockerfile (Custom Image)

# Dockerfile
FROM php:8.2-fpm

# Install extensions
RUN docker-php-ext-install pdo pdo_mysql mysqli mbstring

# Install additional tools
RUN apt-get update && apt-get install -y \
    curl \
    zip \
    unzip \
    && rm -rf /var/lib/apt/lists/*

# Install Composer
COPY --from=composer:latest /usr/bin/composer /usr/bin/composer

# Copy application
COPY . /var/www/html
WORKDIR /var/www/html

# Set permissions
RUN chown -R www-data:www-data /var/www/html

EXPOSE 9000
CMD ["php-fpm"]

Interview Q&A: Docker

Question	Answer
What is Docker?	Platform for building, shipping, and running apps in containers. Lightweight isolated environments
Container vs VM?	Container shares host kernel (lightweight, fast). VM has its own kernel (heavier, more isolation)
What is Docker Compose?	Tool for defining multi-container apps in YAML. Start all services with one command
What is a volume?	Persistent storage for containers. Data survives container restarts/deletion
How to debug a container?	`docker logs`, `docker exec -it container bash`, `docker inspect`
What is a Dockerfile?	Blueprint for building a custom Docker image. Defines base image, dependencies, files, commands

6.4 Git & GitHub

Git Basics for Server Admin

# Install Git
sudo apt install git

# Configure Git
git config --global user.name "Your Name"
git config --global user.email "your@email.com"

# Clone repository
git clone https://github.com/company/website.git
git clone git@github.com:company/website.git         # SSH (preferred for deployments)

# Basic workflow
git status                             # Check changes
git pull origin main                   # Pull latest changes
git add .                              # Stage all changes
git commit -m "Fix: update config"     # Commit
git push origin main                   # Push changes

# Branches
git branch                             # List branches
git branch -a                          # List all (including remote)
git checkout main                      # Switch to main
git checkout -b feature/new-feature    # Create and switch to new branch
git merge feature/new-feature          # Merge branch into current
git branch -d feature/new-feature      # Delete branch

# View history
git log --oneline -10                  # Last 10 commits
git log --oneline --graph              # Visual branch history
git diff                               # View unstaged changes
git diff --cached                      # View staged changes

# Stash changes
git stash                              # Save changes temporarily
git stash pop                          # Restore stashed changes

# Undo changes
git checkout -- file.txt               # Discard file changes
git reset --hard HEAD                  # Discard ALL uncommitted changes
git revert abc1234                     # Create new commit that undoes a commit

Deployment Workflows

# Simple Git Pull Deployment (basic)
# On server:
cd /var/www/html
git pull origin main
# Fix permissions, restart services if needed

# Deploy Script (/usr/local/bin/deploy.sh)
#!/bin/bash
set -e

APP_DIR="/var/www/html"
BRANCH="main"

echo "Starting deployment..."

cd $APP_DIR

# Pull latest code
git fetch origin
git reset --hard origin/$BRANCH

# Install dependencies
composer install --no-dev --optimize-autoloader  # PHP
# OR
npm install --production                          # Node.js

# Build assets (if applicable)
npm run build

# Clear caches
php artisan cache:clear                           # Laravel
php artisan config:cache
php artisan route:cache
php artisan view:cache

# Run migrations
php artisan migrate --force

# Fix permissions
chown -R www-data:www-data $APP_DIR
chmod -R 755 $APP_DIR

# Restart services
sudo systemctl reload php8.2-fpm
sudo systemctl reload nginx

echo "Deployment complete!"

SSH Keys for GitHub (Server Deployment)

# Generate deploy key on server
ssh-keygen -t ed25519 -C "server-deploy-key" -f ~/.ssh/github_deploy

# Add public key to GitHub repo → Settings → Deploy Keys
cat ~/.ssh/github_deploy.pub

# Configure SSH to use this key for GitHub
cat >> ~/.ssh/config << EOF
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_deploy
EOF

# Test connection
ssh -T git@github.com

6.5 Deploying Python & Node.js Applications

Python (Flask/Django) Deployment

# Install Python
sudo apt install python3 python3-pip python3-venv

# Create virtual environment
cd /var/www/myapp
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Install Gunicorn (WSGI server)
pip install gunicorn

# Test run
gunicorn --bind 0.0.0.0:8000 app:app                # Flask
gunicorn --bind 0.0.0.0:8000 myproject.wsgi:application  # Django

# Create systemd service
sudo nano /etc/systemd/system/myapp.service

[Unit]
Description=My Python Application
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/var/www/myapp
Environment="PATH=/var/www/myapp/venv/bin"
ExecStart=/var/www/myapp/venv/bin/gunicorn --workers 3 --bind unix:myapp.sock app:app
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl start myapp
sudo systemctl enable myapp

# Nginx config for Python app
server {
    listen 80;
    server_name myapp.com;

    location / {
        proxy_pass http://unix:/var/www/myapp/myapp.sock;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location /static {
        alias /var/www/myapp/static;
    }
}

Node.js Deployment

# Install Node.js (using nvm - recommended)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
source ~/.bashrc
nvm install 20                         # Install Node.js 20 LTS
nvm use 20

# OR via apt
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt install nodejs

# Deploy app
cd /var/www/nodeapp
npm install --production
npm run build                          # If build step needed

# Process Manager - PM2 (recommended for production)
sudo npm install -g pm2

pm2 start app.js --name "my-app"       # Start app
pm2 start npm --name "my-app" -- start # Start with npm start
pm2 list                               # List all apps
pm2 logs                               # View logs
pm2 restart my-app                     # Restart
pm2 stop my-app                        # Stop
pm2 delete my-app                      # Remove
pm2 monit                              # Real-time monitoring

# PM2 auto-start on reboot
pm2 startup                            # Generate startup script
pm2 save                               # Save current process list

# Nginx config for Node.js
server {
    listen 80;
    server_name nodeapp.com;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_cache_bypass $http_upgrade;
    }
}

6.6 Redis & Elasticsearch

Redis

# Install Redis
sudo apt install redis-server

# Configure
sudo nano /etc/redis/redis.conf
# Key settings:
# bind 127.0.0.1               # Listen on localhost only
# requirepass YourStrongPassword # Set password
# maxmemory 256mb               # Max memory usage
# maxmemory-policy allkeys-lru  # Eviction policy

sudo systemctl restart redis
sudo systemctl enable redis

# Redis CLI
redis-cli
redis-cli -a YourPassword          # With password

# Basic commands
redis-cli ping                      # Test connection → PONG
redis-cli INFO                      # Server info
redis-cli INFO memory               # Memory usage
redis-cli DBSIZE                    # Number of keys
redis-cli FLUSHALL                  # Clear all data (CAREFUL!)
redis-cli MONITOR                   # Real-time command monitoring

# Key operations
redis-cli SET mykey "Hello"
redis-cli GET mykey
redis-cli DEL mykey
redis-cli KEYS "*"                  # List all keys (don't use in production)

Elasticsearch

# Install Elasticsearch
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt update
sudo apt install elasticsearch

# Configure
sudo nano /etc/elasticsearch/elasticsearch.yml
# Key settings:
# cluster.name: my-cluster
# node.name: node-1
# network.host: 127.0.0.1
# http.port: 9200

sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch

# Test
curl -X GET "localhost:9200"          # Check cluster health
curl -X GET "localhost:9200/_cluster/health?pretty"
curl -X GET "localhost:9200/_cat/indices?v"    # List indices

6.7 CI/CD & Automated Deployment

GitHub Actions (Simple CI/CD)

# .github/workflows/deploy.yml
name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy to server
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.SERVER_HOST }}
          username: ${{ secrets.SERVER_USER }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          script: |
            cd /var/www/html
            git pull origin main
            composer install --no-dev
            php artisan migrate --force
            php artisan cache:clear
            sudo systemctl reload php8.2-fpm

Webhook-Based Deployment

# Simple webhook server for auto-deploy on git push
# Install webhook tool
sudo apt install webhook

# /etc/webhook.conf
[
  {
    "id": "deploy",
    "execute-command": "/usr/local/bin/deploy.sh",
    "command-working-directory": "/var/www/html",
    "pass-arguments-to-command": [],
    "trigger-rule": {
      "match": {
        "type": "value",
        "value": "your-webhook-secret",
        "parameter": {
          "source": "header",
          "name": "X-Hub-Signature"
        }
      }
    }
  }
]

6.8 Monitoring Tools (Grafana, Prometheus, Zabbix)

Quick Overview

Tool	Purpose	Key Feature
Prometheus	Metrics collection & storage	Pull-based, time-series DB
Grafana	Visualization & dashboards	Beautiful graphs, alerts
Zabbix	Full monitoring platform	All-in-one, agent-based
Nagios	Infrastructure monitoring	Alerting, legacy but stable
Netdata	Real-time monitoring	Zero config, instant dashboards

Prometheus + Grafana Stack (Most Popular)

# Install Node Exporter (on each server to monitor)
wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
tar xvfz node_exporter-*.tar.gz
sudo mv node_exporter-*/node_exporter /usr/local/bin/

# Create systemd service
sudo nano /etc/systemd/system/node_exporter.service

[Unit]
Description=Node Exporter
After=network.target

[Service]
User=nobody
ExecStart=/usr/local/bin/node_exporter
Restart=always

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
# Metrics available at http://server:9100/metrics

# Install Grafana
sudo apt install -y apt-transport-https software-properties-common
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt install grafana
sudo systemctl start grafana-server
sudo systemctl enable grafana-server
# Access Grafana at http://server:3000 (default: admin/admin)

Quick Monitoring with Netdata (Easiest)

# One-line install
bash <(curl -Ss https://my-netdata.io/kickstart.sh)
# Dashboard available at http://server:19999
# Real-time CPU, RAM, disk, network, processes - zero config

6.9 Disaster Recovery

Disaster Recovery Plan

1. DOCUMENT EVERYTHING
   - Server IPs, hostnames, OS versions
   - All software versions and configurations
   - DNS records for all domains
   - Database credentials
   - SSL certificate details
   - Contact info for providers

2. BACKUP STRATEGY (3-2-1 Rule)
   - 3 copies of all critical data
   - 2 different storage types (local + cloud)
   - 1 offsite copy (different region/provider)

3. AUTOMATED BACKUPS
   - Daily database backups (retained 30 days)
   - Weekly full server backups (retained 3 months)
   - Real-time replication for critical databases

4. RECOVERY PROCEDURES
   - Document step-by-step recovery for each service
   - Test recovery quarterly
   - Maintain runbooks for common failure scenarios

5. MONITORING & ALERTS
   - Uptime monitoring (UptimeRobot, Pingdom)
   - Resource alerts (CPU > 80%, disk > 90%, RAM > 85%)
   - SSL expiry alerts (30 days before)

Recovery Commands Quick Reference

# Server won't boot → Use provider's console/recovery mode

# Restore from backup
# 1. Provision new server
# 2. Install same software stack
# 3. Restore files from backup
tar -xzf www_backup.tar.gz -C /var/www/
# 4. Restore databases
gunzip < all_databases.sql.gz | mysql -u root -p
# 5. Restore configurations
tar -xzf configs.tar.gz -C /etc/
# 6. Fix permissions
chown -R www-data:www-data /var/www/
# 7. Restart all services
systemctl restart nginx php8.2-fpm mysql
# 8. Update DNS if IP changed
# 9. Verify SSL certificates
# 10. Test all websites and services

6.10 Handling High-Traffic Infrastructure

# Performance optimization checklist for high traffic:

# 1. Enable caching
# - OPcache for PHP (bytecode caching)
# - Redis/Memcached for application caching
# - Nginx fastcgi_cache or proxy_cache
# - Cloudflare CDN for static assets

# 2. Optimize Nginx for high traffic
worker_processes auto;
worker_connections 4096;
multi_accept on;
use epoll;

# 3. PHP-FPM tuning
pm = static                            # Or dynamic for variable traffic
pm.max_children = 100                   # Based on available RAM
# Formula: max_children = (Total RAM - Other Services RAM) / PHP avg memory

# 4. MySQL optimization
innodb_buffer_pool_size = 4G           # 60-70% of RAM for dedicated DB
innodb_log_file_size = 512M
max_connections = 500
query_cache_type = 0                   # Disabled in MySQL 8.0

# 5. OS-level tuning
# /etc/sysctl.conf
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
fs.file-max = 65535

# Apply: sysctl -p

# 6. Connection pooling
# Use ProxySQL or PgBouncer for database connection pooling

# 7. Horizontal scaling
# Load balancer (Nginx, HAProxy, AWS ELB) distributing traffic
# Multiple app servers behind load balancer
# Read replicas for database

Interview Q&A: Preferred Skills

Question	Answer
Docker vs VM?	Docker: lightweight, shares kernel, fast startup. VM: full OS, more isolation, heavier
What is PM2?	Node.js process manager. Auto-restarts, clustering, log management
What is Gunicorn?	Python WSGI HTTP server. Runs Python apps, manages worker processes
Git pull vs Git fetch?	`pull` = fetch + merge (updates working code). `fetch` = download changes only (doesn't change files)
What is Redis used for?	In-memory cache, session storage, queues, real-time data. Very fast
What is CI/CD?	Continuous Integration (auto test) / Continuous Deployment (auto deploy). Automates the release pipeline
How to handle sudden traffic spike?	Scale vertically (bigger server), scale horizontally (more servers + load balancer), enable caching, use CDN
What is Prometheus?	Open-source monitoring system. Collects metrics via HTTP pull model, stores in time-series DB
What is Grafana?	Visualization platform. Creates dashboards from Prometheus/other data sources

Linux Server Monitoring, Backup, Security & Troubleshooting: The Practical SysAdmin Playbook

Yashvi Kothari — Wed, 12 Aug 2026 11:24:59 +0000

Linux Server Monitoring, Backup, Security & Troubleshooting Guide

Learn practical Linux server monitoring, backup, security, firewall configuration, and troubleshooting techniques with real commands and interview-ready explanations.

A server rarely fails without leaving clues.

CPU usage rises.

Memory disappears.

Disk space reaches 100%.

A service stops listening.

Nginx returns a 502.

The database refuses connections.

And suddenly someone asks:

“Can you fix production?”

Good system administrators don't panic.

They investigate.

They follow evidence.

They fix the root cause.

This guide covers the practical Linux skills every SysAdmin and DevOps engineer should know: monitoring, backup, security, and troubleshooting.

1. Monitoring: Know What Your Server Is Doing

Monitoring isn't about staring at dashboards all day.

It's about answering four questions:

Is the CPU overloaded?
Is memory exhausted?
Is disk space running out?
Is the network or application behaving abnormally?

CPU Monitoring

Start with:

uptime

You'll see something like:

load average: 0.50, 0.75, 0.80

The three values represent the 1-minute, 5-minute, and 15-minute load averages.

But there's an important interview detail:

Load average is not the same thing as CPU percentage.

A rough rule is to compare load with the number of CPU cores.

For a 4-core server:

Load < 4     → generally comfortable
Load ≈ 4     → CPUs are fully utilized
Load > 4     → work is waiting for CPU/resources

Check CPU information:

lscpu
nproc

Find CPU-heavy processes:

ps aux --sort=-%cpu | head -20

For interactive troubleshooting:

top

or:

htop

2. Memory Monitoring

Use:

free -h

The most important number isn't always free.

Look at:

available

Linux intentionally uses unused RAM for filesystem cache and buffers.

That memory can often be reclaimed when applications need it.

Check memory-heavy processes

ps aux --sort=-%mem | head -20

Check swap

swapon --show

and:

free -h

You can also monitor swap activity:

vmstat 1 5

Pay attention to si and so.

Heavy, continuous swap activity can indicate memory pressure and often results in poor performance.

3. Disk Monitoring

One of the most common production problems is surprisingly simple:

The disk is full.

Start with:

df -h

Then identify where the space went:

du -sh /* 2>/dev/null | sort -rh

Investigate /var:

du -sh /var/* 2>/dev/null | sort -rh | head -20

Logs are frequent offenders:

du -sh /var/log/* 2>/dev/null | sort -rh

Find huge files:

find / -type f -size +100M -exec ls -lh {} \; 2>/dev/null

But disk space isn't the only resource that can run out.

Check inodes

df -i

A filesystem can have free GBs available while still being unable to create files because its inodes are exhausted.

That's a classic interview question.

4. Disk I/O

A server can have:

low CPU
plenty of RAM
plenty of disk space

…and still be painfully slow.

Why?

Disk I/O.

Use:

iostat -x 1 5

For process-level I/O:

iotop

High disk utilization or latency can point toward:

slow disks
excessive logging
database activity
backups
large file operations
overloaded storage

5. Network Monitoring

Check listening services:

ss -tlnp

Get socket statistics:

ss -s

Check network interfaces:

ip addr show
ip link show

Test connectivity:

ping google.com

Trace the route:

traceroute google.com

Or combine latency and route analysis:

mtr google.com

For HTTP:

curl -I https://example.com

This is extremely useful because it tells you whether the server is actually returning an HTTP response.

6. Logs Are Your First Investigation Tool

When something breaks, don't randomly restart everything.

Read the logs.

System logs:

sudo tail -f /var/log/syslog

Authentication logs:

sudo tail -f /var/log/auth.log

Systemd:

sudo journalctl -f

Recent logs:

sudo journalctl --since "1 hour ago"

For Nginx:

sudo tail -f /var/log/nginx/error.log

For Apache:

sudo tail -f /var/log/apache2/error.log

For MySQL:

sudo tail -f /var/log/mysql/error.log

The fastest troubleshooting path is often:

Symptom → service status → logs → dependency → configuration → resource usage

7. Backup: Assume the Server Will Fail

A backup isn't:

“I copied the website somewhere.”

A proper backup strategy considers the entire system.

You should consider backing up:

/var/www/
databases
Nginx/Apache configuration
PHP configuration
MySQL configuration
SSL certificates
cron jobs
user data
custom scripts
package information

For MySQL:

mysqldump -u root -p --all-databases --routines --triggers > all_databases.sql

Compress it:

gzip all_databases.sql

For files:

tar -czf www_backup.tar.gz /var/www/

8. Why rsync Is a SysAdmin Superpower

For large migrations, rsync is usually much more useful than repeatedly copying everything.

Example:

rsync -avz --progress /var/www/ user@newserver:/var/www/

Why?

Because rsync can synchronize only what changed.

That makes it extremely useful for:

server migrations
backups
deployments
synchronization
large file transfers

A strong interview answer:

“I prefer rsync for large transfers because it efficiently synchronizes changed files and can resume transfers.”

9. Remember the 3-2-1 Backup Rule

A practical backup strategy follows:

3 copies of your data

2 different types of storage

1 copy offsite

For example:

Production server
       ↓
Local backup
       ↓
Remote backup
       ↓
Cloud/offsite backup

And one more important principle:

A backup isn't proven until you've restored it.

Test your backups.

A backup that has never been restored is only a theory.

10. Firewall Security

On Ubuntu, UFW provides a simpler interface for firewall management.

Install:

sudo apt install ufw

Before enabling it, make sure SSH is allowed:

sudo ufw allow ssh

Then:

sudo ufw enable

Check:

sudo ufw status verbose

A typical web server may need:

sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https

Then use restrictive defaults:

sudo ufw default deny incoming
sudo ufw default allow outgoing

The principle is simple:

Don't expose services that don't need to be exposed.

11. iptables

For lower-level firewall control:

sudo iptables -L -n -v

Allow established connections:

sudo iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

Allow loopback:

sudo iptables -A INPUT -i lo -j ACCEPT

Allow SSH:

sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

Allow HTTP/HTTPS:

sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT

Then carefully define your default policy.

Important: Never experiment with firewall rules remotely without ensuring you won't lock yourself out.

12. Fail2ban

A firewall controls network access.

Fail2ban adds another layer by watching logs for suspicious authentication behavior and temporarily banning offending IP addresses.

Install:

sudo apt install fail2ban

Check status:

sudo fail2ban-client status

Check SSH protection:

sudo fail2ban-client status sshd

Unban an IP:

sudo fail2ban-client set sshd unbanip 1.2.3.4

Think of it as:

Logs → Detect repeated abuse → Ban IP

13. Server Hardening

A secure server starts with basic discipline.

Keep packages updated

sudo apt update
sudo apt upgrade -y

Remove unnecessary packages

sudo apt autoremove -y

Enable automatic security updates

sudo apt install unattended-upgrades

Audit permissions

find /var/www -type f -perm 0777

World-writable files deserve investigation.

Audit open ports

sudo ss -tlnp

The goal isn't:

“Close every port.”

The goal is:

“Know why every exposed port exists.”

14. Troubleshooting HTTP 500

A 500 Internal Server Error means the server encountered an application-side error.

Don't immediately blame Nginx.

Start with logs:

sudo tail -50 /var/log/nginx/error.log

or:

sudo tail -50 /var/log/apache2/error.log

If PHP is involved:

sudo tail -50 /var/log/php8.2-fpm.log

Then investigate:

application errors
permissions
PHP extensions
PHP configuration
.htaccess
application code
memory limits

The most important habit:

Check the error log first.

15. Troubleshooting 502 Bad Gateway

This error is especially common with reverse proxies.

Conceptually:

Client
  ↓
Nginx
  ↓
Backend

A 502 generally means Nginx cannot successfully communicate with the upstream.

Check:

sudo systemctl status php8.2-fpm

Check the PHP-FPM socket:

ls -la /var/run/php/php8.2-fpm.sock

For a Node.js application:

sudo lsof -i :3000

The critical question is:

“Is the backend running, and is Nginx connecting to the correct socket or port?”

16. 503 Service Unavailable

A 503 generally means the service cannot currently handle the request.

Check:

uptime

Then:

sudo systemctl status nginx
sudo systemctl status apache2
sudo systemctl status php8.2-fpm

Check connections:

ss -s

Possible causes include:

overloaded server
stopped backend
maintenance mode
rate limiting
exhausted application workers
overloaded database

17. 504 Gateway Timeout

A 504 tells a different story.

The backend may be reachable.

But it didn't respond within the required time.

Possible causes:

slow database queries
slow application code
overloaded backend
network latency
insufficient PHP workers
overly aggressive timeout settings

Check:

mysql -u root -p -e "SHOW FULL PROCESSLIST;"

Check resources:

htop
iostat

Only increase timeouts after understanding why the backend is slow.

Increasing a timeout can hide the symptom while making resource consumption worse.

18. DNS Troubleshooting

When a website doesn't open, don't immediately restart Nginx.

Check DNS.

dig example.com

Get the IP:

dig +short example.com

Query a public resolver:

dig @8.8.8.8 example.com

For mail:

dig MX example.com

For SPF:

dig TXT example.com

For DMARC:

dig TXT _dmarc.example.com

The troubleshooting chain becomes:

Domain
  ↓
DNS
  ↓
IP
  ↓
Firewall
  ↓
Port
  ↓
Web server
  ↓
Reverse proxy
  ↓
Application
  ↓
Database

That's how you avoid guessing.

19. SSL Troubleshooting

Check certificate dates:

echo | openssl s_client \
-connect example.com:443 2>/dev/null \
| openssl x509 -noout -dates

Test Let's Encrypt renewal:

sudo certbot renew --dry-run

Common causes of renewal failures:

DNS points somewhere else
port 80 blocked
HTTP challenge inaccessible
incorrect web-server configuration
certificate configuration errors

20. Database Troubleshooting

If MySQL isn't responding:

sudo systemctl status mysql

Check logs:

sudo tail -50 /var/log/mysql/error.log

Test credentials:

mysql -u root -p

Check connections:

mysql -u root -p -e "SHOW STATUS LIKE 'Threads_connected';"

Check maximum connections:

mysql -u root -p -e "SHOW VARIABLES LIKE 'max_connections';"

Don't blindly increase max_connections.

More connections require more resources.

First understand why the connections are being consumed.

21. The SysAdmin Troubleshooting Framework

When production breaks, use this sequence:

Step 1 — Confirm the symptom

curl -I https://example.com

Step 2 — Check the service

systemctl status nginx

Step 3 — Check logs

journalctl -u nginx --since "10 minutes ago"

Step 4 — Check ports

ss -tlnp

Step 5 — Check resources

uptime
free -h
df -h

Step 6 — Check dependencies

Database?

PHP-FPM?

Node.js?

Redis?

External API?

Step 7 — Check configuration

nginx -t

Step 8 — Make the smallest safe change

Avoid changing five things at once.

Step 9 — Verify

curl -I https://example.com

Step 10 — Document the root cause

A production fix is not complete until someone understands why it happened.

22. Interview Questions You Should Be Ready For

How do you check server load?

Use:

uptime

Then compare load average with available CPU cores and investigate CPU, I/O, and other bottlenecks.

How do you find what's filling the disk?

Start with:

df -h

Then:

du -sh /* 2>/dev/null | sort -rh

What's the difference between 502 and 504?

502: The proxy couldn't successfully communicate with the upstream.

504: The proxy waited for the upstream but timed out.

What is rsync?

A file synchronization tool that efficiently transfers changed data and is particularly useful for backups and migrations.

What is the 3-2-1 backup rule?

Three copies, two different storage types, one offsite.

How do you secure a Linux server?

Update the OS, minimize exposed services, configure a firewall, harden SSH, use automatic security updates, monitor authentication, use appropriate intrusion-prevention controls, and maintain tested backups.

What is your first step when a website is down?

Confirm the symptom and check the relevant service and logs before making changes.

Final Lesson

The best SysAdmins don't memorize hundreds of commands.

They understand what the commands are trying to prove.

CPU problem?
    → top / htop / uptime

Memory problem?
    → free / vmstat

Disk problem?
    → df / du / iostat

Network problem?
    → ss / ping / mtr / curl

Service problem?
    → systemctl / journalctl

Application problem?
    → application logs

Security problem?
    → auth logs / firewall / open ports

Backup problem?
    → restore test

HTTP problem?
    → status code → logs → backend → dependencies

That's the difference between knowing Linux commands and actually knowing how to operate Linux servers.

Commands are tools.

Troubleshooting is the skill.

And in production, the engineer who can calmly turn a symptom into evidence into a root cause is the engineer everyone wants on the incident call.

Part 5: Server Monitoring, Backup, Security & Troubleshooting

5.1 Server Monitoring

CPU Monitoring

# Real-time CPU usage
top                                    # Press 'P' to sort by CPU
htop                                   # Better interactive viewer (install: apt install htop)

# CPU info
lscpu                                 # CPU architecture info
nproc                                 # Number of CPU cores
cat /proc/cpuinfo                     # Detailed CPU info

# Load average
uptime                                # Shows load average (1, 5, 15 min)
# Output: load average: 0.50, 0.75, 0.80
# Rule: load average should be < number of CPU cores
# 4 cores → load < 4.0 is OK, > 4.0 means overloaded

# CPU usage snapshot
mpstat                                # CPU usage per core (install: apt install sysstat)
vmstat 1 5                            # System stats every 1 sec, 5 times
sar -u 1 5                            # CPU usage (requires sysstat)

# Top CPU-consuming processes
ps aux --sort=-%cpu | head -20

RAM Monitoring

# Memory usage
free -h                               # Human-readable memory info
# Key: Available (not Free) is what matters

# Detailed memory
cat /proc/meminfo

# Top memory-consuming processes
ps aux --sort=-%mem | head -20

# Check for memory issues
vmstat 1 5                            # Watch si/so columns (swap in/out)
# If si/so are high → server needs more RAM

# Check swap
swapon --show
free -h | grep Swap

Disk Space Monitoring

# Disk usage overview
df -h                                 # All mounted filesystems
df -h /                               # Root partition specifically

# Find what's using space
du -sh /*                             # Size of top-level directories
du -sh /var/* | sort -rh | head -20   # Largest dirs in /var
du -sh /var/log/* | sort -rh          # Log sizes
ncdu /                                # Interactive disk usage tool

# Find large files
find / -type f -size +100M -exec ls -lh {} \;  # Files > 100MB
find /var/log -name "*.log" -size +50M          # Large log files

# Check inodes
df -i                                 # Inode usage (can run out before disk space)

# Monitor disk I/O
iostat -x 1 5                         # Disk I/O stats (install: apt install sysstat)
iotop                                 # Real-time I/O by process (install: apt install iotop)

Bandwidth & Network Monitoring

# Current connections
ss -s                                 # Socket statistics summary
ss -tlnp                              # Listening ports
netstat -tlnp                         # Alternative (legacy)

# Bandwidth usage
iftop                                 # Real-time bandwidth by connection (install: apt install iftop)
nload                                 # Real-time bandwidth graph (install: apt install nload)
vnstat                                # Historical bandwidth (install: apt install vnstat)
vnstat -d                             # Daily traffic
vnstat -m                             # Monthly traffic

# Network interface info
ip addr show                          # IP addresses
ip link show                          # Network interfaces

# Test connectivity
ping google.com                       # Basic connectivity
traceroute google.com                  # Route to destination
mtr google.com                        # Combined ping + traceroute (install: apt install mtr)
curl -I https://example.com           # HTTP response headers
wget --spider https://example.com     # Check if URL is accessible

Server Uptime & Load

# System uptime
uptime
# Output: 14:30:05 up 45 days, 3:22, 2 users, load average: 0.50, 0.75, 0.80

# System info
uname -a                              # Kernel version, architecture
hostnamectl                            # Hostname, OS, kernel, architecture
cat /etc/os-release                    # OS version

# Last reboot
last reboot | head -10
who -b                                # Last boot time

# System resource summary (one command)
echo "=== UPTIME ===" && uptime && echo "=== MEMORY ===" && free -h && echo "=== DISK ===" && df -h / && echo "=== CPU LOAD ===" && cat /proc/loadavg

Log Monitoring

# System logs
sudo tail -f /var/log/syslog                    # System log (live)
sudo tail -f /var/log/auth.log                  # Authentication log
sudo journalctl -f                              # systemd journal (live)
sudo journalctl --since "1 hour ago"            # Recent logs

# Web server logs
sudo tail -f /var/log/apache2/error.log         # Apache errors
sudo tail -f /var/log/nginx/error.log           # Nginx errors
sudo tail -f /var/log/apache2/access.log        # Apache access

# PHP logs
sudo tail -f /var/log/php8.2-fpm.log            # PHP-FPM log

# MySQL logs
sudo tail -f /var/log/mysql/error.log           # MySQL errors

# Analyze access logs
# Top 10 IPs hitting the server
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10

# Top 10 requested URLs
awk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10

# Count 404 errors
grep " 404 " /var/log/nginx/access.log | wc -l

# Count 500 errors
grep " 500 " /var/log/nginx/access.log | wc -l

# Requests per minute/hour
awk '{print $4}' /var/log/nginx/access.log | cut -d: -f1-3 | sort | uniq -c | tail -20

Interview Q&A: Monitoring

Question	Answer
How do you check server load?	`uptime` shows load average. Compare to CPU cores: load > cores = overloaded
What's the difference between free and available memory?	Free = completely unused. Available = free + cache/buffer that can be freed. Available is what matters
How to find what's filling up disk?	`du -sh /* \
How do you monitor in real-time?	{% raw %}`htop` (CPU/RAM), `iftop` (bandwidth), `tail -f` (logs), `iostat` (disk I/O)
What is swap? When is it bad?	Overflow area on disk when RAM is full. High swap usage = server needs more RAM, causes slowness
How to identify a DDoS attack?	Check access logs for unusual traffic from single IPs, high connection count, `ss -s`

5.2 Server Backup, Restore & Migration

Full Server Backup Strategy

# What to backup:
# 1. Website files (/var/www/)
# 2. Databases (mysqldump)
# 3. Server configurations (/etc/nginx, /etc/apache2, /etc/php, /etc/mysql)
# 4. SSL certificates (/etc/letsencrypt)
# 5. Cron jobs (crontab -l)
# 6. User home directories
# 7. Custom scripts

# Full website backup script
#!/bin/bash
DATE=$(date +%Y-%m-%d)
BACKUP_DIR="/backups/$DATE"
mkdir -p $BACKUP_DIR

# 1. Backup website files
tar -czf $BACKUP_DIR/www_backup.tar.gz /var/www/

# 2. Backup all databases
mysqldump -u root -p'password' --all-databases --routines --triggers | gzip > $BACKUP_DIR/all_databases.sql.gz

# 3. Backup configs
tar -czf $BACKUP_DIR/configs.tar.gz /etc/nginx/ /etc/apache2/ /etc/php/ /etc/mysql/ /etc/letsencrypt/

# 4. Backup crontabs
crontab -l > $BACKUP_DIR/root_crontab.txt
crontab -u www-data -l > $BACKUP_DIR/www-data_crontab.txt 2>/dev/null

# 5. Server info
dpkg --get-selections > $BACKUP_DIR/installed_packages.txt
ip addr > $BACKUP_DIR/network_config.txt
cat /etc/hosts > $BACKUP_DIR/hosts.txt

# 6. Sync to remote/cloud
rsync -avz $BACKUP_DIR/ backup@remote-server:/offsite-backups/$DATE/
# OR to S3:
# aws s3 sync $BACKUP_DIR/ s3://my-backups/$DATE/

# 7. Cleanup old backups (keep 30 days)
find /backups/ -maxdepth 1 -type d -mtime +30 -exec rm -rf {} \;

echo "Backup completed: $BACKUP_DIR"

Server Migration Steps

# Step-by-step server migration checklist:

# 1. PREPARE NEW SERVER
sudo apt update && sudo apt upgrade -y
# Install all required software: nginx, php, mysql, etc.
# Configure same PHP version and extensions

# 2. BACKUP OLD SERVER (everything listed above)

# 3. TRANSFER FILES
# Option A: rsync (best for large transfers, supports resume)
rsync -avz --progress -e "ssh -p 22" /var/www/ newserver:/var/www/

# Option B: scp
scp -r /var/www/ user@newserver:/var/www/

# Option C: tar + transfer
tar -czf /tmp/www_backup.tar.gz /var/www/
scp /tmp/www_backup.tar.gz user@newserver:/tmp/
# On new server: tar -xzf /tmp/www_backup.tar.gz -C /

# 4. TRANSFER DATABASES
# On old server:
mysqldump -u root -p --all-databases | gzip > /tmp/all_db.sql.gz
scp /tmp/all_db.sql.gz user@newserver:/tmp/
# On new server:
gunzip < /tmp/all_db.sql.gz | mysql -u root -p

# 5. TRANSFER CONFIGURATIONS
scp -r /etc/nginx/sites-available/ user@newserver:/etc/nginx/sites-available/
scp -r /etc/letsencrypt/ user@newserver:/etc/letsencrypt/
# Copy php.ini, mysql configs, etc.

# 6. FIX PERMISSIONS
sudo chown -R www-data:www-data /var/www/
sudo chmod -R 755 /var/www/

# 7. UPDATE DNS (point to new server IP)
# Lower TTL first (300 seconds), wait for propagation
# Change A records to new server IP
# Wait for propagation (check with dig @8.8.8.8 domain.com)

# 8. VERIFY
# Test all websites
# Test database connections
# Test SSL certificates
# Test email delivery
# Monitor logs for errors

# 9. KEEP OLD SERVER running for a few days as fallback

Interview Q&A: Backup & Migration

Question	Answer
What should be included in a server backup?	Website files, databases, configs (nginx/apache/php/mysql), SSL certs, crontabs, custom scripts
What is rsync? Why use it?	File synchronization tool. Only transfers changed files, supports resume, efficient for large transfers
How often should you backup?	Daily for databases, weekly for full backups. Keep 30 days of daily + 3 months of weekly
What is the 3-2-1 backup rule?	3 copies of data, on 2 different media types, 1 offsite (cloud/remote)
How do you minimize downtime during migration?	Sync data before DNS change, lower TTL, test on new server first, keep old server as fallback
How do you verify a backup is good?	Periodically do test restores to a staging server

5.3 Firewall & Server Security

UFW (Uncomplicated Firewall) - Ubuntu Default

# Install and enable
sudo apt install ufw
sudo ufw enable                        # CAREFUL: make sure SSH is allowed first!

# Check status
sudo ufw status
sudo ufw status verbose
sudo ufw status numbered               # Show rule numbers

# Allow common services
sudo ufw allow ssh                     # Port 22
sudo ufw allow 2222/tcp                # Custom SSH port
sudo ufw allow http                    # Port 80
sudo ufw allow https                   # Port 443
sudo ufw allow 3306/tcp               # MySQL (only if needed remotely)

# Allow from specific IP
sudo ufw allow from 192.168.1.100
sudo ufw allow from 192.168.1.100 to any port 22  # SSH from specific IP only
sudo ufw allow from 192.168.1.0/24 to any port 3306  # MySQL from subnet

# Deny rules
sudo ufw deny 23                       # Deny telnet
sudo ufw deny from 1.2.3.4             # Block specific IP

# Delete rules
sudo ufw delete allow 3306/tcp
sudo ufw delete 5                      # Delete rule by number

# Rate limiting (brute force protection)
sudo ufw limit ssh                     # Limit SSH connections (6 in 30 seconds)

# Default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Reset all rules
sudo ufw reset

# Logging
sudo ufw logging on
sudo ufw logging medium

iptables (Advanced Firewall)

# List rules
sudo iptables -L -n -v
sudo iptables -L -n --line-numbers

# Allow SSH
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Allow HTTP/HTTPS
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Allow established connections
sudo iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow loopback
sudo iptables -A INPUT -i lo -j ACCEPT

# Drop everything else
sudo iptables -A INPUT -j DROP

# Block specific IP
sudo iptables -A INPUT -s 1.2.3.4 -j DROP

# Rate limit SSH
sudo iptables -A INPUT -p tcp --dport 22 -m connlimit --connlimit-above 3 -j DROP

# Save rules (persist after reboot)
sudo apt install iptables-persistent
sudo netfilter-persistent save

Fail2ban (Brute Force Protection)

# Install
sudo apt install fail2ban

# Configure
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

# Key settings in jail.local:
[DEFAULT]
bantime = 3600                         # Ban for 1 hour
findtime = 600                         # Within 10 minutes
maxretry = 5                           # After 5 failed attempts
destemail = admin@example.com
action = %(action_mwl)s                # Ban + email with whois + logs

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3

[apache-auth]
enabled = true
port = http,https
filter = apache-auth
logpath = /var/log/apache2/error.log

[nginx-http-auth]
enabled = true
port = http,https
filter = nginx-http-auth
logpath = /var/log/nginx/error.log

# Service management
sudo systemctl start fail2ban
sudo systemctl enable fail2ban
sudo systemctl status fail2ban

# Check banned IPs
sudo fail2ban-client status
sudo fail2ban-client status sshd

# Unban an IP
sudo fail2ban-client set sshd unbanip 1.2.3.4

Server Hardening Checklist

# 1. Keep system updated
sudo apt update && sudo apt upgrade -y
sudo apt autoremove -y

# 2. Automatic security updates
sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

# 3. SSH hardening (see Part 1 SSH section)

# 4. Disable root login
sudo passwd -l root                   # Lock root password

# 5. Remove unnecessary packages
sudo apt remove telnet ftp             # Remove insecure services

# 6. Secure /tmp
# Mount /tmp with noexec option

# 7. Security headers (in Nginx/Apache)
add_header X-Frame-Options "SAMEORIGIN";
add_header X-Content-Type-Options "nosniff";
add_header X-XSS-Protection "1; mode=block";
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
add_header Content-Security-Policy "default-src 'self'";
add_header Referrer-Policy "strict-origin-when-cross-origin";

# 8. File permissions audit
find /var/www -type f -perm 0777       # Find world-writable files
find /var/www -type d -perm 0777       # Find world-writable directories

# 9. Monitor auth logs
sudo tail -f /var/log/auth.log
sudo grep "Failed password" /var/log/auth.log | tail -20

# 10. Check for rootkits
sudo apt install rkhunter
sudo rkhunter --check

# 11. Audit open ports
sudo ss -tlnp                         # Should only show expected ports
sudo nmap -sT localhost                # Scan own ports

Interview Q&A: Security

Question	Answer
How do you secure a new server?	Update OS, configure firewall (UFW), SSH hardening, fail2ban, disable root, security headers, automatic updates
What is fail2ban?	Monitors logs for failed auth attempts, bans IPs after threshold. Prevents brute force
What is UFW?	Uncomplicated Firewall - simplified interface for iptables on Ubuntu
What ports should be open on a web server?	22 (or custom SSH), 80 (HTTP), 443 (HTTPS). Everything else blocked
How to check for unauthorized access?	Check `/var/log/auth.log`, check `last` command, check running processes, audit open ports
What are security headers?	HTTP headers that protect against XSS, clickjacking, content sniffing. Set in web server config

5.4 Troubleshooting HTTP Errors

500 Internal Server Error

# Most common causes & fixes:

# 1. Check error logs FIRST
sudo tail -50 /var/log/apache2/error.log
sudo tail -50 /var/log/nginx/error.log
sudo tail -50 /var/log/php8.2-fpm.log

# 2. PHP errors
# Check php.ini:
display_errors = On   # Temporarily (NEVER in production)
# Or check PHP error log
tail -f /var/log/php/error.log

# 3. .htaccess issues (Apache)
# Temporarily rename .htaccess to test:
mv /var/www/html/.htaccess /var/www/html/.htaccess.bak
# If site works → .htaccess has syntax error

# 4. File permissions
ls -la /var/www/html/
# Files should be 644, dirs 755, owned by www-data

# 5. PHP memory limit
# Error: "Allowed memory size exhausted"
# Fix in php.ini: memory_limit = 256M

# 6. Missing PHP extensions
php -m  # Check if required extensions are installed

502 Bad Gateway

# Means: Nginx/Apache can't reach the backend (PHP-FPM, Node.js, etc.)

# 1. Check if PHP-FPM is running
sudo systemctl status php8.2-fpm

# 2. Check PHP-FPM socket
ls -la /var/run/php/php8.2-fpm.sock
# If missing → restart PHP-FPM

# 3. Check Nginx config matches PHP-FPM socket
# Nginx should have: fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
# PHP-FPM should have: listen = /var/run/php/php8.2-fpm.sock

# 4. Check backend app (Node.js, Python)
# Is the app running? Is it on the right port?
sudo lsof -i :3000                    # Check if app is listening

# 5. Check PHP-FPM pool config
# Too few workers? Increase pm.max_children

# 6. Check memory - OOM killer may have killed PHP-FPM
sudo dmesg | grep -i "out of memory"
sudo dmesg | grep -i "oom"

503 Service Unavailable

# Means: Server is overloaded or in maintenance

# 1. Check server load
uptime
htop

# 2. Check if service is running
sudo systemctl status nginx
sudo systemctl status apache2
sudo systemctl status php8.2-fpm

# 3. Check connections
ss -s                                 # Socket summary
ss -tlnp | grep :80                   # Connections on port 80

# 4. Check rate limiting
# Nginx: check if limit_req is too aggressive
# Cloudflare: check if WAF is blocking

# 5. Database overloaded
mysql -u root -p -e "SHOW PROCESSLIST;"
# Too many connections? Kill stuck queries

504 Gateway Timeout

# Means: Backend took too long to respond

# 1. Increase timeout in Nginx
proxy_read_timeout 300;
proxy_connect_timeout 300;
fastcgi_read_timeout 300;

# 2. Increase timeout in Apache
ProxyTimeout 300
Timeout 300

# 3. Increase PHP execution time
# php.ini: max_execution_time = 300

# 4. Check for slow database queries
mysql -u root -p -e "SHOW FULL PROCESSLIST;"
# Check slow query log

# 5. Check if server is overloaded
htop                                   # High CPU/RAM usage?
iostat                                 # High disk I/O?

DNS Issues Troubleshooting

# 1. Domain not resolving
dig example.com                        # Check A record
dig @8.8.8.8 example.com              # Check with Google DNS
ping example.com                       # Test connectivity

# 2. Wrong IP
dig +short example.com                 # What IP is it resolving to?
# Update A record in DNS provider

# 3. Propagation delay
# Check propagation: dnschecker.org
# Solution: Wait, or flush local DNS cache
sudo systemd-resolve --flush-caches

# 4. Subdomain not working
dig subdomain.example.com              # Check if A/CNAME record exists
# Add appropriate record in DNS

# 5. Email not working
dig MX example.com                     # Check MX records
dig TXT example.com                    # Check SPF/DKIM

SSL Issues Troubleshooting

# 1. Certificate expired
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -dates
sudo certbot renew

# 2. Mixed content (HTTP on HTTPS page)
# Fix URLs in code/database to use HTTPS
# WordPress: update site URL in wp_options table

# 3. Certificate doesn't match domain
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -subject -issuer
# Reissue certificate for correct domain

# 4. Let's Encrypt renewal failing
sudo certbot renew --dry-run          # Test renewal
# Common issues:
# - Port 80 blocked (needs to be open for HTTP challenge)
# - DNS not pointing to this server
# - .well-known/acme-challenge not accessible

Database Connectivity Issues

# 1. Can't connect to MySQL
sudo systemctl status mysql            # Is it running?
sudo tail -20 /var/log/mysql/error.log # Check error log

# 2. Access denied
mysql -u root -p                       # Test credentials
# Check user@host: SELECT user, host FROM mysql.user;

# 3. Too many connections
mysql -u root -p -e "SHOW STATUS LIKE 'Threads_connected';"
mysql -u root -p -e "SHOW VARIABLES LIKE 'max_connections';"
# Increase max_connections if needed

# 4. MySQL crashed
sudo systemctl restart mysql
sudo tail -50 /var/log/mysql/error.log
# Check for OOM: sudo dmesg | grep -i mysql

Email/SMTP Troubleshooting

# 1. Check mail queue
mailq                                  # View pending emails
sudo postqueue -f                      # Flush/retry mail queue
sudo postsuper -d ALL                  # Delete all queued mail

# 2. Check mail logs
sudo tail -f /var/log/mail.log
sudo grep "status=bounced" /var/log/mail.log | tail -20

# 3. Test SMTP
telnet mail.example.com 25             # Test SMTP connection
openssl s_client -connect mail.example.com:587 -starttls smtp  # Test STARTTLS

# 4. Check SPF/DKIM/DMARC
dig TXT example.com                    # SPF
dig TXT default._domainkey.example.com # DKIM
dig TXT _dmarc.example.com            # DMARC

# 5. Test email delivery
echo "Test" | mail -s "Test Email" user@example.com

# 6. Check if port 25/587 is blocked
sudo ss -tlnp | grep -E "25|587"
sudo ufw status | grep -E "25|587"

# 7. Check blacklist status
# Use: mxtoolbox.com/blacklists.aspx
# Check reverse DNS: dig -x YOUR_SERVER_IP

Interview Q&A: Troubleshooting

Question	Answer
First thing you do when a site is down?	Check if server is reachable (ping), check web server status, check error logs
How to diagnose a 500 error?	Check error logs (apache/nginx/php), check permissions, check .htaccess, check PHP config
Difference between 502 and 504?	502: backend unreachable/crashed. 504: backend is running but too slow to respond
Website is slow. How to diagnose?	Check server load (htop), check slow query log, check PHP-FPM status, check disk I/O, check network
Server ran out of disk space. Quick fix?	Find large files (`du -sh`), clean old logs, clear apt cache (`apt clean`), remove old backups
How to handle DDoS?	Enable Cloudflare, configure rate limiting, use fail2ban, block suspicious IPs, enable UFW rate limiting
How to check if emails are being delivered?	Check mail logs, verify SPF/DKIM/DMARC records, check blacklists, test with mxtoolbox

MySQL & MariaDB Database Administration

Yashvi Kothari — Wed, 12 Aug 2026 11:19:23 +0000

MySQL & MariaDB Database Administration: The Practical Guide Every Linux/DevOps Engineer Should Know

Most people learn MySQL by memorizing SQL commands.

That’s not database administration.

A real DBA or DevOps engineer needs to know five things:

Install it. Secure it. Manage it. Back it up. Recover it.

And when production gets slow?

Find out why.

This guide covers the practical MySQL and MariaDB skills you need to work with databases on Linux servers—and explain them confidently in an interview.

1. Start With Installation

On Ubuntu:

sudo apt update
sudo apt install mysql-server

Then secure the installation:

sudo mysql_secure_installation

This is one of the first things you should do after installation.

Typical hardening includes:

Removing anonymous users
Preventing remote root login
Removing the test database
Setting appropriate root authentication

For MariaDB:

sudo apt install mariadb-server
sudo mysql_secure_installation

Check the service:

sudo systemctl status mysql

Start, stop, restart, and enable it:

sudo systemctl start mysql
sudo systemctl stop mysql
sudo systemctl restart mysql
sudo systemctl enable mysql

Check the version:

mysql --version

mysql -u root -p

On Ubuntu, root may use socket authentication, so this can also work:

sudo mysql

2. Users Are a Security Boundary

Never give every application full database access.

Create users with the minimum permissions they need.

CREATE USER 'appuser'@'localhost'
IDENTIFIED BY 'StrongP@ssw0rd!';

For a remote application:

CREATE USER 'appuser'@'192.168.1.%'
IDENTIFIED BY 'StrongP@ssw0rd!';

Then grant only what is required:

GRANT SELECT, INSERT, UPDATE, DELETE
ON mydb.*
TO 'appuser'@'localhost';

Need full access to one database?

GRANT ALL PRIVILEGES
ON mydb.*
TO 'appuser'@'localhost';

Check permissions:

SHOW GRANTS FOR 'appuser'@'localhost';

Change a password:

ALTER USER 'appuser'@'localhost'
IDENTIFIED BY 'NewP@ssw0rd!';

Remove the user:

DROP USER 'appuser'@'localhost';

The important interview concept

MySQL permissions are not simply about the username.

They are based on:

'user'@'host'

These are different accounts:

'appuser'@'localhost'
'appuser'@'%'
'appuser'@'192.168.1.%'

That distinction matters enormously when troubleshooting:

“Access denied.”

3. Database Administration Starts With Visibility

Before changing anything, inspect the system.

List databases:

SHOW DATABASES;

Create one:

CREATE DATABASE myapp_production
CHARACTER SET utf8mb4
COLLATE utf8mb4_unicode_ci;

Select it:

USE myapp_production;

List tables:

SHOW TABLES;

Inspect a table:

DESCRIBE users;

For the complete table definition:

SHOW CREATE TABLE users;

4. Know Your Database Size

Disk usage becomes a production problem faster than most people expect.

Check table sizes:

SELECT
    table_name,
    ROUND(data_length/1024/1024, 2) AS 'Data (MB)',
    ROUND(index_length/1024/1024, 2) AS 'Index (MB)',
    ROUND(
        (data_length + index_length)/1024/1024,
        2
    ) AS 'Total (MB)'
FROM information_schema.tables
WHERE table_schema = 'myapp_production'
ORDER BY (data_length + index_length) DESC;

Check database sizes:

SELECT
    table_schema AS 'Database',
    ROUND(
        SUM(data_length + index_length) / 1024 / 1024,
        2
    ) AS 'Size (MB)'
FROM information_schema.tables
GROUP BY table_schema
ORDER BY SUM(data_length + index_length) DESC;

This gives you something much more useful than:

“The database seems large.”

You can identify which database and which tables are consuming the space.

5. CRUD Is Basic—But You Must Know It

Insert:

INSERT INTO users (name, email)
VALUES ('John', 'john@example.com');

Read:

SELECT * FROM users;

Filter:

SELECT *
FROM users
WHERE email = 'john@example.com';

Get recent records:

SELECT *
FROM users
ORDER BY created_at DESC
LIMIT 10;

Count:

SELECT COUNT(*)
FROM users;

Update:

UPDATE users
SET name = 'John Doe'
WHERE id = 1;

Delete:

DELETE FROM users
WHERE id = 1;

Production rule

Be extremely careful with:

UPDATE users SET ...

and:

DELETE FROM users;

Without a WHERE clause, you may modify or delete every row.

6. Learn to Troubleshoot Live Queries

When the database is slow, don't guess.

Start here:

SHOW PROCESSLIST;

For more detail:

SHOW FULL PROCESSLIST;

Look for:

Long-running queries
Locked queries
Waiting transactions
Too many connections
Queries stuck in unusual states

If a query is genuinely stuck and you understand the consequences:

KILL process_id;

The important skill isn't memorizing KILL.

It's knowing why you're killing the connection.

7. Backups Are Not Optional

A database without a tested backup is a disaster waiting for a date.

Single database:

mysqldump -u root -p myapp_production \
> /backups/myapp_production_$(date +%Y%m%d).sql

Compressed:

mysqldump -u root -p myapp_production |
gzip > /backups/myapp_production_$(date +%Y%m%d).sql.gz

All databases:

mysqldump -u root -p --all-databases \
> /backups/all_databases_$(date +%Y%m%d).sql

Specific tables:

mysqldump -u root -p myapp_production users orders \
> /backups/tables_backup.sql

Schema only:

mysqldump -u root -p --no-data myapp_production \
> /backups/schema_only.sql

Data only:

mysqldump -u root -p --no-create-info myapp_production \
> /backups/data_only.sql

Include routines, triggers, and events:

mysqldump -u root -p \
--routines \
--triggers \
--events \
myapp_production \
> /backups/full_backup.sql

For larger InnoDB databases:

mysqldump -u root -p \
--single-transaction \
--quick \
--lock-tables=false \
myapp_production |
gzip > /backups/myapp_production_$(date +%Y%m%d).sql.gz

Remember this

Backup ≠ recovery.

A backup becomes valuable only when you can successfully restore it.

8. Restore Like You Mean It

Restore an SQL dump:

mysql -u root -p myapp_production \
< /backups/myapp_production_20240101.sql

Compressed backup:

gunzip < /backups/myapp_production_20240101.sql.gz |
mysql -u root -p myapp_production

Restore all databases:

mysql -u root -p \
< /backups/all_databases_20240101.sql

If the database doesn't exist:

mysql -u root -p -e \
"CREATE DATABASE IF NOT EXISTS myapp_production
CHARACTER SET utf8mb4
COLLATE utf8mb4_unicode_ci"

Then restore:

mysql -u root -p myapp_production \
< /backups/myapp_production_20240101.sql

A good administrator doesn't merely say:

“We have backups.”

They can answer:

“How long does restoration take, where are the backups stored, and when was the last restore test?”

9. Automate Backups

Manual backups eventually become forgotten backups.

A simple script can automate them:

#!/bin/bash

DATE=$(date +%Y-%m-%d_%H-%M)
BACKUP_DIR="/backups/mysql"
RETENTION_DAYS=30

DB_USER="backup_user"
DB_PASS="secure_password"

mkdir -p "$BACKUP_DIR"

DATABASES=$(mysql \
-u "$DB_USER" \
-p"$DB_PASS" \
-e "SHOW DATABASES" \
-s --skip-column-names |
grep -Ev "(information_schema|performance_schema|sys|mysql)")

for DB in $DATABASES; do

    echo "Backing up $DB..."

    mysqldump \
    -u "$DB_USER" \
    -p"$DB_PASS" \
    --single-transaction \
    --quick \
    --routines \
    --triggers \
    "$DB" |
    gzip > "$BACKUP_DIR/${DB}_${DATE}.sql.gz"

    echo "Done: ${DB}_${DATE}.sql.gz"

done

find "$BACKUP_DIR" \
-name "*.sql.gz" \
-mtime +$RETENTION_DAYS \
-delete

echo "Backup completed at $DATE"

Make it executable:

chmod +x /usr/local/bin/mysql_backup.sh

Schedule it:

crontab -e

Example:

0 2 * * * /usr/local/bin/mysql_backup.sh >> /var/log/mysql_backup.log 2>&1

Now the database gets backed up every day at 2 AM.

10. Performance Tuning: Don't Change Random Variables

One of the most important MySQL tuning parameters is:

innodb_buffer_pool_size

The buffer pool caches InnoDB data and indexes in memory.

For a dedicated database server, a common starting point is roughly 50–70% of available RAM, but the correct value depends on the workload and what else runs on the machine.

Example:

[mysqld]

max_connections = 200

wait_timeout = 600
interactive_timeout = 600

innodb_buffer_pool_size = 1G

innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT

tmp_table_size = 64M
max_heap_table_size = 64M

slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 2

bind-address = 127.0.0.1

character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

Important caution

Don't blindly copy production values from a tutorial.

A 1 GB buffer pool makes sense on one server and is completely wrong on another.

Tune based on workload, RAM, concurrency, and measurements.

11. Find Slow Queries

Enable the slow query log:

slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 2

Then inspect:

SHOW VARIABLES LIKE 'slow_query_log%';

SHOW VARIABLES LIKE 'long_query_time';

The goal isn't simply:

“Find queries taking more than 2 seconds.”

The goal is:

Understand why they're slow.

12. EXPLAIN Is One of Your Best Friends

Suppose this query is slow:

SELECT *
FROM users
WHERE email = 'john@example.com';

Run:

EXPLAIN
SELECT *
FROM users
WHERE email = 'john@example.com';

On supported MySQL versions:

EXPLAIN ANALYZE
SELECT *
FROM users
WHERE email = 'john@example.com';

Look for issues such as:

Full table scans
Poor index selection
Excessive rows examined
Bad join strategies

Check existing indexes:

SHOW INDEX FROM users;

Add an index when appropriate:

CREATE INDEX idx_email
ON users(email);

Another example:

ALTER TABLE users
ADD INDEX idx_created(created_at);

The principle is simple:

Don't optimize the query you think is slow. Measure the query that is actually slow.

13. Remote MySQL Access

By default, MySQL is often configured for local access.

To permit remote connections, modify:

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

For example:

bind-address = 0.0.0.0

But don't stop there.

Opening MySQL to the entire internet is a bad idea.

Create a restricted user:

CREATE USER 'remote_user'@'192.168.1.%'
IDENTIFIED BY 'StrongP@ss!';

Grant only the required database permissions:

GRANT ALL PRIVILEGES
ON mydb.*
TO 'remote_user'@'192.168.1.%';

Restrict the firewall:

sudo ufw allow from 192.168.1.0/24 to any port 3306

Restart:

sudo systemctl restart mysql

Better security principle

Instead of:

Internet → MySQL:3306

prefer something like:

Application Server
       ↓
Private Network
       ↓
MySQL Server:3306

Database ports should be exposed only where necessary.

14. Common Production Problems

ERROR 1045: Access denied

Check:

SELECT user, host, plugin
FROM mysql.user;

Then:

SHOW GRANTS FOR 'username'@'host';

Remember:

user@localhost

is not necessarily the same account as:

user@%

Can't connect to socket

Check:

sudo systemctl status mysql

Then:

sudo systemctl start mysql

If necessary, verify the socket path and server configuration.

Too many connections

Check:

SHOW STATUS LIKE 'Threads_connected';
SHOW STATUS LIKE 'Max_used_connections';
SHOW VARIABLES LIKE 'max_connections';

Then investigate:

Connection leaks
Application pooling
Long-running sessions
Incorrect application configuration

Don't automatically increase max_connections.

More connections also mean more memory pressure.

Slow queries

Start with:

SHOW FULL PROCESSLIST;

Then:

EXPLAIN SELECT ...;

Check indexes:

SHOW INDEX FROM users;

And inspect the slow query log.

Disk full

Check:

df -h

Then investigate:

MySQL logs
Binary logs
Old backups
Large tables
Temporary files

A full filesystem can quickly turn a database problem into a complete application outage.

MySQL won't start

Check:

sudo systemctl status mysql

Then inspect logs:

sudo tail -f /var/log/mysql/error.log

Also validate configuration before restarting after configuration changes.

15. MySQL vs MariaDB

MariaDB began as a fork of MySQL and remains highly compatible with MySQL in many common use cases.

But don't assume they are identical.

Over time, the two projects have diverged in:

Features
Storage engines
Authentication
Optimizer behavior
System variables
Replication capabilities
Version-specific syntax

For an interview, a good answer is:

“MariaDB originated as a MySQL fork and maintains substantial compatibility, but modern MySQL and MariaDB have diverged, so I always verify version-specific behavior before migrating or configuring production systems.”

That's much stronger than:

“They're basically the same.”

16. The Interview Questions You Should Be Ready For

How do you back up MySQL?

mysqldump -u root -p mydb |
gzip > backup.sql.gz

How do you restore it?

gunzip < backup.sql.gz |
mysql -u root -p mydb

What is InnoDB?

InnoDB is the primary transactional storage engine used by modern MySQL installations.

It supports features such as:

Transactions
Row-level locking
Foreign keys
Crash recovery

What is `innodb_buffer_pool_size`?

It's the memory area InnoDB uses to cache data and indexes.

For a dedicated database server, it is often one of the most important memory-related tuning parameters.

How do you find slow queries?

Use:

Slow Query Log
        ↓
Identify expensive queries
        ↓
EXPLAIN / EXPLAIN ANALYZE
        ↓
Indexes / query optimization
        ↓
Measure again

What port does MySQL normally use?

How do you check database size?

Query:

information_schema.tables

and calculate:

data_length + index_length

How do you secure MySQL?

A strong answer:

“I would run the secure installation process, remove anonymous users and unnecessary test databases, restrict root access, create least-privilege application accounts, restrict network access to port 3306, use strong authentication, protect backups, and monitor logs.”

The Mental Model

Don't memorize 100 MySQL commands.

Remember the workflow:

INSTALL
   ↓
SECURE
   ↓
CREATE USERS
   ↓
GRANT LEAST PRIVILEGE
   ↓
CREATE DATABASE
   ↓
MONITOR
   ↓
BACKUP
   ↓
TEST RESTORE
   ↓
TUNE
   ↓
TROUBLESHOOT

That's database administration.

And this is the bigger DevOps lesson:

A database administrator isn't paid to know commands.

They're paid to prevent data loss, reduce downtime, control access, diagnose performance problems, and recover when something goes wrong.

Learn the commands.

But master the reasoning behind them.

Part 4: MySQL & MariaDB Database Administration

4.1 Installation & Setup

# Install MySQL
sudo apt update
sudo apt install mysql-server
sudo mysql_secure_installation         # ALWAYS run this after install
# Answer: Set root password, remove anonymous users, disable remote root login, remove test DB

# Install MariaDB (drop-in MySQL replacement)
sudo apt install mariadb-server
sudo mysql_secure_installation

# Service management
sudo systemctl start mysql
sudo systemctl stop mysql
sudo systemctl restart mysql
sudo systemctl status mysql
sudo systemctl enable mysql

# Check version
mysql --version

# Login
mysql -u root -p                       # Login as root with password prompt
mysql -u root                          # On Ubuntu, root uses socket auth by default
sudo mysql                             # If socket auth is configured
mysql -u username -p database_name     # Login to specific database
mysql -h 192.168.1.100 -u user -p     # Remote connection

4.2 User & Permission Management

-- View all users
SELECT user, host, plugin FROM mysql.user;

-- Create a new user
CREATE USER 'appuser'@'localhost' IDENTIFIED BY 'StrongP@ssw0rd!';
CREATE USER 'appuser'@'%' IDENTIFIED BY 'StrongP@ssw0rd!';  -- Allow from any host
CREATE USER 'appuser'@'192.168.1.%' IDENTIFIED BY 'StrongP@ssw0rd!';  -- From subnet

-- Grant permissions
GRANT ALL PRIVILEGES ON mydb.* TO 'appuser'@'localhost';  -- All on specific DB
GRANT SELECT, INSERT, UPDATE, DELETE ON mydb.* TO 'appuser'@'localhost';  -- Specific
GRANT ALL PRIVILEGES ON *.* TO 'admin'@'localhost' WITH GRANT OPTION;  -- Super admin

-- Revoke permissions
REVOKE ALL PRIVILEGES ON mydb.* FROM 'appuser'@'localhost';

-- Show grants for user
SHOW GRANTS FOR 'appuser'@'localhost';

-- Change password
ALTER USER 'appuser'@'localhost' IDENTIFIED BY 'NewP@ssw0rd!';

-- Delete user
DROP USER 'appuser'@'localhost';

-- Apply changes
FLUSH PRIVILEGES;

4.3 Database Operations

-- Show all databases
SHOW DATABASES;

-- Create database
CREATE DATABASE myapp_production CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- Use database
USE myapp_production;

-- Show tables
SHOW TABLES;

-- Describe table structure
DESCRIBE users;
SHOW CREATE TABLE users;

-- Show table sizes
SELECT table_name, 
       ROUND(data_length/1024/1024, 2) AS 'Data (MB)',
       ROUND(index_length/1024/1024, 2) AS 'Index (MB)',
       ROUND((data_length + index_length)/1024/1024, 2) AS 'Total (MB)'
FROM information_schema.tables
WHERE table_schema = 'myapp_production'
ORDER BY (data_length + index_length) DESC;

-- Show database sizes
SELECT table_schema AS 'Database',
       ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS 'Size (MB)'
FROM information_schema.tables
GROUP BY table_schema
ORDER BY SUM(data_length + index_length) DESC;

-- Basic CRUD Operations
-- Create
INSERT INTO users (name, email) VALUES ('John', 'john@example.com');

-- Read
SELECT * FROM users;
SELECT * FROM users WHERE email = 'john@example.com';
SELECT * FROM users ORDER BY created_at DESC LIMIT 10;
SELECT COUNT(*) FROM users;

-- Update
UPDATE users SET name = 'John Doe' WHERE id = 1;

-- Delete
DELETE FROM users WHERE id = 1;

-- Show running queries
SHOW PROCESSLIST;
SHOW FULL PROCESSLIST;

-- Kill a stuck query
KILL process_id;

-- Check server status
SHOW STATUS;
SHOW VARIABLES LIKE 'max_connections';
SHOW VARIABLES LIKE '%buffer%';

4.4 Backup & Restore (CRITICAL)

Backup Methods

# Single database backup
mysqldump -u root -p myapp_production > /backups/myapp_production_$(date +%Y%m%d).sql

# Single database with compression
mysqldump -u root -p myapp_production | gzip > /backups/myapp_production_$(date +%Y%m%d).sql.gz

# ALL databases backup
mysqldump -u root -p --all-databases > /backups/all_databases_$(date +%Y%m%d).sql

# Specific tables only
mysqldump -u root -p myapp_production users orders > /backups/tables_backup.sql

# Structure only (no data)
mysqldump -u root -p --no-data myapp_production > /backups/schema_only.sql

# Data only (no structure)
mysqldump -u root -p --no-create-info myapp_production > /backups/data_only.sql

# With routines, triggers, events
mysqldump -u root -p --routines --triggers --events myapp_production > /backups/full_backup.sql

# Large databases - add options for reliability
mysqldump -u root -p --single-transaction --quick --lock-tables=false myapp_production | gzip > /backups/myapp_production_$(date +%Y%m%d).sql.gz

Restore Methods

# Restore from SQL file
mysql -u root -p myapp_production < /backups/myapp_production_20240101.sql

# Restore from compressed file
gunzip < /backups/myapp_production_20240101.sql.gz | mysql -u root -p myapp_production

# Restore all databases
mysql -u root -p < /backups/all_databases_20240101.sql

# Create database first if it doesn't exist
mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS myapp_production CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
mysql -u root -p myapp_production < /backups/myapp_production_20240101.sql

Automated Backup Script

#!/bin/bash
# /usr/local/bin/mysql_backup.sh

DATE=$(date +%Y-%m-%d_%H-%M)
BACKUP_DIR="/backups/mysql"
RETENTION_DAYS=30
DB_USER="backup_user"
DB_PASS="secure_password"

# Create backup directory
mkdir -p $BACKUP_DIR

# Get list of all databases (excluding system databases)
DATABASES=$(mysql -u $DB_USER -p$DB_PASS -e "SHOW DATABASES" -s --skip-column-names | grep -Ev "(information_schema|performance_schema|sys|mysql)")

# Backup each database
for DB in $DATABASES; do
    echo "Backing up $DB..."
    mysqldump -u $DB_USER -p$DB_PASS --single-transaction --quick --routines --triggers $DB | gzip > $BACKUP_DIR/${DB}_${DATE}.sql.gz
    echo "Done: ${DB}_${DATE}.sql.gz"
done

# Delete old backups
find $BACKUP_DIR -name "*.sql.gz" -mtime +$RETENTION_DAYS -delete

echo "Backup completed at $DATE"

# Make executable and add to cron
chmod +x /usr/local/bin/mysql_backup.sh
# crontab -e
# 0 2 * * * /usr/local/bin/mysql_backup.sh >> /var/log/mysql_backup.log 2>&1

4.5 MySQL Performance Tuning

Key Configuration (`/etc/mysql/mysql.conf.d/mysqld.cnf`)

[mysqld]
# Connection settings
max_connections = 200
wait_timeout = 600
interactive_timeout = 600

# InnoDB settings (most important)
innodb_buffer_pool_size = 1G           # 50-70% of total RAM for dedicated DB server
innodb_log_file_size = 256M
innodb_flush_log_at_trx_commit = 2     # Better performance (1 = safest)
innodb_flush_method = O_DIRECT

# Query cache (MySQL 5.7, removed in 8.0)
# query_cache_type = 1
# query_cache_size = 64M

# Temporary tables
tmp_table_size = 64M
max_heap_table_size = 64M

# Logging
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 2                     # Log queries taking > 2 seconds

# Binary logging (for replication)
# log_bin = /var/log/mysql/mysql-bin.log
# server_id = 1
# expire_logs_days = 14

# Security
bind-address = 127.0.0.1               # Only allow local connections
# bind-address = 0.0.0.0               # Allow remote connections (use with firewall)

# Character set
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

Performance Monitoring

-- Check slow queries
SHOW VARIABLES LIKE 'slow_query_log%';
SHOW VARIABLES LIKE 'long_query_time';

-- Check connection usage
SHOW STATUS LIKE 'Threads_connected';
SHOW STATUS LIKE 'Max_used_connections';
SHOW VARIABLES LIKE 'max_connections';

-- Check InnoDB buffer pool usage
SHOW STATUS LIKE 'Innodb_buffer_pool%';

-- Explain a query (find performance issues)
EXPLAIN SELECT * FROM users WHERE email = 'john@example.com';
EXPLAIN ANALYZE SELECT * FROM users WHERE email = 'john@example.com';

-- Check indexes
SHOW INDEX FROM users;

-- Add index for performance
CREATE INDEX idx_email ON users(email);
ALTER TABLE users ADD INDEX idx_created (created_at);

-- Optimize table (after large deletes)
OPTIMIZE TABLE users;

-- Repair table
REPAIR TABLE users;

-- Check table integrity
CHECK TABLE users;

4.6 MySQL Remote Access Setup

# Step 1: Edit MySQL config
sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf
# Change: bind-address = 0.0.0.0

# Step 2: Create user with remote access
mysql -u root -p
CREATE USER 'remote_user'@'192.168.1.%' IDENTIFIED BY 'StrongP@ss!';
GRANT ALL PRIVILEGES ON mydb.* TO 'remote_user'@'192.168.1.%';
FLUSH PRIVILEGES;

# Step 3: Open firewall port
sudo ufw allow from 192.168.1.0/24 to any port 3306

# Step 4: Restart MySQL
sudo systemctl restart mysql

4.7 Common Issues & Fixes

Issue	Cause	Fix
`ERROR 1045: Access denied`	Wrong credentials or host	Check user@host, password, grants
`ERROR 2002: Can't connect to socket`	MySQL not running or wrong socket	`systemctl start mysql`, check socket path
`Too many connections`	Connection limit reached	Increase `max_connections`, check for leaks, add connection pooling
`Table is marked as crashed`	Corrupted table	`REPAIR TABLE tablename`
`Disk full`	No space for MySQL	Clear old logs, backups, increase disk
`Slow queries`	Missing indexes, bad queries	Enable slow query log, use EXPLAIN, add indexes
`Lock wait timeout exceeded`	Long-running transactions	Find and kill blocking queries with `SHOW PROCESSLIST`
MySQL won't start	Config error or corrupt files	Check `/var/log/mysql/error.log`

Interview Q&A: MySQL/MariaDB

Question	Answer
How do you backup a database?	`mysqldump -u root -p dbname \
How do you restore a database?	{% raw %}`gunzip < backup.sql.gz \
Difference between MySQL and MariaDB?	MariaDB is a fork of MySQL, community-driven, mostly compatible. Created by MySQL founder
What is InnoDB?	Default storage engine. Supports transactions, row-level locking, foreign keys
What is {% raw %}`innodb_buffer_pool_size`?	Memory area for caching data and indexes. Most important MySQL tuning parameter. Set to 50-70% of RAM
How do you find slow queries?	Enable slow query log, set `long_query_time`, use `EXPLAIN` on queries
How do you optimize a slow query?	Check EXPLAIN output, add proper indexes, avoid SELECT *, optimize JOINs
What port does MySQL use?	3306
How to check database size?	Query `information_schema.tables` with SUM of `data_length + index_length`
How do you secure MySQL?	Run `mysql_secure_installation`, remove anonymous users, restrict root, use strong passwords, bind to 127.0.0.1

DNS, SSL & PHP on Linux: Practical DevOps Guide

Yashvi Kothari — Wed, 12 Aug 2026 11:15:34 +0000

DNS, SSL & PHP on Linux: A Practical Guide for DevOps Engineers

Learn DNS records, SPF, DKIM, DMARC, SSL certificates, Let's Encrypt, PHP-FPM, Nginx, and PHP troubleshooting with practical Linux commands.

DNS, SSL & PHP on Linux: What Every DevOps Engineer Should Know

A website can be running perfectly on a server and still be unreachable.

Why?

Because infrastructure has layers.

Your application may be healthy.
Your web server may be running.
Your SSL certificate may be valid.

But if DNS points to the wrong IP, users never reach your server.

And if SSL is misconfigured, they may reach it—but receive a certificate error.

And if PHP-FPM is broken, Nginx may return a 502 Bad Gateway.

This is why DevOps engineers need to understand the entire path:

Domain → DNS → Server → Web Server → SSL/TLS → Application Runtime

Let's break down the most important concepts and commands.

1. Understanding DNS Records

DNS, or Domain Name System, translates human-readable domain names into information computers can use.

For example:

example.com → 192.168.1.100

The most important DNS records to know are:

Record	Purpose
A	Maps a domain to an IPv4 address
AAAA	Maps a domain to an IPv6 address
CNAME	Creates an alias to another domain
MX	Defines mail servers
TXT	Stores text data such as SPF and verification records
NS	Defines authoritative nameservers
SOA	Contains authority information for a DNS zone
PTR	Maps an IP address back to a domain
SRV	Defines the location of specific services
CAA	Controls which certificate authorities can issue certificates

The key distinction to remember:

A record → IP address

CNAME → another domain name

That simple difference appears frequently in DevOps and infrastructure interviews.

2. DNS Troubleshooting With Linux Commands

When a domain isn't resolving correctly, don't immediately blame the application.

Start with DNS.

Check the A record

dig example.com

Or:

dig example.com A

For a cleaner result:

dig +short example.com

Query a specific DNS resolver

Google DNS:

dig @8.8.8.8 example.com

Cloudflare DNS:

dig @1.1.1.1 example.com

This is useful when troubleshooting DNS propagation or resolver-specific problems.

Check MX records

dig example.com MX

Check TXT records

dig example.com TXT

Check nameservers

dig example.com NS

Check reverse DNS

dig -x 192.168.1.100

This checks the PTR record.

3. SPF, DKIM and DMARC

DNS isn't only about websites.

It's also critical for email security.

Three records you should understand are:

SPF

DKIM

DMARC

SPF

SPF specifies which servers are authorized to send email for a domain.

Example:

v=spf1 ip4:192.168.1.100 include:_spf.google.com include:sendgrid.net ~all

The important idea:

SPF answers: “Which servers are allowed to send email for this domain?”

DKIM

DKIM adds a cryptographic signature to outgoing email.

The receiving mail server can use that signature to verify that the message hasn't been improperly modified.

A DKIM record is commonly stored under:

selector._domainkey.example.com

DMARC

DMARC tells receiving mail servers what to do when SPF or DKIM checks fail.

Example:

v=DMARC1; p=quarantine; rua=mailto:dmarc@example.com; pct=100

Possible policies include:

p=none
p=quarantine
p=reject

A useful mental model is:

SPF → Who can send?

DKIM → Was the message cryptographically signed?

DMARC → What should happen if authentication fails?

4. SSL/TLS: Securing Web Traffic

DNS gets users to your server.

SSL/TLS secures the communication between the client and server.

Modern systems use TLS. SSL is the older terminology.

For most Linux web servers, HTTPS traffic is handled on:

HTTP commonly uses:

5. Let's Encrypt and Certbot

One of the most common ways to obtain free TLS certificates is Let's Encrypt.

Certbot automates much of the certificate process.

Install it:

sudo apt update
sudo apt install certbot

Apache

sudo apt install python3-certbot-apache
sudo certbot --apache -d example.com -d www.example.com

Nginx

sudo apt install python3-certbot-nginx
sudo certbot --nginx -d example.com -d www.example.com

Standalone mode

If no web server is running:

sudo certbot certonly --standalone -d example.com

Webroot mode

If a web server is already running:

sudo certbot certonly \
  --webroot \
  -w /var/www/example.com/public \
  -d example.com \
  -d www.example.com

6. SSL Renewal

Getting the certificate is only half the job.

You also need to make sure it doesn't expire.

Check certificates:

sudo certbot certificates

Test renewal:

sudo certbot renew --dry-run

Renew certificates:

sudo certbot renew

You can also verify the Certbot systemd timer:

sudo systemctl status certbot.timer

A good production habit is:

Don't wait for users to discover that your certificate expired.

Automate renewal and test it before you need it.

7. Troubleshooting SSL From the Command Line

When HTTPS isn't behaving correctly, OpenSSL is one of your best friends.

Check the TLS connection:

openssl s_client \
  -connect example.com:443 \
  -servername example.com

Check certificate dates:

echo | openssl s_client \
  -connect example.com:443 2>/dev/null |
  openssl x509 -noout -dates

Check certificate details:

echo | openssl s_client \
  -connect example.com:443 2>/dev/null |
  openssl x509 -noout -text

These commands help you investigate:

Certificate expiration
Certificate details
TLS connections
Certificate configuration

8. Common SSL Errors

`ERR_CERT_HAS_EXPIRED`

The certificate has expired.

Potential solution:

sudo certbot renew

`ERR_CERT_COMMON_NAME_INVALID`

The certificate doesn't match the requested domain.

For example:

Requested:
api.example.com

Certificate:
example.com

The certificate needs to cover the appropriate hostname.

Certificate chain incomplete

The server may be missing an intermediate certificate.

A full certificate chain may be required.

Mixed content

The main website uses HTTPS, but some resources still use HTTP.

For example:

https://example.com

loading:

http://example.com/style.css

That can trigger browser security warnings.

9. Installing PHP on Linux

PHP is commonly used for dynamic web applications.

With Apache:

sudo apt install php libapache2-mod-php

With Nginx, PHP typically runs through PHP-FPM:

sudo apt install php-fpm php-cli

PHP-FPM stands for:

PHP FastCGI Process Manager

It manages PHP worker processes separately from Nginx.

10. Why PHP-FPM Matters With Nginx

Nginx doesn't execute PHP code directly.

Instead, the architecture looks like this:

Client
   ↓
Nginx
   ↓
PHP-FPM
   ↓
PHP Application
   ↓
Database

Nginx receives the HTTP request.

PHP-FPM manages PHP processes.

The PHP application executes.

This separation is one reason PHP-FPM is commonly used with Nginx.

11. Configure PHP-FPM

A common PHP-FPM pool configuration is:

/etc/php/8.2/fpm/pool.d/www.conf

Important settings include:

user = www-data
group = www-data
listen = /var/run/php/php8.2-fpm.sock

PHP-FPM can communicate through a Unix socket or TCP:

listen = /var/run/php/php8.2-fpm.sock

or:

listen = 127.0.0.1:9000

Unix sockets are commonly used when Nginx and PHP-FPM are on the same server.

12. PHP-FPM Process Management

PHP-FPM supports different process manager modes:

pm = dynamic

Other options include:

static
ondemand

For dynamic mode, important settings include:

pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 35
pm.max_requests = 500

These settings control how PHP worker processes are created and managed.

The important operational lesson is:

PHP performance isn't only about PHP code. Process management matters too.

13. PHP Configuration

The main PHP configuration file is php.ini.

Find the configuration being used:

php --ini

You can also inspect specific settings:

php -i | grep memory_limit

php -i | grep upload_max_filesize

Common production settings include:

memory_limit = 256M
upload_max_filesize = 64M
post_max_size = 70M
max_execution_time = 300
max_input_time = 300
max_input_vars = 5000
date.timezone = Asia/Kolkata

For production:

display_errors = Off
log_errors = On

A production server should log errors rather than expose them directly to visitors.

14. OPcache for PHP Performance

PHP can improve performance by caching compiled bytecode.

That's where OPcache comes in.

Example configuration:

opcache.enable = 1
opcache.memory_consumption = 128
opcache.max_accelerated_files = 10000
opcache.revalidate_freq = 2

The principle is simple:

Don't repeatedly do work that can be cached safely.

This is one of the small configuration decisions that can have a meaningful effect on application performance.

15. Multiple PHP Versions

Sometimes different applications require different PHP versions.

You can install multiple PHP-FPM versions:

sudo apt install php7.4-fpm php8.0-fpm php8.1-fpm php8.2-fpm

You can also change the CLI version:

sudo update-alternatives --config php

The bigger advantage is that Nginx can route different websites to different PHP-FPM sockets.

For example:

location ~ \.php$ {
    fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
}

Another site could use:

location ~ \.php$ {
    fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
}

This allows different applications to coexist on the same server with different runtime requirements.

16. Composer

Composer is the package manager commonly used for PHP projects.

Install it and then use commands such as:

composer install

composer update

composer require package/name

composer dump-autoload

The important distinction:

composer install

is generally used to install the dependencies defined by the project.

Whereas:

composer update

updates dependency versions according to the project's dependency constraints.

17. A Practical Troubleshooting Mindset

Here's the most important lesson.

Don't troubleshoot randomly.

Follow the request path.

If a website isn't working, think:

1. Does DNS resolve?
        ↓
2. Does the server respond?
        ↓
3. Is the web server running?
        ↓
4. Is port 80/443 reachable?
        ↓
5. Is TLS configured correctly?
        ↓
6. Is Nginx/Apache routing correctly?
        ↓
7. Is PHP-FPM running?
        ↓
8. Is the PHP application working?
        ↓
9. Can the application reach its database?

This turns troubleshooting from guessing into a process.

Final Takeaway

DevOps isn't about memorizing hundreds of commands.

It's about understanding how systems connect.

DNS tells users where to go.

SSL/TLS protects the connection.

Nginx or Apache handles HTTP traffic.

PHP-FPM executes PHP workloads.

The application talks to its dependencies.

And Linux gives you the tools to inspect every layer.

If you understand that chain, troubleshooting becomes much easier.

Because instead of asking:

“Why is my website down?”

you start asking better questions:

“Where does the request fail?”

That's the mindset that turns a person who knows commands into an engineer who knows how systems work.

Quick Interview Revision

A vs CNAME?
A points to an IPv4 address. CNAME points to another domain name.

What does SPF do?
Defines authorized email senders for a domain.

What does DKIM do?
Provides a cryptographic signature for email authentication.

What does DMARC do?
Defines how receivers should handle SPF/DKIM authentication failures.

What is TLS?
A protocol used to secure network communication, including HTTPS.

What is PHP-FPM?
A process manager for PHP applications, commonly used with Nginx.

How do you check SSL expiry?

sudo certbot certificates

or:

echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -dates

How do you check DNS?

dig example.com

How do you test certificate renewal?

sudo certbot renew --dry-run

The biggest DevOps lesson?

Understand the request path. Troubleshoot the layer where the request actually fails.

Part 3: DNS, SSL Certificates & PHP Environments

3.1 DNS & Domain Configuration

DNS Record Types (MUST KNOW ALL)

Record	Purpose	Example
A	Maps domain to IPv4 address	`example.com → 192.168.1.100`
AAAA	Maps domain to IPv6 address	`example.com → 2001:db8::1`
CNAME	Alias pointing to another domain	`www.example.com → example.com`
MX	Mail server for the domain	`example.com → mail.example.com (priority 10)`
TXT	Text records (SPF, DKIM, verification)	`v=spf1 include:_spf.google.com ~all`
NS	Nameservers for the domain	`example.com → ns1.digitalocean.com`
SOA	Start of Authority (primary NS, admin email)	Auto-managed by DNS provider
PTR	Reverse DNS (IP to domain)	`192.168.1.100 → example.com`
SRV	Service location records	Used for SIP, XMPP, etc.
CAA	Certificate Authority Authorization	Controls who can issue SSL for domain

SPF, DKIM & DMARC (Email Authentication - CRITICAL)

# SPF (Sender Policy Framework) - TXT record
# Specifies which servers can send email for your domain
# Add as TXT record for @ (root domain):
v=spf1 ip4:192.168.1.100 include:_spf.google.com include:sendgrid.net ~all

# Explanation:
# v=spf1          - SPF version 1
# ip4:192.168.1.100 - Allow this IP to send mail
# include:_spf.google.com - Allow Google Workspace to send mail
# include:sendgrid.net - Allow SendGrid to send mail
# ~all            - Soft fail everything else (mark as suspicious)
# -all            - Hard fail everything else (reject)

# DKIM (DomainKeys Identified Mail) - TXT record
# Adds a digital signature to emails to verify they're not tampered
# Usually provided by your email service
# Add as TXT record for selector._domainkey.example.com:
# Name: google._domainkey
# Value: v=DKIM1; k=rsa; p=MIGfMA0GCS... (public key from provider)

# DMARC (Domain-based Message Authentication Reporting & Conformance) - TXT record
# Tells receivers what to do when SPF/DKIM fail
# Add as TXT record for _dmarc.example.com:
v=DMARC1; p=quarantine; rua=mailto:dmarc@example.com; pct=100

# Explanation:
# v=DMARC1        - DMARC version 1
# p=none          - Monitor only (start here)
# p=quarantine    - Send to spam if fails
# p=reject        - Reject email if fails
# rua=mailto:...  - Where to send aggregate reports
# pct=100         - Apply to 100% of emails

DNS Troubleshooting Commands

# Query DNS records
dig example.com                       # A record
dig example.com A                     # A record explicitly
dig example.com AAAA                  # IPv6
dig example.com MX                    # Mail records
dig example.com TXT                   # TXT records (SPF, DKIM)
dig example.com NS                    # Nameservers
dig example.com CNAME                 # CNAME records
dig example.com ANY                   # All records
dig +short example.com                # Short output
dig @8.8.8.8 example.com             # Query specific DNS server (Google)
dig @1.1.1.1 example.com             # Query Cloudflare DNS

# Alternative tools
nslookup example.com                  # Basic DNS lookup
nslookup -type=MX example.com         # MX records
host example.com                      # Simple DNS lookup

# Check DNS propagation
# Use online tools: dnschecker.org, whatsmydns.net

# Check PTR (reverse DNS)
dig -x 192.168.1.100

# Check current DNS cache
systemd-resolve --status              # systemd-resolved stats
systemd-resolve --flush-caches        # Flush DNS cache on Ubuntu

# Trace DNS resolution
dig +trace example.com                # Shows full DNS resolution path

# Check SPF record
dig TXT example.com | grep spf

# Check DKIM record
dig TXT google._domainkey.example.com

# Check DMARC record
dig TXT _dmarc.example.com

Practical DNS Setup Scenarios

# Scenario 1: Point domain to new server
# In DNS provider (Cloudflare/GoDaddy/Route53):
# Type: A    | Name: @   | Value: NEW_SERVER_IP  | TTL: 300
# Type: A    | Name: www | Value: NEW_SERVER_IP  | TTL: 300

# Scenario 2: Subdomain for staging
# Type: A    | Name: staging | Value: STAGING_SERVER_IP | TTL: 300

# Scenario 3: Setup Google Workspace email
# Type: MX   | Name: @   | Value: aspmx.l.google.com      | Priority: 1
# Type: MX   | Name: @   | Value: alt1.aspmx.l.google.com  | Priority: 5
# Type: MX   | Name: @   | Value: alt2.aspmx.l.google.com  | Priority: 5
# Type: TXT  | Name: @   | Value: v=spf1 include:_spf.google.com ~all

# Scenario 4: Cloudflare CDN setup
# Change nameservers to Cloudflare NS at domain registrar
# Cloudflare manages all DNS records after that

Interview Q&A: DNS

Question	Answer
What is DNS propagation?	Time for DNS changes to spread globally (can take up to 48 hours, usually 15 min - 4 hours)
How to speed up DNS propagation?	Lower TTL before making changes (e.g., 300 seconds), then change, then raise TTL back
A record vs CNAME?	A maps to IP, CNAME maps to another domain name. Root domain (@) cannot have CNAME
What is TTL?	Time To Live - how long DNS resolvers cache the record (in seconds)
What happens if MX records are wrong?	Email for your domain won't be delivered
How to verify DNS changes?	`dig @8.8.8.8 example.com`, use dnschecker.org, whatsmydns.net
What is reverse DNS?	Maps IP to domain name (PTR record). Important for email delivery
Why is SPF important?	Prevents email spoofing by specifying authorized mail servers
What does DKIM do?	Adds cryptographic signature to emails proving they're not tampered
What does DMARC do?	Tells receiving servers what to do when SPF/DKIM checks fail

3.2 SSL Certificate Management

Let's Encrypt (Free SSL - Most Common)

# Install Certbot
sudo apt update
sudo apt install certbot

# For Apache
sudo apt install python3-certbot-apache
sudo certbot --apache -d example.com -d www.example.com

# For Nginx
sudo apt install python3-certbot-nginx
sudo certbot --nginx -d example.com -d www.example.com

# Standalone (when no web server is running)
sudo certbot certonly --standalone -d example.com

# Webroot (web server running, no plugin)
sudo certbot certonly --webroot -w /var/www/example.com/public -d example.com -d www.example.com

# Wildcard certificate (covers *.example.com)
sudo certbot certonly --manual --preferred-challenges dns -d "*.example.com" -d example.com
# This requires adding a TXT record to DNS

# Check certificate expiry
sudo certbot certificates

# Renew all certificates
sudo certbot renew

# Renew with dry run (test)
sudo certbot renew --dry-run

# Auto-renewal cron (usually set up automatically)
# Verify timer exists:
sudo systemctl status certbot.timer

# Or add to crontab:
0 0,12 * * * certbot renew --quiet --post-hook "systemctl reload nginx"

Manual SSL Installation (Paid certificates)

# SSL files you'll receive:
# 1. certificate.crt     - Your domain certificate
# 2. ca_bundle.crt        - Intermediate/CA bundle
# 3. private.key          - Private key (you generated this)

# Step 1: Generate CSR (Certificate Signing Request)
openssl req -new -newkey rsa:2048 -nodes -keyout example.com.key -out example.com.csr
# Fill in: Country, State, City, Org, Common Name (example.com)

# Step 2: Submit CSR to certificate provider (GoDaddy, DigiCert, Comodo, etc.)

# Step 3: Download and install certificates
sudo mkdir -p /etc/ssl/example.com
sudo cp example.com.crt /etc/ssl/example.com/
sudo cp ca_bundle.crt /etc/ssl/example.com/
sudo cp example.com.key /etc/ssl/example.com/
sudo chmod 600 /etc/ssl/example.com/example.com.key

# Step 4: Configure in Nginx
ssl_certificate /etc/ssl/example.com/example.com.crt;
ssl_certificate_key /etc/ssl/example.com/example.com.key;
# Note: you may need to combine cert + CA bundle:
cat example.com.crt ca_bundle.crt > fullchain.crt

# Step 4b: Configure in Apache
SSLCertificateFile /etc/ssl/example.com/example.com.crt
SSLCertificateKeyFile /etc/ssl/example.com/example.com.key
SSLCACertificateFile /etc/ssl/example.com/ca_bundle.crt

SSL Troubleshooting

# Check SSL certificate from command line
openssl s_client -connect example.com:443 -servername example.com

# Check certificate expiry date
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -dates

# Check certificate details
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -text

# Verify certificate chain
openssl verify -CAfile ca_bundle.crt example.com.crt

# Check if private key matches certificate
openssl x509 -noout -modulus -in cert.crt | openssl md5
openssl rsa -noout -modulus -in private.key | openssl md5
# Both MD5 hashes should match!

# Online tools for SSL checking:
# ssllabs.com/ssltest
# sslshopper.com/ssl-checker.html

Common SSL Issues & Fixes

Issue	Cause	Fix
ERR_CERT_HAS_EXPIRED	Certificate expired	Renew with `certbot renew` or get new cert
ERR_CERT_COMMON_NAME_INVALID	Domain doesn't match cert	Issue cert for correct domain name
Mixed content warnings	HTTP resources on HTTPS page	Update all URLs to HTTPS in code/DB
Certificate chain incomplete	Missing intermediate cert	Add CA bundle/intermediate certificate
SSL handshake failed	Protocol/cipher mismatch	Update ssl_protocols, check TLS version
Let's Encrypt rate limit	Too many certs in short time	Wait, use staging for testing

Interview Q&A: SSL

Question	Answer
What is SSL/TLS?	Encryption protocol securing data between browser and server. SSL is deprecated, TLS is current (1.2/1.3)
How does Let's Encrypt work?	Free, automated CA. Validates domain ownership via HTTP challenge or DNS challenge, issues 90-day certs
How do you handle SSL renewal?	Certbot auto-renewal via systemd timer or cron job, runs twice daily
What is a wildcard SSL?	Covers all subdomains: *.example.com. Requires DNS challenge for Let's Encrypt
What is HSTS?	HTTP Strict Transport Security. Forces browsers to always use HTTPS
What is a CSR?	Certificate Signing Request. Contains public key + domain info, sent to CA to get certificate
How to troubleshoot "certificate not trusted"?	Check chain is complete, intermediate certs included, cert matches domain
What's the difference between DV, OV, EV certs?	DV: domain validation only. OV: org verified. EV: extended validation (green bar, most trust)

3.3 PHP Server Environment

PHP Installation & Management

# Install PHP (with Apache)
sudo apt install php libapache2-mod-php

# Install PHP (with Nginx - uses PHP-FPM)
sudo apt install php-fpm php-cli

# Install specific PHP version
sudo add-apt-repository ppa:ondrej/php    # Add PHP repository
sudo apt update
sudo apt install php8.2-fpm php8.2-cli

# Install common PHP extensions
sudo apt install php8.2-mysql php8.2-pgsql php8.2-sqlite3  # Database
sudo apt install php8.2-curl php8.2-gd php8.2-mbstring     # Common
sudo apt install php8.2-xml php8.2-zip php8.2-intl         # Processing
sudo apt install php8.2-bcmath php8.2-soap php8.2-redis    # Additional
sudo apt install php8.2-imagick php8.2-opcache              # Performance

# Check PHP version
php -v

# List installed modules
php -m

# Find php.ini location
php --ini
php -i | grep "php.ini"

# Check specific PHP config
php -i | grep memory_limit
php -i | grep upload_max_filesize

PHP Configuration (`php.ini`)

# Find and edit php.ini
# For CLI:
sudo nano /etc/php/8.2/cli/php.ini
# For Apache:
sudo nano /etc/php/8.2/apache2/php.ini
# For FPM (Nginx):
sudo nano /etc/php/8.2/fpm/php.ini

# IMPORTANT settings to know:
memory_limit = 256M                    # Max memory per script
upload_max_filesize = 64M              # Max file upload size
post_max_size = 70M                    # Max POST data (must be > upload_max_filesize)
max_execution_time = 300               # Max script execution time (seconds)
max_input_time = 300                   # Max time to parse input
max_input_vars = 5000                  # Max input variables
date.timezone = Asia/Kolkata           # Timezone

# Error handling
display_errors = Off                   # NEVER On in production
log_errors = On                        # Always log errors
error_log = /var/log/php/error.log     # Error log path
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT

# Session settings
session.save_handler = files
session.save_path = "/var/lib/php/sessions"
session.gc_maxlifetime = 1440

# OPcache (Performance - MUST enable)
opcache.enable = 1
opcache.memory_consumption = 128
opcache.max_accelerated_files = 10000
opcache.revalidate_freq = 2

# After changes:
sudo systemctl restart php8.2-fpm     # For Nginx
sudo systemctl restart apache2         # For Apache

PHP-FPM Configuration (for Nginx)

# Pool config file
sudo nano /etc/php/8.2/fpm/pool.d/www.conf

# Key settings:
user = www-data
group = www-data
listen = /var/run/php/php8.2-fpm.sock  # Unix socket (preferred)
# OR
listen = 127.0.0.1:9000                # TCP socket

# Process manager settings
pm = dynamic                           # dynamic, static, or ondemand
pm.max_children = 50                   # Max worker processes
pm.start_servers = 5                   # Initial workers
pm.min_spare_servers = 5               # Min idle workers
pm.max_spare_servers = 35              # Max idle workers
pm.max_requests = 500                  # Requests before worker restart (prevents memory leaks)

# Status page (for monitoring)
pm.status_path = /php-fpm-status

# Slow log (for debugging)
slowlog = /var/log/php/slow.log
request_slowlog_timeout = 5s           # Log requests taking > 5s

# Restart PHP-FPM
sudo systemctl restart php8.2-fpm
sudo systemctl status php8.2-fpm

Multiple PHP Versions

# Install multiple versions
sudo apt install php7.4-fpm php8.0-fpm php8.1-fpm php8.2-fpm

# Switch CLI version
sudo update-alternatives --set php /usr/bin/php8.2
sudo update-alternatives --config php    # Interactive selection

# Use different PHP versions for different sites in Nginx:
# Site 1 (PHP 7.4):
location ~ \.php$ {
    fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
}

# Site 2 (PHP 8.2):
location ~ \.php$ {
    fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
}

Composer (PHP Package Manager)

# Install Composer
curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer

# Usage
composer install                       # Install dependencies from composer.json
composer update                        # Update dependencies
composer require package/name          # Add new package
composer dump-autoload                 # Regenerate autoloader

Interview Q&A: PHP

Question	Answer
What is PHP-FPM?	FastCGI Process Manager. Manages PHP worker processes. Used with Nginx
Difference between mod_php and PHP-FPM?	mod_php runs inside Apache (simpler). PHP-FPM is separate process (better performance, works with Nginx)
How to increase file upload limit?	Change `upload_max_filesize` and `post_max_size` in php.ini, also `client_max_body_size` in Nginx
What is OPcache?	Caches compiled PHP bytecode in memory, massive performance improvement
How to check which PHP extensions are installed?	`php -m` or `php -i`
What causes a PHP memory limit error?	Script needs more memory than `memory_limit` in php.ini. Increase it or optimize code
How to handle multiple PHP versions?	Install via ondrej PPA, each version runs its own FPM pool, configure per-site in Nginx
Where do PHP errors go?	Check `error_log` in php.ini, or `/var/log/php/error.log`, or site-specific error log
What is `pm = dynamic` vs `static` vs `ondemand`?	Dynamic: scales workers within limits. Static: fixed workers. Ondemand: creates workers only when needed

Apache vs Nginx: Linux Web Server Administration Guide

Yashvi Kothari — Wed, 12 Aug 2026 10:35:49 +0000

Apache vs Nginx: Which Web Server Should a DevOps Engineer Learn?

Most people learning DevOps jump straight into Docker, Kubernetes, Terraform, and CI/CD.

But there is a layer underneath all of them that is easy to overlook:

The web server.

When a browser requests:

https://example.com

something has to receive that request, decide what should happen to it, serve static files, terminate SSL, or forward the request to an application.

That is where web servers such as Apache and Nginx become essential.

Understanding both isn't just useful for interviews.

It helps you understand how real Linux-based applications are deployed.

What Is a Web Server?

A web server accepts HTTP/HTTPS requests and returns responses to clients.

A simplified architecture looks like this:

A web server can therefore do much more than simply serve an index.html file.

It can handle:

HTTP/HTTPS
SSL/TLS termination
Virtual hosting
Reverse proxying
Load balancing
Static file delivery
Access and error logging
Compression
Security headers
Request routing
Apache Web Server Administration

Apache is one of the most established web servers in the Linux ecosystem.

On Ubuntu, installation is straightforward:

sudo apt update
sudo apt install apache2

Start Apache:

sudo systemctl start apache2

Enable it at boot:

sudo systemctl enable apache2

Check its status:

sudo systemctl status apache2

But there is one command every Linux administrator should get into the habit of using:

sudo apache2ctl configtest

It checks the configuration before you reload or restart Apache.

Never blindly restart a production web server after changing its configuration.

Test first.

Apache Virtual Hosts

One of Apache's most important concepts is the Virtual Host.

Virtual hosts allow one server to host multiple websites.

For example:

example.com
blog.example.com
shop.example.com

Each can have its own configuration.

A basic virtual host might look like:

ServerName example.com
ServerAlias www.example.com

DocumentRoot /var/www/example.com/public

<Directory /var/www/example.com/public>
    Options -Indexes +FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

ErrorLog ${APACHE_LOG_DIR}/example.com_error.log
CustomLog ${APACHE_LOG_DIR}/example.com_access.log combined

Then enable the site:

sudo a2ensite example.com.conf

Test the configuration:

sudo apache2ctl configtest

Reload Apache:

sudo systemctl reload apache2

The important pattern is:

Edit
↓
Test
↓
Reload

Not:

Edit
↓
Restart
↓
Hope
Apache as a Reverse Proxy

Apache can also sit in front of an application.

Suppose your Node.js application runs on:

127.0.0.1:3000

Apache can expose it publicly:

Internet
|
v
Apache :80
|
v
Node.js :3000

The configuration uses:

ProxyPreserveHost On
ProxyPass / http://127.0.0.1:3000/
ProxyPassReverse / http://127.0.0.1:3000/

This is a fundamental DevOps pattern.

The application doesn't necessarily need to listen directly on the public interface.

The web server becomes the controlled entry point.

Nginx Web Server Administration

Nginx is another essential tool for Linux and DevOps engineers.

Install it with:

sudo apt update
sudo apt install nginx

Start it:

sudo systemctl start nginx

Enable it at boot:

sudo systemctl enable nginx

Check the service:

sudo systemctl status nginx

Before changing anything in production:

sudo nginx -t

If the configuration is valid, reload:

sudo systemctl reload nginx
Nginx Server Blocks

Nginx uses server blocks for virtual hosting.

A typical configuration looks like:

server {
listen 80;
server_name example.com www.example.com;

root /var/www/example.com/public;
index index.html;

location / {
    try_files $uri $uri/ /index.php?$query_string;
}

}

A common Linux workflow is:

sudo nano /etc/nginx/sites-available/example.com

Then create the symbolic link:

sudo ln -s /etc/nginx/sites-available/example.com \
/etc/nginx/sites-enabled/

Test:

sudo nginx -t

Reload:

sudo systemctl reload nginx
Nginx as a Reverse Proxy

This is where Nginx becomes especially useful in modern application deployments.

Imagine your application runs on:

127.0.0.1:3000

Nginx can receive public traffic on port 80:

User
|
v
Nginx :80
|
v
Application :3000

Example:

server {
listen 80;
server_name nodeapp.com;

location / {
    proxy_pass http://127.0.0.1:3000;

    proxy_http_version 1.1;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

}

Now the user doesn't need to know that the application is running on port 3000.

Nginx handles the public-facing request.

Nginx Load Balancing

Nginx can also distribute traffic across multiple backend servers.

For example:

             +--> App Server 1
             |

Client --> Nginx +--> App Server 2
|
+--> App Server 3

A basic upstream configuration:

upstream backend {
least_conn;

server 127.0.0.1:3001;
server 127.0.0.1:3002;
server 127.0.0.1:3003;

}

Then proxy requests to it:

server {
listen 80;
server_name app.example.com;

location / {
    proxy_pass http://backend;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}

}

Instead of sending every request to one application instance, Nginx can distribute requests across multiple backends.

This becomes important when building applications that need greater availability and scalability.

HTTPS and SSL

Modern applications should use HTTPS.

Both Apache and Nginx can terminate SSL/TLS.

For Nginx, an HTTPS server block can include:

server {
listen 443 ssl http2;
server_name example.com www.example.com;

ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

ssl_protocols TLSv1.2 TLSv1.3;

}

A common architecture is:

Client
|
HTTPS
|
v
Nginx
|
HTTP/internal network
|
v
Application

This allows Nginx to handle TLS while the backend application focuses on application logic.

Apache vs Nginx

So which one should you learn?

The answer isn't simply:

"Nginx is better."

The tools solve overlapping problems but have different strengths.

The practical lesson:

Learn both.

You don't want to be the engineer who knows only one command because every production environment is different.
Apache and Nginx Can Work Together

You don't always have to choose one.

A common architecture is:

Internet
|
v
Nginx
|
+---- Static files
|
+---- Apache
|
+---- PHP/Application

Nginx can act as the public-facing reverse proxy while Apache handles application-specific functionality.

The important thing isn't choosing a "winner."

It's understanding where each component belongs in the request path.

Restart vs Reload: A Small Detail That Matters

This is one of those concepts that looks trivial until you're troubleshooting production.

Restart
sudo systemctl restart nginx

The service is stopped and started again.

Reload
sudo systemctl reload nginx

Nginx re-reads its configuration without completely stopping the service.

That's why configuration changes often follow this pattern:

sudo nginx -t
sudo systemctl reload nginx

The principle is simple:

Validate first. Change second.

Important Linux Web Server Files

If you're preparing for a DevOps interview, know where the configuration lives.

Apache
/etc/apache2/apache2.conf
/etc/apache2/ports.conf
/etc/apache2/sites-available/
/etc/apache2/sites-enabled/
/etc/apache2/mods-available/
/etc/apache2/mods-enabled/
/var/log/apache2/
/var/www/html/
Nginx
/etc/nginx/nginx.conf
/etc/nginx/sites-available/
/etc/nginx/sites-enabled/
/etc/nginx/conf.d/
/var/log/nginx/
/var/www/html/

Knowing these paths makes troubleshooting much faster.

The Most Important Troubleshooting Habit

When a web server stops working, don't randomly restart services.

Work from the bottom up.

Check the service sudo systemctl status nginx
Validate configuration sudo nginx -t
Check listening ports sudo ss -tulpn
Check logs sudo tail -f /var/log/nginx/error.log
Test locally curl http://127.0.0.1
Check connectivity

Then investigate:

firewall rules
DNS
security groups
backend availability
application ports
file permissions

This approach is much more reliable than guessing.

What DevOps Engineers Should Actually Learn

Don't memorize hundreds of configuration directives.

Understand the request flow:

DNS
↓
Load Balancer
↓
Nginx / Apache
↓
Reverse Proxy
↓
Application
↓
Database

Then learn how to troubleshoot each layer.

Because production incidents rarely announce themselves with:

"The problem is on line 47 of nginx.conf."

Instead, you get:

502 Bad Gateway
Connection refused
403 Forbidden
404 Not Found
500 Internal Server Error

Your job is to find which layer failed and why.

Final Takeaway

Apache and Nginx aren't just "web server packages."

They are fundamental building blocks of Linux infrastructure.

If you're serious about DevOps, learn how to:

Install and manage Apache and Nginx
Configure virtual hosts/server blocks
Serve static websites
Configure HTTPS
Use reverse proxies
Configure load balancing
Read access and error logs
Test configurations safely
Understand reload vs restart
Troubleshoot ports and connectivity

The deeper lesson is even more important:

DevOps isn't about memorizing commands.

It's about understanding how systems communicate.

Once you understand the request path, the commands start making sense.

And that's when Linux administration stops feeling like a collection of random commands—and starts becoming engineering.

Frequently Asked Questions
Is Apache better than Nginx?

Not universally. Apache is particularly useful when .htaccess and per-directory configuration are important. Nginx is commonly used for high-concurrency workloads, static content, reverse proxying, and load balancing.

Is Nginx a web server or a reverse proxy?

Both. Nginx can directly serve static content and can also act as a reverse proxy in front of applications.

What command checks Nginx configuration?
sudo nginx -t
What command checks Apache configuration?
sudo apache2ctl configtest
What is a reverse proxy?

A reverse proxy receives client requests and forwards them to backend servers or applications.

Why use Nginx in front of an application?

It can provide a controlled public entry point while handling tasks such as TLS termination, request routing, static content, and load balancing.

Should DevOps engineers learn Apache and Nginx?

Yes. Understanding both makes it easier to work with existing Linux infrastructure and troubleshoot real production environments.

If you remember only one thing:

Don't memorize the web server. Understand the request.

That mindset will take you much further in DevOps than memorizing configuration files.

Part 2: Apache & Nginx Web Server Administration

2.1 Apache Web Server

Installation & Basic Management

# Install Apache
sudo apt update
sudo apt install apache2

# Service management
sudo systemctl start apache2
sudo systemctl stop apache2
sudo systemctl restart apache2
sudo systemctl reload apache2          # Reload config without downtime
sudo systemctl enable apache2          # Start on boot
sudo systemctl status apache2

# Test configuration before restart
sudo apache2ctl configtest             # ALWAYS do this before restart
sudo apachectl -t                      # Alternative

# Check Apache version
apache2 -v

# List enabled modules
apache2ctl -M

Apache Virtual Hosts (CRITICAL - You WILL be asked)

# Create virtual host config
sudo nano /etc/apache2/sites-available/example.com.conf

# /etc/apache2/sites-available/example.com.conf
<VirtualHost *:80>
    ServerName example.com
    ServerAlias www.example.com
    ServerAdmin admin@example.com
    DocumentRoot /var/www/example.com/public

    <Directory /var/www/example.com/public>
        Options -Indexes +FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/example.com_error.log
    CustomLog ${APACHE_LOG_DIR}/example.com_access.log combined
</VirtualHost>

# HTTPS Virtual Host (with SSL)
<VirtualHost *:443>
    ServerName example.com
    ServerAlias www.example.com
    DocumentRoot /var/www/example.com/public

    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem

    <Directory /var/www/example.com/public>
        Options -Indexes +FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/example.com_ssl_error.log
    CustomLog ${APACHE_LOG_DIR}/example.com_ssl_access.log combined
</VirtualHost>

# HTTP to HTTPS redirect
<VirtualHost *:80>
    ServerName example.com
    ServerAlias www.example.com
    Redirect permanent / https://example.com/
</VirtualHost>

# Enable the site
sudo a2ensite example.com.conf

# Disable a site
sudo a2dissite example.com.conf

# Enable required modules
sudo a2enmod rewrite                   # For .htaccess URL rewriting
sudo a2enmod ssl                       # For HTTPS
sudo a2enmod headers                   # For security headers
sudo a2enmod proxy                     # For reverse proxy
sudo a2enmod proxy_http                # For HTTP reverse proxy
sudo a2enmod expires                   # For caching

# Disable a module
sudo a2dismod module_name

# Create document root and set permissions
sudo mkdir -p /var/www/example.com/public
sudo chown -R www-data:www-data /var/www/example.com
sudo chmod -R 755 /var/www/example.com

# Test and reload
sudo apache2ctl configtest
sudo systemctl reload apache2

Apache as Reverse Proxy (for Node.js, Python apps)

# /etc/apache2/sites-available/nodeapp.com.conf
<VirtualHost *:80>
    ServerName nodeapp.com

    ProxyPreserveHost On
    ProxyPass / http://127.0.0.1:3000/
    ProxyPassReverse / http://127.0.0.1:3000/

    ErrorLog ${APACHE_LOG_DIR}/nodeapp_error.log
    CustomLog ${APACHE_LOG_DIR}/nodeapp_access.log combined
</VirtualHost>

Apache .htaccess (Common Rules)

# /var/www/example.com/public/.htaccess

# Enable URL rewriting
RewriteEngine On

# Force HTTPS
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

# Remove www
RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
RewriteRule ^(.*)$ https://%1/$1 [R=301,L]

# Laravel/PHP Framework front controller
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]

# Deny access to sensitive files
<FilesMatch "\.(env|htpasswd|ini|log|sh|sql)$">
    Require all denied
</FilesMatch>

# Security headers
Header always set X-Content-Type-Options "nosniff"
Header always set X-Frame-Options "SAMEORIGIN"
Header always set X-XSS-Protection "1; mode=block"

# Enable compression
<IfModule mod_deflate.c>
    AddOutputFilterByType DEFLATE text/html text/css text/javascript application/javascript application/json
</IfModule>

# Browser caching
<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType image/jpg "access plus 1 year"
    ExpiresByType image/jpeg "access plus 1 year"
    ExpiresByType image/png "access plus 1 year"
    ExpiresByType text/css "access plus 1 month"
    ExpiresByType application/javascript "access plus 1 month"
</IfModule>

Apache Key Files & Directories

Path	Purpose
`/etc/apache2/apache2.conf`	Main config file
`/etc/apache2/ports.conf`	Listening ports
`/etc/apache2/sites-available/`	Available virtual host configs
`/etc/apache2/sites-enabled/`	Enabled virtual hosts (symlinks)
`/etc/apache2/mods-available/`	Available modules
`/etc/apache2/mods-enabled/`	Enabled modules (symlinks)
`/var/log/apache2/error.log`	Error log
`/var/log/apache2/access.log`	Access log
`/var/www/html/`	Default document root

2.2 Nginx Web Server

Installation & Basic Management

# Install Nginx
sudo apt update
sudo apt install nginx

# Service management
sudo systemctl start nginx
sudo systemctl stop nginx
sudo systemctl restart nginx
sudo systemctl reload nginx
sudo systemctl enable nginx
sudo systemctl status nginx

# Test configuration (ALWAYS before restart/reload)
sudo nginx -t

# Check Nginx version
nginx -v                              # Version only
nginx -V                              # Version + compile options

Nginx Server Blocks (Virtual Hosts)

sudo nano /etc/nginx/sites-available/example.com

# /etc/nginx/sites-available/example.com

# HTTP - redirect to HTTPS
server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    return 301 https://example.com$request_uri;
}

# HTTPS
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name example.com www.example.com;

    # Document root
    root /var/www/example.com/public;
    index index.php index.html;

    # SSL
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Security headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

    # Logging
    access_log /var/log/nginx/example.com_access.log;
    error_log /var/log/nginx/example.com_error.log;

    # Main location
    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    # PHP processing (PHP-FPM)
    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        include fastcgi_params;
        fastcgi_read_timeout 300;
    }

    # Deny access to hidden files
    location ~ /\. {
        deny all;
        access_log off;
        log_not_found off;
    }

    # Static file caching
    location ~* \.(jpg|jpeg|png|gif|ico|css|js|woff2|svg)$ {
        expires 30d;
        add_header Cache-Control "public, immutable";
        access_log off;
    }

    # Block sensitive files
    location ~* \.(env|log|sql|sh|bak)$ {
        deny all;
    }

    # Gzip compression
    gzip on;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml text/javascript;
    gzip_min_length 256;
}

# Enable site (create symlink)
sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

# Disable site
sudo rm /etc/nginx/sites-enabled/example.com

# Test and reload
sudo nginx -t
sudo systemctl reload nginx

Nginx as Reverse Proxy (for Node.js, Python, etc.)

# /etc/nginx/sites-available/nodeapp.com
server {
    listen 80;
    server_name nodeapp.com;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
        proxy_read_timeout 300;
        proxy_connect_timeout 300;
    }
}

Nginx Load Balancing

# Load balancing across multiple backend servers
upstream backend {
    least_conn;                        # Load balancing method
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
    server 127.0.0.1:3003;
}

server {
    listen 80;
    server_name app.example.com;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Nginx Key Files & Directories

Path	Purpose
`/etc/nginx/nginx.conf`	Main config file
`/etc/nginx/sites-available/`	Available server block configs
`/etc/nginx/sites-enabled/`	Enabled server blocks (symlinks)
`/etc/nginx/conf.d/`	Additional config files
`/var/log/nginx/error.log`	Error log
`/var/log/nginx/access.log`	Access log
`/var/www/html/`	Default document root

Nginx Performance Tuning (`/etc/nginx/nginx.conf`)

worker_processes auto;                  # Match CPU cores
worker_connections 1024;                # Connections per worker

# In http block:
keepalive_timeout 65;
client_max_body_size 100M;             # Max upload size
server_tokens off;                      # Hide Nginx version

# Buffer sizes
client_body_buffer_size 10K;
client_header_buffer_size 1k;
large_client_header_buffers 4 8k;

# Timeouts
client_body_timeout 12;
client_header_timeout 12;
send_timeout 10;

2.3 Apache vs Nginx Comparison

Feature	Apache	Nginx
Architecture	Process/Thread-based	Event-driven, async
Performance	Good for dynamic content	Better for static content & high concurrency
Config style	`.htaccess` per-directory	Centralized config only
PHP handling	mod_php (built-in)	PHP-FPM (external)
Reverse proxy	mod_proxy	Built-in, excellent
Memory usage	Higher	Lower
Use case	Shared hosting, .htaccess needed	High-traffic, reverse proxy, microservices
Config test	`apache2ctl configtest`	`nginx -t`
Reload	`systemctl reload apache2`	`systemctl reload nginx`

Interview Q&A: Web Servers

Question	Answer
When would you use Apache over Nginx?	When you need `.htaccess` support, mod_php, or per-directory config (shared hosting)
When would you use Nginx over Apache?	High-traffic sites, reverse proxy, static files, lower memory footprint
Can you use both together?	Yes! Nginx as reverse proxy in front of Apache. Nginx handles static files, Apache handles PHP
What is `try_files`?	Nginx directive that checks for files in order, falls back to last option (usually index.php for frameworks)
What's `AllowOverride All` in Apache?	Enables .htaccess files in that directory to override server config
How to test config before restarting?	Apache: `apache2ctl configtest`. Nginx: `nginx -t`
What's the difference between restart and reload?	Restart: stops and starts (brief downtime). Reload: re-reads config without stopping (zero downtime)
What is `proxy_pass`?	Nginx directive to forward requests to another server (reverse proxy)
How do you handle file upload size limits?	Apache: `LimitRequestBody` or PHP `upload_max_filesize`. Nginx: `client_max_body_size`
What is `worker_processes auto`?	Sets Nginx worker processes to match CPU cores automatically

Linux Server Administration: The Skills That Actually Make You Dangerous in Production

Yashvi Kothari — Wed, 12 Aug 2026 08:57:53 +0000

Linux Server Administration: The Skills That Actually Make You Dangerous in Production

Most people learning DevOps start with Kubernetes.

That’s backwards.

Before containers, pipelines, cloud architecture, or Kubernetes, there is a machine.

And that machine is usually running Linux.

If you cannot confidently answer:

What is using port 80?
Why did Nginx fail?
Where did the disk space go?
Which process is consuming CPU?
Who owns this file?
Why can't this user SSH?
Why didn't the cron job run?
Why did the service stop after reboot?

then Kubernetes won't save you.

Linux server administration is the foundation underneath modern infrastructure.

1. Learn to Think Like a Server Administrator

A junior administrator often asks:

"What command should I run?"

A strong administrator asks:

"What evidence do I need?"

That's the difference.

If a website is down, don't randomly restart services.

Build a chain of evidence:

systemctl status nginx

Is the service running?

Then:

sudo ss -tlnp

Is something actually listening on the expected port?

Then:

sudo journalctl -u nginx

What does the service log say?

Then:

df -h

Is the server out of disk space?

Then inspect the application and web-server logs.

The goal isn't to memorize 500 commands.

The goal is to know which command answers which question.

2. Master the Linux Filesystem

You should be able to navigate a server without thinking.

pwd
ls -lah
cd /var/www/html
cd ..
cd ~

Then learn the basic file operations:

cp file.txt /backup/
mv old.txt new.txt
rm file.txt
mkdir -p /var/www/site/logs
touch newfile.txt

And become comfortable reading logs:

less /var/log/syslog
tail -f /var/log/apache2/error.log

That last command is especially important.

Production troubleshooting often means watching a log while reproducing a problem.

3. Learn How to Find Problems

Linux gives you powerful tools for searching.

For example:

grep "error" /var/log/syslog

Search recursively:

grep -r "db_password" /var/www/

Find files:

find / -name "php.ini"

Find old logs:

find /var/www -name "*.log" -mtime +30

And when disk space disappears:

df -h

Then:

du -sh /var/www/*

The pattern is simple:

Detect → Locate → Investigate → Fix → Verify

That's server administration.

4. Permissions Are Not Optional

Linux permissions are one of the first things interviewers test.

Understand:

-rwxrwxrwx

It represents:

Owner | Group | Others

And:

r = 4
w = 2
x = 1

So:

chmod 755 directory

means:

Owner  = rwx
Group  = r-x
Others = r-x

While:

chmod 644 file

means:

Owner  = rw-
Group  = r--
Others = r--

For sensitive files:

chmod 600 .env

And don't forget ownership:

chown www-data:www-data /var/www/html -R

A web application can be perfectly configured and still fail because the process cannot read or write the required files.

5. SSH Is Your Remote Hands

If you're managing Linux servers, SSH becomes one of your most important tools.

Basic connection:

ssh user@server

Custom port:

ssh -p 2222 user@server

Using an AWS private key:

ssh -i ~/.ssh/mykey.pem ubuntu@aws-ip

Better yet, use SSH keys.

ssh-keygen -t ed25519

Then configure hosts:

Host production
    HostName 192.168.1.100
    User deploy
    Port 2222
    IdentityFile ~/.ssh/production_key

Now:

ssh production

Simple.

Repeatable.

Less typing.

6. SSH Hardening Is More Than Changing the Port

Changing port 22 is not a security strategy by itself.

A stronger configuration includes:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
AllowUsers john deploy

And after changing SSH configuration:

sudo sshd -t

Only after validating the configuration should you apply the change.

And keep your current SSH session open while testing a new connection.

That tiny habit can save you from locking yourself out of a server.

7. Cron Turns Scripts Into Automation

A script becomes much more useful when it can run automatically.

Cron format:

* * * * * command
│ │ │ │ │
│ │ │ │ └── day of week
│ │ │ └──── month
│ │ └────── day of month
│ └──────── hour
└────────── minute

For example:

0 2 * * * /usr/local/bin/backup_db.sh

means:

Run the database backup every day at 2 AM.

Every five minutes:

*/5 * * * *

And one of the most important troubleshooting concepts:

>> /var/log/backup.log 2>&1

The command's output gets captured so you can investigate failures later.

Automation without observability is just scheduled confusion.

8. Systemd Controls Your Services

Modern Linux servers rely heavily on systemd.

Know these commands:

systemctl start nginx
systemctl stop nginx
systemctl restart nginx
systemctl reload nginx
systemctl status nginx
systemctl enable nginx

But don't stop at status.

When something fails:

sudo journalctl -u nginx

For live logs:

sudo journalctl -u nginx -f

One of the most useful interview questions is:

What's the difference between restart and reload?

Restart:

Stop the service and start it again.

Reload:

Reload configuration without fully stopping the service.

For services that support it, reload can avoid unnecessary downtime.

9. Processes Tell You What the Machine Is Actually Doing

When the server becomes slow, start looking at processes.

top

or:

htop

For a snapshot:

ps aux

Find a process:

pgrep -a nginx

And when necessary:

kill PID

A graceful termination should normally be preferred over:

kill -9 PID

SIGKILL is the hammer.

Don't use a hammer when a normal shutdown will work.

10. Ports Are One of Your Best Troubleshooting Clues

A service can be "running" and still be unreachable.

Check listening ports:

sudo ss -tlnp

Or:

sudo lsof -i :80

This answers an extremely important question:

"What process is actually listening on this port?"

If your application should listen on port 8080 but nothing is listening there, investigating DNS is probably not your first move.

Follow the evidence.

11. The Real DevOps Skill: Connecting the Dots

Imagine this production incident:

"The website is down."

Don't panic.

Ask:

Is the service running?

systemctl status nginx

Is the port listening?

sudo ss -tlnp

Are there errors?

sudo journalctl -u nginx

Is the disk full?

df -h

Is a process consuming resources?

top

What do the application logs say?

tail -f /var/log/...

Now you're troubleshooting.

Not guessing.

12. The Interview Questions You Should Be Able to Answer

If you're preparing for a Linux/DevOps/SysAdmin interview, know questions such as:

How do you find which process is using a port?

sudo lsof -i :80

or:

sudo ss -tlnp | grep :80

How do you find large files?

du -sh /* | sort -rh | head -20

How do you check whether a service is running?

systemctl status nginx

What does chmod 755 mean?

Owner has rwx; group and others have r-x.

How do you check the OS version?

cat /etc/os-release

How do you troubleshoot a failed service?

systemctl status service
journalctl -u service

What does 2>&1 mean?

It redirects stderr to stdout, allowing both streams to be captured together.

These aren't trivia questions.

They represent real operational tasks.

The Bigger Lesson

DevOps isn't primarily about knowing more tools.

It's about understanding systems.

Linux teaches you:

processes
networking
permissions
services
logs
storage
automation
security
troubleshooting

Once those fundamentals become second nature, tools like Docker, Kubernetes, Jenkins, Terraform, and AWS become much easier to understand.

Because you're no longer memorizing commands.

You're understanding what the infrastructure is doing.

Don't learn Linux to pass an interview.

Learn Linux so that when production breaks at 2 AM, you know where to look.

That's the skill that compounds.

Part 1: Linux Server Administration Fundamentals

[!TIP]
Practice every command on a free VPS (DigitalOcean $4/mo, AWS Free Tier, or a local VM using VirtualBox + Ubuntu Server).

1.1 Essential Linux Commands (Must Know Cold)

File & Directory Operations

# Navigation
pwd                          # Print current directory
ls -la                       # List all files with permissions, sizes
ls -lah                      # Human-readable sizes
cd /var/www/html             # Change directory
cd ..                        # Go up one level
cd ~                         # Go to home directory

# File operations
cp file.txt /backup/         # Copy file
cp -r /source/ /dest/        # Copy directory recursively
mv old.txt new.txt           # Rename/move
rm file.txt                  # Delete file
rm -rf /path/to/dir/         # Delete directory recursively (DANGEROUS)
mkdir -p /var/www/site/logs  # Create nested directories
touch newfile.txt            # Create empty file

# Viewing files
cat /etc/hosts               # View entire file
less /var/log/syslog         # Paginated view (q to quit)
head -n 50 error.log         # First 50 lines
tail -n 100 error.log        # Last 100 lines
tail -f /var/log/apache2/error.log  # Live follow (VERY common in troubleshooting)

# Searching
grep "error" /var/log/syslog          # Search for text in file
grep -r "db_password" /var/www/       # Recursive search
grep -i "warning" error.log           # Case-insensitive
grep -c "404" access.log              # Count matches
find / -name "php.ini"                # Find file by name
find /var/www -name "*.log" -mtime +30 -delete  # Delete logs older than 30 days
locate php.ini                        # Fast search (uses database, run updatedb first)

# Disk & Space
df -h                        # Disk space usage
du -sh /var/www/*             # Directory sizes
du -sh /var/log/*  | sort -rh | head -20  # Top 20 largest dirs in /var/log
ncdu /                       # Interactive disk usage (install: apt install ncdu)

# Text manipulation
sed -i 's/old/new/g' file.txt         # Find and replace in file
awk '{print $1}' access.log           # Print first column
wc -l access.log                      # Count lines
sort access.log | uniq -c | sort -rn  # Count unique lines, sorted

Interview Q&A: Linux Commands

Question	Answer
How do you find which process is using the most CPU?	`top` or `htop`, press `P` to sort by CPU
How do you find which process is using a specific port?	`sudo lsof -i :80` or `sudo netstat -tlnp \
How do you find large files consuming disk space?	{% raw %}`du -sh /* \
How do you check if a service is running?	{% raw %}`systemctl status nginx` or `ps aux \
How to check system uptime?	{% raw %}`uptime` command
What does `chmod 755` mean?	Owner: rwx (7), Group: r-x (5), Others: r-x (5)
What does `chmod 644` mean?	Owner: rw- (6), Group: r-- (4), Others: r-- (4)
Difference between `>` and `>>`?	`>` overwrites file, `>>` appends to file
What is a symlink? How to create?	Shortcut/pointer to another file. `ln -s /target /link_name`
How to check OS version?	`cat /etc/os-release` or `lsb_release -a`

1.2 Users, Groups & Permissions

User Management

# Create users
sudo adduser john                     # Interactive user creation (preferred on Ubuntu)
sudo useradd -m -s /bin/bash john     # Non-interactive, creates home dir, sets shell

# Set/change password
sudo passwd john

# Delete user
sudo userdel john                     # Delete user only
sudo userdel -r john                  # Delete user + home directory

# Modify user
sudo usermod -aG sudo john           # Add john to sudo group
sudo usermod -aG www-data john       # Add john to www-data group (for web files)
sudo usermod -s /usr/sbin/nologin john  # Disable shell login (for service accounts)

# View user info
id john                               # Show UID, GID, groups
whoami                                # Current user
groups john                           # List groups for user
cat /etc/passwd                       # All users
cat /etc/group                        # All groups

# Switch user
su - john                             # Switch to john (with john's environment)
sudo -u www-data command              # Run command as www-data user

File Permissions

# Permission format: -rwxrwxrwx (owner/group/others)
# r=4, w=2, x=1

# Common permissions for web servers
chmod 755 /var/www/html               # Directories: rwxr-xr-x
chmod 644 /var/www/html/index.html    # Files: rw-r--r--
chmod 600 /var/www/html/.env          # Sensitive files: rw-------
chmod 700 /home/john/.ssh             # SSH directory
chmod 600 /home/john/.ssh/authorized_keys  # SSH keys

# Ownership
chown www-data:www-data /var/www/html -R   # Change owner+group recursively
chown john:john file.txt                    # Change owner and group

# Special permissions
chmod +s /usr/bin/program              # SetUID - runs as file owner
chmod g+s /var/www/shared/             # SetGID - new files inherit group
chmod +t /tmp                          # Sticky bit - only owner can delete

Interview Q&A: Users & Permissions

Question	Answer
What permissions should web files have?	Directories: 755, Files: 644, Sensitive (.env, config): 600
What user should own web files?	`www-data:www-data` (for Apache/Nginx on Ubuntu)
What's the difference between `adduser` and `useradd`?	`adduser` is interactive, creates home dir, sets password. `useradd` is low-level, non-interactive
How to give a user sudo access?	`sudo usermod -aG sudo username`
What is `/etc/passwd` vs `/etc/shadow`?	`passwd` has user info (public), `shadow` has encrypted passwords (root only)
How do you prevent a user from logging in via SSH?	Set shell to `/usr/sbin/nologin` or `/bin/false`, or use `DenyUsers` in sshd_config
What is umask?	Default permission mask. umask 022 = new files get 644, dirs get 755

1.3 SSH (Secure Shell)

SSH Configuration & Usage

# Connect to server
ssh root@192.168.1.100                # Basic connection
ssh -p 2222 john@server.com           # Custom port
ssh -i ~/.ssh/mykey.pem ubuntu@aws-ip # Using private key (AWS)

# SSH Key Generation (on your local machine)
ssh-keygen -t ed25519 -C "john@company.com"   # Modern, preferred
ssh-keygen -t rsa -b 4096 -C "john@company.com"  # RSA alternative

# Copy public key to server
ssh-copy-id john@192.168.1.100
# OR manually:
cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys  # On server

# SSH Config file (~/.ssh/config) - saves time
# Create/edit: nano ~/.ssh/config
Host production
    HostName 192.168.1.100
    User john
    Port 2222
    IdentityFile ~/.ssh/production_key

Host staging
    HostName 192.168.1.101
    User deploy
    Port 22
    IdentityFile ~/.ssh/staging_key

# Now you can just type:
ssh production
ssh staging

SSH Hardening (Server-side: `/etc/ssh/sshd_config`)

# Edit SSH config
sudo nano /etc/ssh/sshd_config

# IMPORTANT changes:
Port 2222                         # Change default port (not 22)
PermitRootLogin no                # Disable root SSH login
PasswordAuthentication no         # Force key-based auth only
PubkeyAuthentication yes          # Enable key authentication
MaxAuthTries 3                    # Limit login attempts
AllowUsers john deploy            # Whitelist specific users
ClientAliveInterval 300           # Timeout after 5 min idle
ClientAliveCountMax 2             # Disconnect after 2 missed keepalives
Protocol 2                        # Use SSH protocol 2 only

# After changes, ALWAYS test before disconnecting:
sudo sshd -t                      # Test config for syntax errors
sudo systemctl restart sshd       # Apply changes

# IMPORTANT: Keep current session open, test new connection in another terminal!

Interview Q&A: SSH

Question	Answer
How do you harden SSH?	Change port, disable root login, disable password auth, use key-based auth, whitelist users, use fail2ban
What's the difference between SSH and SFTP?	SSH = remote shell access. SFTP = secure file transfer over SSH (port 22)
How do you transfer files via SSH?	`scp file.txt user@server:/path/` or use SFTP client (FileZilla, WinSCP)
What port does SSH use?	Default: 22 (should be changed in production)
What do you do if you're locked out of SSH?	Access via hosting provider's console/VNC, check sshd_config, check firewall rules, check fail2ban
Difference between SSH key types?	ed25519 (modern, fast, secure), RSA (legacy, use 4096-bit minimum)

1.4 SFTP & FTP

# SFTP (Secure - uses SSH, port 22) - PREFERRED
sftp john@server.com
sftp -P 2222 john@server.com          # Custom SSH port

# Inside SFTP session:
put localfile.txt /remote/path/        # Upload
get /remote/file.txt ./local/          # Download
ls                                      # List remote files
lls                                     # List local files
cd /var/www/html                        # Change remote directory
lcd ~/Desktop                           # Change local directory

# FTP (Insecure - port 21) - AVOID if possible
# If needed, use vsftpd:
sudo apt install vsftpd
sudo nano /etc/vsftpd.conf
# Key settings:
# anonymous_enable=NO
# local_enable=YES
# write_enable=YES
# chroot_local_user=YES     # Jail users to home directory
# ssl_enable=YES             # Enable FTPS (FTP over SSL)

sudo systemctl restart vsftpd

# SCP (Secure Copy)
scp file.txt john@server:/var/www/html/           # Upload file
scp -r /local/dir/ john@server:/remote/dir/       # Upload directory
scp john@server:/var/log/error.log ./              # Download file
scp -P 2222 file.txt john@server:/path/           # Custom port

Interview Q&A: File Transfer

Question	Answer
SFTP vs FTP vs SCP?	SFTP: secure, over SSH. FTP: insecure, legacy. SCP: secure copy, simpler but no resume
Why prefer SFTP over FTP?	SFTP encrypts data in transit, uses SSH port (no extra ports), supports key auth
What port does FTP use?	21 (control), 20 (data in active mode), random high ports (passive mode)

1.5 Cron Jobs

Setting Up Cron Jobs

# Edit crontab for current user
crontab -e

# Edit crontab for specific user (as root)
sudo crontab -u www-data -e

# View crontab
crontab -l
sudo crontab -u www-data -l

# Cron format:
# ┌───────── minute (0-59)
# │ ┌─────── hour (0-23)
# │ │ ┌───── day of month (1-31)
# │ │ │ ┌─── month (1-12)
# │ │ │ │ ┌─ day of week (0-7, 0 and 7 = Sunday)
# │ │ │ │ │
# * * * * * command_to_run

# PRACTICAL EXAMPLES:

# Database backup every day at 2 AM
0 2 * * * /usr/local/bin/backup_db.sh >> /var/log/backup.log 2>&1

# Website backup every Sunday at 3 AM
0 3 * * 0 /usr/local/bin/backup_website.sh >> /var/log/backup.log 2>&1

# Clear tmp files every hour
0 * * * * find /tmp -type f -mtime +1 -delete

# Check disk space every 30 minutes
*/30 * * * * /usr/local/bin/check_disk.sh

# SSL certificate auto-renew (Let's Encrypt) - twice daily
0 0,12 * * * certbot renew --quiet

# Restart PHP-FPM every night at 4 AM (performance)
0 4 * * * systemctl restart php8.2-fpm

# Run Laravel scheduler every minute
* * * * * cd /var/www/html && php artisan schedule:run >> /dev/null 2>&1

# Rotate logs weekly
0 0 * * 0 logrotate /etc/logrotate.conf

Sample Backup Script (`/usr/local/bin/backup_db.sh`)

#!/bin/bash
# Database backup script
DATE=$(date +%Y-%m-%d_%H-%M)
BACKUP_DIR="/backups/mysql"
DB_NAME="production_db"
DB_USER="backup_user"
DB_PASS="secure_password"

# Create backup directory
mkdir -p $BACKUP_DIR

# Dump database
mysqldump -u $DB_USER -p$DB_PASS $DB_NAME | gzip > $BACKUP_DIR/${DB_NAME}_${DATE}.sql.gz

# Delete backups older than 30 days
find $BACKUP_DIR -name "*.sql.gz" -mtime +30 -delete

# Log the backup
echo "[$DATE] Backup completed: ${DB_NAME}_${DATE}.sql.gz" >> /var/log/backup.log

# Make script executable
chmod +x /usr/local/bin/backup_db.sh

Interview Q&A: Cron Jobs

Question	Answer
What does `/5 * * *` mean?	Every 5 minutes
What does `0 2 * * *` mean?	Every day at 2:00 AM
What does `0 0 * * 0` mean?	Every Sunday at midnight
What does `2>&1` mean?	Redirects stderr to stdout (captures all output)
Where are system cron jobs stored?	`/etc/crontab`, `/etc/cron.d/`, `/etc/cron.daily/`, `/etc/cron.hourly/`
How to debug a cron job not running?	Check `/var/log/syslog \
What's the difference between {% raw %}`crontab -e` and `/etc/crontab`?	`crontab -e` is per-user, `/etc/crontab` is system-wide and includes username field

1.6 Systemd & Service Management

# Service management
sudo systemctl start nginx              # Start service
sudo systemctl stop nginx               # Stop service
sudo systemctl restart nginx             # Restart service
sudo systemctl reload nginx              # Reload config without downtime
sudo systemctl status nginx              # Check status
sudo systemctl enable nginx              # Start on boot
sudo systemctl disable nginx             # Don't start on boot
sudo systemctl is-active nginx           # Check if running
sudo systemctl is-enabled nginx          # Check if enabled on boot

# List all services
systemctl list-units --type=service
systemctl list-units --type=service --state=running

# View service logs
sudo journalctl -u nginx                # All logs for nginx
sudo journalctl -u nginx -f             # Follow logs live
sudo journalctl -u nginx --since "1 hour ago"
sudo journalctl -u nginx --since "2024-01-01" --until "2024-01-02"

# Custom service file example (/etc/systemd/system/myapp.service)
[Unit]
Description=My Python Application
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/var/www/myapp
ExecStart=/var/www/myapp/venv/bin/python app.py
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

# After creating/modifying service file:
sudo systemctl daemon-reload            # Reload systemd
sudo systemctl enable myapp             # Enable on boot
sudo systemctl start myapp              # Start the service

Interview Q&A: Services

Question	Answer
Difference between `restart` and `reload`?	`restart` stops and starts (brief downtime). `reload` reloads config without stopping (no downtime, preferred)
How to make a service start on boot?	`systemctl enable servicename`
How to check why a service failed?	`systemctl status servicename` and `journalctl -u servicename`
What is systemd?	Init system and service manager for Linux. Manages services, boot process, logging

1.7 Process Management

# View processes
top                                  # Real-time process viewer
htop                                 # Better interactive viewer (install: apt install htop)
ps aux                               # Snapshot of all processes
ps aux | grep nginx                  # Find specific process
pgrep -a nginx                       # Find process by name

# Kill processes
kill PID                             # Graceful termination (SIGTERM)
kill -9 PID                          # Force kill (SIGKILL)
killall nginx                        # Kill all processes by name
pkill -f "python app.py"             # Kill by full command match

# Background processes
command &                            # Run in background
nohup command &                      # Run in background, survive logout
screen -S session_name               # Create named screen session
screen -r session_name               # Reattach to session
tmux new -s session_name             # Create named tmux session
tmux attach -t session_name          # Reattach to session

# Check what's using a port
sudo lsof -i :80                     # What's on port 80?
sudo lsof -i :443                    # What's on port 443?
sudo ss -tlnp                        # All listening ports
sudo netstat -tlnp                   # All listening ports (legacy)

No Canvas. No SVG. No JavaScript. Just CSS and a Pressure Cooker Full of Pasta.

Yashvi Kothari — Thu, 06 Aug 2026 14:04:33 +0000

This is a submission for Frontend Challenge - Comfort Food Edition, CSS Art.

Inspiration

You've never looked at a pressure cooker and thought, "I should code that."

I did. And the result taught me more about creativity than any design tool ever has.

Let me show you what I built — and more importantly, why you should care even if you never write a line of CSS.

Demo

https://github.com/yashvikothari/yashvikothari.github.io/tree/master/lovepastaincss

yashvikothari.github.io

Journey

Challenge
A frontend community challenge landed in my feed: "CSS Art: Comfort Food Edition."

The rules were simple:

Pure HTML and CSS only.
No JavaScript.
No SVG.
No images.
Let food be your inspiration.
Most people reached for cupcakes. Pizza slices. Maybe a donut.

I reached for the dish I grew up eating: Elbow Pasta Masala cooked in a pressure cooker. Indian-style. Spicy tomato gravy. Butter. Cheese. The works.

Not because it was easy. Because constraints are where creativity lives.

The Elbow Macaroni Problem
Here's the thing about elbow pasta — it's a curve.

CSS doesn't do curves the way you'd think. There's no draw-macaroni() function. No arc tool. No pen path.

So I had to think differently.

The solution was embarrassingly elegant:

Take a square div. Give it border-radius: 50% — now it's a circle. But here's the trick: only paint two adjacent borders (top and right). Set the other two to transparent.

What's left? A thick, curved arc. A perfect elbow macaroni.

css

.elbow {
width: 28px;
height: 28px;
border-radius: 50%;
border: 8px solid #f0c060; /* pasta gold */
border-bottom-color: transparent;
border-left-color: transparent;
}
Four lines. That's the whole pasta.

Then I added a ::before for the specular light ridge and a ::after for the tomato sauce tint. Rotate each piece differently. Scatter 14 of them across a gravy pool.

Suddenly, it looked like food.

The Bigger Lesson: Constraints Create Leverage
Naval says: "Specific knowledge is found by pursuing your genuine curiosity."

Here's what I'd add: Specific creativity is found by embracing genuine constraints.

When you can't use images, you learn gradients. When you can't use JavaScript, you discover the :checked pseudo-class hack. When you can't use SVG, you learn that clip-path and border-radius can build almost anything.

Every limitation I hit became a forcing function for a better idea:

The cooker isn't just art. It's a state machine. You can:

Open and close the pressure cooker lid
Trigger a whistling steam explosion
Adjust the gas stove flame (high, simmer, off)
Toggle cheese, herbs, and garlic toast garnishes
Switch to a cozy night-kitchen mode
All without a single line of JavaScript.

Why This Matters Beyond Code
This isn't a CSS tutorial. This is a lesson about creative leverage.
The tool doesn't make the art. The limitation does.

The Takeaway (in one line)
Don't ask for more tools. Ask for fewer. Then watch what your brain invents.

Try It Yourself
The full source is two files. index.html and styles.css. No build step. No dependencies. No npm install. Open it in a browser and click "Open Lid."

If a pressure cooker full of elbow pasta can be beautiful, so can whatever you're building today — even inside your constraints.

Especially inside your constraints.

AWS Storage Gateway(Hybrid Storage Solution)

Yashvi Kothari — Sun, 04 Jan 2026 04:59:22 +0000

AWS Storage Gateway and its architecture

Which type of storage gateway to use for your use case, and

How to create a storage gateway using the EC2 platform

Using AWS Storage Gateway(Hybrid Storage Solution)

AWS Storage Gateway is a powerful service that enables hybrid cloud storage by seamlessly connecting on-premises environments with AWS cloud storage. In this blog, we’ll explore AWS Storage Gateway and its architecture, helping you understand how it works and how it fits into modern cloud and hybrid infrastructures.

We’ll also break down the different types of AWS Storage Gateway—File Gateway, Volume Gateway, and Tape Gateway—and discuss which gateway type to choose based on your specific use case.

To make things practical, the blog will walk you through creating and deploying an AWS Storage Gateway using the Amazon EC2 platform, demonstrating how to set up and configure the gateway step by step.

Prerequisites

To get the most out of this guide, you should have a solid understanding of AWS storage services, including:

Amazon S3

Amazon FSx

Amazon EBS

Amazon Glacier

Basic familiarity with Amazon EC2 is also recommended, as it will help you better understand the deployment and configuration steps covered in the hands-on sections.

The AWS Storage Gateway: The Missing Bridge Between On‑Prem and Cloud Fluency

Every hybrid cloud story begins at the same line: “We still have data on‑prem.”

But the real question AWS architects ask isn’t why — it’s how cleanly can we bridge that world to S3, FSx, or Glacier?

There’s a reason the AWS Well‑Architected Framework flags data movement and hybrid storage patterns in almost every cost and operations review. When on‑prem systems start talking to AWS, you either build a bridge — or a bottleneck.

What the Storage Gateway Really Is

Storage Gateway isn’t just a connector. It’s a translator layer that turns your local protocols — NFS, SMB, iSCSI — into native AWS storage semantics.

At its core, it’s a lightweight virtual machine or a hardware appliance that establishes a secure, encrypted data path to AWS. You can deploy it in three ways:

As a VM on VMware ESXi, Microsoft Hyper‑V, or Linux KVM — ideal for existing datacenter setups.
As a hardware appliance, pre‑configured and shipped by AWS for plug‑and‑play deployment.
As an EC2 instance inside your VPC or within VMware Cloud on AWS — perfect for hybrid use cases with cloud‑resident workloads.

Once deployed, the gateway allocates ~150 MB of local disk space for caching. This cache doesn’t just stage uploads — it works as a low‑latency read buffer, shaving milliseconds off retrieval time for your most frequently accessed objects.

The Four Personalities of Storage Gateway

Each gateway type reflects a different file access pattern, not a marketing SKU.

S3 File Gateway

Talk SMB or NFS. Store natively in S3. Perfect for backup workloads, shared datasets, or applications that don’t speak API‑native S3.
FSx File Gateway

Serve SMB workloads that demand Windows‑native semantics — think ACLs, DFS namespaces, and domain integration — backed by FSx for Windows File Server.
Tape Gateway

The quiet killer of legacy tape libraries. It abstracts your physical tape system into iSCSI‑based virtual tapes, with automatic tiering into Amazon S3 Glacier Flexible Retrieval or Deep Archive.
Volume Gateway

The bridge for block‑based applications. It exports iSCSI block volumes backed by EBS snapshots or S3. Great for backup targets or lift‑and‑shift migrations.

Ask yourself: which protocol do your on‑prem workloads truly speak — files, blocks, or tapes?

That one question reveals your gateway type.

The Economics That Actually Matter

AWS pricing psychology: you’re not paying for the box. You’re paying for where your bits live and how they move.

Three line items drive your AWS Storage Gateway cost model:

Storage pricing — Mirrors the backing service (S3, FSx, etc.).
Request pricing — Matches the standard S3 or FSx operation costs.
Data transfer pricing — Free inbound to AWS, but outbound traffic to on‑prem is billed per terabyte transferred.

Pro tip: moving data out of AWS through the Storage Gateway has the same economics as any AWS data egress — so architect for data gravity, not convenience.

When to Use It — and When to Rethink It

If your workload can’t or won’t move fully to the cloud (yet), the Storage Gateway gives you hybrid leverage without rearchitecting everything overnight.

But if every request ends up bouncing between S3 and on‑prem, you may need to rethink placement, not performance.

The best use cases we see in the wild:

Archival storage for compliance-heavy workloads.
Hybrid file shares across geographies.
Tape backup elimination projects.
Gradual cloud adoption: data first, compute later.

Thought

Every cloud architect eventually realizes this truth: hybrid isn’t transitional — it’s strategic.

The AWS Storage Gateway exists not to extend your datacenter, but to teach it cloud fluency — one NFS mount, one SMB share, one iSCSI target at a time.

Choosing the Right AWS Storage Gateway — A Principal Engineer’s Take

Most hybrid storage decisions don’t fail in architecture. They fail in clarity.
The AWS console gives you four gateway types — S3 File, FSx File, Tape, and Volume. But behind each type lies a philosophy of data motion: how your workloads talk to AWS, how latency collapses, and how cost behaves over time.

Why Storage Gateway Exists

AWS built Storage Gateway for one quiet but urgent reason: most enterprises aren't cloud‑only — they're latency‑bound.
They need to keep data gravity close to compute without giving up cloud durability or cost efficiency.

Storage Gateway bridges that gap. It speaks on‑premises languages (NFS, SMB, iSCSI) while translating natively to AWS storage APIs (S3, FSx, EBS, Glacier).
Think of it as a bilingual edge device — fluent in both legacy infrastructure and cloud semantics.

Scenario 1: Backups and Archives That Refuse to Die On‑Prem

You’ve got Hadoop clusters and SQL databases sitting in your datacenter.
You want backups in the cloud. You don’t want to rebuild your entire pipeline.

Here’s your moment of clarity:

Use S3 File Gateway when you want object control. Use FSx File Gateway when you want Windows consistency. Use Volume Gateway when you want disaster recovery agility.

The trade‑offs are architectural, not cosmetic.

S3 File Gateway:

Stores data natively in S3, with direct access to S3 Object Lock, versioning, and lifecycle management.
Ideal for large files, logs, and database backups that can benefit from cost tiering into Glacier.
Local cache (up to 64 TB) keeps active data hot — ideal for low‑latency reads in analytics or backup workloads.
FSx File Gateway:

Joins your Microsoft Active Directory and feels like any other SMB file system.
Optimized for multi‑user, mixed file workloads — project shares, home directories, or Exchange backups.
Plays well when your org runs Windows clients or third‑party tools expecting full NTFS parity (shadow copies, permissions, DFS).
Volume Gateway:

Perfect for structured data in block storage form.
Lets you back up volumes as EBS snapshots, which you can later restore to EC2 for cloud migration or disaster recovery.
Comes in two flavors:
Cached volumes: Primary data in AWS, frequent reads cached locally.
Stored volumes: Primary data on‑prem, async backups to AWS. (Think “DR‑first architecture.”)
You can decide which to deploy based on one question:
Do you need object features, file semantics, or block storage integration?

Scenario 2: The Tape Archive Nobody Wants to Touch

Here’s the truth every sysadmin quietly knows: tapes die faster than budgets get renewed.

Tape Gateway solves this elegantly. Not with buzzwords — with compatibility.

It creates a virtual tape library (VTL) that plugs directly into your existing backup software (CommVault, NetBackup, IBM, etc.) over iSCSI. For the backup tool, nothing changes. But instead of shipping tapes to dusty warehouses, you’re sending them to Amazon S3 and auto‑archiving to S3 Glacier.

This is hybrid done right: zero cultural resistance, immediate operational gain.

When to Choose Each

The Hidden “Why Now”

Once your backup data lives in AWS, you’ve quietly unlocked optionality.
Today, it’s backup. Tomorrow, it’s pipeline input for analytics or ML.
That’s the real value — AWS turns what used to be retained data into active data.

If your hybrid architecture still treats storage as passive, you’re leaving compounding value on the table.

Final Thought

Each gateway type reflects a philosophy of transition — from legacy to leverage.
The best AWS builders know that hybrid isn't a stopgap; it’s a design state.
Storage Gateway doesn’t replace your infrastructure. It teaches it to speak cloud fluently.

To Do: Hands-On

Configure an AWS S3 File Storage Gateway to test end to end.

Connect an EC2-based NFS file system to an Amazon S3 bucket and verify that files created on EC2 are automatically stored in S3.

Storage Gateway Configuration

Opened AWS Storage Gateway from the AWS Console

Created a new gateway named sgw-demo

Selected S3 File Gateway

Chose Amazon EC2 as the hosting platform

Selected the appropriate time zone

Gateway Deployment on EC2

Launched an EC2 instance for the gateway:

Instance name: sgw-demo-instance

Instance type: m5.xlarge (as recommended)

Public IP enabled

Configured a security group with required ports:

22 (SSH)

80 (Gateway activation)

2049, 111, 20048 (NFS)

Added an extra EBS volume (150 GiB+) for cache storage

Gateway Activation

Used the public IP address of the EC2 instance to activate the gateway

Selected public internet connectivity

Disabled CloudWatch logs and alarms to reduce cost

Allocated cache storage successfully

NFS File Share Creation

Created a new NFS file share

Linked it to an existing S3 bucket (sgw-bucket-99)

Selected S3 Standard as the storage class

Left cache and permission settings as default

Mounting the File Share

Connected to a separate EC2 instance (application server)

Created a mount directory: /web-files

Mounted the NFS file share using the provided Linux mount command

Verified the mount using df -h

Testing the Setup

Created a file index.html inside /web-files

Added sample content to the file

Checked the S3 bucket and confirmed the file appeared there

Downloaded the file from S3 to verify the content

✅ Final Outcome

The NFS file share and S3 bucket were successfully linked

Any file created on the EC2-mounted file system was automatically stored in Amazon S3

This confirms a 1:1 mapping between the NFS file system and the S3 bucket

In simple terms:
Files written on EC2 using NFS were transparently backed up to Amazon S3 using AWS Storage Gateway.

Flow:

On‑prem data (files, volumes, or backups) flows to the Storage Gateway.

The gateway caches frequently accessed data locally for low‑latency access.

Data is securely transferred to AWS storage services (S3, FSx, Glacier, or EBS).

AWS maintains scalable, durable storage and offers lifecycle management, versioning, or snapshotting depending on gateway type.

Administrators monitor and control the entire setup through the AWS Management Console or APIs.

Image Credits Alana Layton

enable FIPS

connect ssh and copy that example command

Secure Identity and Access Management with AWS IAM

Yashvi Kothari — Thu, 25 Dec 2025 14:11:14 +0000

Understand what is meant identity and access management and the difference between authentication, authorization, and access control

Learn the components of IAM as well as its reporting features

Intended Audience
AWS Administrators
Security Engineers
Security Architects
Anyone who is looking to increase their knowledge of the IAM service in preparation for an AWS certification

The First Rule of Infrastructure: Identity is the Perimeter

Most teams treat AWS Identity and Access Management (IAM) as a configuration hurdle. They see it as a series of checkboxes to get a service running.

They are wrong.

In a cloud-native world, the network perimeter is a ghost. Your real firewall isn't an IP range; it’s your IAM policy. If you haven't mastered the distinction between who a user is and what they can do, you aren't managing an environment—you're managing a liability.

The Two Pillars: Authentication vs. Authorization

Confusion here is the root of most "Access Denied" loops. Let’s strip the jargon.

1. Identity Management (The "Who")

Authentication is the process of proving you exist.

The Claim: You provide a unique identifier (a username). IAM ensures this is a singleton within your account; there is no room for ambiguity.
The Proof: You verify that claim. Usually, this is a password. In high-trust environments, it’s a password plus MFA. If you aren't using MFA, you don't have a security policy; you have a suggestion.

2. Access Management (The "What")

Once we know who you are, authorization determines your reach.

The Permission: This is the logic that lives in your JSON policies.
The Scope: Does this identity need FullAccess to EC2, or just ReadOnly to a specific RDS instance?

Status Signal: Principal Engineers don’t grant permissions to "make it work." They grant permissions to make it secure.

The Mechanics of Control

Access control is the delivery vehicle for your security logic. It’s how the "Who" meets the "What." You have three primary levers:

Standard Credentials: The basic handshake of username and password.
Multi-Factor Authentication (MFA): The second layer that turns a stolen password from a catastrophe into a non-event.
Federation: The hallmark of a mature enterprise. Why manage 500 IAM users when you can delegate trust to an external Identity Provider (IdP)? You don't want more identities; you want better ones.

Why This Matters for the Long Game

IAM is the nervous system of your AWS account. It governs, manages, and audits every heartbeat of your infrastructure.

What if your scaling issues aren’t about capacity, but about throttled IAM calls or misconfigured roles? What if your biggest security risk isn't an external hacker, but an internal developer with AdministratorAccess and a leaked API key?

There’s a reason the AWS Well-Architected Framework starts with Identity. If you get the foundation wrong, the rest of the house is just waiting for a reason to fall.

Build for the identity you need, not the convenience you want.

The Architect’s Burden: Why IAM is Your Only Real Firewall

In the cloud, distance is an illusion. Your resources aren't "somewhere else"—they are behind an API.

If you aren't managing your Identity and Access Management (IAM) with surgical precision, you aren't running an infrastructure; you’re running a risk. Without IAM, your AWS account is an open room in a crowded city. With it, it’s a vault. But here is the truth: AWS provides the vault; you provide the combination.

There’s a reason the AWS Well-Architected Framework treats Security as its second pillar. The responsibility for "tight" security is yours alone. It’s the difference between a system that scales and a system that leaks.

1. The Dashboard: Your Command Center

When you step into the IAM Console, you aren't just looking at settings; you're looking at your security posture in a single pane of glass.

The Sign-In URL: This is your front door. Customizing it isn't just about branding—it’s about operational clarity. In a multi-account environment, knowing exactly which door you’re standing at prevents the kind of "oops" that ends up on the morning news.
The Resource Summary: A lean, high-level heartbeat. How many users? How many roles? If these numbers are climbing without a known project, you have an IAM sprawl problem.

2. The Identity Stack: Users, Groups, and Roles

Most engineers confuse these three. Principals do not.

Users (The Permanent): Objects representing a person or an application. They have long-term credentials. Use them sparingly.
Groups (The Logical): You don't give permissions to people; you give permissions to functions. Put the user in the "SRE" group, and let the group hold the policy. It’s cleaner, it’s faster, it’s more of simplicity.
Roles (The Temporal): This is the gold standard. Roles use STS (Security Token Service) to grant temporary, rotating credentials. Why manage a password that can be stolen when you can use a token that expires in an hour?

3. Policies: The Logic of Permission

IAM policies are written in JSON. They are the "code" in "Infrastructure as Code."

Managed Policies: Your library. Use AWS Managed Policies for speed, but pivot to Customer Managed Policies for the Principle of Least Privilege.
Inline Policies: The "strictly local" option. Harder to manage, harder to audit. Ask yourself: if this policy is so special it only applies to one user, is your architecture too complex?

4. Advanced Guardrails: SCPs and Access Analysis

As your organization grows, manual reviews become impossible. You need automated governance.

Service Control Policies (SCPs): These are the "Maximum Permissions." Think of them as a fence. If the SCP says "No S3," it doesn't matter if an IAM user has AdministratorAccess—they are blocked. SCPs don't grant; they restrict.
Access Analyzer: Your silent auditor. It flags resources—like S3 buckets or IAM roles—that are accessible from outside your "Zone of Trust."
The Credential Report: A simple CSV that tells a story. When was the last time Dev-Account-1 was used? If it’s been 90 days, why does it still exist?

The Mastery Shift

What if your security bottlenecks aren't caused by "too much security," but by an outdated understanding of how IAM flows?

Once those IAM roles are assumed, misconfigured trust relationships become silent attack vectors. It’s not about having more rules; it’s about having the right ones.

DEV Community: Yashvi Kothari

Hands-On Lab Guide: L1 Support Engineer Skills

🛠️ Hands-On Lab Guide: L1 Support Engineer Skills

Lab 1: Windows Server + IIS Setup (Core L1 Skill)

Objective

Step-by-Step

🔥 Troubleshooting Exercises

Lab 2: Linux Server + Log Analysis (Core L1 Skill)

Objective

Step-by-Step

Lab 3: AWS CloudWatch Monitoring (Monitoring Skill)

Objective

Step-by-Step

Lab 4: REST API Troubleshooting with Postman + curl

Objective

Step-by-Step (No EC2 needed — use your local machine)

Lab 5: Writing an Incident Report + RCA

Objective

Template

Lab Checklist — Track Your Progress

Quick Reference: Key Commands

Windows (PowerShell)

Linux (Bash)

AWS CLI (Bonus)

The L1 Support Engineer Nobody Talks About (But Every Company Desperately Needs)

The L1 Support Engineer Nobody Talks About (But Every Company Desperately Needs)

What Is an L1 Support Engineer, Really?

The 7 Skills That Separate Good L1 Engineers From Great Ones

1. You Understand the Stack (Even If You Didn't Build It)

2. You Can Read Logs Like a Detective Reads Crime Scenes

3. You Speak REST API

4. You Know Enough SQL to Be Dangerous (In a Good Way)

5. You Understand Environments (And Never Confuse Them)

6. You Master the Incident Priority Matrix

7. You Monitor Before Things Break

The Career Path Nobody Draws For You

The Mindset Shift

Your 30-Day Action Plan

The Bottom Line

How to Think Like a System Administrator: The Practical Interview Playbook

The SysAdmin Mental Model

Scenario #1: The Website Is Down

The interview answer

Scenario #2: The Server Is Slow

Scenario #3: SSL Certificate Expired

Scenario #4: Disk Is 95% Full

Scenario #5: Database Connection Errors

Scenario #6: Migrating 15 Websites

Before migration

Scenario #7: The Server Is Under Attack

The 10 Commands I Want Every SysAdmin to Know

The Two-Week Interview Plan

Week 1

Week 2

Build Your Own Production-Like Lab

The Golden Rule of Troubleshooting

1. Systematic thinking

2. Security

3. Backups

4. Communication

5. Prevention

Final Interview Advice

Part 7: Scenario Questions, Study Plan & Quick Reference

7.1 Scenario-Based Interview Questions (MOST IMPORTANT)

Scenario 1: Website is Down

Scenario 2: Server is Slow

Scenario 3: SSL Certificate Expired

Scenario 4: Disk Space Full

Scenario 5: Database Connection Errors

Scenario 6: Server Migration

Scenario 7: Server Under Attack

More Rapid-Fire Scenario Questions

7.2 Two-Week Study Plan (Zero to Interview-Ready)

Week 1: Core Skills

Week 2: Advanced Skills

Daily Routine

7.3 Hands-On Lab Setup

Option A: Local VM (Free)

Option B: DigitalOcean (Paid but realistic)

7.4 Critical Commands Cheat Sheet