DEV Community: Janak Shrestha

Clone Git Repository on Storage Server

Janak Shrestha — Tue, 06 Oct 2026 05:25:01 +0000

The DevOps team established a new Git repository last week, which remains unused at present. However, the Nautilus application development team now requires a copy of this repository on the Storage Server in the Stratos DC. Follow the provided details to clone the repository:

The repository to be cloned is located at /opt/cluster.git

Clone this Git repository to the /usr/src/kodekloudrepos directory. Perform this task using the natasha user, and ensure that no modifications are made to the repository or existing directories, such as changing permissions or making unauthorized alterations.

Step 1: SSH Into the Storage Server

From the jump host, connect to the Storage Server as natasha:

ssh natasha@ststor01        # password: Bl@kW

If the hostname doesn't resolve, use the server's IP address instead:

ssh natasha@10.244.49.64    # password: Bl@kW

Once connected, you'll see a prompt like:

[natasha@ststor01 ~]$

Step 2: Navigate to the Target Directory

Move into the directory where the repository should be cloned:

cd /usr/src/kodekloudrepos

Verify you're in the right place:

pwd

Expected output:

/usr/src/kodekloudrepos

Step 3: Clone the Repository

Clone the bare repository from /opt/cluster.git into the current directory:

git clone /opt/cluster.git

Expected output:

Cloning into 'cluster'...
warning: You appear to have cloned an empty repository.
done.

ℹ️ Note: The "empty repository" warning is completely normal. It simply means the repository has no commits yet — the clone itself was successful.

This creates a new folder named cluster inside /usr/src/kodekloudrepos.

Step 4: Verify the Clone

Move into the cloned directory:

cd cluster

Check the remote configuration:

git remote -v

Expected output:

origin  /opt/cluster.git (fetch)
origin  /opt/cluster.git (push)

Confirm the directory contents:

ls -la

You'll see the .git/ directory — this is the working copy of the repository.

Step 5: Confirm Ownership

Ensure everything is owned by natasha, since the clone was performed under that user:

ls -la /usr/src/kodekloudrepos

Expected output:

drwxr-xr-x 3 natasha natasha 4096 Oct  6 05:18 cluster

And inside the cloned folder:

ls -la /usr/src/kodekloudrepos/cluster

Expected output:

drwxr-xr-x 3 natasha natasha 4096 Oct  6 05:18 .
drwxr-xr-x 3 natasha natasha 4096 Oct  6 05:18 ..
drwxr-xr-x 6 natasha natasha 4096 Oct  6 05:18 .git

Every entry should show natasha natasha — confirming the clone is correctly owned. ✅

Step 6 (Optional): Document the Clone with a Heredoc

If you want to leave a log or note file describing the clone, use a heredoc to write it without opening an editor:

cat <<'EOF' > clone-info.txt
Repository cloned: /opt/cluster.git
Destination:      /usr/src/kodekloudrepos/cluster
Cloned by user:   natasha
Date:             $(date)
Notes:
  - Bare source repo, non-bare working copy.
  - No permissions or ownership were modified.
EOF

Verify:

cat clone-info.txt

⚠️ Only create this file if your task allows adding new files. If the instructions say "no modifications," skip this step.

Recap: What We Did

Step	Action
1	SSH'd into the Storage Server as `natasha`
2	Navigated to `/usr/src/kodekloudrepos`
3	Cloned `/opt/cluster.git` with `git clone`
4	Verified the remote with `git remote -v`
5	Confirmed ownership with `ls -la`
6	(Optional) Documented the clone using a heredoc

Common Pitfalls to Avoid

Pitfall	Why It's a Problem
Using `git init` instead of `git clone`	Creates a brand-new empty repo, not a copy of the source
Cloning as the wrong user	Files will be owned by that user, breaking team access
Adding `sudo` unnecessarily	Files become root-owned, causing permission issues
Worrying about "empty repository" warning	This is expected for repos without commits
Modifying permissions with `chmod`/`chown`	Often prohibited; not needed when cloning as the right user

Key Takeaways

Clone as the correct user — ownership follows whoever runs the command.
git clone <path> works with local paths, not just URLs.
The "empty repository" warning is not an error.
Always verify with git remote -v and ls -la.
Use heredocs (<<'EOF') to write files cleanly without opening an editor.
Avoid unnecessary sudo — it changes file ownership.

Set Up Git Repository on Storage Server

Janak Shrestha — Mon, 05 Oct 2026 09:25:00 +0000

The Nautilus development team has provided requirements to the DevOps team for a new application development project, specifically requesting the establishment of a Git repository. Follow the instructions below to create the Git repository on the Storage server in the Stratos DC:

Utilize yum to install the git package on the Storage Server.
Create a bare repository named /opt/games.git (ensure exact name usage).

Step 1: SSH Into the Remote Server

First, connect to your server from your local machine (or jump host):

ssh natasha@ststor01

When prompted, enter the user's password. Once connected, you'll see something like:

[natasha@ststor01 ~]$

💡 Tip: If your server has a hostname that doesn't resolve, use the IP address instead:
ssh natasha@10.244.164.10

Step 2: Elevate to Root (or Use sudo)

Installing system packages and writing to /opt requires elevated privileges. Switch to the root user:

sudo -i

Your prompt will change to:

[root@ststor01 ~]#

⚠️ Security Note: In production environments, prefer using sudo for individual commands rather than staying logged in as root. We use sudo -i here for simplicity.

Step 3: Install Git Using yum

Now install the git package with yum:

yum install -y git

You'll see yum resolve dependencies and install Git along with required packages. Here's what a successful install looks like:

Dependencies resolved.
================================================================================
 Package              Architecture   Version             Repository      Size
================================================================================
Installing:
 git                  x86_64         2.52.0-1.el9        appstream       39 k
Installing dependencies:
 git-core             x86_64         2.52.0-1.el9        appstream      5.0 M
 git-core-doc         noarch         2.52.0-1.el9        appstream      3.1 M
 less                 x86_64         590-6.el9           baseos         162 k
 perl-Error           noarch         1:0.17029-7.el9     appstream       42 k
 perl-Git             noarch         2.52.0-1.el9        appstream       37 k
 perl-TermReadKey     x86_64         2.38-11.el9         appstream       37 k
 perl-lib             noarch         0.65-484.el9        baseos          13 k
================================================================================
...
Complete!

Verify the Installation

Confirm Git is installed and check the version:

git --version

Expected output:

git version 2.52.0

✅ Git is now installed.

Step 4: Create the Bare Repository

This is the core step. We'll use git init --bare to create a repository without a working directory — the correct approach for a central server repo.

git init --bare /opt/games.git

You'll see output similar to:

hint: Using 'master' as the name for the initial branch. This default branch name
hint: will change to "main" in Git 3.0. To configure the initial branch name
hint: to use in all of your new repositories, which will suppress this warning,
hint: call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
hint:
hint: Disable this message with "git config set advice.defaultBranchName false"
Initialized empty Git repository in /opt/games.git/

🤔 What Is a "Bare" Repository?

Feature	Normal Repo	Bare Repo
Working directory	✅ Yes	❌ No
`.git` folder	Nested inside project	Repo is the folder
Purpose	Local development	Central server / remote
Created with	`git init`	`git init --bare`

A bare repo is what you'd typically use as the remote on GitHub, GitLab, or a self-hosted Git server. It has no files to edit — only the version history.

ℹ️ About the "master" Hint Message

Don't panic — that's just a hint, not an error. Git is letting you know its default branch name is master and will become main in Git 3.0. Your repository is fully functional.

To suppress the hint globally in the future:

git config --global init.defaultBranch main

Step 5: Verify the Repository

Confirm the repo was created with the correct structure:

ls -la /opt/games.git

You should see:

total 36
drwxr-xr-x 6 root root 4096 Oct  5 09:20 .
drwxr-xr-x 1 root root 4096 Oct  5 09:20 ..
-rw-r--r-- 1 root root   23 Oct  5 09:20 HEAD
-rw-r--r-- 1 root root   66 Oct  5 09:20 config
-rw-r--r-- 1 root root   73 Oct  5 09:20 description
drwxr-xr-x 2 root root 4096 Oct  5 09:20 hooks
drwxr-xr-x 2 root root 4096 Oct  5 09:20 info
drwxr-xr-x 4 root root 4096 Oct  5 09:20 objects
drwxr-xr-x 4 root root 4096 Oct  5 09:20 refs

What Each Entry Means

Entry	Purpose
`HEAD`	Points to the default branch (`master`)
`config`	Repository configuration settings
`description`	Human-readable description (used by GitWeb)
`hooks/`	Server-side hook scripts (pre-receive, post-receive, etc.)
`info/`	Contains `exclude` patterns and refs info
`objects/`	Stores all commits, trees, and blobs
`refs/`	Branch and tag references

Notice there are no source files — that's exactly what makes this a bare repo. ✅

Confirm "Bare" Mode Is Enabled

Check the config file:

cat /opt/games.git/config

Expected output:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = true

The key line is bare = true — this confirms Git treats it as a server-side repository. ✅

Step 6 (Optional): Set Ownership for Team Access

If multiple team members need to push/pull, adjust ownership and permissions:

chown -R natasha:natasha /opt/games.git
chmod -R 755 /opt/games.git

For a shared team, consider creating a dedicated git group:

groupadd git
usermod -aG git natasha
chown -R :git /opt/games.git
chmod -R 775 /opt/games.git

Step 7 (Optional): Test the Repository From Another Machine

From any other server with SSH access and Git installed, try cloning:

git clone natasha@ststor01:/opt/games.git

You'll see:

Cloning into 'games'...
warning: You appear to have cloned an empty repository.

This warning is expected — you've cloned a fresh, empty bare repo. Everything is working. 🎉

Recap: What We Did

Step	Action
1	SSH'd into the remote server
2	Elevated privileges with `sudo -i`
3	Installed Git via `yum install -y git`
4	Created the bare repo with `git init --bare /opt/games.git`
5	Verified the repo structure and `bare = true` config
6	(Optional) Set ownership for team access
7	(Optional) Tested cloning from another machine

Common Pitfalls to Avoid

❌ Forgetting --bare → creates a normal repo, which breaks push workflows (you'll see "refusing to update checked out branch" errors).
❌ Using the wrong path → task requirements often specify exact paths like /opt/games.git; a typo means failure.
❌ Not using sudo → permission denied when writing to /opt.
❌ Confusing the hint message with an error → the master hint is harmless.

Conclusion

You now have a fully functional bare Git repository hosted on a remote Linux server, ready to be used as a central code hub for your team. This setup forms the backbone of self-hosted Git workflows and is a fundamental skill for any DevOps engineer.

From here, you can:

Add Git hooks for CI/CD automation
Configure SSH key authentication for passwordless access
Front it with Gitolite or Gitea for a full-featured experience
Wire it into a Jenkins pipeline for automated builds

Application Security

Janak Shrestha — Tue, 29 Sep 2026 06:24:53 +0000

We have a backup management application UI hosted on Nautilus's backup server in Stratos DC. That backup management application code is deployed under Apache on the backup server itself, and Nginx is running as a reverse proxy on the same server. Apache and Nginx ports are 8088 and 8099, respectively. We have to install the iptables firewall on the server. Make the appropriate changes to fulfill the requirements mentioned below:

We want to open all incoming connections to Nginx's port and block all incoming connections to Apache's port. Also make sure rules are per
Tags: Linux, iptables, Firewall, Security, Nginx, Apache, System Administration, Networkingmanent.

Configuring iptables Firewall on Linux: Allow Nginx and Block Apache

A Complete Step-by-Step Guide for System Administrators

Introduction

Firewall configuration is a critical aspect of server security. In Linux environments, iptables is a powerful tool for controlling incoming and outgoing network traffic. It allows administrators to define rules that determine which connections are allowed and which are blocked.

This guide walks through a real-world scenario where the backup management application on the Nautilus backup server needed specific firewall rules. The goal was to allow all incoming connections to Nginx (port 8099) while blocking all incoming connections to Apache (port 8088), with rules made permanent.

Environment Details

Component	Value
Server	stbkp01
User	clint
Password	H@wk3y3
Nginx Port	8099
Apache Port	8088
Purpose	Backup management application UI

Understanding iptables

What is iptables

iptables is a command-line firewall utility that uses policy chains to allow or block network traffic. It is part of the Netfilter framework in the Linux kernel and provides a flexible way to define firewall rules.

Key Concepts

Concept	Description
Table	Collection of chains (filter, nat, mangle)
Chain	List of rules (INPUT, OUTPUT, FORWARD)
Rule	Condition and action for matching packets
Target	Action taken when a rule matches (ACCEPT, DROP, REJECT)
Policy	Default action if no rule matches

iptables Chains

Chain	Purpose
INPUT	Incoming packets to the local system
OUTPUT	Outgoing packets from the local system
FORWARD	Packets routed through the system

Common Targets

Target	Description
ACCEPT	Allow the packet to pass
DROP	Silently discard the packet
REJECT	Discard and send an error response
LOG	Log the packet and continue

Prerequisites

Before beginning, ensure the following are available:

SSH access to the backup server with sudo or root privileges
Knowledge of the ports used by Nginx and Apache
Basic understanding of Linux networking

Step 1: Connect to the Backup Server

Access the server using SSH.

ssh clint@stbkp01
Password: H@wk3y3

Switch to root to perform administrative tasks.

sudo su -
Password: H@wk3y3

Step 2: Install iptables

Install the iptables packages using the package manager.

yum install -y iptables iptables-services

The installation may include additional packages depending on the system.

Package	Purpose
iptables	Core firewall utility
iptables-services	Service files for systemd
iptables-legacy	Legacy compatibility layer

Verify the installation.

rpm -q iptables-services

Step 3: Check Current iptables Rules

View the current rules before making changes.

iptables -L -n

If the output shows empty chains, no rules are currently defined.

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Step 4: Flush Existing Rules

Clear any existing rules to start with a clean configuration.

iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X

Command Breakdown

Command	Purpose
iptables -F	Flush all rules in the filter table
iptables -X	Delete all user-defined chains
iptables -t nat -F	Flush rules in the nat table
iptables -t nat -X	Delete user-defined chains in nat table

Step 5: Set Default Policies

Set the default policy for each chain. For this task, the default policy is ACCEPT, and specific rules will handle blocking.

iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

Why Start with ACCEPT

Starting with ACCEPT ensures you do not lock yourself out of the server. Specific DROP rules are then added for the ports that need blocking.

Step 6: Allow Established and Related Connections

Allow responses to outgoing connections. Without this rule, the server cannot receive return traffic.

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

This rule is essential for normal network operations, including SSH sessions.

Step 7: Allow Loopback Interface

Allow traffic on the loopback interface for local communication.

iptables -A INPUT -i lo -j ACCEPT

Many services communicate internally via 127.0.0.1. Blocking loopback can cause unexpected failures.

Step 8: Allow SSH

Allow SSH connections to prevent being locked out of the server.

iptables -A INPUT -p tcp --dport 22 -j ACCEPT

Always allow SSH before adding any DROP rules.

Step 9: Allow Nginx Port

Allow all incoming connections to Nginx on port 8099.

iptables -A INPUT -p tcp --dport 8099 -j ACCEPT

Command Breakdown

Option	Description
-A INPUT	Append rule to INPUT chain
-p tcp	Match TCP protocol
--dport 8099	Match destination port 8099
-j ACCEPT	Accept matching packets

Step 10: Block Apache Port

Block all incoming connections to Apache on port 8088.

iptables -A INPUT -p tcp --dport 8088 -j DROP

Why DROP Instead of REJECT

Target	Behavior	Use Case
DROP	Silently discards packets	Security (hides service existence)
REJECT	Sends error response	User-friendly (tells client port is closed)

DROP is preferred for security because it does not reveal that a service is running on the port.

Step 11: Save the Rules Permanently

Save the rules so they persist across reboots.

service iptables save

On some systems, the service command may not be available. Use iptables-save instead.

iptables-save > /etc/sysconfig/iptables

Verify the saved rules.

cat /etc/sysconfig/iptables

Expected output.

*filter
:INPUT ACCEPT [34:2440]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [32:4704]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8099 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8088 -j DROP
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
COMMIT

Step 12: Enable the iptables Service

Enable the iptables service to load rules on boot.

systemctl enable iptables
systemctl start iptables
systemctl status iptables

Expected output.

● iptables.service - IPv4 firewall with iptables
     Loaded: loaded (/usr/lib/systemd/system/iptables.service; enabled)
     Active: active (exited) since Tue 2026-09-29 06:17:03 UTC

Step 13: Verify the Rules

List all rules with line numbers.

iptables -L -n --line-numbers

Expected output.

Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
2    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
3    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:8099
4    DROP       tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:8088
5    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:22

Step 14: Test the Configuration

Test Nginx (Should Work)

curl -I http://localhost:8099/

Expected output.

HTTP/1.1 200 OK
Server: nginx/1.20.1
Content-Type: text/html
Content-Length: 2713881

Test Apache (Should Be Blocked)

curl -I http://localhost:8088/ --connect-timeout 3

Expected output (external connection).

curl: (7) Failed to connect to localhost port 8088: Connection timed out

Important Note on Local Testing

Testing from localhost may not accurately reflect the DROP rule due to how the loopback interface is handled. To properly test the block, run the curl command from an external host such as the jump host.

# From jump host
curl -I http://stbkp01:8088/ --connect-timeout 3

Complete Workflow Summary

# 1. Connect to server
ssh clint@stbkp01
sudo su -

# 2. Install iptables
yum install -y iptables iptables-services

# 3. Flush existing rules
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X

# 4. Set default policies
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

# 5. Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# 6. Allow loopback
iptables -A INPUT -i lo -j ACCEPT

# 7. Allow SSH
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# 8. Allow Nginx
iptables -A INPUT -p tcp --dport 8099 -j ACCEPT

# 9. Block Apache
iptables -A INPUT -p tcp --dport 8088 -j DROP

# 10. Save rules
iptables-save > /etc/sysconfig/iptables

# 11. Enable service
systemctl enable iptables
systemctl start iptables

# 12. Verify
iptables -L -n --line-numbers

Verification Checklist

Check	Command	Expected
iptables installed	rpm -q iptables-services	Installed
Nginx port allowed	iptables -L -n \| grep 8099	ACCEPT
Apache port blocked	iptables -L -n \| grep 8088	DROP
Rules saved	cat /etc/sysconfig/iptables	Rules present
Service enabled	systemctl is-enabled iptables	enabled
Nginx accessible	curl -I http://localhost:8099/	HTTP 200
Apache blocked	curl -I http://stbkp01:8088/	Timeout

Troubleshooting Common Issues

Issue 1: service Command Not Found

On newer systems, the service command may be deprecated. Use iptables-save instead.

iptables-save > /etc/sysconfig/iptables

Issue 2: Rules Not Persisting After Reboot

Ensure the iptables service is enabled and the rules are saved.

systemctl enable iptables
iptables-save > /etc/sysconfig/iptables

Issue 3: Locked Out of Server

If you lose SSH access, access the server through the console and flush the rules.

iptables -F

Issue 4: Nginx Not Accessible

Check rule order. Ensure the Nginx ACCEPT rule comes before any DROP rule.

iptables -L -n --line-numbers

Issue 5: Apache Still Accessible Externally

Verify the DROP rule is present and positioned correctly.

iptables -L INPUT -n -v --line-numbers

Best Practices

Always allow SSH before adding any DROP rules.
Test rules from an external host to confirm blocking works.
Save rules immediately after making changes.
Enable the iptables service for persistence.
Use DROP instead of REJECT for security-sensitive ports.
Document firewall rules for audit purposes.
Monitor firewall logs for blocked connection attempts.

Conclusion

This guide covered the complete process of configuring iptables on the backup server to allow Nginx traffic and block Apache traffic permanently. The key steps were installing iptables, flushing existing rules, setting default policies, adding specific allow and block rules, and saving the configuration.

The rules were verified with iptables -L -n --line-numbers and tested with curl. The iptables service was enabled to ensure the rules persist across reboots.

By following these steps, system administrators can implement precise firewall policies that enhance server security while maintaining required service accessibility.

Quick Reference

Task	Command
Install iptables	yum install -y iptables iptables-services
Flush rules	iptables -F
Allow port	iptables -A INPUT -p tcp --dport PORT -j ACCEPT
Block port	iptables -A INPUT -p tcp --dport PORT -j DROP
Save rules	iptables-save > /etc/sysconfig/iptables
Enable service	systemctl enable iptables
Start service	systemctl start iptables
List rules	iptables -L -n --line-numbers
Test Nginx	curl -I http://localhost:8099/
Test Apache	curl -I http://stbkp01:8088/

This guide was created based on a real-world iptables configuration task on the Nautilus Backup Server in the Stratos Datacenter.

Linux LogRotate

Janak Shrestha — Mon, 28 Sep 2026 11:57:15 +0000

The Nautilus DevOps team is ready to launch a new application, which they will deploy on app servers in Stratos Datacenter. They are expecting significant traffic/usage of tomcat on app servers after that. This will generate massive logs, creating huge log files. To utilise the storage efficiently, they need to compress the log files and need to rotate old logs. Check the requirements shared below:

a. In all app servers install tomcat package.

b. Using logrotate configure tomcat logs rotation to monthly and keep only 3 rotated logs.

(If by default log rotation is set, then please update configuration as needed)

Introduction

Log files are essential for troubleshooting, auditing, and monitoring applications. However, without proper management, log files can grow uncontrollably, consuming disk space and potentially causing service outages. LogRotate is a Linux utility that automates the rotation, compression, and removal of log files, ensuring efficient use of storage.

This guide walks through a real-world scenario where the Nautilus DevOps team needed to install Tomcat on three application servers and configure logrotate to rotate Tomcat logs monthly while keeping only three rotated logs.

Environment Details

Server	Hostname	User	Password
App Server 1	stapp01	tony	Ir0nM@n
App Server 2	stapp02	steve	Am3ric@
App Server 3	stapp03	banner	BigGr33n
Jump Host	jump-host	thor	mjolnir123

Understanding LogRotate

What is LogRotate

LogRotate is a Linux utility that manages log files by rotating them based on size or time. It prevents log files from growing indefinitely, compresses old logs to save space, and removes the oldest logs based on a retention policy.

Key Concepts

Concept	Description
Rotation	Moving the current log to a new file and starting fresh
Compression	Reducing the size of rotated logs using gzip
Retention	Number of rotated logs to keep before deletion
Frequency	How often rotation occurs (daily, weekly, monthly)

Why Use LogRotate

LogRotate provides several benefits. It prevents disk space exhaustion by keeping log files at manageable sizes. It preserves historical data for troubleshooting and auditing. It automates routine maintenance, reducing manual intervention. It compresses old logs to save storage space. It integrates with systemd timers for reliable scheduling.

Prerequisites

Before beginning, ensure the following are available. You need SSH access to all three app servers with sudo or root privileges. You should have basic knowledge of Linux commands. The servers should have network access to package repositories.

Step 1: Connect to App Server 1

Access the server using SSH.

ssh tony@stapp01
Password: Ir0nM@n

Switch to root to perform administrative tasks.

sudo su -
Password: Ir0nM@n

Step 2: Install Tomcat

Install Tomcat using the package manager. On CentOS Stream 9, Tomcat is available in the AppStream repository.

yum install -y tomcat

The installation pulls in several dependencies.

Package	Purpose
tomcat	Main Tomcat server
tomcat-lib	Core libraries
tomcat-servlet-4.0-api	Servlet API
tomcat-jsp-2.3-api	JSP API
tomcat-el-3.0-api	Expression Language API
ecj	Eclipse Compiler for Java
apr	Apache Portable Runtime
tomcat-native	Native library for performance

Verify the installation.

rpm -q tomcat

Expected output.

tomcat-9.0.120-1.el9.noarch

Step 3: Start and Enable Tomcat

Start the Tomcat service and enable it to start on boot.

systemctl start tomcat
systemctl enable tomcat
systemctl status tomcat

The status output should show the service is active and running.

● tomcat.service - Apache Tomcat Web Application Container
     Loaded: loaded (/usr/lib/systemd/system/tomcat.service; enabled; preset: disabled)
     Active: active (running) since Mon 2026-09-28 11:49:38 UTC; 359ms ago
   Main PID: 6545 (java)

Why Start the Service

Starting Tomcat generates the log files that logrotate will manage. Without running the service, the log directory remains empty.

Step 4: Verify Log Files are Generated

Check the Tomcat log directory.

ls -la /var/log/tomcat/

Expected output.

-rw-r--r-- 1 tomcat tomcat 7321 Sep 28 11:49 catalina.2026-09-28.log
-rw-r--r-- 1 tomcat tomcat    0 Sep 28 11:49 host-manager.2026-09-28.log
-rw-r--r-- 1 tomcat tomcat    0 Sep 28 11:49 localhost.2026-09-28.log
-rw-r--r-- 1 tomcat tomcat    0 Sep 28 11:49 localhost_access_log.2026-09-28.txt
-rw-r--r-- 1 tomcat tomcat    0 Sep 28 11:49 manager.2026-09-28.log

Log File	Purpose
catalina.log	Main Tomcat log
host-manager.log	Host Manager application log
localhost.log	Localhost application log
localhost_access_log.txt	HTTP access log
manager.log	Manager application log

Step 5: Install LogRotate

Check if logrotate is installed.

rpm -q logrotate

If not installed, install it.

yum install -y logrotate

The installation creates a systemd timer for logrotate.

Created symlink /etc/systemd/system/timers.target.wants/logrotate.timer → /usr/lib/systemd/system/logrotate.timer.

Step 6: Create the LogRotate Configuration

Create a configuration file for Tomcat at /etc/logrotate.d/tomcat.

vi /etc/logrotate.d/tomcat

Add the following configuration.

/var/log/tomcat/*.log {
    monthly
    rotate 3
    compress
    missingok
    notifempty
    copytruncate
}

Configuration Directive Breakdown

Directive	Purpose
monthly	Rotate logs once per month
rotate 3	Keep only 3 rotated log files
compress	Compress rotated logs with gzip
missingok	Do not error if log file is missing
notifempty	Do not rotate empty log files
copytruncate	Copy then truncate original log

Why copytruncate is Essential

Tomcat keeps log files open while running. Without copytruncate, logrotate would rename the log file, but Tomcat would continue writing to the renamed file. The new log file would remain empty. The copytruncate directive copies the log file and then truncates the original, allowing Tomcat to continue writing to the same file without interruption.

Step 7: Verify the Configuration

Test the configuration with a dry run.

logrotate -d /etc/logrotate.d/tomcat

Expected output.

reading config file /etc/logrotate.d/tomcat
Reading state from file: /var/lib/logrotate/logrotate.status
Allocating hash table for state file, size 64 entries

Handling 1 logs

rotating pattern: /var/log/tomcat/*.log  monthly (3 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/tomcat/catalina.2026-09-28.log
  Now: 2026-09-28 11:52
  Last rotated at 2026-09-28 11:00
  log does not need rotating (log has already been rotated)

The dry run shows what logrotate would do without making changes.

Step 8: Apply to App Server 2

Connect to stapp02 and repeat the process.

ssh steve@stapp02
sudo su -
Password: Am3ric@

yum install -y tomcat
yum install -y logrotate

systemctl start tomcat
systemctl enable tomcat

cat > /etc/logrotate.d/tomcat << 'EOF'
/var/log/tomcat/*.log {
    monthly
    rotate 3
    compress
    missingok
    notifempty
    copytruncate
}
EOF

logrotate -d /etc/logrotate.d/tomcat

Step 9: Apply to App Server 3

Connect to stapp03 and repeat the process.

ssh banner@stapp03
sudo su -
Password: BigGr33n

yum install -y tomcat
yum install -y logrotate

systemctl start tomcat
systemctl enable tomcat

cat > /etc/logrotate.d/tomcat << 'EOF'
/var/log/tomcat/*.log {
    monthly
    rotate 3
    compress
    missingok
    notifempty
    copytruncate
}
EOF

logrotate -d /etc/logrotate.d/tomcat

Step 10: Verify All Servers

Run verification commands on all servers.

# Check Tomcat service
ssh tony@stapp01 "systemctl is-active tomcat"
ssh steve@stapp02 "systemctl is-active tomcat"
ssh banner@stapp03 "systemctl is-active tomcat"

# Check logrotate configuration
ssh tony@stapp01 "cat /etc/logrotate.d/tomcat"
ssh steve@stapp02 "cat /etc/logrotate.d/tomcat"
ssh banner@stapp03 "cat /etc/logrotate.d/tomcat"

# Check log files
ssh tony@stapp01 "sudo ls -la /var/log/tomcat/"
ssh steve@stapp02 "sudo ls -la /var/log/tomcat/"
ssh banner@stapp03 "sudo ls -la /var/log/tomcat/"

Understanding the Rotation Process

Before Rotation

/var/log/tomcat/catalina.2026-09-28.log

After Rotation

/var/log/tomcat/catalina.2026-09-28.log.1.gz

After Multiple Rotations

/var/log/tomcat/catalina.2026-09-28.log.1.gz
/var/log/tomcat/catalina.2026-09-28.log.2.gz
/var/log/tomcat/catalina.2026-09-28.log.3.gz

When the fourth rotation occurs, the oldest file (.3.gz) is deleted, and the others are renamed to make room for the new rotation.

Verification Checklist

Check	Command	Expected
Tomcat installed	rpm -q tomcat	tomcat-9.0.120
Tomcat running	systemctl is-active tomcat	active
logrotate installed	rpm -q logrotate	logrotate-3.18.0
Config file exists	cat /etc/logrotate.d/tomcat	Configuration content
Dry run passes	logrotate -d /etc/logrotate.d/tomcat	No errors
Log files exist	ls -la /var/log/tomcat/	Log files listed

Troubleshooting Common Issues

Issue 1: logrotate Command Not Found

Install the logrotate package.

yum install -y logrotate

Issue 2: State File Warning

When running logrotate for the first time, you may see a warning about the state file. This is normal. LogRotate creates the state file automatically when it runs for the first time during a real rotation cycle.

Issue 3: Logs Not Rotating

Check that the logrotate timer is active.

systemctl status logrotate.timer

Check the logrotate status.

cat /var/lib/logrotate/logrotate.status

Issue 4: Tomcat Not Writing to New Logs

Ensure copytruncate is in the configuration. Without it, Tomcat may continue writing to the old log file.

Issue 5: Configuration Syntax Errors

Verify the configuration syntax.

logrotate -d /etc/logrotate.d/tomcat

Fix any syntax errors before the next rotation cycle.

Best Practices

Use copytruncate for applications that keep log files open.
Set appropriate rotation frequency based on log volume.
Configure compression to save disk space.
Set retention based on compliance and storage requirements.
Test configuration with dry run before applying.
Monitor logrotate status regularly.
Document log rotation policies for audit purposes.

Conclusion

This guide covered the complete process of installing Tomcat on three application servers and configuring logrotate for monthly rotation with three retained logs. The key steps were installing the packages, starting the service to generate logs, creating the logrotate configuration, and verifying the setup.

The copytruncate directive is essential for Tomcat because it keeps log files open while running. Without it, Tomcat would continue writing to the renamed log file, and the new log file would remain empty.

LogRotate is a powerful tool for managing log files efficiently. By configuring it properly, system administrators can ensure that log files do not consume excessive disk space while preserving historical data for troubleshooting and auditing.

Quick Reference

Task	Command
Install Tomcat	yum install -y tomcat
Start Tomcat	systemctl start tomcat
Enable Tomcat	systemctl enable tomcat
Install logrotate	yum install -y logrotate
Create config	vi /etc/logrotate.d/tomcat
Test config	logrotate -d /etc/logrotate.d/tomcat
Check logs	ls -la /var/log/tomcat/
Check status	cat /var/lib/logrotate/logrotate.status

Configuration File Summary

/var/log/tomcat/*.log {
    monthly
    rotate 3
    compress
    missingok
    notifempty
    copytruncate
}

This guide was created based on a real-world logrotate implementation on the Nautilus Application Servers in the Stratos Datacenter.

Linux GPG Encryption

Janak Shrestha — Sun, 27 Sep 2026 14:17:06 +0000

We have confidential data that needs to be transferred to a remote location, so we need to encrypt that data.We also need to decrypt data we received from a remote location in order to understand its content.

On storage server in Stratos Datacenter we have private and public keys stored at /home/*_key.asc. Use these keys to perform the following actions.

Encrypt /home/encrypt_me.txt to /home/encrypted_me.asc.
Decrypt /home/decrypt_me.asc to /home/decrypted_me.txt. (Passphrase for decryption and encryption is kodekloud).
The user ID you can use is kodekloud@kodekloud.com.

Introduction

GPG (GNU Privacy Guard) is a powerful encryption tool used to secure data during transfer and storage. It supports both symmetric encryption (using a passphrase) and asymmetric encryption (using public and private key pairs). In enterprise environments, GPG is commonly used to protect confidential data before transferring it to remote locations.

This guide walks through a real-world GPG encryption and decryption task on the Nautilus Storage Server in the Stratos Datacenter. The task involved importing keys, encrypting a file with a recipient's public key, and decrypting a file using a passphrase.

Environment Details

Server	Hostname	User	Password
Storage Server	ststor01	natasha	Bl@kW
Jump Host	jump-host	thor	mjolnir123

File	Purpose
/home/public_key.asc	Public key for encryption
/home/private_key.asc	Private key for decryption
/home/encrypt_me.txt	File to encrypt
/home/decrypt_me.asc	File to decrypt
kodekloud@kodekloud.com	User ID for encryption
kodekloud	Passphrase for decryption

Step 1: Connect to the Storage Server

Access the storage server using SSH.

ssh natasha@ststor01
Password: Bl@kW

Switch to root to perform administrative tasks.

sudo su -
Password: Bl@kW

Step 2: Verify the Available Keys

Check the key files available in the home directory.

ls -la /home/*_key.asc

Expected output:

-rw-r--r-- 1 root root 3590 Sep 27 13:43 /home/private_key.asc
-rw-r--r-- 1 root root 1723 Sep 27 13:43 /home/public_key.asc

The public key is used for encryption, and the private key is used for decryption.

Step 3: Import the Public Key

Import the public key so GPG can use it for encryption.

gpg --import /home/public_key.asc

Expected output:

gpg: key 8F17F26ECCE3AF51: public key "kodekloud <kodekloud@kodekloud.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1

Verify the public key is imported.

gpg --list-keys

Expected output:

/root/.gnupg/pubring.kbx
------------------------
pub   rsa2048 2020-01-19 [SC]
      FEA85011C456B5E9AE5A516F8F17F26ECCE3AF51
uid           [ unknown] kodekloud <kodekloud@kodekloud.com>
sub   rsa2048 2020-01-19 [E]

Step 4: Import the Private Key

Import the private key. This key is protected with a passphrase.

gpg --import /home/private_key.asc

When prompted, enter the passphrase.

Passphrase: kodekloud

Expected output:

gpg: key 8F17F26ECCE3AF51: "kodekloud <kodekloud@kodekloud.com>" not changed
gpg: key 8F17F26ECCE3AF51: secret key imported
gpg: Total number processed: 1
gpg:              unchanged: 1
gpg:       secret keys read: 1
gpg:   secret keys imported: 1

Verify the secret key is imported.

gpg --list-secret-keys

Expected output:

/root/.gnupg/pubring.kbx
------------------------
sec   rsa2048 2020-01-19 [SC]
      FEA85011C456B5E9AE5A516F8F17F26ECCE3AF51
uid           [ unknown] kodekloud <kodekloud@kodekloud.com>
ssb   rsa2048 2020-01-19 [E]

Step 5: Encrypt the File

Encrypt /home/encrypt_me.txt using the recipient's public key.

gpg --encrypt --armor --recipient kodekloud@kodekloud.com --output /home/encrypted_me.asc /home/encrypt_me.txt

When prompted about the key trust, answer yes.

It is NOT certain that the key belongs to the person named
in the user ID.  If you *really* know what you are doing,
you may answer the next question with yes.

Use this key anyway? (y/N) y

Command Breakdown

Option	Description
--encrypt	Encrypt the file
--armor	Create ASCII armored output (.asc format)
--recipient	Specify the recipient's user ID
--output	Output file path
/home/encrypt_me.txt	Input file to encrypt

Verify the encrypted file.

ls -la /home/encrypted_me.asc
head -5 /home/encrypted_me.asc

Expected output:

-rw-r--r-- 1 root root 651 Sep 27 14:10 /home/encrypted_me.asc

-----BEGIN PGP MESSAGE-----

hQEMA91rhQaGXAcNAQf9FKNgOTXnf1l19NE12N8iKxweVSkBHULQNQbl6VHAB6xV
7dsOkT79xGIjMD45ktzI56eczklbZg6l6sFfwcdjFDheO8YkSiPPN526HIl4hFoW
tvP1l+H6kvi1qqFnQxb6H96zZLgaUSkxyGK/kBBV1KuPxJiNH4xGOrr6J0DhfN6g

Step 6: Decrypt the File

Decrypt /home/decrypt_me.asc to /home/decrypted_me.txt.

gpg --output /home/decrypted_me.txt --decrypt /home/decrypt_me.asc

When prompted, enter the passphrase.

Passphrase: kodekloud

If successful, the decrypted file will be created.

gpg: AES.CFB encrypted data
gpg: encrypted with 1 passphrase
gpg: Signature made ...
gpg: Good signature from "kodekloud <kodekloud@kodekloud.com>"

Verify the decrypted file.

ls -la /home/decrypted_me.txt
cat /home/decrypted_me.txt

Expected output:

-rw-r--r-- 1 root root 80 Sep 27 14:01 /home/decrypted_me.txt
Welcome to xFusionCorp Industries. This is KodeKloud System Administration Lab

Understanding the Process

Public Key Cryptography

GPG uses asymmetric cryptography, which involves two keys:

Key	Purpose	Shared With
Public key	Encrypt data	Anyone
Private key	Decrypt data	Only the owner

When you encrypt a file with someone's public key, only their private key can decrypt it. This ensures confidentiality during transfer.

Symmetric Encryption

GPG also supports symmetric encryption, which uses a single passphrase for both encryption and decryption. This is simpler but requires securely sharing the passphrase.

Key Trust

When importing a key, GPG marks it as untrusted because it cannot verify the key belongs to the stated owner. You must manually confirm trust by answering "yes" when prompted.

Troubleshooting

Bad Session Key

If decryption fails with Bad session key, the passphrase does not match. Possible causes include:

The passphrase is incorrect
The file was encrypted with a different passphrase
The file is corrupted

Solution: Verify the passphrase from the task description. If the file was encrypted with a known passphrase, use the exact value.

Public Key Not Found

If encryption fails with public key not found:

gpg --import /home/public_key.asc
gpg --list-keys

Ensure the key is imported before encrypting.

Secret Key Not Found

If decryption fails with No secret key:

gpg --import /home/private_key.asc
gpg --list-secret-keys

Ensure the private key is imported before decrypting.

File Format Issues

If the file does not have the expected format, check it with:

gpg --list-packets /home/decrypt_me.asc

This shows the packet structure and helps identify whether the file is symmetrically or asymmetrically encrypted.

Summary

Task	Status
Import public key	Complete
Import private key	Complete
Encrypt `/home/encrypt_me.txt`	Complete
Decrypt `/home/decrypt_me.asc`	Complete

Final File Status

-rw-r--r-- 1 root root 651 Sep 27 14:10 /home/encrypted_me.asc
-rw-r--r-- 1 root root  80 Sep 27 14:01 /home/decrypted_me.txt

Decrypted Content

Welcome to xFusionCorp Industries. This is KodeKloud System Administration Lab

Key GPG Commands

Command	Purpose
`gpg --import key.asc`	Import a key
`gpg --list-keys`	List public keys
`gpg --list-secret-keys`	List private keys
`gpg --encrypt --armor --recipient user@domain --output file.asc file.txt`	Encrypt a file
`gpg --decrypt --output file.txt file.asc`	Decrypt a file
`gpg --symmetric --passphrase pass --output file.asc file.txt`	Symmetric encryption
`gpg --list-packets file.asc`	View file structure

Best Practices

Always verify keys after import with gpg --list-keys.
Use --armor to create ASCII-armored output that is safe for email and text transfer.
Use the --recipient flag with the exact user ID to encrypt for a specific person.
Store private keys securely and protect them with a strong passphrase.
Test GPG functionality with a small file before working with critical data.
When decryption fails, check the file format with gpg --list-packets.
If the passphrase is wrong, verify it from a trusted source.

Conclusion

This guide covered the complete process of importing GPG keys, encrypting a file with a recipient's public key, and decrypting a file with a passphrase. The tasks were completed successfully on the Nautilus Storage Server.

The key takeaways are understanding the difference between public and private keys, knowing how to import keys, and using the correct GPG commands for encryption and decryption. When issues arise, the gpg --list-packets command is invaluable for understanding the file structure.

Quick Reference

Task	Command
Import key	`gpg --import key.asc`
List keys	`gpg --list-keys`
List secret keys	`gpg --list-secret-keys`
Encrypt	`gpg --encrypt --armor --recipient user@domain --output out.asc in.txt`
Decrypt	`gpg --output out.txt --decrypt in.asc`
Symmetric encrypt	`gpg --symmetric --passphrase pass --output out.asc in.txt`
View packets	`gpg --list-packets file.asc`

Apache Troubleshooting

Janak Shrestha — Sat, 26 Sep 2026 12:17:26 +0000

xFusionCorp Industries uses some monitoring tools to check the status of every service, application, etc running on the systems. Recently, the monitoring system identified that Apache service is not running on some of the Nautilus Application Servers in Stratos Datacenter.

Identify the faulty Nautilus Application Server and fix the issue. Also, make sure Apache service is up and running on all Nautilus Application Servers. Do not try to stop any kind of firewall that is already running.
Apache is running on 8089 port on all Nautilus Application Servers and its document root must be /var/www/html on all app servers. o
Finally you can test from jump host using curl command to access Apache on all app servers and it should be reachable and you should get some static page. E.g. curl http://stapp01:8089/.

Environment Details

Server	Hostname	User	Password
App Server 1	stapp01	tony	Ir0nM@n
App Server 2	stapp02	steve	Am3ric@
App Server 3	stapp03	banner	BigGr33n
Jump Host	jump-host	thor	mjolnir123

Step 1: Identify the Faulty Server

Connect to the jump host and test each app server.

ssh thor@jump-host
Password: mjolnir123

Test Apache on each server.

curl -I http://stapp01:8089/
curl -I http://stapp02:8089/
curl -I http://stapp03:8089/

In this scenario, all three servers returned connection refused, indicating Apache was not running on any of them.

curl: (7) Failed to connect to stapp01 port 8089: Connection refused
curl: (7) Failed to connect to stapp02 port 8089: Connection refused
curl: (7) Failed to connect to stapp03 port 8089: Connection refused

Step 2: Troubleshoot App Server 1 (stapp01)

Connect to stapp01 and switch to root.

ssh tony@stapp01
sudo su -

Check the Service Status

systemctl status httpd

The output showed the service was inactive.

○ httpd.service - The Apache HTTP Server
     Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled)
     Active: inactive (dead)

Check the Configuration for Syntax Errors

httpd -t

The output revealed the root cause.

AH00526: Syntax error on line 47 of /etc/httpd/conf/httpd.conf:
Invalid command 'Listen 808989', perhaps misspelled or defined by a module not included in the server configuration

The Listen directive had a typo: Listen 808989 instead of Listen 8089.

Examine the Problematic Line

sed -n '40,55p' /etc/httpd/conf/httpd.conf

The line was:

"Listen 808989"

The line had quotes around it, which prevented standard sed commands from matching it.

Check DocumentRoot

grep "DocumentRoot" /etc/httpd/conf/httpd.conf

The output showed:

DocumentRoot /var/www/html;

The semicolon at the end made the path invalid.

Fix the Configuration

Remove the problematic line and ensure correct directives.

# Remove the line containing "Listen 808989"
sed -i '/Listen 808989/d' /etc/httpd/conf/httpd.conf

# Remove any other Listen lines with quotes
sed -i '/"Listen/d' /etc/httpd/conf/httpd.conf

# Ensure correct Listen directive
grep -q "^Listen 8089" /etc/httpd/conf/httpd.conf || echo "Listen 8089" >> /etc/httpd/conf/httpd.conf

# Fix DocumentRoot
sed -i 's|^DocumentRoot.*|DocumentRoot "/var/www/html"|' /etc/httpd/conf/httpd.conf

Verify the Fix

grep -n "^Listen" /etc/httpd/conf/httpd.conf
grep -n "^DocumentRoot" /etc/httpd/conf/httpd.conf
httpd -t

Expected output:

358:Listen 8089
124:DocumentRoot "/var/www/html"
Syntax OK

Start and Enable Apache

systemctl start httpd
systemctl enable httpd
systemctl status httpd

The service started successfully.

● httpd.service - The Apache HTTP Server
     Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
     Active: active (running)
   Main PID: 11989 (httpd)
     Status: "Started, listening on: port 8089"

Step 3: Troubleshoot App Server 2 (stapp02)

Connect to stapp02 and switch to root.

ssh steve@stapp02
sudo su -

Check the Configuration

grep -n "^Listen" /etc/httpd/conf/httpd.conf
grep -n "^DocumentRoot" /etc/httpd/conf/httpd.conf
grep -n "^ServerRoot" /etc/httpd/conf/httpd.conf

The output showed two issues:

47:Listen 808989
124:DocumentRoot "/var/www/html"
34:ServerRoot "/etc/httpd;"

The Listen directive had a typo, and the ServerRoot had a semicolon making it invalid.

Check the Syntax

httpd -t

The output revealed the ServerRoot issue.

httpd: Syntax error on line 34 of /etc/httpd/conf/httpd.conf: ServerRoot must be a valid directory

Fix the Configuration

# Remove the problematic Listen line
sed -i '/Listen 808989/d' /etc/httpd/conf/httpd.conf
sed -i '/"Listen/d' /etc/httpd/conf/httpd.conf

# Ensure correct Listen directive
grep -q "^Listen 8089" /etc/httpd/conf/httpd.conf || echo "Listen 8089" >> /etc/httpd/conf/httpd.conf

# Fix ServerRoot
sed -i 's|^ServerRoot.*|ServerRoot "/etc/httpd"|' /etc/httpd/conf/httpd.conf

# Fix DocumentRoot
sed -i 's|^DocumentRoot.*|DocumentRoot "/var/www/html"|' /etc/httpd/conf/httpd.conf

# Ensure index.html
echo "Welcome to xFusionCorp Industries" > /var/www/html/index.html

Verify and Start Apache

grep -n "^ServerRoot" /etc/httpd/conf/httpd.conf
grep -n "^Listen" /etc/httpd/conf/httpd.conf
grep -n "^DocumentRoot" /etc/httpd/conf/httpd.conf
httpd -t
systemctl start httpd
systemctl enable httpd
systemctl status httpd

The service started successfully.

● httpd.service - The Apache HTTP Server
     Active: active (running)
     Status: "Started, listening on: port 8089"

Step 4: Troubleshoot App Server 3 (stapp03)

Connect to stapp03 and switch to root.

ssh banner@stapp03
sudo su -

Check the Configuration

grep -n "^ServerRoot" /etc/httpd/conf/httpd.conf
grep -n "^Listen" /etc/httpd/conf/httpd.conf
grep -n "^DocumentRoot" /etc/httpd/conf/httpd.conf

Apply the Same Fixes

# Fix ServerRoot
sed -i 's|^ServerRoot.*|ServerRoot "/etc/httpd"|' /etc/httpd/conf/httpd.conf

# Fix Listen
sed -i '/Listen 808989/d' /etc/httpd/conf/httpd.conf
sed -i '/"Listen/d' /etc/httpd/conf/httpd.conf
grep -q "^Listen 8089" /etc/httpd/conf/httpd.conf || echo "Listen 8089" >> /etc/httpd/conf/httpd.conf

# Fix DocumentRoot
sed -i 's|^DocumentRoot.*|DocumentRoot "/var/www/html"|' /etc/httpd/conf/httpd.conf

# Ensure index.html
echo "Welcome to xFusionCorp Industries" > /var/www/html/index.html

# Verify
grep -n "^ServerRoot" /etc/httpd/conf/httpd.conf
grep -n "^Listen" /etc/httpd/conf/httpd.conf
grep -n "^DocumentRoot" /etc/httpd/conf/httpd.conf

# Check syntax
httpd -t

# Start and enable
systemctl start httpd
systemctl enable httpd
systemctl status httpd

The service started successfully.

Step 5: Verify from Jump Host

Return to the jump host and test all servers.

curl http://stapp01:8089/
curl http://stapp02:8089/
curl http://stapp03:8089/

Expected output:

Welcome to xFusionCorp Industries!
Welcome to xFusionCorp Industries
Welcome to xFusionCorp Industries

All three servers returned the static page content successfully.

Summary of Issues and Fixes

Server	Issue	Fix
stapp01	`"Listen 808989"` typo with quotes	Removed line, added `Listen 8089`
stapp01	`DocumentRoot /var/www/html;`	Fixed to `DocumentRoot "/var/www/html"`
stapp02	`Listen 808989` typo	Removed line, added `Listen 8089`
stapp02	`ServerRoot "/etc/httpd;"`	Fixed to `ServerRoot "/etc/httpd"`
stapp03	Same issues as stapp02	Applied same fixes

Verification Checklist

Check	stapp01	stapp02	stapp03
Apache running	Active	Active	Active
Port 8089	Listening	Listening	Listening
DocumentRoot	/var/www/html	/var/www/html	/var/www/html
Index page	Present	Present	Present
curl test	200 OK	200 OK	200 OK

Common Apache Configuration Errors

Error	Cause	Fix
`Invalid command 'Listen ...'`	Typo in port number	Correct the Listen directive
`ServerRoot must be a valid directory`	Semicolon or wrong path	Set `ServerRoot "/etc/httpd"`
`DocumentRoot` not working	Semicolon in path	Set `DocumentRoot "/var/www/html"`
Service fails to start	Syntax error	Run `httpd -t` to identify
Port already in use	Another service on port	Change port or stop conflicting service

Best Practices

Always run httpd -t before restarting Apache to catch syntax errors.
Use grep -n to view configuration lines with line numbers.
Remove problematic lines entirely rather than trying to edit them in place.
Verify the ServerRoot, Listen, and DocumentRoot directives are correct.
Ensure the index.html file exists in the document root.
Check the service status after starting Apache.
Test from a remote host to confirm the service is reachable.

Useful Commands

Task	Command
Check syntax	`httpd -t`
Check service	`systemctl status httpd`
Start service	`systemctl start httpd`
Enable service	`systemctl enable httpd`
Check listening ports	`ss -tlnp \
View configuration	{% raw %}`grep -n "^Listen" /etc/httpd/conf/httpd.conf`
Remove a line	`sed -i '/pattern/d' /etc/httpd/conf/httpd.conf`
Replace a line	`sed -i 's\

Conclusion

This guide covered a complete Apache troubleshooting workflow on three application servers. The root causes were configuration errors: typos in the Listen directive, semicolons in ServerRoot and DocumentRoot paths, and quotes around directive values.

By systematically checking the service status, configuration syntax, and specific directives, the issues were identified and fixed. After correcting the configuration and restarting Apache, all three servers responded correctly on port 8089.

The key takeaway is that Apache configuration errors are often caused by small typos or stray characters. Using {% raw %}httpd -t to check syntax and grep to inspect specific directives makes troubleshooting efficient and reliable.

Jenkins Parameterized Builds

Janak Shrestha — Sat, 26 Sep 2026 11:44:08 +0000

A new DevOps Engineer has joined the team and he will be assigned some Jenkins related tasks. Before that, the team wanted to test a simple parameterized job to understand basic functionality of parameterized builds. He is given a simple parameterized job to build in Jenkins. Please find more details below:

Click on the Jenkins button on the top bar to access the Jenkins UI. Login using username admin and password Adm!n321.

Create a parameterized job which should be named as parameterized-job
Add a string parameter named Stage; its default value should be Build.
Add a choice parameter named env; its choices should be Development, Staging and Production.
Configure job to execute a shell command, which should echo both parameter values (you are passing in the job).
Build the Jenkins job at least once with choice parameter value Development to make sure it passes. Note:
You might need to install some plugins and restart Jenkins service. So, we recommend clicking on Restart Jenkins when installation is complete and no jobs are running on plugin installation/update page i.e update centre. Also, Jenkins UI sometimes gets stuck when Jenkins service restarts in the back end. In this case, please make sure to refresh the UI page.
For these kind of scenarios requiring changes to be done in a web UI, please take screenshots so that you can share it with us for review in case your task is marked incomplete. You may also consider using a screen recording software such as loom.com to record and share your work.

Step 1: Access the Jenkins UI

Click the Jenkins button on the top bar of your KodeKloud lab environment. This opens the Jenkins UI in a new tab.

Field	Value
Username	`admin`
Password	`Adm!n321`

You'll land on the Jenkins dashboard.

Step 2: Create the Job

From the dashboard:

Click New Item in the left sidebar
In the Enter an item name field, type: parameterized-job
Select Freestyle project
Click OK

You'll be taken to the job configuration page.

Step 3: Enable Parameterized Build

At the top of the configuration page, in the General section:

Check the box: ☑️ This project is parameterized

Once checked, an Add Parameter dropdown button will appear. This is where we'll define our parameters.

Step 4: Add the String Parameter (`Stage`)

Click Add Parameter → select String Parameter
Fill in the fields:

Field	Value
Name	`Stage`
Default Value	`Build`
Description	(optional) `Enter the stage name`

What is a String Parameter?
It accepts free-form text input. You can set a default value that pre-fills the input box when a user runs the job.

Step 5: Add the Choice Parameter (`env`)

Click Add Parameter again → select Choice Parameter
Fill in the fields:

Field	Value
Name	`env`
Choices	See below
Description	(optional) `Select the environment`

For the Choices field, enter each option on its own line:

Development
Staging
Production

⚠️ Important: Do NOT use commas. Each choice must be on a separate line. Jenkins will parse them into a dropdown list. The order matters — Development should be first so it appears at the top.

What is a Choice Parameter?
It presents a dropdown of predefined options. Users pick one; no free-text input is allowed.

Final Parameter Order

Your configuration should show:

Parameters:
├── Stage (String)  → default: "Build"
└── env   (Choice)  → Development | Staging | Production

Step 6: Configure the Shell Command

Scroll down to the Build section:

Click Add build step → select Execute shell
In the Command text area, enter:

echo "Stage: $Stage"
echo "Environment: $env"

How Parameter Injection Works

Jenkins exposes parameters as environment variables during the build. So $Stage and $env are available just like any shell variable. The syntax is case-sensitive — $env won't match $ENV or $Env.

Step 7: Save the Job

Click Save at the bottom of the page
You'll be redirected to the job page

The job page will now show a Build with Parameters option in the left sidebar (instead of just "Build Now").

Step 8: Build with Parameters

Click Build with Parameters in the left sidebar
You'll see a form:
- Stage: text field pre-filled with Build
- env: dropdown with Development, Staging, Production
Select Development from the env dropdown
Leave Stage as Build (the default)
Click Build

Step 9: Verify the Build Output

In the left sidebar, Build History will show #1 appear. Click on #1 → Console Output.

You should see:

Started by user admin
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/parameterized-job
[parameterized-job] $ /bin/sh -xe /tmp/jenkins4687125526416196371.sh
+ echo Stage: Build
Stage: Build
+ echo Environment: Development
Environment: Development
Finished: SUCCESS

Reading the Output

Line	Meaning
`+ echo Stage: Build`	Shell command Jenkins executed
`Stage: Build`	The string parameter value printed
`+ echo Environment: Development`	Second shell command
`Environment: Development`	The choice parameter value printed
`Finished: SUCCESS`	Build completed without errors

The + prefix is the shell's trace mode (sh -x), which Jenkins enables by default to show every command it runs. It's incredibly useful for debugging.

Troubleshooting

Issue	Solution
"Add Parameter" button missing	Make sure This project is parameterized is checked
Choice parameter only shows one option	Each choice must be on a separate line, not comma-separated
`$Stage` prints empty	Variable name must match exactly — Jenkins is case-sensitive
"Build with Parameters" not visible	Refresh the job page; ensure parameters were saved correctly
Build fails	Read the Console Output — the `+` lines show exactly what ran

Summary

#	Task	Result
1	Created `parameterized-job`	Freestyle project ✅
2	Added String Parameter `Stage`	Default: `Build` ✅
3	Added Choice Parameter `env`	Development / Staging / Production ✅
4	Configured shell command	Echoed both values ✅
5	Built with `env = Development`	Build #1 SUCCESS ✅

Key Takeaways

Parameterized builds eliminate job duplication. One job, many configurations — perfect for multi-environment deployments.
String vs. Choice has a clear purpose. Use String for free-text input with a sensible default; use Choice to constrain input to valid options.
Parameters become environment variables. Any script step can reference them via $NAME.
Order matters in the UI. The order you add parameters is the order users see them in the "Build with Parameters" form.
The Console Output is your best friend. The + lines reveal exactly what Jenkins executed — invaluable when debugging.

Real-World Applications

This simple pattern scales into serious CI/CD workflows:

Deploy to a specific environment: Choice parameter for env, then use it in a deployment script
Build a specific branch: String parameter for BRANCH, then git checkout $BRANCH
Run with configurable versions: String parameter for VERSION, then docker build -t app:$VERSION
Release channels: Choice parameter for CHANNEL (stable/beta/nightly)

Once you're comfortable with parameters, the next step is chaining jobs together with the Parameterized Trigger plugin — but that's a topic for another blog.

Jenkins Views

Janak Shrestha — Fri, 25 Sep 2026 11:09:21 +0000

The DevOps team of xFusionCorp Industries is planning to create a number of Jenkins jobs for different tasks. So to easily manage the jobs within Jenkins UI they decided to create different views for all Jenkins jobs based on usage/nature of these jobs, - for example nautilus-crons view for all cron jobs. Based on the requirements shared below please perform the below mentioned task:

Click on the Jenkins button on the top bar to access the Jenkins UI. Login using username admin and password Adm!n321.

Create a Jenkins job named nautilus-test-job.
Configure this job to run a simple bash command i.e echo "hello world!!".
Create a view named nautilus-crons (it must be a global view of type List View) and make sure nautilus-test-job and nautilus-cron-job (which is already present on Jenkins) jobs are listed under this new view.
Schedule this newly created job to build periodically at every minute i.e * * * * * (please make sure to use the cron expression exactly same how it is mentioned here).
Make sure the job builds successfully.

Note:

You might need to install some plugins and restart Jenkins service. So, we recommend clicking on Restart Jenkins when installation is complete and no jobs are running on plugin installation/update page i.e update centre. Also, Jenkins UI sometimes gets stuck when Jenkins service restarts in the back end. In this case please make sure to refresh the UI page.
For these kind of scenarios requiring changes to be done in a web UI, please take screenshots so that you can share it with us for review in case your task is marked incomplete. You may also consider using a screen recording software such as loom.com to record and share your work.

Step 1: Access the Jenkins UI

Click the Jenkins button on the top bar of your KodeKloud lab environment. This opens the Jenkins UI in a new tab.

Username: admin
Password: Adm!n321

Once logged in, you'll land on the Jenkins dashboard. Take a moment to note the existing jobs and the "All" view tab at the top.

Step 2: Create the `nautilus-test-job`

From the dashboard:

Click New Item in the left sidebar
In the Enter an item name field, type: nautilus-test-job
Select Freestyle project
Click OK

You'll be taken to the job configuration page.

Step 3: Configure the Build Command

Scroll down to the Build section:

Click Add build step
Select Execute shell
In the Command text area, type exactly:

echo "hello world!!"

Tip: Use double quotes exactly as shown. Using single quotes may cause inconsistent behavior during evaluation.

Step 4: Schedule the Job (Every Minute)

Still on the job configuration page, scroll up to the Build Triggers section:

Check the box: Build periodically
In the Schedule text field, enter exactly:

* * * * *

Understanding the Cron Expression

Field	Value	Meaning
Minute	`*`	Every minute
Hour	`*`	Every hour
Day of Month	`*`	Every day of the month
Month	`*`	Every month
Day of Week	`*`	Every day of the week

This means Jenkins will trigger the job once per minute, continuously.

⚠️ Important: The expression must be exactly * * * * * with single spaces between each asterisk. Any extra spaces or characters will cause the schedule to fail validation or behave incorrectly.

Step 5: Save and Trigger a Manual Build

Click Save at the bottom of the page
You'll be redirected to the job page
Click Build Now in the left sidebar
Watch the Build History box on the left — a blue ball will appear

Verify the Build Output

Click on the build number (e.g., #1) → Console Output. You should see:

Started by user admin
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/nautilus-test-job
[nautilus-test-job] $ /bin/sh -xe /tmp/jenkins...sh
+ echo 'hello world!!'
hello world!!
Finished: SUCCESS

If you see hello world!! and Finished: SUCCESS, the job is working.

Step 6: Create the `nautilus-crons` List View

Now let's organize our jobs. From the Jenkins dashboard:

Look at the top-left area, next to the All tab
Click the small + button — this is the New View button

Note: The "+" button is easy to miss because it's small and unlabeled. It's located right next to the "All" tab. If you can't find it, you can also navigate directly to <jenkins-url>/newView.

On the New View page:

Name: nautilus-crons
Type: Select List View (radio button)
- ❌ Do NOT select "My View" — this creates a personal view, not a global one
Click Create

Step 7: Add Jobs to the View

On the view configuration page:

Under Job Filters, make sure All is selected
Scroll down to the Jobs section
Check the boxes next to:
- ☑️ nautilus-cron-job (existing job)
- ☑️ nautilus-test-job (newly created)
Optionally customize columns (Last Success, Last Failure, Last Duration are useful)
Click Save

Step 8: Verify the View

After saving, you'll be redirected to the nautilus-crons view. Confirm:

✅ A new tab named nautilus-crons appears next to All
✅ Both jobs are listed:
- nautilus-cron-job
- nautilus-test-job

Step 9: Confirm Automated Builds Are Working

Wait 1–2 minutes, then check nautilus-test-job again.

You should see:

✅ Build History growing (#1, #2, #3, ...)
✅ Each new build shows "Started by timer" in the console output
✅ Last Success column shows a recent timestamp
✅ A green checkmark next to the job name

This confirms the cron schedule is working correctly.

Sample Console Output from an Automated Build

Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/nautilus-test-job
[nautilus-test-job] $ /bin/sh -xe /tmp/jenkins...sh
+ echo 'hello world!!'
hello world!!
Finished: SUCCESS

The key indicator here is "Started by timer" — that's proof Jenkins is triggering the job based on the cron schedule, not manual clicks.

Add Response Headers in Apache

Janak Shrestha — Thu, 24 Sep 2026 12:04:41 +0000

We are working on hardening Apache web server on all app servers. As a part of this process we want to add some of the Apache response headers for security purpose. We are testing the settings one by one on all app servers. As per details mentioned below enable these headers for Apache:

Install httpd package on App Server 2 using yum and configure it to run on 3000 port, make sure to start its service.
Create an index.html file under Apache's default document root i.e /var/www/html and add below given content in it. Welcome to the xFusionCorp Industries!
Configure Apache to enable below mentioned headers: X-XSS-Protection header with value 1; mode=block X-Frame-Options header with value SAMEORIGIN X-Content-Type-Options header with value nosniff

Note: You can test using curl on the given app server as LBR URL will not work for this task.

Configuring Apache with Security Headers on a Custom Port

A Deep Dive for System Administrators

Introduction

Web server hardening is a critical part of securing modern infrastructure. One of the most effective ways to harden Apache is by adding HTTP response headers that instruct browsers to enforce security policies. These headers protect against common web vulnerabilities such as cross-site scripting (XSS), clickjacking, and MIME-type sniffing attacks.

This guide walks through a complete implementation on a CentOS Stream 9 server. The task involves installing Apache, running it on a custom port, creating a test page, and enabling three specific security headers. Each step is explained in depth to build a strong understanding of the underlying concepts.

Environment Details

Component	Value
Server	stapp02
User	steve
Password	Am3ric@
OS	CentOS Stream 9
Apache Version	2.4.62
Custom Port	3000
Document Root	/var/www/html

Understanding the Security Headers

Before diving into the configuration, it is essential to understand what each header does and why it matters.

X-XSS-Protection

This header enables the browser's built-in Cross-Site Scripting (XSS) filter. When set to 1; mode=block, the browser detects reflected XSS attacks and blocks the page from rendering rather than attempting to sanitize it. While modern browsers rely more on Content Security Policy, this header still provides protection for older browsers.

X-Frame-Options

This header prevents clickjacking attacks by controlling whether a page can be embedded in a frame or iframe. Setting it to SAMEORIGIN allows only pages from the same origin to embed the content. Attackers cannot trick users into clicking hidden elements on a different site.

X-Content-Type-Options

This header set to nosniff prevents browsers from guessing the MIME type of a file. Without this header, a browser might interpret a text file as JavaScript or HTML, leading to security vulnerabilities. Setting it to nosniff forces the browser to trust the server's declared Content-Type.

Summary Table

Header	Value	Protection
X-XSS-Protection	1; mode=block	Cross-site scripting
X-Frame-Options	SAMEORIGIN	Clickjacking
X-Content-Type-Options	nosniff	MIME sniffing

Step 1: Connect to the Server

Access the server using SSH.

ssh steve@stapp02
Password: Am3ric@

Switch to root to perform administrative tasks.

sudo su -
Password: Am3ric@

The sudo command provides elevated privileges required for installing packages and editing system configuration files.

Step 2: Install the httpd Package

Install Apache using the yum package manager.

yum install -y httpd

The installation pulls in several dependencies.

Package	Purpose
httpd	Main Apache server
httpd-core	Core server binary
httpd-tools	Utilities like apachectl
apr	Apache Portable Runtime
apr-util	Utility library for APR
mod_http2	HTTP/2 protocol support
mod_lua	Lua scripting support
mailcap	MIME type handling

After installation, the httpd.service is registered with systemd.

Step 3: Configure Apache to Listen on Port 3000

By default, Apache listens on port 80. The task requires port 3000. The Listen directive in the main configuration file controls this.

Edit the file at /etc/httpd/conf/httpd.conf.

sed -i 's/^Listen 80/Listen 3000/' /etc/httpd/conf/httpd.conf

Verify the change.

grep "^Listen" /etc/httpd/conf/httpd.conf

Expected output.

Listen 3000

Why Use sed

The sed command performs a search and replace in place. The ^ anchors the match to the start of the line, ensuring only the Listen directive is changed and not other occurrences of 80.

Step 4: Create the Index Page

Create a simple index.html file in the document root.

echo "Welcome to the xFusionCorp Industries!" > /var/www/html/index.html

Verify the content.

cat /var/www/html/index.html

The document root /var/www/html is the default location from which Apache serves files. The DirectoryIndex directive in the configuration tells Apache to look for index.html when a directory is requested.

Step 5: Configure Security Headers

Apache uses the mod_headers module to manipulate HTTP headers. This module is enabled by default on CentOS Stream 9.

Create a dedicated configuration file in the conf.d directory.

cat > /etc/httpd/conf.d/security-headers.conf << 'EOF'
Header always set X-XSS-Protection "1; mode=block"
Header always set X-Frame-Options "SAMEORIGIN"
Header always set X-Content-Type-Options "nosniff"
EOF

Why Use a Separate Configuration File

Keeping custom configurations in /etc/httpd/conf.d/ is a best practice. Files in this directory are automatically included by the main configuration. This approach avoids modifying the main file and makes it easier to manage changes.

Understanding the Directives

Directive	Meaning
Header always set	Adds the header to all responses
X-XSS-Protection	Header name
"1; mode=block"	Header value (quoted)

The always keyword ensures the header is added even to error responses and redirects.

Step 6: Start and Enable the Apache Service

Start the service and configure it to start on boot.

systemctl start httpd
systemctl enable httpd
systemctl status httpd

The status output confirms the service is running and listening on port 3000.

● httpd.service - The Apache HTTP Server
     Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; preset: disabled)
     Active: active (running)
   Main PID: 29475 (httpd)
     Status: "Started, listening on: port 3000"

Understanding systemctl

Command	Purpose
systemctl start	Starts the service immediately
systemctl enable	Configures the service to start on boot
systemctl status	Shows the current state

Step 7: Verify Firewall Status

Check if firewalld is active.

systemctl status firewalld

In this environment, firewalld was not installed. If it were active, port 3000 would need to be allowed.

firewall-cmd --zone=public --add-port=3000/tcp --permanent
firewall-cmd --reload

Step 8: Verify the Headers

Test the configuration using curl.

curl -I http://localhost:3000/

The output shows the HTTP response headers.

HTTP/1.1 200 OK
Date: Thu, 24 Sep 2026 12:01:11 GMT
Server: Apache/2.4.62 (CentOS Stream)
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Last-Modified: Thu, 24 Sep 2026 11:59:33 GMT
ETag: "27-65c39553de0df"
Accept-Ranges: bytes
Content-Length: 39
Content-Type: text/html; charset=UTF-8

All three headers are present with the correct values.

Filter the output for specific headers.

curl -I http://localhost:3000/ | grep -E "X-XSS|X-Frame|X-Content"

Expected output.

X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff

Complete Workflow Summary

# 1. Connect to the server
ssh steve@stapp02
sudo su -

# 2. Install httpd
yum install -y httpd

# 3. Configure port 3000
sed -i 's/^Listen 80/Listen 3000/' /etc/httpd/conf/httpd.conf
grep "^Listen" /etc/httpd/conf/httpd.conf

# 4. Create index.html
echo "Welcome to the xFusionCorp Industries!" > /var/www/html/index.html

# 5. Configure security headers
cat > /etc/httpd/conf.d/security-headers.conf << 'EOF'
Header always set X-XSS-Protection "1; mode=block"
Header always set X-Frame-Options "SAMEORIGIN"
Header always set X-Content-Type-Options "nosniff"
EOF

# 6. Start and enable Apache
systemctl start httpd
systemctl enable httpd
systemctl status httpd

# 7. Configure firewall (if applicable)
systemctl status firewalld
# If active:
# firewall-cmd --zone=public --add-port=3000/tcp --permanent
# firewall-cmd --reload

# 8. Verify
curl -I http://localhost:3000/
curl -I http://localhost:3000/ | grep -E "X-XSS|X-Frame|X-Content"

Verification Checklist

Check	Command	Expected
Apache installed	`rpm -qa \	grep httpd`
Listening on 3000	`ss -tlnp \	grep :3000`
Index page	`cat /var/www/html/index.html`	Welcome message
Headers file	`cat /etc/httpd/conf.d/security-headers.conf`	Three headers
Service running	`systemctl is-active httpd`	active
Service enabled	`systemctl is-enabled httpd`	enabled
Headers present	`curl -I http://localhost:3000/`	All three headers

Troubleshooting

Apache Fails to Start

Check the configuration syntax.

httpd -t

Check the error log.

tail -20 /var/log/httpd/error_log
journalctl -u httpd -n 50

Headers Not Appearing

Verify that mod_headers is enabled.

httpd -M | grep headers

If not enabled, uncomment the LoadModule line in /etc/httpd/conf.modules.d/00-base.conf.

Port Already in Use

Check what is listening on port 3000.

ss -tlnp | grep :3000

Stop the conflicting service or choose a different port.

Firewall Blocking Access

Check firewalld status and allow the port if needed.

systemctl status firewalld
firewall-cmd --zone=public --add-port=3000/tcp --permanent
firewall-cmd --reload

SELinux Blocking Port

If SELinux is enforcing, Apache may not be able to bind to port 3000.

semanage port -a -t http_port_t -p tcp 3000

Best Practices

Use a separate configuration file in /etc/httpd/conf.d/ for custom headers.
Always test configuration with httpd -t before restarting.
Use Header always set to ensure headers appear on all responses.
Document the purpose of each header for future reference.
Check both the service status and the actual HTTP response after changes.
Consider Content Security Policy (CSP) for modern XSS protection.
Regularly review and update security headers as standards evolve.

Conclusion

This guide demonstrated the complete process of installing Apache, configuring it to run on a custom port, creating a test page, and enabling three important security headers. Each step was explained with the underlying concepts to build a deep understanding.

The key takeaways are that HTTP response headers provide browser-enforced security controls, Apache configuration is modular and can be extended via files in /etc/httpd/conf.d/, and verification with tools like curl confirms that the configuration is working as intended.

By applying these headers, the web server gains protection against cross-site scripting, clickjacking, and MIME-type sniffing attacks, contributing to a more secure application environment.

Quick Reference

Task	Command
Install httpd	`yum install -y httpd`
Change port	`sed -i 's/^Listen 80/Listen 3000/' /etc/httpd/conf/httpd.conf`
Create index	`echo "Welcome..." > /var/www/html/index.html`
Add headers	Create `/etc/httpd/conf.d/security-headers.conf`
Start service	`systemctl start httpd`
Enable service	`systemctl enable httpd`
Test headers	`curl -I http://localhost:3000/`
Check syntax	`httpd -t`

Linux Bash Scripts

Janak Shrestha — Wed, 23 Sep 2026 10:49:58 +0000

The production support team of xFusionCorp Industries is working on developing some bash scripts to automate different day to day tasks. One is to create a bash script for archiving website content files. They have a static website running on App Server 3 in Stratos Datacenter, and they need to create a bash script named blog_archive.sh which should accomplish the following tasks. (Also remember to place the script under the /scripts directory on App Server 3).

a. Create a zip archive named xfusioncorp_blog.zip of /var/www/html/blog directory.

b. Save the archive in the /archives/ directory on the App Server 3. This is a temporary storage, as archives from this location will be cleaned on a weekly basis. Therefore, the archive should also be copied to the Nautilus Storage Server so it can be retrieved later for validation purposes.

c. Copy the created archive to the Nautilus Storage Server server in the /archives/ location.

d. Please make sure script won't ask for password while copying the archive file. Additionally, the respective server user (for example, tony in case of App Server 1) must be able to run it.

e. Do not use sudo inside the script.

Note:

The zip package must be installed on given App Server before executing the script. This package is essential for creating the zip archive of the website files. Install it manually outside the script.

Environment Details

Component	Value
App Server	stapp03
App Server User	banner
App Server Password	BigGr33n
Storage Server	ststor01
Storage Server User	natasha
Storage Server Password	Bl@kW
Website Directory	/var/www/html/blog
Script Location	/scripts/blog_archive.sh
Archive Location	/archives/xfusioncorp_blog.zip

Step 1: Connect to App Server 3

Access the server using SSH.

ssh banner@stapp03
Password: BigGr33n

Switch to root to perform administrative tasks.

sudo su -
Password: BigGr33n

Step 2: Install the zip Package

The zip package is required to create the archive. Install it manually outside the script as specified in the task requirements.

yum install -y zip

Verify the installation.

rpm -qa | grep zip

If the package is already installed, the output will show the version.

Step 3: Create Required Directories

Create the /scripts directory for the script and the /archives directory for the archive.

mkdir -p /scripts
mkdir -p /archives

Set ownership so the banner user can write to these directories.

chown -R banner:banner /scripts
chown -R banner:banner /archives

Step 4: Generate SSH Keys for Passwordless Copy

The script must copy the archive to the storage server without prompting for a password. This requires SSH key-based authentication.

Switch to the banner user.

su - banner

Generate an SSH key pair.

ssh-keygen -t rsa -b 4096 -N "" -f ~/.ssh/id_rsa

Copy the public key to the storage server.

ssh-copy-id natasha@ststor01
Password: Bl@kW

Test the passwordless login.

ssh natasha@ststor01 "hostname"

If the command returns the storage server hostname without prompting for a password, the key authentication is working.

Step 5: Create the Bash Script

Create the script file at /scripts/blog_archive.sh.

vi /scripts/blog_archive.sh

Add the following content.

#!/bin/bash

# Create a zip archive of the blog directory
zip -r /archives/xfusioncorp_blog.zip /var/www/html/blog

# Copy the archive to the Nautilus Storage Server
scp /archives/xfusioncorp_blog.zip natasha@ststor01:/archives/

Save the file and exit the editor.

Step 6: Make the Script Executable

Set execute permissions on the script.

chmod +x /scripts/blog_archive.sh

Verify the permissions.

ls -la /scripts/blog_archive.sh

Step 7: Run the Script as the banner User

The task requires that the banner user can run the script. Execute it as banner.

su - banner -c "/scripts/blog_archive.sh"

The output shows the files being added to the archive and the copy progress.

  adding: var/www/html/blog/ (stored 0%)
  adding: var/www/html/blog/index.html (stored 0%)
  adding: var/www/html/blog/.gitkeep (stored 0%)
xfusioncorp_blog.zip                                   100%  588     1.0MB/s   00:00

Step 8: Verify the Archive

Check that the archive was created locally.

ls -la /archives/xfusioncorp_blog.zip

Expected output:

-rw-r--r-- 1 banner banner 588 Sep 23 10:38 /archives/xfusioncorp_blog.zip

Check that the archive was copied to the storage server.

ssh natasha@ststor01 "ls -la /archives/xfusioncorp_blog.zip"

Expected output:

-rw-r--r-- 1 natasha natasha 588 Sep 23 10:38 /archives/xfusioncorp_blog.zip

Verify the contents of the archive.

unzip -l /archives/xfusioncorp_blog.zip

Complete Workflow Summary

# 1. Connect to App Server 3
ssh banner@stapp03
sudo su -

# 2. Install zip
yum install -y zip

# 3. Create directories
mkdir -p /scripts /archives
chown -R banner:banner /scripts /archives

# 4. Generate SSH keys as banner
su - banner
ssh-keygen -t rsa -b 4096 -N "" -f ~/.ssh/id_rsa
ssh-copy-id natasha@ststor01

# 5. Create the script
vi /scripts/blog_archive.sh
# Add: zip -r /archives/xfusioncorp_blog.zip /var/www/html/blog
# Add: scp /archives/xfusioncorp_blog.zip natasha@ststor01:/archives/

# 6. Make executable
chmod +x /scripts/blog_archive.sh

# 7. Run the script as banner
su - banner -c "/scripts/blog_archive.sh"

# 8. Verify
ls -la /archives/xfusioncorp_blog.zip
ssh natasha@ststor01 "ls -la /archives/xfusioncorp_blog.zip"

Script Content

#!/bin/bash

# Create a zip archive of the blog directory
zip -r /archives/xfusioncorp_blog.zip /var/www/html/blog

# Copy the archive to the Nautilus Storage Server
scp /archives/xfusioncorp_blog.zip natasha@ststor01:/archives/

Key Requirements Explained

Requirement	How It Is Met
Script named `blog_archive.sh` under `/scripts`	Created at `/scripts/blog_archive.sh`
Zip archive of `/var/www/html/blog`	`zip -r /archives/xfusioncorp_blog.zip /var/www/html/blog`
Save archive to `/archives/` on App Server 3	Archive saved locally
Copy to Nautilus Storage Server	`scp /archives/xfusioncorp_blog.zip natasha@ststor01:/archives/`
No password prompt during copy	SSH key-based authentication configured
Run as the banner user	Script owned by banner, keys generated for banner
No sudo inside the script	Script uses only zip and scp

Troubleshooting

Issue	Cause	Fix
`zip: command not found`	zip package not installed	`yum install -y zip`
`Permission denied` when running script	Script not executable	`chmod +x /scripts/blog_archive.sh`
Password prompt during scp	SSH keys not configured for banner	Regenerate keys as banner user
Host key verification failed	Host key not accepted	Run `ssh natasha@ststor01` first to accept the key
Archive not copied	SSH connectivity issue	Test with `ssh natasha@ststor01 "echo test"`
`chown: Operation not permitted`	Not running as root	Run `chown` commands as root

Best Practices

Always install required packages outside the script, not inside.
Generate SSH keys as the user who will run the script.
Test passwordless SSH before relying on it in a script.
Set appropriate ownership on script and archive directories.
Keep the script simple and focused on a single task.
Avoid using sudo inside scripts when not necessary.
Verify both the local and remote copies of the archive.

Conclusion

This guide covered the creation of a bash script that archives website content and copies it to a remote server. The script uses zip to create the archive and scp to transfer it, with SSH key-based authentication enabling passwordless operation.

The key steps were installing the zip package, creating the required directories, generating SSH keys for the banner user, writing the script, and verifying that the archive was created and copied successfully.

Quick Reference

Task	Command
Install zip	`yum install -y zip`
Create directories	`mkdir -p /scripts /archives`
Generate SSH key	`ssh-keygen -t rsa -b 4096 -N "" -f ~/.ssh/id_rsa`
Copy SSH key	`ssh-copy-id natasha@ststor01`
Create archive	`zip -r /archives/xfusioncorp_blog.zip /var/www/html/blog`
Copy to remote	`scp /archives/xfusioncorp_blog.zip natasha@ststor01:/archives/`
Run script	`su - banner -c "/scripts/blog_archive.sh"`
Verify local	`ls -la /archives/xfusioncorp_blog.zip`
Verify remote	`ssh natasha@ststor01 "ls -la /archives/xfusioncorp_blog.zip"`

MariaDB Troubleshooting

Janak Shrestha — Tue, 22 Sep 2026 11:48:08 +0000

There is a critical issue going on with the Nautilus application in Stratos DC. The production support team identified that the application is unable to connect to the database. After digging into the issue, the team found that mariadb service is down on the database server.

Look into the issue and fix the same.

Step 1: Connect to the Database Server

Access the database server using SSH.

ssh peter@stdb01
Password: Sp!dy

Switch to root to perform administrative tasks.

sudo su -
Password: Sp!dy

Step 2: Check the MariaDB Service Status

The first step is to check whether the service is running.

systemctl status mariadb

The output showed the service was inactive and dead.

○ mariadb.service - MariaDB 10.5 database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; preset: disabled)
     Active: inactive (dead)
       Docs: man:mariadbd(8)
             https://mariadb.com/kb/en/library/systemd/

The service was not running and was also disabled, meaning it would not start on boot.

Step 3: Attempt to Start and Enable the Service

Try to start and enable the service to see what error occurs.

systemctl enable --now mariadb

The output showed the service failed to start.

Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xeu mariadb.service" for details.

Step 4: Check the Logs

Check the system logs for MariaDB.

journalctl -u mariadb -n 50

The logs showed the initialization process but did not immediately reveal the root cause.

Sep 22 11:41:00 stdb01 systemd[1]: Starting MariaDB 10.5 database server...
Sep 22 11:41:00 stdb01 mariadb-prepare-db-dir[22278]: Initializing MariaDB database
...
Sep 22 11:41:01 stdb01 systemd[1]: mariadb.service: Main process exited, code=exited, st...
Sep 22 11:41:01 stdb01 systemd[1]: mariadb.service: Failed with result 'exit-code'.
Sep 22 11:41:01 stdb01 systemd[1]: Failed to start MariaDB 10.5 database server.

Step 5: Check the MariaDB Error Log

The MariaDB error log contains more detailed information.

cat /var/log/mariadb/mariadb.log

The error log clearly identified the root cause.

2026-09-22 11:41:01 0 [Note] Server socket created on IP: '::'.
2026-09-22 11:41:01 0 [ERROR] mariadbd: Can't create/write to file '/run/mariadb/mariadb.pid' (Errcode: 13 "Permission denied")
2026-09-22 11:41:01 0 [ERROR] Can't start server: can't create PID file: Permission denied

Root Cause Identified

MariaDB cannot create its PID file at /run/mariadb/mariadb.pid because of a permission issue. The directory /run/mariadb/ has incorrect ownership or permissions.

Step 6: Check the Directory Permissions

Inspect the /run/mariadb/ directory.

ls -la /run/mariadb/

The output showed the directory was owned by root:mysql with permissions 755.

drwxr-xr-x 2 root mysql 40 Sep 22 10:23 .

MariaDB runs as the mysql user. While the group is correct, the directory is owned by root, and the mysql user cannot create files in it because the owner permissions do not include write for the mysql user (the owner is root, and root has write, but mysql is not root).

Wait, 755 means the owner has rwx, group has r-x, and others have r-x. Since the owner is root, the mysql user only has group permissions (r-x), which do not include write. Therefore, mysql cannot create the PID file.

Step 7: Fix the Directory Ownership

Change the ownership of the directory to mysql:mysql.

chown -R mysql:mysql /run/mariadb
chmod 755 /run/mariadb

Verify the change.

ls -la /run/mariadb/

Expected output:

drwxr-xr-x 2 mysql mysql 40 Sep 22 10:23 .

Now the mysql user owns the directory and has write permission.

Step 8: Start the MariaDB Service

Start the service again.

systemctl start mariadb
systemctl status mariadb

The service started successfully.

● mariadb.service - MariaDB 10.5 database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: disabled)
     Active: active (running) since Tue 2026-09-22 11:43:18 UTC; 18ms ago
   Main PID: 23135 (mariadbd)
     Status: "Taking your SQL requests now..."

Step 9: Verify the Service

Check that the service is listening on port 3306.

ss -tlnp | grep 3306

Expected output:

LISTEN 0 80 *:3306 *:* users:(("mariadbd",pid=23135,fd=19))

Test the database connection.

mysql -u root -e "SELECT 1;"

Expected output:

+---+
| 1 |
+---+
| 1 |
+---+

Step 10: Ensure the Service is Enabled

Confirm the service is enabled to start on boot.

systemctl is-enabled mariadb

Expected output:

enabled

Complete Troubleshooting Workflow

# 1. Connect to the database server
ssh peter@stdb01
sudo su -

# 2. Check service status
systemctl status mariadb

# 3. Attempt to start and enable
systemctl enable --now mariadb

# 4. Check logs
journalctl -u mariadb -n 50

# 5. Check MariaDB error log
cat /var/log/mariadb/mariadb.log

# 6. Check directory permissions
ls -la /run/mariadb/

# 7. Fix ownership
chown -R mysql:mysql /run/mariadb
chmod 755 /run/mariadb

# 8. Start service
systemctl start mariadb
systemctl status mariadb

# 9. Verify port
ss -tlnp | grep 3306

# 10. Test connection
mysql -u root -e "SELECT 1;"

# 11. Check enabled
systemctl is-enabled mariadb

Summary of the Fix

Issue	Cause	Fix
MariaDB fails to start	Permission denied on `/run/mariadb/mariadb.pid`	`chown -R mysql:mysql /run/mariadb`
Directory owned by root	MariaDB runs as mysql user	Change ownership to mysql:mysql
Service disabled	Not enabled on boot	`systemctl enable mariadb`

Common MariaDB Issues and Fixes

Issue	Symptom	Fix
Permission denied on PID file	`Can't create/write to file '/run/mariadb/mariadb.pid'`	`chown -R mysql:mysql /run/mariadb`
InnoDB corruption	`ib_logfile0 not found`	Remove `ib_logfile*` and restart
Port already in use	`Address already in use`	Find and stop conflicting process
Data directory permissions	`Can't open file`	`chown -R mysql:mysql /var/lib/mysql`
Configuration errors	`Unknown variable`	Check `my.cnf` files

Verification Checklist

Check	Command	Expected
Service Status	`systemctl status mariadb`	active (running)
Port Listening	`ss -tlnp \	grep 3306`
Connection Test	`mysql -u root -e "SELECT 1;"`	Returns 1
Enabled on Boot	`systemctl is-enabled mariadb`	enabled
Error Log	`tail /var/log/mariadb/mariadb.log`	No errors

Best Practices

Always check the service status first.
Review both system logs and application-specific logs.
Identify the exact error message before making changes.
Fix permissions based on the user the service runs as.
Verify the service after starting it.
Ensure the service is enabled for boot persistence.
Document the fix for future reference.

Conclusion

This guide covered a complete MariaDB troubleshooting workflow. The root cause was a permission issue on the /run/mariadb/ directory, which prevented MariaDB from creating its PID file. By changing the ownership to mysql:mysql, the service started successfully.

The key steps were checking the service status, reviewing logs, identifying the permission error, fixing the directory ownership, and verifying the service is running and listening on port 3306.

Haproxy LBR Troubleshooting

Janak Shrestha — Mon, 21 Sep 2026 11:20:33 +0000

xFusionCorp Industries has an application running on Nautlitus infrastructure in Stratos Datacenter. The monitoring tool recognised that there is an issue with the haproxy service on the LBR server. That needs to fixed to make the application work properly.

Troubleshoot and fix the issue, and make sure haproxy service is running on the Nautilus LBR server. Once fixed, make sure you are able to access the website by running the command curl http://stlb01:80/ in the terminal.

Understanding the Environment

Component	Details
LBR Server	stlb01
LBR User	loki
LBR Password	Mischi3f
App Server 1	stapp01 (Apache on port 8080)
App Server 2	stapp02 (Apache on port 8080)
App Server 3	stapp03 (Apache on port 8080)
HAProxy Port	80

Step 1: Connect to the LBR Server

Access the load balancer server using SSH.

ssh loki@stlb01
Password: Mischi3f

Switch to root to perform administrative tasks.

sudo su -
Password: Mischi3f

Step 2: Check the HAProxy Service Status

The first step is to check whether the HAProxy service is running.

systemctl status haproxy

In this scenario, the output showed the service had failed.

× haproxy.service - HAProxy Load Balancer
     Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; preset: disabled)
    Drop-In: /etc/systemd/system/haproxy.service.d
             └─override.conf
     Active: failed (Result: exit-code)

The service was in a failed state, indicating a configuration or runtime issue.

Step 3: Check HAProxy Logs

Check the system logs for HAProxy to identify the root cause.

journalctl -u haproxy -n 50

The logs showed the service was trying to execute the haproxy binary with a configuration file but failing with an exit code.

haproxy.service: Executing: /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg ...
haproxy.service: Control process exited, code=exited, status=...
haproxy.service: Failed with result 'exit-code'.

Step 4: Verify the Configuration Syntax

Run the HAProxy configuration check to identify syntax errors.

haproxy -c -f /etc/haproxy/haproxy.cfg

The output revealed multiple errors.

[ALERT] (3689) : config : parsing [/etc/haproxy/haproxy.cfg:39] : please use the 'bind' keyword for listening addresses.
[ALERT] (3689) : config : parsing [/etc/haproxy/haproxy.cfg:40] : 'listen' or 'defaults' expected.
[ALERT] (3689) : config : parsing [/etc/haproxy/haproxy.cfg:41] : 'listen' or 'defaults' expected.
[ALERT] (3689) : config : parsing [/etc/haproxy/haproxy.cfg:43] : 'listen' or 'defaults' expected.
[ALERT] (3689) : config : parsing [/etc/haproxy/haproxy.cfg:44] : 'listen' or 'defaults' expected.
[ALERT] (3689) : config : parsing [/etc/haproxy/haproxy.cfg:60]: Missing LF on last line, file might have been truncated at position 36.
[ALERT] (3689) : config : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
[ALERT] (3689) : config : Fatal errors found in configuration.

Root Cause Identified

Deprecated Syntax: The frontend main *:80 line used old syntax. Modern HAProxy requires frontend main followed by bind *:80.
Commented-Out Backend: The backend app section was commented out, leaving orphaned balance and server lines that caused parsing errors.
Missing Newline: The file was truncated without a newline at the end.

Step 5: Examine the Configuration File

View the current configuration to understand the issues.

cat /etc/haproxy/haproxy.cfg

The problematic sections were identified.

frontend  main *:80
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    ...

#backend app
    balance     roundrobin
    server  app1 stapp01:3000 check
    server  app2 stapp02:3000 check
    server  app3 stapp03:3000 check

The backend app was commented out, and the server definitions were on port 3000.

Step 6: Check the Drop-In Override

A systemd drop-in override was present at /etc/systemd/system/haproxy.service.d/override.conf.

cat /etc/systemd/system/haproxy.service.d/override.conf

[Service]
ExecStartPre=
ExecStartPre=/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c -q
ExecStart=
ExecStart=/usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock

This override forces HAProxy to use only the main configuration file, which is fine. The issue was in the main configuration itself.

Step 7: Determine the Correct Backend Port

Before fixing the configuration, verify which port the app servers are listening on.

curl -I http://stapp01:8087
curl -I http://stapp02:8087
curl -I http://stapp03:8087

Result: Connection refused on port 8087.

curl -I http://stapp01:8080
curl -I http://stapp02:8080
curl -I http://stapp03:8080

Result: HTTP 200 OK on port 8080.

The app servers are listening on port 8080, not 3000 or 8087.

Step 8: Replace the Configuration with a Valid Version

Replace the configuration file with a clean, valid version.

cat > /etc/haproxy/haproxy.cfg << 'EOF'
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats

defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

frontend main
    bind *:80
    default_backend app

backend app
    balance roundrobin
    server stapp01 stapp01:8080 check
    server stapp02 stapp02:8080 check
    server stapp03 stapp03:8080 check
EOF

Key Changes

Original	Fixed
`frontend main *:80`	`frontend main` + `bind *:80`
`#backend app` (commented)	`backend app` (uncommented)
Orphaned `balance` and `server` lines	Properly nested under `backend app`
Port 3000	Port 8080 (correct port)
Missing newline at end	File ends with newline

Step 9: Verify the Configuration

Run the configuration check again.

haproxy -c -f /etc/haproxy/haproxy.cfg

Expected output:

Configuration file is valid

Step 10: Restart HAProxy

Restart the service to apply the changes.

systemctl restart haproxy
systemctl status haproxy

The output should show the service is active and running.

● haproxy.service - HAProxy Load Balancer
     Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; preset: disabled)
    Drop-In: /etc/systemd/system/haproxy.service.d
             └─override.conf
     Active: active (running) since Mon 2026-09-21 11:16:59 UTC; 23ms ago
   Main PID: 3968 (haproxy)
     Status: "Ready."

Step 11: Verify Listening Ports

Confirm HAProxy is listening on port 80.

ss -tlnp | grep :80

Expected output:

LISTEN 0 3000 0.0.0.0:80 0.0.0.0:* users:(("haproxy",pid=3982,fd=9))

Step 12: Test the Website

Test the website using curl.

curl http://stlb01:80/

Expected output:

Welcome to xFusionCorp Industries!

Complete Troubleshooting Workflow

# 1. Connect to the LBR server
ssh loki@stlb01
sudo su -

# 2. Check service status
systemctl status haproxy

# 3. Check logs
journalctl -u haproxy -n 50

# 4. Check configuration syntax
haproxy -c -f /etc/haproxy/haproxy.cfg

# 5. Examine the configuration file
cat /etc/haproxy/haproxy.cfg

# 6. Check the drop-in override
cat /etc/systemd/system/haproxy.service.d/override.conf

# 7. Determine the correct backend port
curl -I http://stapp01:8080
curl -I http://stapp02:8080
curl -I http://stapp03:8080

# 8. Replace configuration with valid version
cat > /etc/haproxy/haproxy.cfg << 'EOF'
... (valid configuration) ...
EOF

# 9. Verify configuration
haproxy -c -f /etc/haproxy/haproxy.cfg

# 10. Restart HAProxy
systemctl restart haproxy
systemctl status haproxy

# 11. Verify listening ports
ss -tlnp | grep :80

# 12. Test the website
curl http://stlb01:80/

Common HAProxy Issues and Fixes

Issue	Symptom	Fix
Deprecated syntax	`please use the 'bind' keyword`	Use `frontend main` + `bind *:80`
Commented backend	`'listen' or 'defaults' expected`	Uncomment `backend app`
Wrong backend port	503 Service Unavailable	Verify port with `curl` and update config
Missing newline	`Missing LF on last line`	Ensure file ends with a newline
Duplicate sections	Fatal errors	Ensure unique section names
Port conflict	Service fails to start	Check `ss -tlnp \
Stats socket missing	Service fails	Ensure {% raw %}`stats socket /var/lib/haproxy/stats` present

Verification Checklist

Check	Command	Expected
Configuration valid	`haproxy -c -f /etc/haproxy/haproxy.cfg`	Configuration file is valid
Service running	`systemctl status haproxy`	active (running)
Listening on port 80	`ss -tlnp \	grep :80`
Website accessible	`curl http://stlb01:80/`	Welcome message
Backends reachable	`curl -I http://stapp01:8080`	HTTP 200 OK
Logs clean	`journalctl -u haproxy -n 30`	No errors

Best Practices

Always verify configuration before restarting: Use haproxy -c -f to check syntax.
Use modern syntax: Use bind instead of the deprecated frontend name *:port syntax.
Verify backend ports: Test each backend with curl before configuring HAProxy.
Keep the stats socket: The stats socket /var/lib/haproxy/stats line is required for monitoring.
Check logs regularly: Use journalctl -u haproxy to identify issues early.
Maintain backups: Back up the configuration before making changes.
Document changes: Keep a record of configuration changes for auditing.

Conclusion

This guide covered a complete HAProxy troubleshooting workflow. The main issues were:

Deprecated frontend syntax that caused parsing errors.
Commented-out backend that left orphaned directives.
Incorrect backend port (3000 instead of 8080) that caused 503 errors.

By systematically checking the service status, logs, and configuration, the root cause was identified and fixed. After replacing the configuration with a valid version, HAProxy started successfully and the website became accessible.

The key takeaways are to always verify configuration syntax before restarting, use modern HAProxy syntax, and confirm backend ports by testing them directly.

Quick Reference

Task	Command
Check config	`haproxy -c -f /etc/haproxy/haproxy.cfg`
Check status	`systemctl status haproxy`
Check logs	`journalctl -u haproxy -n 50`
Check ports	`ss -tlnp \
Test website	{% raw %}`curl http://stlb01:80/`
Test backend	`curl -I http://stapp01:8080`
Restart service	`systemctl restart haproxy`

This guide was created based on a real-world HAProxy troubleshooting scenario on the Nautilus LBR server in the Stratos Datacenter.