100 Days of DevOps: Day 12

#linux #bash #programming #devops

Resolving an Apache Connectivity Issue in Stratos Datacenter

In the world of system administration, encountering connectivity issues with crucial services like Apache is a common occurrence. This article details a recent incident in the Stratos Datacenter where an Apache service on an application server, stapp01, was unreachable. The problem was successfully diagnosed and resolved by systematically checking the service status, identifying a port conflict, and correctly configuring firewall rules.

Step 1: Diagnosing the Apache Service Failure

The initial investigation began on stapp01. A check of the Apache service status revealed it was in a failed state. The error logs clearly indicated a port conflict: another process was already using the required port.

[tony@stapp01 ~]$ sudo systemctl status httpd
. . .
Aug 15 07:39:47 stapp01.stratos.xfusioncorp.com httpd[493]: (98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:8087

Step 2: Resolving the Port Conflict

To fix the port conflict, the process currently using port 8087 had to be identified and terminated.

The netstat command was used to find the process ID (PID) listening on the port.

[tony@stapp01 ~]$ sudo netstat -tulpn | grep 8087
tcp        0      0 127.0.0.1:8087          0.0.0.0:* LISTEN      432/sendmail: accep

The output showed that a sendmail process with PID 432 was the culprit. It was terminated using the kill command.
```
sudo kill 432
```
With the conflicting process gone, the Apache service was successfully restarted.
```
sudo systemctl start httpd
```

Step 3: Adding and Correcting the Firewall Rule

Even with Apache running, it remained unreachable from the jump host. This pointed to a firewall issue. A check of the iptables rules showed a global REJECT rule that was blocking all incoming traffic. The specific ACCEPT rule for port 8087 was missing, and it was essential to add it before the REJECT rule.

The new rule was inserted at position 5 in the INPUT chain, placing it before the general REJECT rule.
```
sudo iptables -I INPUT 5 -p tcp --dport 8087 -j ACCEPT
```
Finally, the iptables configuration was saved to make the changes permanent.
```
sudo iptables-save | sudo tee /etc/sysconfig/iptables
```

Conclusion

Following these steps, the Apache service was successfully configured to run on port 8087, and the firewall was correctly configured to allow external access. The curl command from the jump host returns the expected response, confirming the issue is fully resolved. This case highlights the importance of a methodical approach to troubleshooting, starting from the service itself and then carefully configuring network components like the firewall in the correct order.