DEV Community

Nnamdi Felix Ibe
Nnamdi Felix Ibe

Posted on

100 Days of DevOps and Cloud (AWS), Day 12: A Port Conflict Killed httpd, and an Attached Volume Isn't a Mounted One

Two services cannot listen on the same port. So when Apache refuses to start, the most common reason is boring and specific, something else grabbed the port first. Day 12 was about finding that something and clearing it, then handling the AWS storage version of a job that looks finished but isn't.

One Linux task, one AWS task. Work out why httpd will not come up on its port and fix the conflict, then attach an EBS volume to an EC2 instance. The tasks come from the KodeKloud Engineer platform.

httpd: find the process on the port, then clear it

When httpd fails to bind its port, you need to know what is already sitting there. netstat shows every listening socket and the process behind it:

# Install net-tools if netstat isn't present
yum install -y net-tools

# List listening TCP/UDP ports with the owning PID
netstat -tulpn

# Narrow to the port you care about
netstat -tulpn | grep <port>
Enter fullscreen mode Exit fullscreen mode

The -tulpn flags are worth learning as one unit: TCP and UDP, listening sockets, numeric ports, and the PID. That last piece, the PID, is what lets you act. Once you know which process is squatting on the port, clear it and start httpd:

# Stop the conflicting process
sudo kill -9 <PID>

# Now httpd can bind the port
systemctl enable httpd
systemctl start httpd
systemctl status httpd
Enter fullscreen mode Exit fullscreen mode

A quick word on kill -9. It sends SIGKILL, which ends a process instantly with no chance to clean up. It works, but it is a blunt instrument. Try a plain kill first, which sends SIGTERM and lets the process shut down gracefully, and reach for -9 only when that gets ignored. With the port free, the firewall still has to allow the traffic in:

# Allow inbound traffic on the port, inserted at the top of INPUT
iptables -I INPUT -p tcp --destination-port <port> -j ACCEPT

# Persist the rules so they survive a reboot
service iptables save
systemctl enable iptables
Enter fullscreen mode Exit fullscreen mode

iptables -I inserts the ACCEPT rule at the top of the INPUT chain, so it is checked before any DROP rule further down. And the step people forget, service iptables save, writes the rules to disk. Skip it, and every rule you just added disappears on the next reboot.

EBS: attaching is not mounting

The AWS task was to attach an existing EBS volume to a running instance:

# Attach the volume (same AZ as the instance, always)
aws ec2 attach-volume \
  --volume-id vol-xxxxxxxxxx \
  --instance-id i-xxxxxxxxxx \
  --device /dev/sdf

# Check the attachment state
aws ec2 describe-volumes --volume-ids vol-xxxxxxxxxx
Enter fullscreen mode Exit fullscreen mode

Two things to know. First, the volume and the instance have to be in the same availability zone, the same rule that governs snapshots and network interfaces. Second, and this is the one that catches people, attaching is not mounting. The API call succeeds, and the volume is now wired to the instance, but the operating system still sees a raw disk. It is not usable until you go inside the instance and finish the job:

# Inside the instance
lsblk                        # see the new disk
sudo mkfs -t xfs /dev/xvdf   # format it, only if it is blank
sudo mount /dev/xvdf /data   # mount it somewhere
Enter fullscreen mode Exit fullscreen mode

And a modern wrinkle worth knowing. You asked AWS for /dev/sdf, but on current Nitro-based instances, the OS often shows the volume as an NVMe device like /dev/nvme1n1 instead. The name you request and the name the kernel assigns can differ, so trust lsblk over the device string you passed to the API.

The shared lesson

Both tasks have the same shape. The command returning success is not the same as the job being done. httpd needed the port cleared and the firewall opened. The EBS volume needed formatting and mounting after the attach. Cloud APIs and system services both hand you a green result at the halfway mark, and knowing that is what separates "it says it worked" from "it actually works."

So here is the Day 12 question. Would you rather trust the success message and move on, or know exactly which half of the job that success message actually covers?

Day 12 down. Eighty-eight to go.

Top comments (0)