DEV Community

Cover image for Securing Your Home Server — UFW, Fail2Ban, SSH Hardening, and Lessons Learned
Sasanka Rath
Sasanka Rath

Posted on

Securing Your Home Server — UFW, Fail2Ban, SSH Hardening, and Lessons Learned

#linux #security #selfhosted #tutorial

In Part 4, your server went live on the internet. Which means within 24 hours, bots from around the world started probing it for weaknesses. This isn't paranoia — it's reality. Any public-facing server gets hit with hundreds of automated attacks daily.

This is the final part of the series. We'll lock everything down so you can sleep at night.

What we'll cover:

  • UFW firewall — only allow what you need
  • Fail2Ban — auto-ban IPs that try to brute-force your server
  • SSH hardening — custom port, key-only login, no root access
  • Basic monitoring — know when something breaks
  • Lessons learned after running this for a while

Step 1: Set up UFW firewall properly

UFW (Uncomplicated Firewall) blocks every port except the ones you explicitly allow. Default-deny is the only sane approach for a public server.

# Start fresh
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH (we'll change this port in Step 3)
sudo ufw allow 22/tcp

# Allow Nextcloud on local network
sudo ufw allow 8888/tcp

# Allow Samba on local network
sudo ufw allow 137/udp
sudo ufw allow 138/udp
sudo ufw allow 139/tcp
sudo ufw allow 445/tcp

# Allow all traffic on Tailscale interface
sudo ufw allow in on tailscale0

# Enable UFW
sudo ufw enable
Enter fullscreen mode Exit fullscreen mode

Check the rules:

sudo ufw status verbose
Enter fullscreen mode Exit fullscreen mode

You should see only the ports you opened. Everything else is blocked.

⚠️ Don't enable UFW over an SSH session without allowing SSH first. You'll lock yourself out. Always sudo ufw allow 22/tcp before sudo ufw enable.

Step 2: Install and configure Fail2Ban

Fail2Ban watches your log files and bans IPs that show malicious behavior (failed logins, brute-force attempts, etc.).

sudo apt install fail2ban -y
Enter fullscreen mode Exit fullscreen mode

Create a local config (never edit jail.conf directly — it gets overwritten on updates):

sudo nano /etc/fail2ban/jail.local
Enter fullscreen mode Exit fullscreen mode 
[DEFAULT]
# Ban for 1 hour after 5 failed attempts in 10 minutes
bantime = 3600
findtime = 600
maxretry = 5

# Don't ban localhost or your local network
ignoreip = 127.0.0.1/8 192.168.1.0/24

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3

[nextcloud]
enabled = true
port = 80,443,8888
protocol = tcp
filter = nextcloud
maxretry = 5
bantime = 3600
findtime = 600
logpath = /srv/nas/nextcloud/nextcloud.log
Enter fullscreen mode Exit fullscreen mode

Create the Nextcloud filter:

sudo nano /etc/fail2ban/filter.d/nextcloud.conf
Enter fullscreen mode Exit fullscreen mode 
[Definition]
failregex = ^.*Login failed: .* \(Remote IP: <HOST>\).*$
            ^.*"remoteAddr":"<HOST>".*Trusted domain error.*$
ignoreregex =
Enter fullscreen mode Exit fullscreen mode

Start Fail2Ban:

sudo systemctl enable fail2ban
sudo systemctl restart fail2ban
sudo systemctl status fail2ban
Enter fullscreen mode Exit fullscreen mode

Check banned IPs:

sudo fail2ban-client status sshd
Enter fullscreen mode Exit fullscreen mode

Within a day or two, you'll see dozens of IPs banned — these are the bots constantly probing your server.

Step 3: Harden SSH

SSH is the front door to your server. Default settings are a huge target for bots.

Change the SSH port 

sudo nano /etc/ssh/sshd_config
Enter fullscreen mode Exit fullscreen mode

Find and change:

Port 2299
Enter fullscreen mode Exit fullscreen mode

Pick any port between 1024-49151 (I'm using 2299 as an example). This alone reduces bot attacks by ~90% since most bots only scan port 22.

Disable root login

Find or add:

PermitRootLogin no
Enter fullscreen mode Exit fullscreen mode

Disable password authentication (use keys only)

Before doing this, make sure your SSH key works. Generate one if you haven't:

# On your local machine (Mac/Linux/Windows with OpenSSH)
ssh-keygen -t ed25519 -f ~/.ssh/homeserver_key
Enter fullscreen mode Exit fullscreen mode

Copy the public key to your server:

ssh-copy-id -i ~/.ssh/homeserver_key.pub your-username@192.168.1.100
Enter fullscreen mode Exit fullscreen mode

Test it works:

ssh -i ~/.ssh/homeserver_key your-username@192.168.1.100
Enter fullscreen mode Exit fullscreen mode

Once you confirm key login works, disable passwords in sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes
Enter fullscreen mode Exit fullscreen mode

Apply the changes 

# Update UFW for the new SSH port
sudo ufw allow 2299/tcp
sudo ufw delete allow 22/tcp

# Restart SSH
sudo systemctl restart sshd
Enter fullscreen mode Exit fullscreen mode

From now on, SSH with:

ssh -p 2299 -i ~/.ssh/homeserver_key your-username@192.168.1.100
Enter fullscreen mode Exit fullscreen mode

Step 4: Enable automatic security updates

Unpatched servers are the easiest targets. Set up automatic updates for security patches:

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure -plow unattended-upgrades
Enter fullscreen mode Exit fullscreen mode

Select Yes when prompted. Your server now automatically installs security updates overnight.

Step 5: Set up basic monitoring

You want to know when something breaks — not find out from a family member saying "the photos site is down."

Install Uptime Kuma

Uptime Kuma is a simple self-hosted monitoring tool. Add it to your Docker setup:

cd ~/server
nano docker-compose.yml
Enter fullscreen mode Exit fullscreen mode

Add this service below nextcloud:

  uptime-kuma:
    image: louislam/uptime-kuma:latest
    container_name: uptime-kuma
    restart: always
    ports:
      - "3001:3001"
    volumes:
      - ./uptime-kuma:/app/data
Enter fullscreen mode Exit fullscreen mode

Apply:

docker compose up -d
Enter fullscreen mode Exit fullscreen mode

Open http://192.168.1.100:3001, create an admin account, and add monitors for:

  • https://files.yourdomain.com (public endpoint)
  • http://192.168.1.100:8888 (local Nextcloud)
  • Your Tailscale IP (private endpoint)

Set up email or Telegram notifications so you get pinged when anything goes down.

Step 6: Protect your Oracle VPS the same way

Don't forget — your Oracle VPS is also public-facing. SSH into it and apply the same hardening:

# Install Fail2Ban
sudo apt install fail2ban -y

# Same SSH hardening (key-only, change port, disable root)
# Same UFW rules — only open 80, 443, 7000

sudo ufw default deny incoming
sudo ufw allow 2299/tcp    # your custom SSH port
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 7000/tcp
sudo ufw enable
Enter fullscreen mode Exit fullscreen mode

Both servers need the same level of security. The chain is only as strong as its weakest link.

Lessons learned after running this for a while

Things I wish I'd known on day one:

Use Ethernet if the server is anywhere near your router. WiFi drops happen. They're rare but always at the worst time — usually when you're traveling and need a file. Wired connections are boringly reliable.

Set BIOS to auto-power-on. After a power cut, you want the server to boot itself. Look for "AC Power Recovery" or "Restore on AC Power Loss" in BIOS and set to "Always On." All your services (Docker, frpc, Tailscale) will auto-start on boot, so full recovery takes 2-3 minutes with zero intervention.

Back up Nextcloud's config directory. If your server's drive fails, the data is gone but losing the config directory means you can't even rebuild. I rsync ~/server/config to an external drive weekly.

Tailscale is the unsung hero. If I had to cut any part of this stack, the public VPS path would go. Tailscale alone covers 95% of real-world use. The VPS exists only so family members can open a link in a browser.

Don't expose services you don't need. Every open port is a potential vulnerability. If you don't need Samba accessible remotely, don't expose it. Only Nextcloud goes through the public tunnel.

Docker makes recovery easy. When something breaks, I can nuke the container and rebuild from docker-compose.yml in 30 seconds. All state lives in mounted volumes. This is the biggest practical advantage of running services in containers.

Monitor what matters. Disk space is the #1 thing to watch. Photos pile up fast. When your drive hits 90%, things start failing silently. Uptime Kuma + a simple disk space check saves you from nasty surprises.

The finished product

After 5 posts, you have:

✅ A complete personal cloud on an old laptop
✅ Phone auto-backup replacing Google Photos
✅ Desktop sync replacing Google Drive
✅ NAS file sharing via Samba
✅ Private remote access via Tailscale
✅ Public access via custom domain with SSL
✅ Firewall, brute-force protection, and SSH hardening
✅ Automated monitoring
✅ Total recurring cost: ~₹850/year for a domain

All running on hardware that was collecting dust.

What's next

The series is done, but the server isn't. Things I'm planning to add over the coming months:

  • Immich — self-hosted Google Photos alternative with face recognition and AI search
  • Automated off-site backup — nightly rsync to an external drive + occasional snapshots to a cheap cloud
  • Jellyfin — self-hosted media server for streaming movies to any device
  • Second storage drive — a 4TB USB drive for expansion

I'll write about each of these as I add them. If any of it sounds interesting, follow me here.

All config files from this entire series are on GitHub:
👉 github.com/sasrath/homecloud

Thanks for following along with this series. Whether you built the whole thing or just read for context, I hope it was useful.

Questions, feedback, or your own war stories from self-hosting? Drop them in the comments. I read and reply to everything.

Your cloud. Your data. Your house. 🏠

Top comments (0)