DEV Community

Cover image for Hardening SSH for Production Servers
FlatlinerAtheist | CLI-NERDS
FlatlinerAtheist | CLI-NERDS

Posted on Originally published at clinerds.com

Hardening SSH for Production Servers

#security #ssh #linux

SSH is the gateway to your server, so it is the first service to lock down on a new machine. By the end of this guide you will have key-only logins, no direct root login, a short list of accounts allowed in, and fail2ban banning addresses that keep failing, with every change tested before you close your session. This is Part 1 of the Linux Server Baseline series (Part 2 covers file permissions, Part 3 covers cron). Commands target Ubuntu 24.04 and Debian 12, with RHEL differences noted where they matter.

Set up key-based login first

Once password logins are off, a key is the only way in over the network, so the key has to work before you touch the server config. Generate an ed25519 key on your workstation, not on the server:

ssh-keygen -t ed25519 -C "alice@laptop"
Enter fullscreen mode Exit fullscreen mode

Accept the default path (~/.ssh/id_ed25519) and set a passphrase. Then copy the public half to the account you will log in as:

ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@203.0.113.10
Enter fullscreen mode Exit fullscreen mode

ssh-copy-id uses your password one last time and appends the key to ~/.ssh/authorized_keys on the server. Confirm the key works on its own by telling the client not to fall back to a password:

ssh -o PreferredAuthentications=publickey deploy@203.0.113.10
Enter fullscreen mode Exit fullscreen mode

If you get a shell, key authentication works. While you are logged in, run sudo whoami and confirm it prints root, because root login is about to be disabled and this account is how you will administer the box.

For the rest of this guide, keep one SSH session open and run every test from a second terminal. Reloading sshd does not drop established sessions, so the open one is your way back if a change goes wrong. Also, if your admin account has no local password (common on cloud images), set one with sudo passwd deploy so the provider's console login works if you need it.

Put your settings in a drop-in file

On Ubuntu 24.04 and Debian 12, /etc/ssh/sshd_config starts with Include /etc/ssh/sshd_config.d/*.conf. For most keywords sshd keeps the first value it reads, and it reads the included files in lexical order before the rest of the main file. Cloud images often ship a file such as 50-cloud-init.conf, and it may contain PasswordAuthentication yes. If you add PasswordAuthentication no to the bottom of sshd_config, or to a file named 60-hardening.conf, the cloud-init value is read first and yours is ignored.

Check what is already there:

grep -rniE 'include|passwordauthentication' /etc/ssh/sshd_config /etc/ssh/sshd_config.d/
Enter fullscreen mode Exit fullscreen mode

Naming your file 00-hardening.conf puts it ahead of everything else. Create it with sudoedit /etc/ssh/sshd_config.d/00-hardening.conf:

PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
MaxAuthTries 3
LoginGraceTime 30
AllowUsers deploy alice
X11Forwarding no
Enter fullscreen mode Exit fullscreen mode

What each line does:

  • PasswordAuthentication no and KbdInteractiveAuthentication no together stop password prompts. With UsePAM yes (the Debian and Ubuntu default), keyboard-interactive auth can still ask for a password through PAM, so both are needed. KbdInteractiveAuthentication is the current name; older guides call it ChallengeResponseAuthentication, which is now a deprecated alias.
  • PermitRootLogin no replaces the default prohibit-password, which still allows root in with a key. Admins log in as themselves and use sudo, so the logs show who did what.
  • PubkeyAuthentication yes is already the default. Stating it keeps the file self-explanatory.
  • MaxAuthTries 3 limits authentication attempts per connection (default 6). Every key the client offers counts as one attempt.
  • LoginGraceTime 30 disconnects clients that have not authenticated within 30 seconds (default 120).
  • AllowUsers refuses every account not listed, even with a valid key. On a box with many admins, AllowGroups sshusers is easier to maintain. Pick one: if you set both, a user has to pass both checks. Make sure your own account is on the list.
  • X11Forwarding no overrides the yes in Debian and Ubuntu's stock config. Servers rarely need it.

Validate and reload without locking yourself out

Test the syntax before anything else. No output means the config parsed:

sudo sshd -t
Enter fullscreen mode Exit fullscreen mode

Then ask sshd for the effective configuration, which shows the value that actually won after all files were read:

sudo sshd -T | grep -i passwordauthentication
Enter fullscreen mode Exit fullscreen mode 
passwordauthentication no
Enter fullscreen mode Exit fullscreen mode

Repeat the grep for permitrootlogin, kbdinteractiveauthentication, maxauthtries, and allowusers. If any shows the old value, an earlier file is setting it (see troubleshooting below).

Reload the service. It is called ssh on Ubuntu and Debian and sshd on RHEL and Fedora:

sudo systemctl reload ssh
Enter fullscreen mode Exit fullscreen mode

Now, from a second terminal, log in again. Then confirm passwords are refused by disabling key auth on the client:

ssh -o PubkeyAuthentication=no deploy@203.0.113.10
Enter fullscreen mode Exit fullscreen mode 
deploy@203.0.113.10: Permission denied (publickey).
Enter fullscreen mode Exit fullscreen mode

A login as root should fail the same way. If the new login does not work, go back to the session you kept open, move the file aside with sudo mv /etc/ssh/sshd_config.d/00-hardening.conf /root/, reload, and work out what went wrong before trying again.

Change the port for less log noise

Moving SSH off port 22 cuts the volume of automated login attempts in your logs. It is not real security: a port scan of your address still finds the new port, and key-only auth is what actually protects you. Do it if the noise bothers you, and open the firewall before you move sshd.

sudo ufw allow 2222/tcp
sudo ufw status
Enter fullscreen mode Exit fullscreen mode

Add Port 2222 to 00-hardening.conf and run sudo sshd -t. How you apply it depends on whether SSH is socket-activated, which is the default on Ubuntu 23.04 and later, including 24.04:

systemctl is-enabled ssh.socket
Enter fullscreen mode Exit fullscreen mode

If that prints enabled, systemd owns the listening socket and generates its port from your sshd config, so regenerate and restart the socket:

sudo systemctl daemon-reload
sudo systemctl restart ssh.socket
Enter fullscreen mode Exit fullscreen mode

If it prints anything else, such as disabled on Debian 12, restart the service with sudo systemctl restart ssh. Either way, existing sessions stay up. Confirm the new listener:

sudo ss -tln | grep -E ':(22|2222)\s'
Enter fullscreen mode Exit fullscreen mode

From a second terminal, run ssh -p 2222 deploy@203.0.113.10. To confirm the port is reachable from outside your provider's network, test it with the port checker. Only then remove the old rule with sudo ufw delete allow 22/tcp (or sudo ufw delete allow OpenSSH, depending on how it was added).

On RHEL, Rocky, and Alma, SELinux only lets sshd bind ports labelled ssh_port_t, and the firewall is firewalld. Label and open the port before restarting sshd:

sudo semanage port -a -t ssh_port_t -p tcp 2222
sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --reload
Enter fullscreen mode Exit fullscreen mode

semanage comes from the policycoreutils-python-utils package.

Ban repeated failures with fail2ban

MaxAuthTries limits guesses per connection. fail2ban works across connections: it watches for failed logins and adds a firewall rule that blocks an address after too many. With passwords disabled it mostly saves log space and CPU, but it is cheap to run.

sudo apt install fail2ban python3-systemd
Enter fullscreen mode Exit fullscreen mode

Leave jail.conf alone so package upgrades can update it cleanly, and put your settings in /etc/fail2ban/jail.local:

[DEFAULT]
ignoreip = 127.0.0.1/8 ::1 198.51.100.7

[sshd]
enabled = true
port = 2222
backend = systemd
maxretry = 5
findtime = 10m
bantime = 1h
Enter fullscreen mode Exit fullscreen mode

An address that fails maxretry times within findtime is banned for bantime. Set port to your SSH port (use ssh if you stayed on 22), or bans will block a port nobody is using. Put your office or VPN address in ignoreip so you cannot ban yourself.

backend = systemd makes fail2ban read the journal instead of a log file. Default Debian 12 installs have no /var/log/auth.log because rsyslog is not installed, and without this line the sshd jail fails to start. It works on Ubuntu 24.04 as well. Restart and check the jail:

sudo systemctl restart fail2ban
sudo fail2ban-client status sshd
Enter fullscreen mode Exit fullscreen mode

The output lists currently failed and total banned counts plus the banned IP list. If an address you need gets banned, release it with sudo fail2ban-client set sshd unbanip 198.51.100.23.

Troubleshooting common lockouts

Locked out completely

Use your provider's web console, serial console, or rescue mode. These log in locally without going through sshd or the firewall, which is why the local password matters. Once in, move the drop-in aside, run sudo sshd -t, and reload. If the firewall is the problem, sudo ufw allow 22/tcp gets you back.

Permission denied (publickey)

Check the username first. Cloud images create a distro-specific default user (ubuntu on Ubuntu, for example), and any account left out of AllowUsers is refused. Next, check file permissions. sshd ignores authorized_keys if ~/.ssh or the file can be written by other users, or if the home directory is group- or world-writable. The fix for an account named deploy:

sudo chmod 700 /home/deploy/.ssh
sudo chmod 600 /home/deploy/.ssh/authorized_keys
sudo chown -R deploy:deploy /home/deploy/.ssh
Enter fullscreen mode Exit fullscreen mode

If octal modes are unfamiliar, Linux File Permissions and chmod, Explained covers what these numbers mean. The server log usually names the reason, such as Authentication refused: bad ownership or modes:

sudo journalctl -u ssh -n 30
Enter fullscreen mode Exit fullscreen mode

Use -u sshd on RHEL. On the client side, ssh -v shows which keys were offered.

Too many authentication failures

With MaxAuthTries 3, an agent holding several keys can use up the attempts before it reaches the right one. Offer only the key you mean:

ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519 deploy@203.0.113.10
Enter fullscreen mode Exit fullscreen mode

A setting is ignored

If sshd -T shows the old value, another file sets it first. Find it with sudo grep -rni passwordauthentication /etc/ssh/sshd_config.d/, then remove the line or rename your file so it sorts earlier. On Ubuntu, a port change that does not show up in ss usually means daemon-reload and the ssh.socket restart were skipped. If the old port is still listed after both, run sudo systemctl restart ssh as well.

Checklist before you log out

  • Key login works from a fresh terminal, and password login is refused.
  • Your admin account has a local password for console access.
  • sudo sshd -t is clean and sudo sshd -T shows the values you set.
  • The firewall allows your SSH port and the port checker can reach it.
  • sudo fail2ban-client status sshd shows the jail running.

Next in the series, Linux File Permissions and chmod, Explained covers the modes that decide whether sshd trusts your key files.

Top comments (0)