SSH hardening checklist for production servers

My Production SSH Configuration Was a Mess. Here's How I Fixed It.

SSH used to be a simple tool for me. Set it up, maybe change the port, and that was it. It worked fine until I inherited a fleet of production servers. Suddenly, "just SSH" felt like a security vulnerability. I’d spend hours poring over logs, seeing failed login attempts from every corner of the globe, and realizing my SSH setup was more of an open invitation than a secure gateway.

This isn't a guide to the most extreme, lock-it-down-to-the-last-byte SSH hardening. Instead, these are the practical steps I took on real production machines to shrink our attack surface and, frankly, sleep better at night. It's the kind of advice I wish I'd had when I started.

The Password Problem: A Welcome Mat for Attackers

The first thing that became glaringly obvious was the sheer volume of brute-force password attempts hitting every single server. My initial, naive thought was, "Just change the password!" That's akin to putting a tiny bandage on a gaping wound. The fundamental issue is allowing password authentication at all.

Therefore, the absolute first step is to disable password authentication. This might sound alarming. "But what if I lose my key?" you might worry. We'll cover key management. For now, trust me on this: if you're relying on passwords for SSH access, your door is wide open.

Here's the crucial line in /etc/ssh/sshd_config to modify:

PasswordAuthentication no

Crucially, ensure you have a working method to log in before making this change. I always do this during a planned maintenance window or when I have direct console access. It’s also wise to test your SSH key login from a separate terminal session before restarting the SSH service.

Key-Based Authentication: The Only Secure Path

If you're disabling password authentication, you must have robust key-based authentication in place. This means relying solely on SSH keys. I won't delve into the intricacies of ssh-keygen here, as it's well-documented elsewhere. The critical aspect for production environments is effective key management.

1. Generate Strong Keys: Avoid using default settings when generating keys. I strongly prefer Ed25519 keys. They are modern, performant, and generally considered more secure than older RSA keys.

ssh-keygen -t ed25519 -C "[e-mail]"

The -C flag is optional but incredibly useful for identifying keys later on.

2. Distribute Public Keys Securely: Your public key needs to reside on the servers, typically in ~/.ssh/authorized_keys. Your private key remains on your local machine (or your secure connection point). I've seen teams simply cat their public key into the authorized_keys file. This is a prime candidate for disaster, especially if you accidentally paste incorrect information or mishandle file permissions.

A much safer and more reliable method is to use ssh-copy-id. It correctly sets permissions and significantly reduces the risk of errors.

ssh-copy-id -i ~/.ssh/my_ed25519.pub user@your_server_ip

3. Restrict Key Usage: This is where you gain significant control. You can add specific options to the authorized_keys file to limit what a particular public key can do. This is invaluable for service accounts or shared access scenarios.

For instance, if a key is only intended to execute a specific script, you can enforce this:

command="/path/to/your/script.sh",no-port-forwarding,no-pty,no-user-rc,no-agent-forwarding ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI… [e-mail]

This configuration means that any login using that specific key will only execute /path/to/your/script.sh. It grants no shell access, no port forwarding, and no other command execution. This is a massive security improvement. I learned the hard way how a compromised key could grant far more access than intended.

Limiting SSH Access: Granular Control

Not everyone needs root privileges. Not every user needs to SSH into every server.

1. Use Dedicated User Accounts: Avoid logging in directly as root. Instead, create specific user accounts for yourself and your team members. Then, leverage sudo for elevated privileges. This creates a clear audit trail and minimizes the potential damage if an account is compromised.

2. Implement Group-Based Access: Define SSH access based on user groups. For example, a "developers" group might have access to staging environments but not production. An "operations" group could have access to a broader range of systems.

On the server, you can configure sshd to permit logins only from specified groups:

AllowGroups ops developers

This setting goes into sshd_config. Remember to restart the SSH service after making this change.

3. Explicitly Disable Root Login: Even if you're diligently using sudo, it's a best practice to explicitly disallow direct root SSH logins. This is another important line in sshd_config:

PermitRootLogin no

Port, Protocol, and Other Essential Tweaks

Beyond authentication and user management, several smaller, yet impactful, configuration adjustments can enhance security.

1. Change the Default Port: While often labeled "security by obscurity," changing the default SSH port (22) significantly reduces the noise from automated scanning bots. If your SSH server isn't on port 22, it's less likely to be found by opportunistic scans. Choose a high, unused port and, critically, document it for your team.

Port 2222 # Select a port not in common use

You'll then need to specify this port when connecting:

ssh -p 2222 user@your_server_ip

Crucially, ensure your firewall rules are updated to allow traffic on this new port.

2. Protocol Version: SSH has two major versions, 1 and 2. SSHv1 is outdated and insecure. Always ensure you are only allowing SSHv2.

Protocol 2

This is typically the default, but it's always worth verifying.

3. Disable X11 Forwarding (If Unnecessary): Unless you specifically need to run graphical applications over SSH, disable X11 forwarding. It presents an unnecessary potential attack vector.

X11Forwarding no

4. Limit Login Attempts: This is another effective measure against brute-force attacks, even after disabling password authentication. You can limit the number of login attempts permitted per user or IP address.

MaxAuthTries 3
MaxSessions 2

MaxAuthTries restricts authentication attempts per connection, while MaxSessions limits concurrent open sessions for a single connection. These settings help prevent resource exhaustion.

5. Implement Idle Timeout: I've encountered lingering sessions that remained open for days because someone forgot to log out. While a minor risk, it's easily mitigated.

ClientAliveInterval 300  # Send a keepalive signal every 300 seconds (5 minutes)
ClientAliveCountMax 3    # Disconnect after 3 missed keepalives (effectively 15 minutes of inactivity)

This configuration forces a disconnection if the client becomes unresponsive for a defined period.

Logging and Monitoring: Seeing What's Happening

Robust configuration is only effective if you have visibility into system activity.

1. Verbose Logging: Ensure your SSH server is configured to log sufficient detail. The default LogLevel is often INFO. For enhanced security monitoring, I recommend increasing this.

LogLevel VERBOSE

This provides more granular insights into connection attempts, authentication methods, and user actions. It's essential to also check your system's syslog configuration (rsyslog or syslog-ng) to ensure these logs are stored and rotated appropriately.

2. Fail2Ban: This is an indispensable tool. Fail2Ban actively scans log files (including your SSH logs) and automatically bans IP addresses exhibiting malicious behavior – excessive failed login attempts, probing for vulnerabilities, etc. It's remarkably effective at thwarting automated attacks.

Installation is typically straightforward:

sudo apt update && sudo apt install fail2ban # For Debian/Ubuntu
sudo yum install epel-release && sudo yum install fail2ban # For CentOS/RHEL

You will need to configure Fail2Ban to monitor your SSH logs and define your banning parameters. The default configuration often provides a solid starting point.

Ongoing Challenges

Even with these improvements, managing SSH keys at scale remains a persistent challenge. When an employee departs, the task of hunting down and removing their public keys from every server can be daunting. Tools like Ansible or Chef are invaluable for automating key distribution and revocation, which is the direction we're moving. However, adopting these tools introduces more infrastructure to manage, and sometimes the perceived speed of manual processes leads to complications down the line.

Furthermore, the documentation for some sshd_config options can be less than clear, requiring you to piece together information from various man pages and online forums. It’s a time-consuming process.

The aim here isn't to make SSH unusable, but to significantly raise the bar for potential attackers. By disabling password authentication, enforcing restricted key-based logins, and implementing granular access controls, you demonstrably strengthen your security posture. This investment of a few hours can save you days of potential headaches.