SSH Security Best Practices: 7 Ways to Harden Your Linux Server
INTRODUCTION
SSH (Secure Shell) is the gateway to your Linux servers, making it a prime target for attackers. A compromised SSH service can lead to complete server takeover, data breaches, and unauthorized access to your entire infrastructure.
After managing Linux servers on Azure, AWS, and GCP for [X years/months], I've developed a security checklist that significantly reduces SSH attack surface. Here are 7 essential practices I implement on every production server.
Note: These steps apply to Ubuntu, Debian, CentOS, RHEL, and most Linux distributions.
Practice #1: Disable Root Login
The Risk:
Allowing direct root SSH login is one of the most dangerous misconfigurations. Attackers constantly scan for servers with root access enabled.
Why This Matters:
- Attackers know the username (root)
- Only need to guess the password
- Root has unlimited privileges
- No audit trail of individual user actions
How to Disable Root Login:
Step 1: Create a sudo user (if you don't have one)
Add new user
sudo adduser [your-username]
Add to sudo group
sudo usermod -aG sudo [your-username]
Test sudo access
su - [your-username]
sudo whoami # Should output: root
Step 2: Disable root login in SSH config
sudo nano /etc/ssh/sshd_config
Find and change this line:
PermitRootLogin no
Step 3: Restart SSH service
Ubuntu/Debian:
sudo systemctl restart sshd
CentOS/RHEL:
sudo systemctl restart sshd
Step 4: Test from another terminal (DON'T close your current session!)
This should now fail:
ssh root@your-server-ip
Result: Root SSH login blocked. Users must log in with personal accounts and use sudo.
Practice #2: Use SSH Key Authentication (Disable Password Auth)
The Risk:
Password authentication is vulnerable to brute force attacks. Attackers run automated tools trying thousands of password combinations.
Why SSH Keys Are Better:
- 2048-bit (or higher) encryption
- Nearly impossible to brute force
- No password to steal or leak
- Can be easily revoked
How to Set Up SSH Key Authentication:
On your local machine (if you don't have a key):
Generate SSH key pair
ssh-keygen -t rsa -b 4096 -C "your-email@example.com"
Press Enter to save in default location (~/.ssh/id_rsa)
Set a strong passphrase (optional but recommended)
Copy your public key to the server:
Method 1: Using ssh-copy-id (easiest)
ssh-copy-id username@server-ip
Method 2: Manual copy
On local machine:
cat ~/.ssh/id_rsa.pub
Copy the output, then on the server:
mkdir -p ~/.ssh
echo "your-public-key-here" >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
Test key-based login:
Should work without password:
ssh username@server-ip
Disable password authentication:
sudo nano /etc/ssh/sshd_config
# Change these lines:
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
# Restart SSH:
sudo systemctl restart sshd
**IMPORTANT:** Keep a session open until you confirm key-based login works!
**Result:** Only SSH key authentication allowed. Brute force attacks become ineffective.
---
## **Practice #3: Change the Default SSH Port**
**The Risk:**
Automated bots constantly scan port 22 (default SSH port). Changing it reduces noise and automated attacks.
**Note:** This is "security through obscurity" - not a replacement for other measures, but adds an extra layer.
**How to Change SSH Port:**
Step 1: Choose a port number
# Pick a high port number (1024-65535)
# Example: 2222, 2345, 22022
# Avoid common ports: 8080, 3000, etc.
Step 2: Update SSH config
sudo nano /etc/ssh/sshd_config
# Find and change:
Port 2222 # Or your chosen port
Step 3: Update firewall rules
**Ubuntu/Debian (UFW):**
# Allow new port:
sudo ufw allow 2222/tcp
# Remove old rule (after testing!):
sudo ufw delete allow 22/tcp
CentOS/RHEL (firewalld):
Add new port:
sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --reload
Remove old port (after testing!):
sudo firewall-cmd --permanent --remove-service=ssh
sudo firewall-cmd --reload
Step 4: Update cloud security groups (if applicable)
[Fill in specific steps for Azure NSG, AWS Security Groups, or GCP Firewall]
Azure NSG:
- Go to Network Security Group
- Add inbound rule for port 2222
- After testing, remove port 22 rule
AWS Security Group:
- Add inbound rule: TCP, port 2222, your IP
- After testing, remove port 22 rule
Step 5: Restart SSH and test
sudo systemctl restart sshd
From local machine:
ssh -p 2222 username@server-ip
Update your SSH config for convenience:
On local machine (~/.ssh/config):
Host myserver
HostName server-ip
Port 2222
User username
Now you can just type:
ssh myserver
Result: 90%+ reduction in automated attacks on SSH service.
Practice #4: Implement Fail2Ban
The Risk:
Even with strong security, some attacks persist. Fail2Ban automatically blocks IPs after failed login attempts.
What Fail2Ban Does:
- Monitors authentication logs
- Blocks IPs after X failed attempts
- Automatically unblocks after set time
- Works with firewall rules
How to Set Up Fail2Ban:
Step 1: Install Fail2Ban
Ubuntu/Debian:
sudo apt update
sudo apt install fail2ban -y
CentOS/RHEL:
sudo yum install epel-release -y
sudo yum install fail2ban -y
Step 2: Configure Fail2Ban for SSH
Create local config:
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Edit configuration:
sudo nano /etc/fail2ban/jail.local
Add/modify these settings:
[sshd]
enabled = true
port = 2222 # Or your SSH port
filter = sshd
logpath = /var/log/auth.log # Ubuntu/Debian
logpath = /var/log/secure # CentOS/RHEL
maxretry = 3 # Failed attempts before ban
bantime = 3600 # Ban duration in seconds (1 hour)
findtime = 600 # Time window for failed attempts (10 min)
Step 3: Start and enable Fail2Ban
sudo systemctl start fail2ban
sudo systemctl enable fail2ban
Step 4: Check Fail2Ban status
Check service status:
sudo systemctl status fail2ban
Check banned IPs:
sudo fail2ban-client status sshd
Unban an IP (if needed):
sudo fail2ban-client set sshd unbanip IP_ADDRESS
Result: Automated protection against brute force attacks. Failed login attempts = automatic IP ban.
Practice #5: Use Two-Factor Authentication (2FA)
The Risk:
Even with SSH keys, a compromised key can grant full access. 2FA adds another layer.
How 2FA Works:
- Something you have (SSH key)
- Something you know (2FA code)
- Both required for access
How to Set Up Google Authenticator for SSH:
Step 1: Install Google Authenticator
Ubuntu/Debian:
sudo apt install libpam-google-authenticator -y
CentOS/RHEL:
sudo yum install google-authenticator -y
Step 2: Configure Google Authenticator for your user
Run as your regular user (not root):
google-authenticator
Answer the questions:
- Do you want authentication tokens to be time-based? YES
- Scan QR code with Google Authenticator app
- Save emergency scratch codes in safe place
- Do you want to disallow multiple uses? YES
- Increase time window? NO (unless needed)
- Enable rate-limiting? YES
Step 3: Configure SSH to use 2FA
Edit PAM configuration:
sudo nano /etc/pam.d/sshd
Add this line at the top:
auth required pam_google_authenticator.so
Step 4: Update SSH daemon config
sudo nano /etc/ssh/sshd_config
# Add or modify these lines:
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive
Step 5: Restart SSH
sudo systemctl restart sshd
**Testing (from another terminal!):**
ssh username@server-ip
# You'll need:
# 1. Your SSH key
# 2. Verification code from Google Authenticator app
**Result:** Even if SSH key is compromised, attacker needs your phone to log in.
**Optional:** Exempt trusted IPs from 2FA
# Edit /etc/pam.d/sshd:
auth [success=done default=ignore] pam_succeed_if.so user ingroup no2fa
auth required pam_google_authenticator.so
# Create group:
sudo groupadd no2fa
sudo usermod -aG no2fa username
---
## **Practice #6: Restrict SSH Access by IP**
**The Risk:**
If you always connect from known IPs (office, home, VPN), why allow connections from anywhere?
**How to Whitelist IP Addresses:**
**Method 1: Using sshd_config (Simple)**
sudo nano /etc/ssh/sshd_config
# Add at the end:
AllowUsers username@192.168.1.100
AllowUsers username@10.0.0.0/8
AllowUsers username@203.0.113.0/24
# Or allow specific IPs for all users:
# Match Address 192.168.1.0/24,10.0.0.0/8
# AllowUsers *
sudo systemctl restart sshd
**Method 2: Using Firewall (More flexible)**
**UFW (Ubuntu/Debian):**
bash
Remove general SSH allow:
sudo ufw delete allow 2222/tcp
Add specific IPs:
sudo ufw allow from 192.168.1.100 to any port 2222 proto tcp
sudo ufw allow from 203.0.113.0/24 to any port 2222 proto tcp
firewalld (CentOS/RHEL):
Create rich rules:
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.100" port port="2222" protocol="tcp" accept'
sudo firewall-cmd --reload
Method 3: Cloud Security Groups (Best for cloud)
[Fill in your specific cloud platform steps]
Azure:
Network Security Group → Inbound rules
- Source: IP Addresses
- Source IP: Your.IP.Address/32
- Destination port: 2222
- Action: Allow
- Priority: 100
AWS Security Groups:
Edit inbound rules
- Type: Custom TCP
- Port: 2222
- Source: My IP (auto-fills your current IP)
Result: SSH access only from your trusted IPs. All other connections blocked.
Practice #7: Enable and Monitor SSH Logs
The Risk:
Without logging, you won't know if someone is attacking or has gained access to your server.
Why Logging Matters:
- Detect attack patterns
- Forensic analysis after incidents
- Compliance requirements
- Early warning of compromises
How to Set Up Comprehensive SSH Logging:
Step 1: Ensure logging is enabled
sudo nano /etc/ssh/sshd_config
Verify these settings:
SyslogFacility AUTH
LogLevel VERBOSE # Or INFO for less detail
sudo systemctl restart sshd
Step 2: View SSH logs
Ubuntu/Debian:
sudo tail -f /var/log/auth.log
CentOS/RHEL:
sudo tail -f /var/log/secure
Using journalctl:
sudo journalctl -u sshd -f
Step 3: Set up log monitoring script
bash
# Create monitoring script:
sudo nano /usr/local/bin/ssh-monitor.sh
**Add this script:**
#!/bin/bash
# Monitor SSH failed login attempts
LOGFILE="/var/log/auth.log" # Change for CentOS: /var/log/secure
THRESHOLD=5
EMAIL="your-email@example.com"
# Count failed attempts in last hour
FAILED=$(grep "Failed password" $LOGFILE | grep "$(date '+%b %e %H')" | wc -l)
if [ $FAILED -gt $THRESHOLD ]; then
echo "WARNING: $FAILED failed SSH attempts in the last hour" | mail -s "SSH Alert" $EMAIL
fi
Make executable and add to cron:
sudo chmod +x /usr/local/bin/ssh-monitor.sh
# Run every hour:
sudo crontab -e
# Add:
0 * * * * /usr/local/bin/ssh-monitor.sh
Step 4: Useful commands for log analysis
# Show all failed login attempts:
sudo grep "Failed password" /var/log/auth.log
# Show successful logins:
sudo grep "Accepted publickey" /var/log/auth.log
# Count attacks by IP:
sudo grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -nr
# Show last 10 successful SSH logins:
sudo last -10
# Show current SSH connections:
who
# or
w
**Result:** Full visibility into SSH access attempts, successful logins, and attack patterns.
---
## **BONUS: Quick Security Audit Checklist**
Run these commands to verify your SSH security:
# Check SSH config:
sudo sshd -T | grep -E 'permitrootlogin|passwordauthentication|port|challengeresponse'
# Verify SSH is running:
sudo systemctl status sshd
# Check who's currently connected:
who
# View recent SSH activity:
sudo last | head -20
# Check for suspicious failed attempts:
sudo grep "Failed password" /var/log/auth.log | tail -50
# Verify Fail2Ban is active:
sudo fail2ban-client status sshd
---
## **THE RESULTS**
After implementing all 7 practices:
✅ Zero successful brute force attacks (blocked by Fail2Ban and key auth)
✅ 95% reduction in attack attempts (changed port + IP whitelisting)
✅ Complete audit trail of all access (comprehensive logging)
✅ Additional 2FA layer prevents compromised key exploits
✅ Peace of mind that servers are hardened against SSH attacks
**Time investment:** 2-3 hours to set up
**Security improvement:** Significant reduction in attack surface
**Maintenance:** Minimal (just monitor logs periodically)
---
## **KEY TAKEAWAYS**
- Never allow root login via SSH - use sudo instead
- SSH keys are far superior to password authentication
- Changing the default port significantly reduces automated attacks
- Fail2Ban provides automated defense against brute force
- 2FA adds critical protection even if SSH keys are compromised
- IP whitelisting is the most effective restriction if feasible
- Logging is essential - you can't protect what you can't see
Implement these seven practices in order - each builds on the previous one. Even implementing just the first three will dramatically improve your SSH security.
## **CONCLUSION**
SSH is the primary entry point to your Linux servers, making it the most critical service to secure. These seven practices form a defense-in-depth strategy that protects against the vast majority of SSH attacks.
Start with practices #1 and #2 (disable root login and use SSH keys) - these provide the biggest security improvement with minimal effort. Then gradually implement the remaining practices based on your risk tolerance and requirements.
Have you implemented SSH hardening on your servers? What additional practices do you use? Share your experiences in the comments below!
Top comments (0)