DEV Community

Cover image for SSH and Remote Access Configuration in Linux
Yeasin's
Yeasin's

Posted on

SSH and Remote Access Configuration in Linux

#linux #ssh #sysadmin

Almost every server in the world is managed remotely. Nobody walks into a data center, plugs in a monitor, and types commands. Instead, administrators connect over the network using SSH (Secure Shell) — the standard, encrypted way to control one computer from another.

If you are learning Linux system administration, this is one of the first real skills you need. In this guide, we will cover what SSH is, how it works under the hood, how to install and configure the SSH server on Debian and Ubuntu, how to set up passwordless login with keys, and — most importantly — how to lock down your server so it is safe on the public internet.

What Is SSH and Why Does It Matter?

SSH is a protocol that gives you a secure, encrypted terminal on a remote machine. When you connect to a server with SSH, it is as if you are sitting in front of it with a keyboard — except you could be on the other side of the world.

Before SSH, people used tools like telnet, which sent everything — including your password — in plain text. Anyone watching the network traffic could read it. SSH was created to fix exactly this problem.

With SSH, everything is encrypted:

  • Your password and commands
  • The files you transfer
  • The entire session traffic

SSH is used for far more than terminals, though. Once you know it, you will use it for:

  • Administering remote servers (your daily driver)
  • Securely copying files (scp, rsync, sftp)
  • Tunneling other traffic through an encrypted channel (port forwarding)
  • Git operations with remote repositories
  • Automated deployments from CI/CD pipelines

How SSH Works: The Basic Flow

Before the commands, it helps to understand the mechanics. An SSH connection always has two sides:

  • SSH client — the machine you are sitting at (your laptop, or even Termux on a phone)
  • SSH server (sshd) — the remote machine you want to log into

The flow looks like this:

  1. You run ssh user@server-ip on the client.
  2. The client and server perform a key exchange and agree on encryption keys for this session.
  3. The server proves its identity using its host key (stored in /etc/ssh/ on the server).
  4. The client verifies your identity — with a password or, better, a public/private key pair.
  5. An encrypted session opens, and you get a shell on the remote machine.

Note: The first time you connect to any server, SSH asks you to confirm the server's fingerprint. It then saves it to ~/.ssh/known_hosts. From then on, if that fingerprint ever changes, SSH refuses to connect and warns you loudly — that is protection against man-in-the-middle attacks, not a bug.

Diagram of SSH connection flow: client and server connected through an encrypted tunnel

Step 1: Installing the SSH Server

On a fresh Debian or Ubuntu server, the SSH server is usually already installed. Check its status:

sudo systemctl status ssh
Enter fullscreen mode Exit fullscreen mode

If it is not installed:

sudo apt update
sudo apt install openssh-server -y
Enter fullscreen mode Exit fullscreen mode

Then confirm it is running and enabled at boot:

sudo systemctl enable --now ssh
sudo systemctl status ssh
Enter fullscreen mode Exit fullscreen mode 
● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled)
     Active: active (running) since Sat 2026-10-17 09:00:00 UTC
Enter fullscreen mode Exit fullscreen mode

The SSH server listens on port 22 by default. Verify it:

sudo ss -tlnp | grep sshd
Enter fullscreen mode Exit fullscreen mode 
LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=812,fd=3))
Enter fullscreen mode Exit fullscreen mode

Note on Ubuntu: Since Ubuntu 22.10, the service may be named ssh.socket instead of ssh.service (socket activation). The commands above work either way, but if you ever wonder why sshd is not listed as a running service, that is why.

Step 2: Connecting for the First Time

From your client machine:

ssh yeasin@203.0.113.10
Enter fullscreen mode Exit fullscreen mode 
The authenticity of host '203.0.113.10' can't be established.
ED25519 key fingerprint is SHA256\:xxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
yeasin@203.0.113.10's password:
Enter fullscreen mode Exit fullscreen mode

Enter the password, and you are in. If you can ping the server but the connection is refused, the most common cause is a firewall — which brings us to an important step:

sudo ufw allow ssh
Enter fullscreen mode Exit fullscreen mode

On Debian, ufw may not be installed; install it with sudo apt install ufw. Without this rule, remote SSH traffic never reaches the server.

Step 3: Key-Based Authentication (Do This First)

Passwords can be guessed, reused, or leaked. SSH keys solve this. A key pair consists of:

  • A private key (~/.ssh/id_ed25519) — never leaves your machine. Never share it, never copy it to a server.
  • A public key (~/.ssh/id_ed25519.pub) — safe to share; you place it on every server you want to access.

The server can verify you own the private key without you ever sending it. That is the whole trick: the private key proves your identity without ever traveling over the network.

Generate a key pair

On your client machine:

ssh-keygen -t ed25519 -C "yeasin@laptop"
Enter fullscreen mode Exit fullscreen mode 
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/yeasin/.ssh/id_ed25519):
Enter passphrase (empty for no passphrase):
Your identification has been saved in /home/yeasin/.ssh/id_ed25519
Your public key has been saved in /home/yeasin/.ssh/id_ed25519.pub
Enter fullscreen mode Exit fullscreen mode
  • -t ed25519 — the modern, fast, and secure key type. Prefer it over RSA.
  • The passphrase protects your private key if your laptop is stolen. Recommended.

Copy the public key to the server 

ssh-copy-id yeasin@203.0.113.10
Enter fullscreen mode Exit fullscreen mode

That is it. Log in again and you will not be asked for a password:

ssh yeasin@203.0.113.10
Enter fullscreen mode Exit fullscreen mode

Tip: If ssh-copy-id is unavailable (for example on Termux), do it manually by appending your public key to the server's ~/.ssh/authorized_keys file. Make sure the .ssh directory has permission 700 and the file has 600 — SSH ignores keys if the permissions are too open.

Step 4: Understanding and Configuring sshd

All server-side configuration lives in one file:

/etc/ssh/sshd_config
Enter fullscreen mode Exit fullscreen mode

Before changing anything, create a backup:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
Enter fullscreen mode Exit fullscreen mode

After every change, test your configuration before restarting:

sudo sshd -t
Enter fullscreen mode Exit fullscreen mode

No output means everything is fine. Then apply the changes:

sudo systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Warning: Never close your current SSH session while testing new SSH settings. Keep it open, open a second terminal, and confirm you can still log in. If you locked yourself out, your open session is your rescue line.

The most important settings 

sudo nano /etc/ssh/sshd_config
Enter fullscreen mode Exit fullscreen mode
Setting Safe value What it does
Port 22 e.g. Port 2222 Changes the port scanners expect
PermitRootLogin no no Blocks direct root login
PasswordAuthentication no no Disables password login (keys only)
PubkeyAuthentication yes yes Enables key-based login
AllowUsers yeasin deploy your users Only listed users may connect at all
MaxAuthTries 3 3 Limits login attempts per connection 
Port 2222
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
AllowUsers yeasin
MaxAuthTries 3
Enter fullscreen mode Exit fullscreen mode

Each of these lines removes one attack path:

  • PermitRootLogin no — attackers always try root first. Even with a good password, you do not want them hammering the most powerful account.
  • PasswordAuthentication no — once your keys work, this is the single biggest security upgrade. Nothing can be brute-forced if passwords are not accepted.
  • AllowUsers — even a valid password for an unused account (like guest) is now useless.

Note: On modern Ubuntu (22.04+), password authentication may be controlled by a drop-in file at /etc/ssh/sshd_config.d/*.conf, which overrides the main config. Check with grep -ri PasswordAuthentication /etc/ssh/ to make sure your change actually takes effect.

Changing the default port

Changing the port (e.g., Port 2222) is security through obscurity — it reduces automated bot noise dramatically, but a targeted attacker will scan all ports anyway. Treat it as noise reduction, not real security:

sudo ufw allow 2222/tcp
Enter fullscreen mode Exit fullscreen mode 
ssh -p 2222 yeasin@203.0.113.10
Enter fullscreen mode Exit fullscreen mode

Order matters: open the firewall for the new port before restarting sshd, or you will lock yourself out.

Step 5: The Client-Side Config File

Tired of typing ssh -p 2222 yeasin@203.0.113.10? SSH has a built-in solution. Create ~/.ssh/config on your client:

nano ~/.ssh/config
Enter fullscreen mode Exit fullscreen mode 
Host myserver
    HostName 203.0.113.10
    User yeasin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
Enter fullscreen mode Exit fullscreen mode

Now connecting is simply:

ssh myserver
Enter fullscreen mode Exit fullscreen mode

This also makes scp, rsync, and sftp use the same settings automatically. If you manage more than one server, this file will save you hours.

Step 6: Extra Hardening for Public Servers

If your server has a public IP, bots are already knocking on port 22 — check for yourself:

sudo journalctl -u ssh | grep -i failed | tail
Enter fullscreen mode Exit fullscreen mode

You will likely see login attempts for root, admin, and test from IP addresses all over the world. Two more layers stop this completely.

1. Fail2ban — automatic attacker blocking

Fail2ban watches the logs and bans IPs after repeated failed attempts:

sudo apt install fail2ban -y
sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd
Enter fullscreen mode Exit fullscreen mode 
Status
|- Number of matches:  14
|- Currently banned:   2
`- Total banned:        9
Enter fullscreen mode Exit fullscreen mode

On Debian and Ubuntu, fail2ban ships with a ready-made sshd jail; default is 5 failed attempts leading to a 10-minute ban. Adjust it in /etc/fail2ban/jail.local.

2. Firewall — allow only what you need 

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 2222/tcp
sudo ufw enable
sudo ufw status verbose
Enter fullscreen mode Exit fullscreen mode

Warning: Before running ufw enable on a remote server, make sure your SSH port is allowed. Locking yourself out of a remote machine is a rite of passage — but not one you need to experience.

Common Mistakes and Troubleshooting

1. "Connection refused"
The server is reachable, but nothing is listening on that port. Check: is ssh running (sudo systemctl status ssh)? Is the firewall allowing the port? Are you connecting to the right port?

2. "Connection timed out"
Packets are not reaching the server. Check the IP address, cloud provider security groups (AWS/GCP/DigitalOcean firewalls are separate from ufw), and any network firewall in between.

3. Key login silently falls back to password
Usually permissions. On the server:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
Enter fullscreen mode Exit fullscreen mode

And check the logs — they tell you exactly why a login failed:

sudo journalctl -u ssh -n 20
Enter fullscreen mode Exit fullscreen mode

4. Locked yourself out after changing the port
This is why you never close the working session during changes. If it does happen: on a VPS, use the provider's web console to fix sshd_config; on a local machine, plug in a keyboard and monitor.

5. "Too many authentication failures"
Your client may be offering many keys. Fix it in ~/.ssh/config with IdentitiesOnly yes on that host, so only the right key is offered.

Summary

  • SSH is the encrypted standard for remote administration — everything you type and transfer is protected end to end.
  • Install the server with openssh-server, and immediately allow it through ufw.
  • Replace password login with ed25519 key pairs as soon as possible — it is both more secure and more convenient.
  • Harden the server in /etc/ssh/sshd_config: PermitRootLogin no, PasswordAuthentication no, and AllowUsers for your accounts.
  • Always test with sshd -t and keep your current session open while applying changes.
  • Use ~/.ssh/config on the client to turn long commands into ssh myserver.
  • Add fail2ban on public servers; bots are already trying your door.

Remote access is the gateway skill: once SSH feels natural, managing servers, deploying applications, and automating infrastructure all start from that same ssh command.

Top comments (0)