How to SSH Using Private Key in Linux

Secure Shell (SSH) protects remote terminal sessions, file transfers, port forwarding, and remote commands. With public-key authentication, the server checks your private key against a public key stored in your account instead of asking for the account password on every connection.

The private key stays on your client and must never be shared. The public key is the part you copy to a server.

Step 1: Install the OpenSSH client

Most desktop Linux installations already include the client. Check before installing anything:

ssh -V
ssh-keygen -h 2>&1 | head -n 2

If it is missing, install the client package for your distribution:

# Debian or Ubuntu
sudo apt update
sudo apt install openssh-client

# Arch or Manjaro
sudo pacman -S openssh

# Fedora, RHEL, Rocky, or AlmaLinux
sudo dnf install openssh-clients

The remote machine also needs an SSH server, commonly from the openssh-server package. Installing a client on your laptop does not enable inbound SSH on that laptop.

Step 2: Generate a key pair

For a new key on a current OpenSSH installation, Ed25519 is a compact and widely supported choice:

ssh-keygen -t ed25519 -C "your-name@your-computer"

The comment helps you recognize the key later; it is not an authentication secret. If an older system does not support Ed25519, RSA with 4096 bits remains a common compatibility option:

ssh-keygen -t rsa -b 4096

ssh-keygen asks where to save the key. Press Enter for the default, such as ~/.ssh/id_ed25519, or enter a new path for a separate key. It does not silently overwrite an existing key: if the destination exists, it asks for confirmation. Cancel and choose another filename unless you intentionally want to replace that key.

Use a strong passphrase. Without one, anyone who obtains the private-key file can immediately use it. A passphrase adds protection and can be combined with an SSH agent so you do not type it for every connection.

Generation creates two files:

• id_ed25519 — the private key; keep it confidential.
• id_ed25519.pub — the public key; safe to install on servers you control.

Step 3: Install the public key on the server

The simplest method is ssh-copy-id, which normally uses the existing password login once:

ssh-copy-id -i ~/.ssh/id_ed25519.pub remote_user@remote_ip

For a nondefault SSH port:

ssh-copy-id -i ~/.ssh/id_ed25519.pub -p 2222 remote_user@remote_ip

The command appends the public key to ~/.ssh/authorized_keys in the remote user's account. Run it once per key and server; repeatedly copying the same key can create duplicate entries.

If your provider gives you a web console instead of password access, paste the contents of the .pub file into a new line in that user's ~/.ssh/authorized_keys. Never paste the private key into a website, ticket, chat message, or server.

Set permissions on the server if you created the files manually:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

On the client, keep the private key readable only by its owner:

chmod 600 ~/.ssh/id_ed25519

Step 4: Connect with the private key

SSH tries the standard default keys automatically:

ssh remote_user@remote_ip

For a key with a custom name or location, select it explicitly:

ssh -i ~/.ssh/clinerds_ed25519 remote_user@remote_ip

You may be asked for the key passphrase, which is different from the remote account password. To avoid repeated passphrase prompts during a session, use an agent:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/clinerds_ed25519

Many desktop environments already run an agent, so check ssh-add -l before starting another one.

Create a reusable SSH alias

Instead of remembering the user, address, port, and key, add a host block to ~/.ssh/config on the client:

Host myserver
    HostName 192.0.2.25
    User remote_user
    Port 22
    IdentityFile ~/.ssh/clinerds_ed25519
    IdentitiesOnly yes

Protect the file with chmod 600 ~/.ssh/config, then connect with:

ssh myserver

This alias is local configuration, not something you need to copy to the server.

Troubleshooting key authentication

Start with the verbose client output:

ssh -vvv -i ~/.ssh/clinerds_ed25519 remote_user@remote_ip

Then check the usual causes:

• Permission denied (publickey): confirm the username, selected key, and that the matching public line exists on the server.
• Bad permissions or ignored key: check ownership and permissions on the remote home directory, .ssh, and authorized_keys. OpenSSH can reject files writable by other users.
• The server offers no public-key authentication: inspect the active sshd configuration and logs. Use journalctl -u ssh on many Debian/Ubuntu systems or journalctl -u sshd on RHEL-family systems.
• A passphrase prompt appears: that is expected for an encrypted key and does not mean public-key authentication failed.

Avoid “fixing” authentication with chmod 777; broad permissions are both insecure and often rejected by SSH.

Should you disable password login?

Key-only login can reduce password-guessing attacks, but test it before changing the server. Open one successful key-based session and keep it connected. From a second terminal, confirm that you can still log in and use sudo.

Only then set PasswordAuthentication no in a drop-in such as /etc/ssh/sshd_config.d/99-key-only.conf, validate the configuration, and reload the service:

sudo sshd -t
sudo systemctl reload ssh     # Debian/Ubuntu
# or: sudo systemctl reload sshd  # RHEL-family

Keep console or provider recovery access available. Never disable passwords during your first remote experiment without a verified second way in.