How to Log In to a Remote Linux Machine Using SSH Keys

Photo by Jordan Harrison on Unsplash

SSH (Secure Shell) is a common way to securely connect to remote Linux machines. Instead of using passwords, SSH keys make logging in safer and easier. This guide will show you how to install an SSH server, create SSH keys, add your public key to the remote server and then connect using your SSH keys.

1. Installing the SSH Server

To enable SSH access, the remote Linux machine must have an SSH server installed.

For Debian-based Systems (e.g., Ubuntu)

sudo apt update
sudo apt install openssh-server -y
sudo systemctl enable ssh
sudo systemctl start ssh

For Red Hat-based Systems (e.g., CentOS, Rocky Linux)

sudo yum update -y
sudo yum install -y openssh-server
sudo systemctl enable sshd
sudo systemctl start sshd

Verify SSH Server is Running

sudo systemctl status ssh # For Ubuntu/Debian
sudo systemctl status sshd # For CentOS/RHEL

If the server is running you should see active (running) as shown below.

2. Generating SSH Keys on your Local Machine

SSH keys consist of a private key (kept secure on your local machine) and a public key (placed on the remote server). If you haven’t generated SSH keys before, check out this detailed guide:

🔗 Generating SSH Keys on Windows, Linux, and macOS

Once you’ve generated your SSH keys, proceed to the next step to configure the server for SSH key-based authentication by adding your public key to the remote server.

3. Adding your Public Key to the Remote Machine

Now, we need to copy the public key to the remote server so it can allow your key to be used to log in.

Copy your Public Key from the Local Machine

On your local machine, display the public key by running cat ~/.ssh/id_rsa.pub or opening the .pub file in Notepad, and copy the contents.

Prepare the Remote Server

On the remote machine, create the .ssh directory and authorized_keys file if they do not already exist.

mkdir -p ~/.ssh
touch ~/.ssh/authorized_keys

Add the Public Key to Authorized Keys

Paste the copied key into the ~/.ssh/authorized_keys file on the remote machine. If there's already another key in the file, make sure to add your key on a new line.

Alternatively, you can use echo command on the remote machine and append it directly:

echo "PASTE_PUBLIC_KEY_HERE" >> ~/.ssh/authorized_keys

Set the correct permissions to ensure security:

Some systems will now allow you to use your SSH keys if the permissions are too open or incorrect. Run the following commands to set the recommended permissions.

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

4. Connecting to the Remote Machine Using SSH Keys

Now that the remote machine is ready for SSH-based authentication and your public key has successfully been added, you can connect from your local machine:

ssh username@remote_host

If your private key exists in a non-default location, then provide the path to the private key by using the following command:

ssh -i /path/to/private_key username@remote_host

Conclusion

You have now set up your remote server for SSH access and learned how to log in securely using SSH keys, eliminating the need to enter a password each time. This method enhances security and convenience, making remote access more efficient. With SSH keys in place, you can now connect to your server seamlessly and focus on what matters most.