DEV Community

Cover image for 🔐 SSH Essentials Every DevOps Engineer Should Know
Md Mohiuddin
Md Mohiuddin

Posted on

🔐 SSH Essentials Every DevOps Engineer Should Know

#devops #linux #ssh #beginers

If you're learning DevOps, Cloud Computing, or System Administration, SSH is one of the most important skills you'll ever learn.

Almost every cloud server, Kubernetes node, CI/CD runner, and Linux machine is managed remotely through SSH. Without SSH, you simply can't access or manage production infrastructure effectively.

As part of my DevOps learning journey, I spent time understanding how SSH works, how key-based authentication improves security, and how to securely connect to remote servers.

In this article, I'll cover:

✅ What SSH is
✅ How SSH works
✅ Public vs Private Keys
✅ Generating SSH keys
✅ Connecting to remote servers
✅ Managing SSH configurations
✅ Secure file transfers with SCP

Let's get started.

What Is SSH?

SSH (Secure Shell) is a secure network protocol that allows you to remotely access and manage another computer over a network.

Using SSH, you can:

  • Access a remote Linux server
  • Execute commands remotely
  • Manage cloud infrastructure
  • Transfer files securely
  • Automate deployments

Before SSH existed, tools like Telnet sent everything in plain text, including passwords.

SSH solved this problem by encrypting all communication between your computer and the remote server.

Why SSH Is Important for DevOps

Every modern DevOps workflow relies on SSH.

Some common examples include:

  • Connecting to AWS EC2 instances
  • Managing Linux servers
  • Accessing Kubernetes nodes
  • Deploying applications
  • Running automation scripts
  • Configuring CI/CD pipelines

In real-world DevOps work, you'll frequently need to answer questions like:

  • Is the server running properly?
  • Why is my application failing?
  • How do I deploy new code?
  • How can I transfer files securely?

SSH is usually the first tool you'll use to solve these problems.

How SSH Works

SSH works using encryption and authentication.

A typical SSH connection happens in two stages.

Step 1: Server Verification

When connecting to a server for the first time, SSH verifies the server's identity.

You may see a message like:

The authenticity of host '203.0.113.5' can't be established.
ED25519 key fingerprint is SHA256:xxxxxx.
Are you sure you want to continue connecting (yes/no)?
Enter fullscreen mode Exit fullscreen mode

If you trust the server, type:

yes
Enter fullscreen mode Exit fullscreen mode

SSH then stores the server fingerprint locally in:

~/.ssh/known_hosts
Enter fullscreen mode Exit fullscreen mode

Future connections are checked against this fingerprint automatically.

Step 2: Authentication

After the encrypted channel is established, SSH verifies who you are.

There are two common methods.

Password Authentication 

Client → Password → Server
Enter fullscreen mode Exit fullscreen mode

The password travels through the encrypted connection.

While secure, passwords can still be:

  • Guessed
  • Reused
  • Phished
  • Brute-forced

Key-Based Authentication

This is the industry standard.

You create:

  • A private key
  • A public key

The public key is stored on the server.

The private key remains on your computer.

When connecting, SSH verifies that you own the matching private key without ever sending it across the network.

Benefits include:

  • Stronger security
  • No password typing
  • Resistant to brute-force attacks
  • Easy automation for CI/CD pipelines

Generating an SSH Key Pair

Create a new SSH key using:

ssh-keygen -t ed25519 -C "your_email@example.com"
Enter fullscreen mode Exit fullscreen mode

Understanding the Options

Option Purpose
-t ed25519 Modern and secure key type
-C Comment to identify the key

You'll be prompted for:

  • File location
  • Passphrase

Press Enter to accept the default location.

Generated Files

View your SSH directory:

ls -la ~/.ssh
Enter fullscreen mode Exit fullscreen mode

Example output:

id_ed25519
id_ed25519.pub
Enter fullscreen mode Exit fullscreen mode
File Purpose
id_ed25519 Private key (secret)
id_ed25519.pub Public key (safe to share)

Important Security Rule

Never share:

id_ed25519
Enter fullscreen mode Exit fullscreen mode

Never commit it to GitHub.

Never send it through email or chat.

Only share:

id_ed25519.pub
Enter fullscreen mode Exit fullscreen mode

The .pub file is designed to be distributed.

Understanding Public and Private Keys

Think of it like this:

Public Key  → Lock
Private Key → Key
Enter fullscreen mode Exit fullscreen mode

You can give your lock to anyone.

Only the person holding the matching key can unlock it.

This is essentially how SSH authentication works.

Adding Your Public Key to a Server

SSH servers store authorized public keys inside:

~/.ssh/authorized_keys
Enter fullscreen mode Exit fullscreen mode

Any public key listed there can authenticate as that user.

Method 1: Cloud Provider

When launching an AWS EC2 instance, AWS can automatically place your public key on the server.

This is the most common method when learning cloud infrastructure.

Method 2: ssh-copy-id

If you already have password access:

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@server_ip
Enter fullscreen mode Exit fullscreen mode

This automatically installs your public key.

Method 3: Manual Method 

cat ~/.ssh/id_ed25519.pub | ssh username@server_ip "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
Enter fullscreen mode Exit fullscreen mode

This manually appends your public key to the server's authorized keys file.

Connecting to a Remote Server

Basic SSH connection:

ssh username@server_ip
Enter fullscreen mode Exit fullscreen mode

Example:

ssh ubuntu@203.0.113.5
Enter fullscreen mode Exit fullscreen mode

Once connected, you'll receive a remote terminal session.

Using a Specific Key

If your key is stored in a different location:

ssh -i ~/.ssh/my-aws-key.pem ubuntu@203.0.113.5
Enter fullscreen mode Exit fullscreen mode

This tells SSH exactly which key to use.

Common SSH Permission Error

You may encounter:

Permissions 0644 for 'my-aws-key.pem' are too open.
Enter fullscreen mode Exit fullscreen mode

SSH refuses to use keys that other users can read.

Fix it:

chmod 400 my-aws-key.pem
Enter fullscreen mode Exit fullscreen mode

This makes the file readable only by you.

Simplifying Connections with SSH Config

Instead of remembering long commands, create:

~/.ssh/config
Enter fullscreen mode Exit fullscreen mode

Example:

Host my-ec2-server
    HostName 203.0.113.5
    User ubuntu
    IdentityFile ~/.ssh/my-aws-key.pem
Enter fullscreen mode Exit fullscreen mode

Now connect using:

ssh my-ec2-server
Enter fullscreen mode Exit fullscreen mode

This becomes incredibly useful when managing multiple servers.

Transferring Files with SCP

SCP (Secure Copy) uses SSH to transfer files securely.

Copy a Local File to a Server 

scp -i ~/.ssh/my-key.pem local-file.txt ubuntu@203.0.113.5:/home/ubuntu/
Enter fullscreen mode Exit fullscreen mode

Copy a Remote File to Your Computer 

scp -i ~/.ssh/my-key.pem ubuntu@203.0.113.5:/home/ubuntu/file.txt ./
Enter fullscreen mode Exit fullscreen mode

Copy an Entire Directory 

scp -r -i ~/.ssh/my-key.pem ./project/ ubuntu@203.0.113.5:/home/ubuntu/
Enter fullscreen mode Exit fullscreen mode

The -r flag means recursive.

SCP Syntax Made Simple

Think of SCP exactly like the Linux cp command:

scp source destination
Enter fullscreen mode Exit fullscreen mode

The only difference is that remote locations use:

user@host:/path
Enter fullscreen mode Exit fullscreen mode

instead of a normal local path.

Quick Command Reference

Command Purpose
ssh-keygen Generate SSH keys
ssh Connect to a remote server
ssh-copy-id Install public key on a server
chmod 400 key.pem Secure private key permissions
scp Transfer files securely
scp -r Transfer directories
cat ~/.ssh/known_hosts View trusted hosts
cat ~/.ssh/authorized_keys View authorized keys
ssh -i key.pem Connect using a specific key

Final Thoughts

SSH is one of the foundational skills every DevOps engineer must master.

Whether you're managing cloud servers, deploying applications, configuring CI/CD pipelines, or troubleshooting production systems, SSH is the secure gateway that gives you access.

Learning SSH early will make your journey into AWS, Docker, Kubernetes, Terraform, and automation much easier.

I'm currently documenting my DevOps learning journey and sharing beginner-friendly notes as I learn.

If you have any SSH tips or best practices, feel free to share them in the comments.

Happy Learning! 🚀🐧

Top comments (0)