DEV Community

Mh Asif Kamal
Mh Asif Kamal

Posted on

# Real-World SSH: From Your Laptop to the Linux Kernel 🚀

#linux #networking #security #tutorial

If you work in tech, you use SSH every day. But for a lot of developers, it's just a black box. Let’s pull back the curtain and trace a real-world hands-on sequence on an Ubuntu server to see exactly how the network handoff and Linux PAM (Pluggable Authentication Modules) pipeline work under the hood.

1. The Core Concept: Public vs. Private Keys

Instead of using passwords, secure modern SSH relies on asymmetric cryptography:

  • The Public Key: Think of this like a padlock. You can hand it out to anyone or leave it on a public server. It doesn't matter who sees it.
  • The Private Key: This is the physical key that opens the padlock. It stays safely on your laptop, and you never share it.

2. Hands-On: Solving "Permission Denied" in Real-Time 🛠️

Step 1: The Error

Connecting to a cloud instance without an authorized key fails immediately:

mhasifkamal@Mhs-MacBook-Air work % ssh mhasifkamal@34.131.76.231
mhasifkamal@34.131.76.231: Permission denied (publickey).
Enter fullscreen mode Exit fullscreen mode

Step 2: Generating the Match

Run ssh-keygen. On modern operating systems, it automatically defaults to the highly secure ED25519 algorithm:

mhasifkamal@Mhs-MacBook-Air work % ssh-keygen
Generating public/private ed25519 key pair.
Enter file in which to save the key (/Users/mhasifkamal/.ssh/id_ed25519): ssh_key
Your identification has been saved in ssh_key
Your public key has been saved in ssh_key.pub
Enter fullscreen mode Exit fullscreen mode

Step 3: Verifying the Files

Your system sets strict, private read/write privileges (-rw-------) for the private key, while leaving the .pub lock file readable:

mhasifkamal@Mhs-MacBook-Air work % ls -l
-rw-------  1 mhasifkamal  staff  432 Jun 25 21:42 ssh_key
-rw-r--r--  1 mhasifkamal  staff  115 Jun 25 21:42 ssh_key.pub

mhasifkamal@Mhs-MacBook-Air work % cat ssh_key.pub 
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFHgEWeyk6UwHRs4VknwryjptBQcG/dYufjCJu8oq+Qe...
Enter fullscreen mode Exit fullscreen mode

Step 4: The Payoff

Point directly to your private key file using the -i flag to unlock entry:

mhasifkamal@Mhs-MacBook-Air work % ssh -i ssh_key mhasifkamal@34.131.76.231 
Welcome to Ubuntu 24.04.4 LTS (GNU/Linux 6.17.0-1018-gcp x86_64)
mhasifkamal@instance-20260620-113027:~$
Enter fullscreen mode Exit fullscreen mode

3. Where the Lock Lives 📂

Once you log in, navigate to your home directory's hidden .ssh folder. All authorized locks are stored in a single text file:

mhasifkamal@instance-20260620-113027:~$ cd .ssh/
mhasifkamal@instance-20260620-113027:~/.ssh$ ls
authorized_keys
mhasifkamal@instance-20260620-113027:~/.ssh$ sudo vi authorized_keys
Enter fullscreen mode Exit fullscreen mode

Cloud setup panels and commands like ssh-copy-id simply append your public key string into this authorized_keys file.

4. Toggling Passwords & The Cloud Override Trap 🕸️

Let's create a traditional password user named test-user on the server:

mhasifkamal@instance-20260620-113027:~/.ssh$ sudo adduser test-user
New password: 
passwd: password updated successfully
Enter fullscreen mode Exit fullscreen mode

Trying to log in immediately from your laptop will still fail:

mhasifkamal@Mhs-MacBook-Air work % ssh test-user@34.131.76.231
test-user@34.131.76.231: Permission denied (publickey).
Enter fullscreen mode Exit fullscreen mode

The Fix

To allow temporary password logins, you must modify the incoming SSH daemon settings. Avoid ssh_config (which handles outbound client connections) and focus on the drop-in cloud override files inside /etc/ssh/sshd_config.d/:

mhasifkamal@instance-20260620-113027:/etc/ssh$ ls /etc/ssh/sshd_config.d/
50-cloudimg-settings.conf  60-cloudimg-settings.conf
Enter fullscreen mode Exit fullscreen mode

Open these overrides, change the parameter to yes, and restart the daemon:

sudo vi /etc/ssh/sshd_config.d/*
# Set: PasswordAuthentication yes

sudo systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Now, the server prompts for the password and allows authentication cleanly:

mhasifkamal@Mhs-MacBook-Air work % ssh test-user@34.131.76.231
test-user@34.131.76.231's password: 
Welcome to Ubuntu 24.04.4 LTS!
Enter fullscreen mode Exit fullscreen mode

5. Under the Hood: The Complete Architecture Flow 🔄

Here is exactly how the network handoff transitions directly into Linux's internal PAM (Pluggable Authentication Modules) pipeline:

[ LAPTOP ]                                                [ UBUNTU SERVER ]
    │                                                            │
    ├─── 1. Protocol Handshake (Agree on SSH versions) ─────────>│
    │                                                            │
    <─── 2. Diffie-Hellman Key Exchange (Secure Tunnel Built) ──>│
    │                                                            │
    <─── 3. Asymmetric Key Challenge (Puzzle sent to client) ────┤
    │                                                            │
    ├─── 4. Challenge Solved (Proof verified by sshd) ──────────>│
    │                                                            │
    │                                                   [ Hand-off to PAM Engine ]
    │                                                            │
    │                                                   ┌────────┴────────┐
    │                                                   │  /etc/pam.d/    │
    │                                                   └────────┬────────┘
    │                                                            │
    │                                                   🔒 1. common-auth
    │                                                      (Checks credentials via pam_unix.so)
    │                                                            │
    │                                                   📋 2. common-account
    │                                                      (Checks password expiration/locks)
    │                                                            │
    │                                                   🛠️ 3. common-session
    │                                                      (Sets env, umask, & log limits)
    │                                                            │
    <─── 5. Shell Opened (PTY Allocated to user) ────────────────┘
Enter fullscreen mode Exit fullscreen mode

The PAM Blueprint

Modern Linux environments manage application access dynamically via files in /etc/pam.d/. (The old master file /etc/pam.conf is ignored).

When we look inside the system authentication rules (cat /etc/pam.d/common-auth), we see the literal logical circuit board:

auth    [success=1 default=ignore]      pam_unix.so nullok
auth    requisite                       pam_deny.so
auth    required                        pam_permit.so
Enter fullscreen mode Exit fullscreen mode
  • pam_unix.so: Checks credentials against system hashes. If successful, [success=1] jumps past the next line.
  • pam_deny.so: The trap line. If your credentials fail, PAM stops here, breaks the circuit, and kicks you out with a Permission denied error.
  • pam_permit.so: The green light. Passes a success flag back to the environment, allowing common-session to spin up your environment variables and launch your bash or zsh shell.

Wrap Up 🎯

SSH builds the secure tunnel across the web, but PAM controls your access rights once you cross the threshold. Hardening your server by setting your drop-in configs back to PasswordAuthentication no keeps the automated brute-force bots completely out of your architecture.

What does your local ~/.ssh/config file look like? Let's chat in the comments! 👇

#sre #ssh #linux #devops #security #sysadmin #tutorial

Top comments (0)