DEV Community

Kjetil Furås
Kjetil Furås

Posted on • Originally published at kjetilfuras.com

How I Manage My SSH Keys Across Multiple Servers and GitHub

#security #tutorial #devops #linux

Originally published at kjetilfuras.com

Managing SSH keys across multiple servers and platforms can get messy fast — especially when you have a homelab, VPS, GitHub repos, and more. I used to juggle one or two keys across everything… until I hit name collisions, agent confusion, and “wrong key” errors.

Now I generate one key per service or server, store them with meaningful filenames, and use an organized ~/.ssh/config to control how each is used.

Here’s how I do it — and how you can too.

My SSH Key Directory Structure 

~/.ssh/
├── id_ed25519_homelab
├── id_ed25519_homelab.pub
├── id_ed25519_hetzner
├── id_ed25519_hetzner.pub
├── id_ed25519_github
├── id_ed25519_github.pub
├── config
Enter fullscreen mode Exit fullscreen mode

My Key Setup Process

1. Generate a Separate Key for Each Server 

ssh-keygen -t ed25519 -C "docker-01" -f ~/.ssh/id_ed25519_docker_01
ssh-keygen -t ed25519 -C "pve-1" -f ~/.ssh/id_ed25519_pve_1
ssh-keygen -t ed25519 -C "hetzner" -f ~/.ssh/id_ed25519_hetzner_example
ssh-keygen -t ed25519 -C "github"  -f ~/.ssh/id_ed25519_github_example
Enter fullscreen mode Exit fullscreen mode

Tip: You can skip the passphrase for convenience or add one for security.

2. Copy Your Public Keys to the Remote Hosts 

ssh-copy-id -i ~/.ssh/id_ed25519_docker_01.pub myuser@10.xx.xx.xx
ssh-copy-id -i ~/.ssh/id_ed25519_pve_1.pub root@10.xx.xx.xx
ssh-copy-id -i ~/.ssh/id_ed25519_hetzner.pub root@your-vps-ip
Enter fullscreen mode Exit fullscreen mode

3. Configure Your ~/.ssh/config 

vim ~/.ssh/config
Enter fullscreen mode Exit fullscreen mode

Example:

# Docker server
Host docker-01
  HostName 10.xx.xx.xx
  User myuser
  IdentityFile ~/.ssh/id_ed25519_docker_01

# Proxmox node
Host pve-1
  HostName 10.xx.xx.xx
  User root
  IdentityFile ~/.ssh/id_ed25519_pve_1

# Hetzner VPS
Host hetzner
  HostName 65.21.xx.xx
  User root
  IdentityFile ~/.ssh/id_ed25519_hetzner

# GitHub
Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_github
  IdentitiesOnly yes
Enter fullscreen mode Exit fullscreen mode

IdentitiesOnly yes prevents SSH from trying other keys — helps avoid confusion when multiple keys are loaded.

4. Add SSH Key to GitHub

4.1. Copy the Public Key

  • macOS: 
pbcopy < ~/.ssh/id_ed25519_github.pub
Enter fullscreen mode Exit fullscreen mode
  • Linux: 
xclip -sel clip < ~/.ssh/id_ed25519_github.pub
# or 
wl-copy < ~/.ssh/id_ed25519_github.pub
Enter fullscreen mode Exit fullscreen mode

4.2. Add to GitHub

  • Visit GitHub SSH settings

  • Click “New SSH key”

  • Give it a meaningful name

  • Paste the key

  • Click “Add SSH key”

Test Your Setup

Linux VMs 

ssh docker-01
ssh pve-1
ssh hetzner
Enter fullscreen mode Exit fullscreen mode

GitHub 

ssh -T git@github.com
Enter fullscreen mode Exit fullscreen mode

If everything’s set up correctly, GitHub will reply:

Hi kjetil! You've successfully authenticated, but GitHub does not provide shell access.
Enter fullscreen mode Exit fullscreen mode

Optional: Disable Password Prompt for sudo

If you’re tired of typing your password every time you use sudo, you can edit the sudoers file:

sudo visudo
Enter fullscreen mode Exit fullscreen mode

Look for this line:

# Allow members of group sudo to execute any command
%sudo  ALL=(ALL:ALL) ALL
Enter fullscreen mode Exit fullscreen mode

Change it to:

# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) NOPASSWD: ALL
Enter fullscreen mode Exit fullscreen mode

Just be careful — removing password prompts reduces security, especially on multi-user or production systems.

Final Thoughts

This setup has completely decluttered my SSH workflow — no more guessing which key is being used, or why I can’t connect. Whether you’re running a homelab, cloud servers, or just want GitHub SSH access done right, this pattern is clean, scalable, and easy to maintain.

Found this helpful? I share more real-world fixes and homelab tips here on the blog and on GitHub.

Top comments (0)