DEV Community

Rasika Dangamuwa
Rasika Dangamuwa

Posted on

Why Your SSH Keys Break in 2026: Ed25519 vs RSA, OpenSSH Formats, and 5 Traps Every Developer Hits

#devops #security #linux #programming

Every developer has run into the dreaded Permission denied (publickey) or Load key "id_rsa": invalid format error while connecting to a remote server, configuring a CI/CD runner, or pushing to Git.

Over the last few OpenSSH releases, SSH authentication standards have undergone major security shifts. Yet countless internal wikis, Dockerfiles, and deployment scripts still paste legacy ssh-keygen snippets from 2012.

Here is a breakdown of why modern SSH authentication breaks in production, how cryptographic standards have evolved, and the five traps you should watch out for.

1. The RSA-SHA1 Deprecation Trap

Since OpenSSH 8.8, the ssh-rsa signature scheme (which uses the SHA-1 hash algorithm) has been disabled by default because SHA-1 is cryptographically broken against chosen-prefix collisions.

Here is the catch that confuses many engineers: even if you generated a strong 4096-bit RSA key, your connection might still fail if the client or server negotiation falls back to the legacy SHA-1 signature algorithm rather than rsa-sha2-512 or rsa-sha2-256.

If you connect to an older appliance or legacy Git server from a modern Linux client, you might see:

debug1: send_pubkey_test: no mutual signature algorithm
Permission denied (publickey).
Enter fullscreen mode Exit fullscreen mode

While you can temporarily patch this in ~/.ssh/config using:

Host legacy-server.internal
    PubkeyAcceptedKeyTypes +ssh-rsa
    HostkeyAlgorithms +ssh-rsa
Enter fullscreen mode Exit fullscreen mode

The correct long-term solution is to migrate away from RSA entirely and move to Ed25519.

2. Ed25519 vs RSA vs ECDSA: The Modern Standard

When generating a new SSH keypair today, the recommended default is Ed25519:

ssh-keygen -t ed25519 -a 100 -C "dev@company.com"
Enter fullscreen mode Exit fullscreen mode

Why is Ed25519 preferred over RSA and ECDSA?

  • Key Size & Performance: Ed25519 public keys are just 68 characters in base64 (32 bytes raw), compared to 700+ characters for RSA-4096. Key generation, signing, and verification are orders of magnitude faster.
  • Side-Channel Resistance: Ed25519 arithmetic operations execute in constant time, preventing timing attacks.
  • Deterministic Signatures: Standard ECDSA (such as NIST P-256) relies on a cryptographically secure random number ($k$) for every signature. If the entropy source fails or generates a biased nonce, the private key can be extracted mathematically from two signatures. Ed25519 generates the nonce deterministically from the private key and message hash.

3. OpenSSH Key Format vs PKCS#8 vs PEM

OpenSSH introduced its own proprietary private key container format starting in OpenSSH 6.5, making it the default in version 7.8:

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAA...
-----END OPENSSH PRIVATE KEY-----
Enter fullscreen mode Exit fullscreen mode

Older legacy libraries (such as older versions of Python's paramiko, Java JSch, or legacy cloud provisioning agents) only understand OpenSSL PEM format:

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA0...
-----END RSA PRIVATE KEY-----
Enter fullscreen mode Exit fullscreen mode

If your automated pipeline throws InvalidKeyException or unsupported key type, your tooling cannot parse the modern OpenSSH container. You can export a PEM-compatible format using:

ssh-keygen -p -m PEM -f ~/.ssh/id_rsa
Enter fullscreen mode Exit fullscreen mode

When provisioning environments or verifying key structures without local terminal access, using a client-side SSH key generator allows you to inspect WebCrypto key generation, wire encodings, and fingerprint hashes in-browser without exposing credentials to a server.

4. Silent Permission Failures and StrictModes

By default, the SSH daemon enables StrictModes yes in /etc/ssh/sshd_config. This setting checks file ownership and permissions before reading authentication files. If permissions are too permissive, authentication fails silently without specific client error messages.

The required permissions are:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
Enter fullscreen mode Exit fullscreen mode

The hidden catch: sshd also checks the permissions of the user's home directory (/home/username) and parent path. If /home/username is writable by group or others (chmod 775 or 777), sshd will refuse all key authentication.

5. Missing Trailing Newlines in authorized_keys

A common CI automation bug occurs when appending public keys via deployment scripts:

# Buggy script:
echo -n "$NEW_PUBLIC_KEY" >> ~/.ssh/authorized_keys
Enter fullscreen mode Exit fullscreen mode

If the existing authorized_keys file lacks a trailing newline, the new key gets appended directly onto the end of the previous key line. This results in one long corrupted line containing two invalid public keys, breaking access for both users simultaneously.

Always append public keys with explicit newlines:

(echo ""; cat new_key.pub; echo "") >> ~/.ssh/authorized_keys
Enter fullscreen mode Exit fullscreen mode

Key Takeaways

  1. Default to ssh-keygen -t ed25519 -a 100 for all new systems.
  2. If you must maintain RSA keys, ensure your infrastructure supports rsa-sha2-512.
  3. Verify directory permissions down to the user's home folder when debugging authentication rejections.
  4. For quick local testing or generating zero-install keypairs, tools like the Nutilz SSH Key Generator provide instant client-side Ed25519/RSA generation, MD5/SHA256 fingerprint calculations, and PEM outputs with zero server-side storage.

Top comments (0)