How to Manage Multiple SSH Key Pairs

Most developers will interact with resources that use SSH keys instead of passwords. I recently overheard someone say that he uses the same SSH key for all of his accounts, which is a bad idea from a security perspective. The more places a single key is authorized, the more valuable that key becomes. If that key gets compromised, more targets are put at risk. There are other reasons I might have multiple key pairs:

• Team resources that share the same key
• Older systems that don't support the newest ed25519 encryption algorithm
• Separate keys for each consulting client I have

When I initially started managing multiple SSH key/password combinations on my personal machine, I learned best practices from a variety of sources. I’m writing this information down in one place for the benefit of others. My current OS of choice is MacOS, but these instructions should work for any *nix system. Atlassian recommends users replace their SSH keys once a year for security. Following these steps will ensure that you can.

First: Generate a new key

Open terminal and navigate to ~/.ssh to generate a new SSH key:

ssh-keygen -t ed25519 -f ~/.ssh/key_name -C "[e-mail]"

Here is what each flag means:

• -t specifies the algorithm that makes the key.
• -f specifies a custom name for the key (assuming you're in the ~/.ssh directory), and an alternate location if it’s in the form of a path. The key_name is the name of the key. Make this as specific as possible.
• -C is an option to include a comment. It can be anything but is usually in the form user@host (who generated the key)

I always use a key name that is specific and makes sense to me. This makes key management easier in the long term. You should use a passphrase when prompted.

Second: Create known_hosts file

When you complete the first step two files are created: key_name and key_name.pub. The first is your private key and the second (with the .pub extension) is your public key.

Create a known_hosts file for each account you have because it makes diagnosing issues easier when you have multiple keys. Ideally the name of this file is similar enough to the key name that you aren’t confused later.

touch known_hosts_keyname 

Third: Set up the config file

The config file sets options for each host. Create the config file if it doesn’t already exist and then open it for editing. I label each key for visual neatness and to avoid confusion as the list of keys gets longer over time. Create a comment using the # at the start of a line to label each host. The text in the picture below is available here to save you some typing.

Here is the breakdown of what each line means:

• Host is a pattern matcher that is used to differentiate between these sets of configurations. Keep it the same as the HostName so it matches hosts in connections correctly without additional specification.
• The URL on the HostName line is the base URL where the repository resides. For example, if you have a personal account on github with personal projects, the URL will be github.com.
• User for git based systems will be git. The value of User will be different if you connect to something else (i.e. ec2-user for connecting to an Amazon AWS EC2 instance)
• IdentityFile asks for the location of the identity key we made. Type in the respective path here.
• AddKeysToAgent allows a private key that is used during authentication to be added to ssh-agent if it is running
• UseKeychain (macOS only) allows the computer to remember the password each time it restarts
• UserKnownHostsFile specifies an exact location to store all hosts you connect to when you’re using that profile. Provide the respective paths here and choose a unique known hosts file name (see step 2 above) so that troubleshooting and key maintenance over time is easier.
• IdentitiesOnly specifies that only the keys provided must be used to connect to a host, even if another service like the ssh-agent offers a key for use.

Fourth: Add keys to ssh agent

Add keys to ssh agent if passphrase was used. Skip to the next step if you didn't use a passphrase. Start the ssh agent in the terminal:

eval "$(ssh-agent -s)"

Add private keys to the agent in terminal:

ssh-add --apple-use-keychain ~/.ssh/key_name

Note that the --apple-use-keychain option works only on Mac for keychain access. (In versions of MacOS prior to Monterey, use -K rather than --apple-use-keychain).

Fifth: Finishing up

If you're using a service like Bitbucket or Github, add public keys to the clipboard and paste them into the appropriate account (i.e. Bitbucket):

cat key_name.pub | pbcopy 

Finally, verify the configuration in the terminal:

ssh -T [e-mail] 

With multiple keys, I have the option of creating new keys as needed to keep each connection secure. If I have a single compromised key, then I only worry about changing that single key. My config file makes it easy for me to use multiple keys.

References
OpenSSH documentation
Bitbucket documentation
Github documentation

Comments

[brandon_wallace]

Nice article. I recommend adding a comment and a file name to ssh-keygen.

-C "<comment>"
-f /path/to/key

Examples:

$ ssh-keygen -t rsa -b 4096 -C "[e-mail]" -f ~/.ssh/id_rsa_home
$ ssh-keygen -t ed25519 -C "[e-mail]" -f ~/.ssh/id_ed25519_work

[Scott Sawyer]

Good post. Another ssh_config trick i sometimes use is ForwardAgent, so i can connect through to a third party service directly from a tunneled server using my local ssh_agent. (Like if you have to connect to github from a remote server)

[lesha 🟨⬛️]

If I have a single compromised key

The private key should never ever leave your computer. If you have single compromised key that means someone's been in your system and you might as well treat them all as compromised

Therefore, by having multiple keys you don't gain any additional security, you only gain additional headache each time you restart the system and have to ssh-add each one of them

[Alexey]

I'd recommend using

ssh-keygen -t ed25519 -f keyname

ed25519 is much better than RSA - more secure, shorter keys.

E.g. see medium.com/risan/upgrade-your-ssh-...

[Joseph Midura]

This is an excellent point. While researching this article, I was surprised to see that Bitbucket documentation uses RSA by default (April 2021). I'll likely change my first step to recommend ed25519 when I have a chance to do some edits.

[Phil Ashby]

Thanks Joseph, this really is something I should do more often than I do (to reduce the attack surface of a compromised key)! I tend to separate by activity area (ie: work, home, project) rather than individual targets.

Also worth noting is that password managers frequently integrate via ssh-agent (eg: techrepublic.com/article/how-to-in...), which can help keep everything tidy and safely enable portability (how many times have I left my offline key store at home.. sheesh!).

As a final security comment, ssh private keys are only as safe as their pass-phrases, so when you say "Use a passphrase when prompted.", that's likely the weakest link in this setup - possibly another good reason to delegate to a password manager with a strong pass phrase.

[Joseph Midura]

I agree that password managers are a must, and I struggled with how in depth to make this post to keep it beginner friendly. I'll use your suggestion for an optional/advanced section when I have time to do some edits.

[Daniel Uber]

One thing to bear in mind is that the ssh config setup (specifying which identities to use for which hosts) is important here. I've seen situations where having too many personal keys (in your ~/.ssh/ directory) caused login failures (when not IdentityFile is passed, the ssh command will offer them one by one to the remote, and the remote will interpret this as a brute force/too many failed logins attempt).

It's possible adding a host: * section with a preferred default key may be all that's needed to address that.

[Dinko Pehar]

Just add here, I also use simple utility called stormssh. It's a simple command utility for managing ssh config file easily.

Example:

storm add --id_file ~/.ssh/your_key production-server [e-mail]

And config file in .ssh folder looks like:

Host production-server
    hostname your-domain.com
    user dev
    port 22
    identityfile "/home/dinko/.ssh/your_key"

Then you can connect to server as ssh production-server.

[than0s]

Nice article. However, I still don't understand the use of multiple key pairs as mitigation of a compromised private key. The private key should exist strictly on a single device, while the public one may exist on many. If a private key is compromised that would mean the same for all the rest existing ones on the same device.

[Mac]

Very useful article: everybody has a use of a reference on how to create and configure private keys. 👍🏻

The only thing I don't understand is: what angle of attack would require the use of different keys for every account? It seems to me that one key by device is enough:

• the only practical means to get access to my private keys is to get access to my device.
• if my keys are insecure (wrong algorithm, not enough bits) it will most probably affect all of them anyway.
• if I lose my device I easily know what keys to deactivate on all my accounts.

Am I missing something ?

[Joseph Midura]

Thanks. I've had my private keys on one or more work computers in addition to my personal laptop. If the machine isn't mine, I want to be able to limit access to the key. I've also shared keys with others for team resources, which also makes multiple key management necessary.

[armyofda12mnkeys]

Note I have a personal github and a work github account... and I used something like this instead:

//personal account for github.com, note can also choose to leave this as 'Host github.com' if dont' have other keys and want this to be the default
Host github-my_personal_username
HostName github.com
User git
IdentityFile ~/.ssh/id_rsa-github-my_personal_username
// IdentitiesOnly yes

//work account for github.com
Host github-my_work_username
HostName github.com
User git
IdentityFile ~/.ssh/id_rsa-github-my_work_username
// IdentitiesOnly yes

and just change the clone url slight when needed:
$ git clone git@github-my_work_username:me/repo.git