ssh-agent

#ssh #security #unix #devops

Technical Beauty — Episode 31

You have typed your passphrase four times this morning. Once to pull from GitHub. Once to deploy to staging. Once to SSH into production. Once because you mistyped it the third time and had to start over. By lunch, you will have typed it twelve more times, and by the end of the week you will have created a key without a passphrase because life, one feels, is too short.

Congratulations. Your private key is now a plaintext file on disk. Anyone who reads ~/.ssh/id_ed25519 owns every server you can reach. This is not a hypothetical. This is a Tuesday.

In 1995, a password-sniffing attack hit the network at Helsinki University of Technology. Tatu Ylonen, a researcher there, decided this was unacceptable and wrote SSH that same year. As part of that implementation, he wrote ssh-agent: a process that holds your private keys in memory and signs authentication challenges on your behalf. The key never leaves the process. Not to the client. Not to the server. Not to the wire. Never.

Thirty-one years later, that agent is still the authentication backbone of modern software delivery. It has been rewritten, hardened, extended, and maintained by the OpenSSH team (forked from Ylonen's ssh 1.2.12 in 1999, first stable release with OpenBSD 2.6 on 1 December 1999). The current source is at version 1.324, with contributions from Markus Friedl, Aaron Campbell, and Theo de Raadt, among others. The design, however, has not fundamentally changed. It did not need to.

The Design

The entire agent is 2,624 lines of C in a single file: ssh-agent.c. Key storage, socket management, the full agent protocol, PKCS#11 smart card support, FIDO/U2F hardware key support, agent forwarding with destination constraints, and locking. In one file. Shorter than most React components one has had the pleasure of reviewing.

The API is a Unix domain socket. When the agent starts, it creates a socket, sets its permissions to owner-only (umask(0177)), and prints two environment variables:

SSH_AUTH_SOCK=/tmp/ssh-XXXXXXXXXX/agent.12345
SSH_AGENT_PID=12345

That is the entire interface. No configuration file. No service manager. No daemon registration. No YAML. The socket exists. Programs that know SSH_AUTH_SOCK can talk to it. Programs that do not, cannot. ls -la $SSH_AUTH_SOCK tells you everything you need to know about your authentication state. One does find this rather refreshing.

The protocol is a packetised request-response exchange with five core operations:

Operation	Client sends	Agent returns
List keys	`REQUEST_IDENTITIES`	List of public keys
Sign data	`SIGN_REQUEST` (data + key reference)	Signature bytes
Add key	`ADD_IDENTITY` (private key + constraints)	Success/failure
Remove key	`REMOVE_IDENTITY`	Success/failure
Lock/unlock	`LOCK` / `UNLOCK` (passphrase)	Success/failure

Five operations. The IETF draft describing the protocol (draft-ietf-sshm-ssh-agent, progressing to RFC) is shorter than most framework tutorials. The entire specification fits in your head, which is, one suspects, rather the point.

The Elegance

The authentication flow:

Your SSH client connects to a server.
The server sends a challenge (data to be signed).
The client forwards the challenge to the agent via the Unix socket.
The agent signs the challenge with the private key using sshkey_sign().
The agent returns only the signature.
The client sends the signature to the server.

Step four is where the beauty lives. The agent calls the signing function internally. The result, a sequence of signature bytes, is sent back to the client. The private key material never crosses a process boundary. It is never serialised to any output channel. It exists in exactly one place: the agent's memory, in an in-memory linked list of Identity structures.

When the agent process terminates, the operating system reclaims the memory. The keys are gone. No cleanup script. No cache file to purge. No "secure delete" to trust. No residual state. The process was the vault, and the vault is demolished.

The agent also monitors its parent process. If the parent shell dies (detected by getppid() returning 1), the agent cleans up its socket and exits. No orphan processes accumulating in your process table. One does appreciate software that tidies up after itself.

The Constraints

ssh-add is the interface for managing keys in the agent:

ssh-add ~/.ssh/id_ed25519          # Add a key (passphrase prompt)
ssh-add -t 3600                     # Key expires after 1 hour
ssh-add -c                          # Confirm each signing operation
ssh-add -l                          # List loaded key fingerprints
ssh-add -D                          # Delete all keys

The -t flag stores an absolute expiry time in the identity's death field. After that time, the key is automatically removed. No cron job. No external timer. The constraint is part of the key's identity.

The -c flag requires interactive confirmation (via ssh-askpass) before every signing operation. The agent calls confirm_key() before executing sshkey_sign(). You see who is asking, and you decide whether to sign. For forwarded agents, this is the difference between convenience and a security incident.

The locking mechanism (ssh-agent -x or the LOCK protocol message) makes all keys inaccessible until the agent is unlocked with the correct passphrase. The passphrase is stored as a salted hash, not plaintext. For stepping away from your desk, this is rather more civilised than terminating the agent and re-adding all keys when you return.

The Security

The agent takes security seriously at the implementation level:

Anti-tracing: platform_disable_tracing(0) at startup prevents other processes from attaching a debugger to read key material from agent memory.
Privilege dropping: setegid(getgid()) and setgid(getgid()) at startup. The agent runs with minimal privileges.
Memory hygiene: sshkey_free() for key material. freezero() for PINs (zeroes memory before freeing). explicit_bzero() for lock password hashes, which prevents the compiler from optimising away the zeroing because the variable is "no longer used."
Socket permissions: Created with umask(0177). Owner-only access. The directory is created with mkdtemp() for additional protection.

These are not features listed on a marketing page. These are manners. The kind of quiet, disciplined engineering that distinguishes software built by people who understand what they are protecting.

Agent Forwarding

Agent forwarding (ssh -A or ForwardAgent yes) allows a remote machine to use your local agent. The remote sshd creates a Unix socket, sets SSH_AUTH_SOCK, and tunnels requests back through the SSH connection to your local agent. You can hop from server to server without copying keys.

The risk: anyone with root on the remote machine can access the forwarded socket while your session is active. They can use your agent to authenticate to any server your keys can reach.

The mitigations are characteristically minimal. ssh-add -c requires confirmation per use. ProxyJump avoids forwarding entirely by routing through an intermediate host without giving it agent access. Recent OpenSSH versions add destination constraints: keys can be restricted to specific hosts, so a compromised jump host cannot use the forwarded agent to reach arbitrary targets.

The design philosophy is consistent: provide the mechanism, make the risks visible, let the operator decide. No hand-holding. No default that pretends to be safe. Honest tooling for people who read the man page.

The Proof

Every CI/CD system on earth uses ssh-agent. GitHub Actions has a dedicated webfactory/ssh-agent action. GitLab's official documentation recommends eval $(ssh-agent -s) as the standard pattern for pipeline SSH key management. Jenkins, CircleCI, Buildkite, Travis CI: all use the agent protocol.

macOS integrated ssh-agent into the system keychain with Leopard in 2007. ssh-add --apple-use-keychain stores passphrases in Keychain, bridging the Unix tool with Apple's credential infrastructure. GNOME and KDE auto-start agent-compatible daemons. Windows 10 added an OpenSSH agent as a system service in 2018. 1Password and Bitwarden now implement the agent protocol natively.

An API designed by a Finnish researcher in 1995, implemented in a single C file, communicating over a Unix socket named by an environment variable, is the authentication backbone of modern software delivery. One does find this rather beautiful.

The Principle

ssh-agent is 2,624 lines of C. One file. One socket. One environment variable. No configuration file. No YAML. No cloud dependency. No subscription. The private key never leaves the process, the process never outlives the session, and the session never trusts more than it must.

Tatu Ylonen wrote it because typing passphrases was tedious. The best Unix tools often begin this way: someone finds a task annoying, writes a small programme to solve it, and designs it with enough discipline that it still works thirty-one years later. No rewrite. No framework migration. No breaking changes.

Technical beauty emerges from reduction. ssh-agent reduced authentication to five operations, one socket, and the guarantee that the secret never leaves the room.

Read the full article on vivianvoss.net →

By Vivian Voss — System Architect & Software Developer. Follow me on LinkedIn for daily technical writing.