An ~/.ssh/config That Scales: Hosts, ProxyJump, Keepalive and Per-Host Keys

~/.ssh/config is the highest-leverage file in a developer's home directory. Ten lines replace a page of remembered flags, and a well-written config keeps working when the server list grows from three machines to fifty.

This is the structure that holds up, plus the parts that quietly break.

The basics that everything builds on

Host db-prod
  HostName 10.20.4.11
  User deploy
  Port 2222
  IdentityFile ~/.ssh/id_ed25519_deploy

Now ssh db-prod does what ssh deploy@10.20.4.11 -p 2222 -i ~/.ssh/id_ed25519_deploy did. Two rules make the rest predictable:

• Host is a pattern, not a name. Host db-* matches any alias starting with db-.
• The first matching value wins. SSH reads the file top to bottom and keeps the first value it finds for each keyword. Put specific blocks above wildcard blocks, or the wildcard wins and you will spend an hour wondering why User is wrong.

That second rule is why the most common config mistake looks like this:

Host *
  User deploy

Host db-prod
  User postgres

db-prod still connects as deploy, because User deploy was set first. Move Host * to the bottom.

Jump hosts without retyping them

Any machine behind a bastion deserves a permanent alias:

Host bastion
  HostName bastion.example.com
  User ops

Host db-prod
  HostName 10.20.4.11
  User deploy
  ProxyJump bastion

Chains work: ProxyJump bastion1,bastion2. Each hop is resolved from the previous one, so db-prod keeps its private HostName and nothing about it needs to be routable.

The old ProxyCommand nc -X connect -x ... recipe still appears in tutorials. ProxyJump replaced it in OpenSSH 7.3 (2016) and is faster: it reuses a single transport rather than piping through a subprocess.

Keepalive, so idle sessions survive

A session that dies after five minutes of inactivity is usually NAT or a firewall dropping the flow, not the server:

Host *
  ServerAliveInterval 30
  ServerAliveCountMax 3
  TCPKeepAlive yes

ServerAliveInterval sends a lightweight keepalive every 30 seconds; ServerAliveCountMax 3 drops the connection after three unanswered ones. Ninety seconds of silence means the link is genuinely gone, which is far more useful than a hung terminal.

For connections you open dozens of times a day, add multiplexing:

Host *
  ControlMaster auto
  ControlPath ~/.ssh/cm-%r@%h:%p
  ControlPersist 10m

The first connection pays the handshake; the rest reuse the socket. scp, git fetch and a shell on the same host all share one authenticated session. The catch is that a stale socket leaves you with Control socket connect: Connection refused long after the network changed — remove ~/.ssh/cm-* when that happens.

Per-host keys, and why they get ignored

Host github-work
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_work
  IdentitiesOnly yes

Without IdentitiesOnly yes, the agent offers every key it holds, and servers with MaxAuthTries set to a low number reject the connection before it reaches the right key. -v shows Offering public key: ... repeated until the server gives up. That single line prevents it.

The same block is how you keep two GitHub accounts apart: git@github-work:org/repo.git picks the work key, [e-mail]:org/repo.git picks the personal one.

Wildcards for groups of machines

Host *.staging.example.com
  User deploy
  StrictHostKeyChecking accept-new

Host 10.20.*
  User ops
  ForwardAgent no

StrictHostKeyChecking accept-new trusts a host on first sight and refuses if the key later changes. That is the sane middle ground for ephemeral CI-style hosts: no prompt on first connect, still protected against a swapped key.

ForwardAgent no on a private range is a deliberate choice — agent forwarding lets anyone with root on that host use your key.

Forwarding that lives in the config

Tunnels you open every morning belong in the config:

Host db-prod
  HostName 10.20.4.11
  ProxyJump bastion
  LocalForward 13306 127.0.0.1:3306

Now ssh db-prod gives you a shell and a working local database port. For tunnel-only hosts, add the alias:

Host db-tunnel
  HostName 10.20.4.11
  ProxyJump bastion
  LocalForward 13306 127.0.0.1:3306
  ExitOnForwardFailure yes
  RequestTTY no
  RemoteCommand cat

ExitOnForwardFailure yes is underrated: without it, SSH happily connects, fails to bind the port, and leaves you with a shell and no tunnel.

Debugging order

When an alias misbehaves, this sequence finds the cause in under a minute:

ssh -G db-prod          # print the effective config after all merging
ssh -v db-prod 2>&1 | grep -i offering   # which keys were offered
ssh -vvv db-prod        # full handshake, including ProxyJump hops

ssh -G is the important one. It shows what SSH actually decided after reading every file, which resolves "I set it but it is not applied" instantly.

Write it once, generate it the same way

The keywords are stable, but the block for a jump host plus forwarding plus a specific key has a lot of surface for a typo. I generate the skeleton with SSH config example generator and then paste it into ~/.ssh/config — it covers Host, HostName, User, Port, IdentityFile, ProxyJump and LocalForward in the right order.

If you connect to a lot of these hosts rather than a handful, the config file stops being enough and you want a session manager that reads it directly. That is a separate problem, but it starts from the same file, so getting the config right is never wasted work.

The checklist

• Specific Host blocks above Host *, always.
• IdentitiesOnly yes on every block with an IdentityFile.
• ProxyJump instead of ProxyCommand.
• ServerAliveInterval 30 plus ControlMaster for anything used daily.
• ExitOnForwardFailure yes on tunnel-only hosts.
• ssh -G before you file a bug against the server.